Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Gitlab #35

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

RytisBa
Copy link
Contributor

@RytisBa RytisBa commented Nov 4, 2023

I have added support for GitLab merge requests as well as made some minor adjustments to the code.

@johnlk Let me know if you think Gitlab support could be added to this project or should I maintain a separate fork for it (maybe the project was intended only as GitHub workflow, in that case, I'll close this PR)

Changes:

  • A workflow for docker image build was added
  • Multi stage build for Dockefile. By moving docpars installation to a separate layer we can save some space on wget package. The final image is 15.6Mb locally and 7.8Mb when pushed to registry.
  • Added git_provider variable and renamed GITHUB_TOKEN to git_token
  • Added Gitlab provider support
  • Fixed prompt code block formatting to escape special characters (`) as script was complaining about "relevant-coding-language" function not being present
  • Added primary script termination handling from function so the script fails with error if GPT/Gitlab API returns an error
  • Added color into error/info messages
  • Moved default env variable setting into utils.sh so they can be re-used by Gitlab and not pushed by workflow

I'm not sure about the following:

  1. Since we will need docker image for Gitlab CI to work, I have added docker build workflow but I have not used Github actions so maybe it needs adjusting to fit this project flow?
  2. While debugging locally, I could not get files_to_ignore to work with multiple files, I think the current workflow is only passing first file for exclusion and anything after that is ignored. I have tested it by creating a simple variable:
export FILES_TO_IGNORE='"README.md" "assets/*" "package-lock.json"'

And then running entrypoint.sh script we get Invalid arguments. error when trying to parse variable using docpars.
Could you verify if it is working as intended in main branch on not a bug on my side?

I'm also attaching a few screenshots of the integration from the GitLab UI.
image
image

Copy link
Contributor

@johnlk johnlk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great. Thanks for opening up this change. I would recommend breaking this up into two changes. There are the quality of life changes like the docker build stages update, the cleaner logging utilities, and prompt character fixes, which I can approve immediately.

For the the other stuff like the new docker workflow and gitlab tweaks I just had a couple questions

steps:
- uses: actions/checkout@v4
- name: Log in to the Container registry
uses: docker/login-action@65b78e6e13532edd9afa3aa52ac7964289d1a9c1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why the hash after the @? I would expect some version number on these actions

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No real reason, just used what was recommended from GitHub documentation page:
https://docs.github.com/en/actions/publishing-packages/publishing-docker-images#publishing-images-to-github-packages

# GitHub recommends pinning actions to a commit SHA.
# To get a newer version, you will need to update the SHA.

I'll make change to update to versions instead of commit SHA.

@@ -0,0 +1,36 @@
name: Create and publish a Docker image
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The currently workflow I follow is to publish a release and Github will host that version for us. What's the intent behind having the docker image workflow? Is this more relevant for gitlab?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is correct. GitLab needs to have an image we can reference for the CI/CD job to start. You can see the reference in README.md part, here:

  image:
    name: ghcr.io/Integral-Healthcare/robin-ai-reviewer
    entrypoint: [""]

If we don't have access to this specific image, we would have to resort to using an image that already has Git installed. Then, we'd need to clone this repository and perform the build within the runner. However, this approach would necessitate the user having Docker-in-Docker (DinD) support in the runner, as well as Git and Docker installed. The recommended workflow, on the other hand, is to mount an image that contains the tools the user wants to use. Then, within the CI process, we can execute scripts from within that mounted image. In this specific tool's case, we would initiate the 'entrypoint.sh' script from the runner.

I also believe this approach can save time during the build phase. If GitHub workflows were to adopt this method, it would eliminate the need for DinD in the GitHub runner when building the image. Furthermore, using a lighter image should result in faster job start times. I haven't had direct experience with GitHub Actions, so I can't confirm if this practice is widely adopted or if people typically build images from within the runners for their workflows.

@@ -1,10 +1,18 @@
FROM alpine:3.15
FROM alpine:3.18 as docpars
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sweet, thanks for doing this multi-stage docker update

| `files_to_ignore` | No | (empty string) | A whitespace delimited list of files to ignore. |
| Name | Required | Default Value | Description |
|--|--|--|--|
| `open_ai_api_key` | Yes | N/A | An API key from Open AI's [developer portal](https://platform.openai.com/account/api-keys). |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Intention for the all caps OPEN_AI_API_KEY was to emphasize that it was a required argument

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Understood, I can change it here to have capital letters, but what about the usage from within the script? Should it also accept OPEN_AI_API_KEY as a parameter, or for consistency, stay as open_ai_api_key? It might be confusing if documentation is showing capitalized parameter while the entrypoint script accepts lower case argument.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whatever we elect for, we should be consistent. We should update the entry point script to accept an upper case argument in my opinion

echoerr "API request failed: $error"
exit 1
if [ "$error" != "null" ]; then
kill -s TERM $TOP_PID
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does kill -s TERM $TOP_PID present an issue if $TOP_PID env var doesn't exist?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had to add this so we can exit the script when something fails in the downstream. Previously, error from within the function would still continue to execute the main script and report it as a success. So, for example, if the API key for OpenAI was incorrect, it would print the error message and still report workflow/job as successful since the main script executed successfully. Maybe that is not an issue in Github actions, but in GitLab it would falsely report the result in the UI.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting, that sounds like a gitlab issue. So long as it doesn't introduce issues with the github action flow, it seems reasonable enough

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think GitHub workflow works the same, for example, here you have an error from bash script but it continued to show successful execution:
https://github.com/Integral-Healthcare/robin-ai-reviewer/actions/runs/6723617742/job/18274076022#step:3:131

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're completely right. Good catch!

@johnlk
Copy link
Contributor

johnlk commented Nov 5, 2023

@RytisBa

I'm not sure about the following:

  1. Since we will need docker image for Gitlab CI to work, I have added docker build workflow but I have not used Github actions so maybe it needs adjusting to fit this project flow?

Interesting, is it possible to define a .gitlab/ workflow directory?

  1. While debugging locally, I could not get files_to_ignore to work with multiple files, I think the current workflow is only passing first file for exclusion and anything after that is ignored.

That could be a newly discovered bug, it seems like your adjustments to the main.sh file with [] wrappers could've been a fix? Or is this still open? I'll have to play around on my side

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants