Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create reusable image deploy workflow #3

Open
wants to merge 43 commits into
base: main
Choose a base branch
from

Conversation

kathy-t
Copy link

@kathy-t kathy-t commented Nov 19, 2024

https://ucsc-cgl.atlassian.net/browse/SEAB-6771

This PR:

  • Adds a reusable deploy_image.yaml workflow that uploads an image to quay and the image digest to S3.
  • Adds a reusable deploy_artifacts.yaml workflow that uses maven to deploy artifacts to artifactory, and optionally invokes deploy_image.yaml to deploy an image.
    • This workflow works for both tags and branches. We had a lot of duplicated code between the two so I just combined them into one workflow.
    • For a tag, it'll create a tagged release and is essentially the Tagged release deploy that we're used to.
    • For a branch, it'll create a snapshot. A develop snapshot functions exactly the same as before, ex: 1.17.0-SNAPSHOT. A feature branch snapshot is named 1.17.0-<branch-name-where-slashes-are-subbed-with-hyphens>-SNAPSHOT like 1.17.0-feature-seab-6771-reusable-image-deploy-SNAPSHOT.

See the following PRs where these workflows are used:

run: echo "::set-output name=sha7::$(echo ${GITHUB_SHA} | cut -c1-7)"

- name: Copy checksum files
run: aws s3 cp image-digest.txt s3://${{ secrets.AWS_BUCKET }}/${{ env.S3_FOLDER }}-${{ steps.slug.outputs.sha7 }}/${{ inputs.quayRepository }}/image-digest.txt
Copy link
Author

@kathy-t kathy-t Nov 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: the image digest is now uploaded to a folder with the quay image name. For example, s3://***/1.16.0-alpha.1-ae8db5d/dockstore-support/image-digest.txt (this folder doesn't exist so don't look for it 🙂)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alot of these steps are copied from https://github.com/dockstore/dockstore/blob/b604742ab675b6ca47629e7e54a71478c6247191/.github/workflows/deploy_tagged.yml#L1 with a few modifications so that it can be reused by multiple repositories

distribution: 'adopt'

- name: Set changelist
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This allows us to call the tagged release workflow without providing the changelist version. This info is already part of the tag so this step is just figuring out the changelist version from the tag

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neat!
Just to think it through, does this need error handling in the case that someone accidentally makes a tag like
1.17-alpha.0

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be clear, not because I want to support it, just to have it die gracefully

@@ -1,54 +1,27 @@
name: Reusable tagged release deploy

on:
workflow_dispatch:
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this section is needed. The calling workflow should have the workflow_dispatch defined so this isn't actually used.

Copy link
Member

@denis-yuen denis-yuen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comments on using this in dockstore/dockstore?

distribution: 'adopt'

- name: Set changelist
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neat!
Just to think it through, does this need error handling in the case that someone accidentally makes a tag like
1.17-alpha.0

Comment on lines +52 to +55
# settings.xml configuration
server-id: ${{ github.ref_type == 'tag' && 'central' || 'snapshots' }}
server-username: DEPLOY_USERNAME
server-password: DEPLOY_TOKEN
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Defining this allows us to remove all the duplicated settings.xml files across the various repos that upload artifacts to artifactory.

set -x
if ${{ github.ref_type == 'tag' }}; then
# Check that the tag follows semantic versioning. Note that the regex is not super strict
if [[ ${{ github.ref_name }} =~ ^[0-9]+\.[0-9]+\.[0-9a-zA-Z-]+$ ]]; then
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a simple check just to make sure that the tag generally follows semantic versioning. There is a much stricter regex out there, but I figured it might be overkill here

set -x
if ${{ github.ref_type == 'tag' }}; then
# Check that the tag follows semantic versioning. Note that the regex is not super strict
if [[ ${{ github.ref_name }} =~ ^[0-9]+\.[0-9]+\.[0-9a-zA-Z-]+(-[0-9a-zA-Z-]+\.[0-9a-zA-Z-]+)*$ ]]; then
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This regex is not as strict as https://semver.org/#is-there-a-suggested-regular-expression-regex-to-check-a-semver-string because that might be overkill/hard to read... thoughts?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As long as it matches the patterns that appear in our refs, should be fine, imho

Copy link
Member

@denis-yuen denis-yuen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need to be followed up with a ticket to update the release page (I think it's on github now ... but lacking images?)

Please link with what you end up updating if so

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants