-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Create reusable image deploy workflow #3
base: main
Are you sure you want to change the base?
Changes from all commits
c4a2bca
19772fc
c8cacde
a223020
99a0126
81681da
328169a
ff4eaa8
47f3dc0
5e496db
03e4140
61f01be
7be3ef1
7f30b40
9ad0263
ae29afc
b207552
a5299e7
b6fe7b8
d97397b
52707c8
986880f
6c8bbd9
0787b01
819ec82
b68ebe0
fcf2d6a
b646684
81c1519
c5bb846
19dcc55
dca27c2
62abc54
27ef0e0
e4c2997
c15245c
dd138e9
cbb7880
2970609
3e6de0b
a2e6a99
e1f3131
09d95a4
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,111 @@ | ||
# Deploys artifacts using maven and optionally deploys an image to quay | ||
name: Reusable artifacts deploy | ||
|
||
on: | ||
workflow_call: | ||
inputs: | ||
createDockerImage: | ||
description: 'Whether to create a Docker image, caller must have Dockerfile in the repository' | ||
required: false | ||
type: boolean | ||
default: false | ||
quayRepository: | ||
description: The quay repository to upload the image to. The repository must belong to the dockstore quay organization. | ||
required: false | ||
type: string | ||
dockerContext: | ||
description: The Docker context containing the Dockerfile of the image to build and push. | ||
required: false | ||
default: . | ||
type: string | ||
|
||
env: | ||
IS_DEVELOP_SNAPSHOT: ${{ github.ref_type == 'branch' && github.ref_name == 'develop' }} | ||
|
||
jobs: | ||
deploy_maven: | ||
name: Maven deploy ${{ github.ref_type == 'tag' && 'tagged' || 'snapshot' }} release | ||
runs-on: ubuntu-22.04 | ||
|
||
steps: | ||
- uses: actions/checkout@v4 | ||
# Step that does that actual cache save and restore | ||
- uses: actions/cache@v4 | ||
with: | ||
path: ~/.m2/repository | ||
key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }} | ||
restore-keys: | | ||
${{ runner.os }}-maven- | ||
|
||
- name: install git secrets | ||
run: | | ||
wget --no-verbose -O git-secrets-1.3.0.tar.gz https://github.com/awslabs/git-secrets/archive/1.3.0.tar.gz | ||
tar -zxf git-secrets-1.3.0.tar.gz | ||
cd git-secrets-1.3.0 | ||
sudo make install | ||
|
||
- name: Set up JDK | ||
uses: actions/setup-java@v4 | ||
with: | ||
java-version: '21.0.2+13.0.LTS' | ||
distribution: 'adopt' | ||
# settings.xml configuration | ||
server-id: ${{ github.ref_type == 'tag' && 'central' || 'snapshots' }} | ||
server-username: DEPLOY_USERNAME | ||
server-password: DEPLOY_TOKEN | ||
|
||
- name: Set changelist version | ||
run: | | ||
set -x | ||
if ${{ github.ref_type == 'tag' }}; then | ||
# Check that the tag follows semantic versioning. Note that the regex is not super strict | ||
if [[ ${{ github.ref_name }} =~ ^[0-9]+\.[0-9]+\.[0-9a-zA-Z-]+(-[0-9a-zA-Z-]+\.[0-9a-zA-Z-]+)*$ ]]; then | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This regex is not as strict as https://semver.org/#is-there-a-suggested-regular-expression-regex-to-check-a-semver-string because that might be overkill/hard to read... thoughts? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. As long as it matches the patterns that appear in our refs, should be fine, imho |
||
# Break up the semantic version tag by the '.' delimiter and return the third field onward. Prefix this with the '.' | ||
# Ex: 1.16.0 -> .0 and 1.16.0-alpha.0 -> .0-alpha.0 | ||
CHANGELIST_VERSION=.$(echo ${{ github.ref_name }} | cut -d. -f 3-) | ||
else | ||
echo "Invalid semantic version for tag ${{ github.ref_name }}" | ||
exit 1 | ||
fi | ||
elif ${{ github.ref_name != 'develop' }}; then | ||
CHANGELIST_VERSION=.0-${{ github.ref_name }}-SNAPSHOT | ||
CHANGELIST_VERSION=${CHANGELIST_VERSION//\//-} | ||
fi | ||
|
||
echo "CHANGELIST_VERSION=${CHANGELIST_VERSION}" >> $GITHUB_ENV | ||
|
||
- name: Store Maven project version | ||
run: | | ||
set -x | ||
if ${{ env.IS_DEVELOP_SNAPSHOT }}; then | ||
echo "maven_project_version=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout)" >> $GITHUB_ENV | ||
else | ||
echo "maven_project_version=$(mvn help:evaluate -Dexpression=project.version -Dchangelist=${{ env.CHANGELIST_VERSION }} -q -DforceStdout)" >> $GITHUB_ENV | ||
fi | ||
|
||
- name: Read exported variable | ||
run: | | ||
echo "${{ env.maven_project_version }}" | ||
|
||
- name: Deploy with mvnw | ||
run: | | ||
git config --global user.email "${{ github.actor }}" | ||
git config --global user.name "${{ github.actor }}" | ||
|
||
set -x | ||
if ${{ env.IS_DEVELOP_SNAPSHOT }}; then | ||
./mvnw --batch-mode deploy -ntp -DskipTests | ||
else | ||
./mvnw --batch-mode deploy -ntp -DskipTests -Dchangelist=${{ env.CHANGELIST_VERSION }} | ||
fi | ||
env: | ||
DEPLOY_USERNAME: ${{ github.ref_type == 'tag' && 'dockstore-bot' || 'dockstore-snapshot-bot' }} | ||
DEPLOY_TOKEN: ${{ github.ref_type == 'tag' && secrets.COLLAB_DEPLOY_TOKEN || secrets.SNAPSHOT_DEPLOY_TOKEN }} | ||
|
||
deploy_image: | ||
if: ${{ inputs.createDockerImage && inputs.quayRepository != '' }} | ||
uses: dockstore/workflow-actions/.github/workflows/deploy_image.yaml@seab-6771/reusable-image-deploy | ||
with: | ||
quayRepository: ${{ inputs.quayRepository }} | ||
dockerContext: ${{ inputs.dockerContext }} | ||
secrets: inherit |
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Alot of these steps are copied from https://github.com/dockstore/dockstore/blob/b604742ab675b6ca47629e7e54a71478c6247191/.github/workflows/deploy_tagged.yml#L1 with a few modifications so that it can be reused by multiple repositories |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,85 @@ | ||
name: Reusable Image Deploy | ||
|
||
on: | ||
workflow_call: | ||
inputs: | ||
quayRepository: | ||
description: The quay repository to upload the image to. The repository must belong to the dockstore quay organization. | ||
required: true | ||
type: string | ||
dockerContext: | ||
description: The Docker context containing the Docker file of the image to build and push. | ||
required: false | ||
default: . | ||
type: string | ||
|
||
env: | ||
DOCKER_IMAGE_NAME: quay.io/dockstore/${{ inputs.quayRepository }} | ||
|
||
permissions: | ||
id-token: write | ||
contents: read | ||
|
||
jobs: | ||
deploy_image: | ||
name: Deploy image to quay.io/dockstore/${{ inputs.quayRepository }} | ||
runs-on: ubuntu-22.04 | ||
|
||
steps: | ||
- uses: actions/checkout@v4 | ||
|
||
# Step that does that actual cache save and restore | ||
- uses: actions/cache@v4 | ||
with: | ||
path: ~/.m2/repository | ||
key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }} | ||
restore-keys: | | ||
${{ runner.os }}-maven- | ||
|
||
- name: Set up JDK | ||
uses: actions/setup-java@v4 | ||
with: | ||
java-version: '21.0.2+13.0.LTS' | ||
distribution: 'adopt' | ||
|
||
- name: Configure AWS Credentials | ||
uses: aws-actions/configure-aws-credentials@v4 | ||
with: | ||
role-to-assume: ${{ secrets.AWS_ROLE_TO_ASSUME }} | ||
role-session-name: docker_checksum_upload_from_github | ||
aws-region: ${{ secrets.AWS_REGION }} | ||
|
||
- name: Login to Quay.io | ||
uses: docker/login-action@v3 | ||
with: | ||
registry: quay.io | ||
username: ${{ secrets.QUAY_USER }} | ||
password: ${{ secrets.QUAY_TOKEN }} | ||
|
||
- name: Build | ||
run: ./mvnw -B clean install -DskipTests | ||
|
||
- name: Set folder name | ||
run: | | ||
S3_FOLDER=${{ github.ref_name }} | ||
echo "S3_FOLDER=${S3_FOLDER//\//_}" >> $GITHUB_ENV | ||
|
||
- name: Build and push | ||
id: docker_build | ||
uses: docker/build-push-action@v6 | ||
with: | ||
context: ${{ inputs.dockerContext }} | ||
push: true | ||
tags: '${{ env.DOCKER_IMAGE_NAME }}:${{ env.S3_FOLDER }}' | ||
|
||
- name: Create checksums | ||
run: | | ||
docker inspect ${{ env.DOCKER_IMAGE_NAME }}:${{ env.S3_FOLDER }} | grep -A 1 RepoDigests | ||
docker inspect ${{ env.DOCKER_IMAGE_NAME }}:${{ env.S3_FOLDER }} | grep -A 1 RepoDigests | grep -oPm1 'sha256:\K\w+' > image-digest.txt | ||
|
||
- name: Get short SHA | ||
id: slug | ||
run: echo "::set-output name=sha7::$(echo ${GITHUB_SHA} | cut -c1-7)" | ||
|
||
- name: Copy checksum files | ||
run: aws s3 cp image-digest.txt s3://${{ secrets.AWS_BUCKET }}/${{ env.S3_FOLDER }}-${{ steps.slug.outputs.sha7 }}/${{ inputs.quayRepository }}/image-digest.txt | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Note: the image digest is now uploaded to a folder with the quay image name. For example, s3://***/1.16.0-alpha.1-ae8db5d/dockstore-support/image-digest.txt (this folder doesn't exist so don't look for it 🙂) |
This file was deleted.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Defining this allows us to remove all the duplicated
settings.xml
files across the various repos that upload artifacts to artifactory.