Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ci: Add bin-image workflow #4752

Merged
merged 3 commits into from
Jan 12, 2024
Merged

ci: Add bin-image workflow #4752

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
6ad07f2
Dockerfile/binary: Output the binary directly
vvoland Jan 3, 2024
ecf338f
scripts/build: Handle VERSION containing git ref
vvoland Jan 4, 2024
15d4c99
ci: Add bin-image workflow
vvoland Jan 3, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

env:
VERSION: ${{ github.ref }}

on:
workflow_dispatch:
push:
Expand Down Expand Up @@ -86,6 +89,50 @@ jobs:
path: /tmp/out/*
if-no-files-found: error

bin-image:
runs-on: ubuntu-20.04
if: ${{ github.event_name != 'pull_request' && github.repository == 'docker/cli' }}
steps:
-
name: Checkout
uses: actions/checkout@v4
vvoland marked this conversation as resolved.
Show resolved Hide resolved
-
name: Set up QEMU
uses: docker/setup-qemu-action@v3
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
-
name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: dockereng/cli-bin
tags: |
type=semver,pattern={{version}}
type=ref,event=branch
type=ref,event=pr
type=sha
-
name: Login to DockerHub
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs to skip login on PR:

Suggested change
name: Login to DockerHub
name: Login to DockerHub
if: github.event_name != 'pull_request'

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it will be removed once the PR is undrafted, this is all a part of a separate debug commit that just enables the worflow to run for this PR: 23c5809

if: github.event_name != 'pull_request'
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_CLIBIN_USERNAME }}
password: ${{ secrets.DOCKERHUB_CLIBIN_TOKEN }}
-
name: Build and push image
uses: docker/bake-action@v4
with:
files: |
./docker-bake.hcl
${{ steps.meta.outputs.bake-file }}
targets: bin-image-cross
push: ${{ github.event_name != 'pull_request' }}
set: |
*.cache-from=type=gha,scope=bin-image
*.cache-to=type=gha,scope=bin-image,mode=max

prepare-plugins:
runs-on: ubuntu-20.04
outputs:
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -124,4 +124,4 @@ FROM scratch AS plugins
COPY --from=build-plugins /out .

FROM scratch AS binary
COPY --from=build /out .
COPY --from=build /out/docker /docker
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious why this COPY had to be changed; did we add more things in /out ? Where possible, I like “copy all the things” so that the part copying things doesn’t need to know what to look for

(basically "/out" being the "contract" / API (for better words)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

scripts/build/binary produces two files, first is docker-<platform>-<arch> which is an actual binary, and the second is a docker which is a symlink to the former.

So without this change the image content would be:

/docker-<platform>-<arch>        # the actual binary
/docker                          # symlink to the above

The COPY copies the /docker which dereferences into the actual binary and puts that directly as /docker in the final image. So the image only contains the docker binary directly.

It's not strictly needed, because it should work with COPY --from=dockereng/cli /docker /docker anyway, but I'm not sure if we really need that symlink in the final image.

Alternatively I could look into changing the scripts/build/binary so that it doesn't output the symlink, but that would probably require changing other things.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, yes, forgot about the symlink; yeah. not a blocker, just something "nice if we can improve"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LOL, and of course I didn't think of docker-ce-packaging using this output, and now failng;

dest=$PWD/build/mac; cd /home/ubuntu/workspace/release-packaging_ce-nightly/packaging/src/github.com/docker/cli/build && for platform in *; do \
	arch=$(echo $platform | cut -d_ -f2); \
	mkdir -p $dest/$arch/docker; \
	cp $platform/docker-darwin-* $dest/$arch/docker/docker && \
	tar -C $dest/$arch -c -z -f $dest/$arch/docker-25.0.0-rc.2.tgz docker; \
done
cp: cannot stat 'darwin_amd64/docker-darwin-*': No such file or directory
cp: cannot stat 'darwin_arm64/docker-darwin-*': No such file or directory

23 changes: 23 additions & 0 deletions docker-bake.hcl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -165,3 +165,26 @@ target "e2e-gencerts" {
dockerfile = "./e2e/testdata/Dockerfile.gencerts"
output = ["./e2e/testdata"]
}

target "docker-metadata-action" {
tags = ["cli-bin:local"]
}

target "bin-image" {
inherits = ["binary", "docker-metadata-action"]
output = ["type=docker"]
}

target "bin-image-cross" {
inherits = ["bin-image"]
output = ["type=image"]
platforms = [
"linux/amd64",
"linux/arm/v6",
"linux/arm/v7",
"linux/arm64",
"linux/ppc64le",
"linux/s390x",
"windows/amd64"
]
}
7 changes: 7 additions & 0 deletions scripts/build/.variables
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,13 @@ else
BUILDTIME=${BUILDTIME:-$(TZ=UTC date -u --date="@${SOURCE_DATE_EPOCH:-$(date +%s)}" +"%Y-%m-%dT%H:%M:%SZ")}
fi

case "$VERSION" in
refs/tags/v*) VERSION=${VERSION#refs/tags/v} ;;
refs/tags/*) VERSION=${VERSION#refs/tags/} ;;
refs/heads/*) VERSION=$(echo "${VERSION#refs/heads/}" | sed -r 's#/+#-#g') ;;
refs/pull/*) VERSION=pr-$(echo "$VERSION" | grep -o '[0-9]\+') ;;
esac

GOOS="$(go env GOOS)"
GOARCH="$(go env GOARCH)"
if [ "${GOARCH}" = "arm" ]; then
Expand Down
Loading