CI: Automate pypi wheel publishing

[WillAyd]

ref #660

This is currently pointing to the test pypi environment and we can trigger manually for now

[WillAyd]

Before we try this I think we will need to enable trusted publishing in pypi (better to start with the test environment)

https://packaging.python.org/en/latest/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows/#configuring-trusted-publishing

[assignUser]

Wheels are not signed during release right? Otherwise I'd say download the wheels from the release and push those. (Or make use of reproducible builds)

[paleolimbot]

Thank you for looking into this!

I have to admit that what I had in mind was from the local end...something like "find the wheels job on the RC branch:

arrow-nanoarrow/dev/release/02-sign.sh

Lines 50 to 55 in e54b7df

	run_id=$(gh run list \
	--repo "${REPOSITORY}" \
	--workflow=packaging.yaml \
	--json 'databaseId,event,headBranch,status' \
	--jq ".[] | select(.event == \"push\" and .headBranch == \"${tag}\") | .databaseId" | \
	head -n 1)

...then download artifacts ( https://cli.github.com/manual/gh_run_download ), then twine to upload.

Something like this can work too, although there is a bit of a chicken-and-egg problem with the Apache vote: the wheels will have already been built against the release commit before the release candidate vote happens, and we need to upload them after the release vote passes. (Not technically a problem since binaries aren't technically voted on, just not strictly necessary).

[assignUser]

You could add the wheels to the github release (though I think they might need to be signed then?) and use these in the workflow (or just use gh to get them as in your example)

the wheels will have already been built against the release commit before the release candidate vote happens, and we need to upload them after the release vote passes.

I don't see the issue? I assume you publish the github release once the vote is finished? That would trigger this workflow and get the wheels voted on and upload them?

[paleolimbot]

As written this PR would build new wheels after the release vote. If possible it would be nice to download the wheels that had already been built instead of building new ones (mostly so that when the release vote ends, all the release steps can happen immediately instead of waiting for a CI job). It is also slightly awkward if that CI job fails for whatever reason.

We could make it a script and put it in this CI job (and then if the CI job fails, you can run it locally)?

[assignUser]

Ah yeah, we were talking about the same thing ^^ +1