Multi-arch image support in CI via splitting builds and using `docker manifest`

[matthewfeickert]

I already know how to do this manually with things like

multi_platform:
	docker pull python:3.10-slim-bullseye
	docker pull tonistiigi/binfmt
ifeq ($(shell uname -m),x86_64)
	docker run --privileged --rm tonistiigi/binfmt --install arm64
else
	docker run --privileged --rm tonistiigi/binfmt --install amd64
endif
	docker buildx create \
		--name buildx_builder \
		--driver docker-container \
		--bootstrap \
		--use
	docker buildx build \
	--file Dockerfile \
	--platform linux/amd64,linux/arm64 \
	--build-arg BASE_IMAGE=python:3.10-slim-bullseye \
	--build-arg HEPMC_VERSION=3.2.5 \
	--build-arg LHAPDF_VERSION=6.5.3 \
	--build-arg FASTJET_VERSION=3.4.0 \
	--build-arg PYTHIA_VERSION=8308 \
	--tag matthewfeickert/pythia-python:pythia8.308 \
	--tag matthewfeickert/pythia-python:latest \
	--push \
	.
	docker buildx stop buildx_builder
	docker buildx rm buildx_builder

but to do this in CI this is too slow (takes almost 6 hours) with

...
    - name: Build and publish to registry
      id: docker_build_latest
      uses: docker/build-push-action@v3
      with:
        context: .
        file: Dockerfile
        tags: ${{ github.repository }}:latest,${{ github.repository }}:pythia8.308,${{ github.repository }}:pythia8.308-hepmc3.2.5-fastjet3.4.0-python3.10
        labels: |
          org.opencontainers.image.source=${{ github.event.repository.html_url }}
          org.opencontainers.image.revision=${{ github.sha }}
        push: true
        platforms: linux/amd64,linux/arm64

taking far too long and can time out.

As pointed out by Anthony Sottile on Twitter, probably the easiest way to get around this is to build separate images in CI (using GitHub Actions for linux/amd64 and Cirrus CI for linux/arm64) with something like

docker buildx build \
	--file Dockerfile \
	--platform linux/amd64 \
	--tag matthewfeickert/pythia-python:latest-amd64 \
	--push \
	.

on GitHub Actions (x86_64) and

docker buildx build \
	--file Dockerfile \
	--platform linux/arm64 \
	--tag matthewfeickert/pythia-python:latest-arm64 \
	--push \
	.

on Cirrus CI (aarch64). Once both of those images are up on the container registry of choice (here Docker Hub but probably also want to do this for ghcr as well later) would then want to in another workflow do

$ docker manifest create matthewfeickert/pythia-python:latest \
    --amend matthewfeickert/pythia-python:latest-amd64 \
    --amend matthewfeickert/pythia-python:latest-arm64
$ docker manifest push matthewfeickert/pythia-python:latest

I assume that as docker manifest create --help doesn't make it clear, there is no way to to multiple manifest tags at once and so to also do a tag of matthewfeickert/pythia-python:pythia8.308 it would require

$ docker manifest create matthewfeickert/pythia-python:pythia8.308 \
    --amend matthewfeickert/pythia-python:latest-amd64 \
    --amend matthewfeickert/pythia-python:latest-arm64
$ docker manifest push matthewfeickert/pythia-python:pythia8.308

[matthewfeickert]

I'm also not sure if I'll need to manually use docker manifest annotate to add in platform architecture and os information, or if that will get automatically picked up from use of the --platform tag in the docker buildx build step.

[asottile]

one slight correction -- you don't need to call docker manifest create twice -- you'd use docker tag instead to copy the name