feat: native multi-platform builds via runner matrix (no QEMU emulation)#435
Open
tgenov wants to merge 7 commits intodevcontainers:mainfrom
Open
feat: native multi-platform builds via runner matrix (no QEMU emulation)#435tgenov wants to merge 7 commits intodevcontainers:mainfrom
tgenov wants to merge 7 commits intodevcontainers:mainfrom
Conversation
Add platformTag and mergeTag inputs to support building on native ARM runners in a matrix strategy, then merging per-platform images into a multi-arch manifest via docker buildx imagetools create. This avoids slow QEMU emulation for multi-platform builds by allowing each matrix job to build natively for its own platform.
The devcontainer CLI rejects --platform without --output. For native single-platform builds (platformTag set), use type=docker to load the image into the local daemon for subsequent docker push.
The devcontainer CLI rejects --platform for docker-compose-based devcontainers. When platformTag is set, the runner is already the correct native architecture, so --platform is unnecessary.
- Mirror platformTag/mergeTag logic in azdo-task (task.json inputs, runMain/runPost in main.ts, createManifest wrapper in docker.ts) - Add unit tests for createManifest in common/__tests__/docker.test.ts - Update docs/github-action.md and docs/azure-devops-task.md input tables - Add native multi-platform builds section to docs/multi-platform-builds.md with examples for both GitHub Actions and Azure DevOps Pipelines
Author
|
@microsoft-github-policy-service agree |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The current multi-platform build support (
platforminput) relies on QEMU emulation through Docker buildx. This works but has significant drawbacks:GitHub Actions and Azure DevOps both offer native ARM runners (
ubuntu-24.04-arm,ARM64pool), making emulation unnecessary if the action supports a matrix-based workflow where each platform builds natively on its own runner.Proposed Solution
Add two new inputs that enable a split build/merge workflow:
platformTag(per-platform build phase)Used in matrix jobs. Each job runs on a native runner for its target platform and sets
platformTagto a suffix (e.g.,linux-amd64,linux-arm64). The action:-{platformTag}to each image tag (e.g.,myimage:latest-linux-amd64)--platformflag needed)mergeTag(manifest merge phase)Used in a final job after all matrix builds complete. The action:
docker buildx imagetools createto combine the per-platform images into a multi-arch manifest under the original tag (e.g.,myimage:latest)Example Workflow (GitHub Actions)
Scope of Work
Per CONTRIBUTING.md, changes must maintain feature parity between GitHub Actions and Azure DevOps, include tests, and compile via
./scripts/build-local.sh.GitHub Action
platformTagandmergeTaginputs toaction.ymlrunMain()ingithub-action/src/main.ts:mergeTagis set (save state for post step)--platformand--outputflags whenplatformTagis set (native runner builds correct arch)-{platformTag}suffix to image tagsplatformTagis setrunPost()ingithub-action/src/main.ts:docker buildx imagetools createwhenmergeTagstate is presentdocker pushwith platform-suffixed tags whenplatformTagstate is presentcreateManifestwrapper togithub-action/src/docker.tsAzure DevOps Task
platformTagandmergeTaginputs toazdo-task/DevcontainersCi/task.jsonrunMain()/runPost()logic inazdo-task/DevcontainersCi/src/main.tscreateManifestwrapper toazdo-task/DevcontainersCi/src/docker.tsCommon
createManifestfunction tocommon/src/docker.tsusingdocker buildx imagetools createTests
createManifestincommon/__tests__/docker.test.tsplatformTag/mergeTaginput handlingDocumentation
docs/github-action.mdwithplatformTagandmergeTaginput descriptionsdocs/azure-devops-task.mdwith equivalent input descriptionsdocs/multi-platform-builds.mdalongside the existing QEMU approachDesign Notes
Why two inputs instead of extending
platform?The existing
platforminput (e.g.,linux/amd64,linux/arm64) triggers a single QEMU-emulated build. The new inputs are orthogonal —platformTagtags the output of a single-platform native build, andmergeTagmerges tagged outputs. This avoids breaking the existing QEMU-based workflow.Why
docker buildx imagetools createinstead ofdocker manifest?imagetools createworks with remote registry images without pulling them locally, and supports OCI image indexes natively. It is the recommended approach for combining multi-platform images that are already pushed to a registry.Runner polymorphism
This design makes the action polymorphic across runner types. Without the new flags, the action behaves as before — a single QEMU-emulated build on one runner. With
platformTagandmergeTag, the same action drives a matrix of native runners where each builds its own architecture natively, then a final job merges the results.platformonlyplatform+platformTag+mergeTagBackwards compatibility
When neither
platformTagnormergeTagis set, the action behaves exactly as before. The existingplatforminput with QEMU emulation continues to work unchanged.Open Questions
Polymorphism vs. separation of concerns
The current design overloads a single action with two distinct multi-platform strategies (QEMU vs. native runners) controlled by input flags. An alternative would be to ship the native runner workflow as a separate action (e.g.,
devcontainers/ci/native-multiplatform) with a dedicated interface, keeping the original action focused on single-runner builds.Polymorphism keeps the surface area small and avoids forcing users to switch actions when migrating from QEMU to native runners. Separation would make each action's contract simpler and avoid the conditional logic around
platformTag/mergeTagthat touches bothrunMainandrunPost. Which trade-off does the project prefer?Note
We are already using the fork internally with GitHub Actions. The DevOps task implementation follows the same patterns but has not been tested in an AzDO pipeline.