Add Release Process for Building Wheels for Mac/Linux aarch64 and x86_64 by posborne · Pull Request #100 · fastly/compute-sdk-python

posborne · 2026-05-28T16:28:30Z

This work is based loosely off what is done by componentize-py and currently daylights as attachments on tags/releases in Github. Future work will build on this to do trusted publishing to pypi.

We target abi3 for broad support of modern python versions as our py03 layer is extremely small and this lets us avoid having to have wheels for a large number of python versions.

Windows support is not included in this pass -- we should be able to support it after a componentize-py bump but I wanted to have that as part of a separate change. As a test, I have created this release. We will likely move/remove this tag/release: https://github.com/fastly/compute-sdk-python/releases/tag/v0.1.0 (once CI completes).

I tested the wheel for macos locally in a project as follows as a one-off and it worked. This will not be required with pypi.

[project]
name = "bottle-app-example"
version = "0.1.0"
description = "Fastly Compute example using Bottle framework"
requires-python = ">=3.12"
dependencies = [
    "bottle>=0.12.25",
    "fastly-compute",
]

[tool.fastly-compute]
entry = "bottle_app"

[tool.uv.sources]
fastly-compute = { url = "https://github.com/fastly/compute-sdk-python/releases/download/v0.1.0/fastly_compute-0.1.0-cp312-abi3-macosx_11_0_arm64.whl" }

I tried to do what I could to achieve some level of reuse between python-ci and release but env vars can't directly be shared, so there are a few things that we may need to update in sync if bumping the (now pinned) nightly version, stable rust version, etc. There's some odd stuff I ended up doing with grabbing wasm-tools with curl that I might revisit. The cross build environment resulted in cargo grabbing stuff that wasn't for the host architecture and this was my workaround for now.

Pins to the commit already resolved in Cargo.lock (81d582a) so that future builds cannot silently pull in upstream HEAD changes.

Builds manylinux_2_28 wheels for Linux x86_64 and aarch64 (via maturin-action cross-compilation containers), native wheels for macOS x86_64 (macos-13) and aarch64 (macos-14), and a Windows x86_64 wheel. On tag push (v*) a pre-release GitHub Release is created with all wheels attached as assets so they can be installed and validated before PyPI integration. On workflow_dispatch the wheels are left as Actions artifacts only. Trusted publishing will be wired up as a follow-on step.

Enable pyo3's stable ABI support so a single wheel per platform works on CPython 3.12 and all future minor versions, avoiding an exploding release matrix. - Add abi3-py312 feature to pyo3 in Cargo.toml - Set features = ["pyo3/abi3-py312"] in [tool.maturin] so maturin emits the cp312-abi3-* wheel tag - Drop -i python3.12 from release workflow build args (not needed with abi3; maturin infers the minimum version from the Cargo feature)

- Make __pycache__ recursive (**/__pycache__/) so subdirectories under examples/, scripts/, fastly_compute/_bindings/, and tests/ are covered - Add *.pyd for Windows compiled extensions alongside the existing *.so - Add *.dSYM/ to ignore macOS debug symbol bundles emitted by maturin develop

Add an optional --tag argument (also read from $VERSION) that strips the leading 'v' and checks it against both in-tree versions. Exits non-zero if any mismatch is found, so it can be used as a hard gate in CI. Existing no-argument behaviour (file consistency check) is unchanged.

Add a check-version job that runs check_version_sync.py --tag against the triggering tag (or workflow_dispatch version input) before any builds start. All three build jobs depend on it, so a tag/file mismatch fails fast without burning build minutes.

Documents the step-by-step release flow: version bump, sync check, tagging, workflow monitoring, wheel validation, and PyPI promotion. Also covers ad-hoc test builds via workflow_dispatch and how to recover from a failed version check.

Centralise RUST_STABLE, MATURIN_VERSION, and PYTHON_VERSION into top-level env: blocks so upgrading any of them requires a single-line change: - release.yml: RUST_STABLE, MATURIN_VERSION, PYTHON_VERSION - python-ci.yml: RUST_STABLE (cross-ref to release.yml for context) Also fixes two issues introduced in recent edits: - Maturin version was inconsistent (v1.13.3 on Linux, v1.11.5 on macOS/Windows) - -i python3.12 was still present on macOS/Windows args despite abi3 not needing it - Hardcoded 1.92.0 in two cache key strings replaced with ${{ env.RUST_STABLE }}

Add .github/actions/setup-rust which installs the pinned stable and nightly toolchains in one step. Both workflows now call this action instead of duplicating the setup-rust-toolchain + rustup nightly install sequence. - python-ci.yml: drops RUST_STABLE / RUST_NIGHTLY from env entirely; calls the action with no overrides (uses defaults) - release.yml: macOS and Windows jobs call the action; Linux keeps before-script-linux (composite actions can't run inside maturin-action containers) but still reads from env vars that are kept in sync with the action defaults via a comment - rust-toolchain.toml: added to pin the stable channel for local dev and document the nightly requirement

- Add concurrency/cancel-in-progress so redundant runs on the same branch are cancelled when a new push arrives (per componentize-py CI) - Add a generate-checksums job that runs after collect-wheels and before create-github-release. It produces checksums.txt (sha256sum of all wheels) which is uploaded as an artifact and attached to the GitHub Release alongside the wheels.

v6 adds Node.js 24 runtime support and a minor credentials storage improvement. No breaking changes for our usage.

- Remove windows support for now; it should be able to be supported with a componentize-py update it appears (and some other work). For now, drop to avoid increasing scope further. - Link the "nightly" rustup name to our pinned nightly. This has to do with how componentize-py's build interacts with rustup.

Linux: the manylinux cross container sets CARGO_BUILD_TARGET to the wheel target (e.g. aarch64), so cargo install would produce an aarch64 binary that can't execute on the x86_64 host. Download the x86_64 pre-built binary from GitHub releases explicitly. macOS: wasm-tools is not pre-installed on GitHub runners; add an install step using the pre-built macOS binary, selecting the correct arch via uname -m (x86_64 on macos-13, arm64 on macos-14).

uname -m returns 'arm64' on Apple Silicon but release assets use 'aarch64'; add a mapping before constructing the download URL. Also comment out the x86_64-apple-darwin (macos-13) matrix entry as those runners are queueing indefinitely. Re-enable when availability improves.

macos-13 is no longer reliably available; macos-26 (Tahoe) is current and available for both Intel and Apple Silicon. Runner OS version has no effect on wheel compatibility — deployment target is set by the Rust target triple, not the runner.

erikrose

Just a couple of questions before we merge it in. I'm fine leaving out Windows for the beta. (For that matter, Mac Intel is going to bite the dust in September too.)

I imagine we can get trusted publishing going for the beta, don't you? We'd just have to stick the artifacts in a different place.

posborne · 2026-05-28T21:42:25Z

I imagine we can get trusted publishing going for the beta, don't you? We'd just have to stick the artifacts in a different place.

Yeah, we can tee the artifacts off at the end to be release attachments and get uploaded to pypi. That should work fine, I expect.

The python script still supports feeding in a tag as an additional check, but the makefile wrapper and instructions around that were realistically overly complicated.

Both of these were keyed off Cargo.lock which changes with version bumps. Given the limited TTL on these caches, they would be unlikely to provide any value in practice for releases which are expected to be relatively infrequent.

…chain

THis change unifies sdist, wheel, and checksum computation into a unified job that can be depended on a bit more easily. This also makes it so that checksums are computed on sdist as well as wheels.

Instead, use APIs on wit-parser and wit-component in order to achieve the same thing. This gives us better control over dependency versions and removes an external build-time dependency on the host machine.

With our latest build.rs changes, we now no longer have a build-time or runtime dependency on shelling out to wasm-tools or wac-cli, so we can remove that comlexity entirely. A github specific script is extracted to aid in installing our pinned nightly with caching around that.

I was a bit overzealous in removing wasm-tools (for now) and it is still used in codegen steps from python; we can, however, avoid the dependency on the release side of things which simplifies things.

erikrose

Looks great! As soon as the CI is happy, I'm happy!

The regex we had before wasn't correct on CI; use a `rustup run nightly rustc` invocation instead which should hopefully be more robust.

Maturin currently sets a default for x86_64 that isn't compatible with what xcode 26 supports. Tell it to target 10.15 instead which will hopefully solve the problem. If not, we'll need to change the runner for x86_64 mac.

posborne · 2026-05-29T16:35:18Z

🎉 After a final round of release build CI teasing, it worked. Merging.

posborne added 17 commits May 27, 2026 11:56

Pin componentize-py git dependency to resolved commit SHA

4c1228e

Pins to the commit already resolved in Cargo.lock (81d582a) so that future builds cannot silently pull in upstream HEAD changes.

Upgrade actions/checkout from v4 to v6

9279655

v6 adds Node.js 24 runtime support and a minor credentials storage improvement. No breaking changes for our usage.

Add wasm-tools/wac-cli to containers for release and pin

a4c046b

Tighten up the release docs in CONTRIBUTING.md

0953502

posborne requested a review from erikrose May 28, 2026 16:28

posborne added 2 commits May 28, 2026 11:35

make format

185fcdf

Add sdist alongside wheels

c45cb6c

erikrose reviewed May 28, 2026

View reviewed changes

posborne added 7 commits May 28, 2026 16:57

Drop make targets for checks, just do with lint

6b324be

The python script still supports feeding in a tag as an additional check, but the makefile wrapper and instructions around that were realistically overly complicated.

release: drop source/binary caches

ad25b7f

Both of these were keyed off Cargo.lock which changes with version bumps. Given the limited TTL on these caches, they would be unlikely to provide any value in practice for releases which are expected to be relatively infrequent.

release: remove confusing/unecessary language related to nightly tool…

42ce28d

…chain

release: include sdist in checksums, unify artifact collection

798847e

THis change unifies sdist, wheel, and checksum computation into a unified job that can be depended on a bit more easily. This also makes it so that checksums are computed on sdist as well as wheels.

Remove wasm-tools shell calls from build.rs

0a64f91

Instead, use APIs on wit-parser and wit-component in order to achieve the same thing. This gives us better control over dependency versions and removes an external build-time dependency on the host machine.

Add back wasm-tools to main ci workflow

d05b79c

I was a bit overzealous in removing wasm-tools (for now) and it is still used in codegen steps from python; we can, however, avoid the dependency on the release side of things which simplifies things.

erikrose approved these changes May 29, 2026

View reviewed changes

posborne added 2 commits May 29, 2026 10:27

Test nightly install differently

c6654b4

The regex we had before wasn't correct on CI; use a `rustup run nightly rustc` invocation instead which should hopefully be more robust.

release: target MACOSX_DEPLOYMENT_TARGET 10.15 for x86_64

00be019

Maturin currently sets a default for x86_64 that isn't compatible with what xcode 26 supports. Tell it to target 10.15 instead which will hopefully solve the problem. If not, we'll need to change the runner for x86_64 mac.

posborne merged commit 37cbeb1 into main May 29, 2026
9 checks passed

posborne deleted the posborne/wheels branch May 29, 2026 16:35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Release Process for Building Wheels for Mac/Linux aarch64 and x86_64#100

Add Release Process for Building Wheels for Mac/Linux aarch64 and x86_64#100
posborne merged 28 commits into
mainfrom
posborne/wheels

posborne commented May 28, 2026

Uh oh!

erikrose left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

posborne commented May 28, 2026

Uh oh!

erikrose left a comment

Uh oh!

posborne commented May 29, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

posborne commented May 28, 2026

Uh oh!

erikrose left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

posborne commented May 28, 2026

Uh oh!

erikrose left a comment

Choose a reason for hiding this comment

Uh oh!

posborne commented May 29, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants