[IsuraManchanayake]

What changed

Each release can now also go out on PyPI, npm and Homebrew. Nothing uploads until the repository variable PUBLISH_ENABLED is true; until then every run is a full dry run.

channel	package	contents
PyPI	superfluid-cli	one wheel per platform (macosx_15_0_arm64, manylinux_2_28_x86_64, manylinux_2_28_aarch64): the release tree plus superfluid, superfluid-workerd and superfluid-noded launchers that execv the binaries
npm	@basecompute/superfluid	the three commands. The release comes in optional per-platform packages, @basecompute/superfluid-{darwin-arm64,linux-x64,linux-arm64}, restricted by os/cpu/libc
Homebrew	basecompute/tap/superfluid	a formula that downloads the release tarball, keeps it whole in libexec and links the commands. It goes to the tap as a PR, never a push

superfluid is already taken on PyPI (a Superfluid Protocol SDK) and on npm.

• .github/workflows/publish.yml runs when a release is published (release: released, since release.yml only drafts one) and by hand for a tag. It downloads the tarballs, checks their .sha256, builds every package, runs twine check --strict, and installs each kind on macos-15, ubuntu-24.04 and ubuntu-24.04-arm (Homebrew on the first two). It uploads only when all of these hold:

  • PUBLISH_ENABLED is true and dry_run is not ticked;
  • the release is published, not a draft or a pre-release;
  • the run's commit is the one the tag names (git ls-remote on refs/tags/vX and refs/tags/vX^{}), so a dispatch from a branch is always a dry run.

  Each upload job first checks what its registry has: all three wheels and all four npm packages, not just the version. A rerun only finishes what is missing. PyPI uses trusted publishing with attestations, npm uses --provenance; both are possible because the repo is public. Actions are pinned to commit SHAs, and Python 3.12.15, Node 24.21.0 and twine 7.0.0 to exact versions.

• packaging/:

  • release.py checks a tarball's SHA-256 and takes only regular files and folders under its one top folder (no symlinks, .., or AppleDouble ._ entries).
  • It checks that every command and worker is a Mach-O/ELF binary for the target, and sets modes by file header.
  • It reads the wheel's platform tag from the binaries: the newest LC_BUILD_VERSION minimum on macOS, the newest GLIBC_2.N on Linux.
  • python/build_wheel.py writes the wheel directly. That keeps the exec bits, makes builds reproducible and needs no network.
  • npm/build_packages.py and homebrew/formula.py build the npm tarballs and the formula; build.py runs all three.
  • try-install.sh installs one kind somewhere throwaway and checks it. publish.yml and local runs both use it.
  • packaging/README.md covers how it fits together and the registry setup.

• self update vs package managers (selfupdate.rs): a package keeps the release layout, bin/ beside libexec/superfluid. So installed_as() read a Homebrew, pip or npm copy as the install script's in-place copy, and self update would have moved it under ~/.superfluid/versions. Confirmed with the 0.1.4 binaries: self list from a pip install said "this superfluid is the install script's copy …".

  • Each package now carries installed-by at the release root (pip: pip install --upgrade superfluid-cli, etc.).
  • self update / self list then say "… was installed by pip; pip install --upgrade superfluid-cli updates it", and the daily "newer release" line names the same command.

• crates.io (release.yml): a new crates-verify job, with no token, runs cargo package --workspace --locked on macos-14, so every crate (MLX's included) builds from its packaged form. The upload job now needs it and keeps --no-verify, so no build script runs while it holds CARGO_REGISTRY_TOKEN. Both jobs pin actions/checkout and dtolnay/rust-toolchain to SHAs, use Rust 1.99.1 instead of stable, and set persist-credentials: false.

• Tests and docs:

  • tests/scripts/test_packaging.py has 20 tests on made-up releases: staging and its refusals, platform tags, wheel RECORD/modes/metadata, the npm manifests, the formula, both launchers (argument and exit-status passthrough, repair message), and contract checks that LINKED/INSTALLED_BY in selfupdate.rs, release.yml's targets and launch.js agree with packaging/.
  • CI runs it in a new packaging scripts job.
  • installation.md and updating.md describe the new channels (published from 0.1.5 on), and the CHANGELOG has 0.1.5 entries.

Why

So superfluid installs with the tools people already use (brew install, pipx install, npm i -g) and updates through them, without self update fighting the package manager. The shipped binaries are all Apache-2.0: llama.cpp, MLX and baseRT are installed at runtime, not bundled. So no platform needs its own licence gate.

It does not touch basert's release-asset contract: the tarballs, compat.json and superfluid-basert-abi-<N>.txt stay as they are, and basert never uses a superfluid on PATH.

How it was tested

All on an M5 Pro Mac against the real v0.1.4 release assets. Nothing was uploaded anywhere.

• python3 packaging/build.py <v0.1.4 assets> out --version 0.1.4 built 3 wheels, 4 npm tarballs and superfluid.rb. twine check --strict (7.0.0) passes all three wheels.

• packaging/try-install.sh pip|npm|brew out 0.1.4, all three pass:

  • pip: a fresh venv (system Python 3.9) picks the macosx_15_0_arm64 wheel by its tag. superfluid --version prints superfluid 0.1.4. --help, superfluid-workerd / superfluid-noded, and python -m superfluid_cli all run. runtimes --json finds the llamacpp and mlx workers inside site-packages/superfluid_cli/release/libexec/superfluid.
  • npm: npm install --global --prefix <tmp> (npm 11.16) of the darwin-arm64 package plus the main one passes the same checks, with the workers under node_modules/@basecompute/superfluid-darwin-arm64/release.
  • brew: a throwaway local tap: brew style and brew audit --strict are clean, then brew install (from the GitHub URL) and brew test pass and the same checks hold under Cellar/superfluid/0.1.4/libexec. It was uninstalled and untapped afterwards.

• Launchers: both pass through exit status 2 from an unknown command, and both put back exec bits that were stripped from the release (found by header).

• installed-by end to end: this branch's superfluid swapped into the 0.1.4 tree and installed as a wheel. self update --dry-run exits 1 with "… was installed by pip; pip install --upgrade superfluid-cli updates it", and self list says the same.

• The plan step's shell was run locally against v0.1.4: it uploads only with PUBLISH_ENABLED=true, dry_run false and the run at abed852 (the tag's commit), and refuses main as a tag.

• cargo package --workspace --locked (the new crates-verify step) packages and builds all 19 crates on macOS.

• python3 -m unittest discover -s tests/scripts: 20 pass on Python 3.9 and 3.14.

• cargo clippy -p superfluid --all-targets -- -D warnings

• cargo test -p superfluid (622 passed, 0 failed)

• Changes a wire protocol, the WAL format or the tick ABI: no.

Not tested:

• Linux installs. The Linux wheels and npm tarballs were only built and inspected (tags manylinux_2_28_x86_64 / _aarch64, ELF arch, modes). publish.yml's ubuntu jobs will be the first real install, and the Linux Homebrew run depends on the ubuntu-24.04 image's /home/linuxbrew.
• publish.yml itself has not run: release: released and workflow_dispatch only fire once it is on main. No upload has been tried.
• The crates-verify job on a hosted macos-14 runner, as opposed to locally.
• The self update refusal inside try-install.sh (it only applies to versions after 0.1.4) was checked by hand as above, not through the script.

Before the first upload (registry setup, for an admin)

1. PyPI: add a pending trusted publisher for superfluid-cli: owner basecompute, repo superfluid, workflow publish.yml, environment pypi.
2. npm: create the basecompute org on npmjs.com (shared with baseRT's planned @basecompute/basert). Make a granular token that can publish to @basecompute and store it as secret NPM_TOKEN in environment npm. Trusted publishing can replace the token once the four packages exist.
3. Homebrew: the tap is public, so brew install basecompute/tap/superfluid works for anyone. Make a fine-grained token (Contents + Pull requests write, tap only) and store it as secret HOMEBREW_TAP_TOKEN in environment homebrew.
4. Environments: create pypi, npm and homebrew, each limited to tags v*, with reviewers if wanted.
5. Set the repository variable PUBLISH_ENABLED=true.

This targets rc-0.1.5 (#33), so 0.1.5 is the first release that can upload: the upload requires the run to be at the tag's commit, and 0.1.4's commit has no publish.yml. 0.1.5 is also the first release whose binaries read installed-by. Once this is on main, a dispatch with tag v0.1.4 dry-runs the whole pipeline.