Skip to content

release: check that each NIF loads, and publish the sources of the tag - #90

Merged
rubas-agent[bot] merged 4 commits into
mainfrom
release-pipeline
Nov 12, 2026
Merged

rubas-agent[bot] merged 4 commits into
mainfrom
release-pipeline

Conversation

@rubas-agent

@rubas-agent rubas-agent Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Before

The Publish to Hex job publishes the tree of the checksum merge, not the tag that the NIF archives were built from. The release build never checks that a NIF loads. mix hex.publish exits 0 when only the docs upload fails, so the job stays green.

AGENTS.md refreshes the checksum file with --no-config, which skips the compile. The download then reads the version from the metadata of the last compile in any checkout, in ~/.cache/rustler_precompiled/metadata. After a compile of 0.5.0 elsewhere:

-- mix rustler_precompiled.download WhisperCpp.Native --all --no-config --ignore-unavailable --print
   versions in the file: v0.5.0-nif

Problem

A fresh checkout writes the checksums of the release before, and --ignore-unavailable hides a variant that the release lacks. A commit that merges between the version bump and the checksum PR ships in the Hex package, next to NIFs that were not built from it. A NIF with an undefined symbol gets a tag, a GitHub release, and a Hex version: the aarch64 NIF of whisper_ct2 0.7.1 does exactly that today (rubas/whisper_ct2#99).

After

Before After
Package sources tree of the checksum merge tag vX.Y.Z and the checksum file of the merge
NIF with an undefined symbol released build fails, no tag
Failed NIF build tag already pushed, the next push skips the version no tag
Checksum merge while another run waits a newer push cancels its run, no publish runs in turn
Precompiled NIF of the runner downloaded, never loaded loaded before the upload
Failed docs upload job green job red
Checksum refresh metadata of any checkout task checksum:download builds first, with the metadata in _build, and fails on a missing asset
-- task checksum:download, with the same stale metadata
   versions in the file: v0.5.1-nif
   equals the committed checksum file of 0.5.1

The same change for the other repos: rubas/cloaked_req#92, rubas/kagi_ex#54, rubas/exmpeg#101, rubas/whisper_ct2#100.

Details

The publish job came with #86, and --no-config with #31. The tag job needed only detect since the first commit, so it pushed the tag while the NIFs still built. Hex 2.7.0 drops the docs error in Mix.Tasks.Hex.Publish.finish_combined_publish/6. RustlerPrecompiled.available_nifs/1 reads metadata-Elixir.WhisperCpp.Native.exs, which only a compile of the module writes. The task sets WHISPER_CPP_BUILD=1, because a compile without it fetches the new NIF of the host, which the checksum file does not list yet.

The job checks out the tag and takes only the checksum file from the merge, as whisper_ct2 does. A change after the tag ships with the next version. "Check that the NIF loads" runs ldd -r on Linux and nm -m -u on macOS. Both bind every symbol, as the BEAM's RTLD_NOW load does, and the check allows only enif_*, which the BEAM defines. The runner has no GPU driver, so the CUDA variants bind libcuda.so.1 to the toolkit's stub. ROCm puts no library path into ld.so.conf.d, so the hipBLAS variant adds $ROCM_PATH/lib.

The concurrency group now has queue: max. With the default single slot, a newer push cancels the pending run, and only the run of the checksum merge publishes.

A publish job that fails through a bug in the job itself cannot re-run with the fixed workflow file. We add no dispatch input for that: a patch release or an owner's mix hex.publish of the tag covers it, and the case has not happened.

Checks: zizmor pedantic is clean. A harness ran this step on all six 0.5.1 NIFs, on the release runners with the CUDA and ROCm toolkits: all pass, and a test NIF that misses MKL_Free_Buffers fails on each runner (https://github.com/rubas/whisper_cpp/actions/runs/43276300228). The publish steps ran on epyc against v0.5.1 with mix hex.publish --dry-run: the tree differs from the tag only in the checksum file, the checksum file equals SHA256SUMS, and the NIF loads. The checksum outputs above come from a private rustler_precompiled cache with the metadata of 0.5.0.

The build fails when a NIF needs a symbol that no library defines. The publish job checks out the tag, takes the checksum file of the merge, loads the precompiled NIF, and fails when Hex reports a failed docs upload.
With --no-config, mix rustler_precompiled.download read the version from the metadata of the last compile in any checkout, and wrote the checksums of that version. task checksum:download builds the NIF first, and fails on a missing asset.
The tag job needed only detect, so a failed build left a tag, and the next push skipped the version. A newer push cancelled the pending run of a checksum merge, and only that run publishes.
rustler_precompiled writes the metadata before the source build, so a compile in another checkout could change it while the build runs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant