feat: self-provisioning nix toolchain with turbo-cached mutation gate - #14
Open
younna-ai-opencode wants to merge 7 commits into
Open
feat: self-provisioning nix toolchain with turbo-cached mutation gate#14younna-ai-opencode wants to merge 7 commits into
younna-ai-opencode wants to merge 7 commits into
Conversation
The cargo-mutants verdict is now a turbo task keyed on exactly the files that determine mutant survival (crate sources, tests, workspace and toolchain manifests). An unchanged tree replays the cached verdict in milliseconds instead of re-running the 3-minute mutant loop; any change to the classifier inputs re-executes. cacheDir is pinned to .turbo/cache so git worktrees stop silently sharing one cache with the main checkout.
flake.nix locks the entire toolchain (cargo, clippy, rustfmt, cargo-mutants, gcc, node, pnpm pinned to packageManager) behind one `nix develop`. CI's gate and mutation jobs provision from the same flake instead of dtolnay/rust-toolchain plus a per-run `cargo install cargo-mutants`, and the mutation job runs the gate through turbo with .turbo/cache persisted by actions/cache, so re-runs and same-input PR builds replay the cached verdict instead of re-executing mutants.
The actions/cache key now hashes the same input set the mutants task hashes (sources, tests, manifests, flake.lock) instead of the commit sha, so unchanged inputs restore an exact key, skip the post-job save, and stop growing the cache store on every commit. flake.lock joins the task inputs: nixpkgs pins cargo-mutants and rustc, so a toolchain bump regenerates a different mutant set and must invalidate the cached verdict. The pnpm store is cached too — pnpm install runs before turbo can replay anything and was a cold fetch on every run.
Review fixes (ce-code-review, validated): - turbo.json: add $TURBO_ROOT$/flake.nix to mutants inputs. flake.nix defines the toolchain; without it in the key a flake.nix-only edit replayed the previous verdict verbatim (probe: hash b956bf4d... stayed put, Cached (Local) = true). With it: miss + full re-execution. - ci.yml: add flake.nix, turbo.json and the npm package.json (which carries the gate script) to the turbo-bag hashFiles key. turbo self-hashes the latter two, so an exact-key restore was suppressing the post-job save on config-only edits: fresh verdicts were re-run every time and never persisted. - ci.yml: restore cargo build-state caching in the mutation job (origin/master had it; the turbo rewrite dropped it) keyed on Cargo.lock + flake.lock, sharing the gate job's cargo- restore prefix. - ci.yml: set CARGO_BUILD_JOBS=4 on the mutation run line for parity with the gate job and deterministic env hashing. Verified: mutants loop miss(117 green, 2m36s) -> hit(10ms) -> flake.nix edit miss(2m36s) -> revert hit(8ms); one-shot gate green (fmt, clippy -D warnings, 90 tests).
…oncept Compound of this session's learning: the turbo-cached mutation gate hashed the toolchain pin (flake.lock) but not the toolchain definition (flake.nix), so flake.nix-only edits replayed stale verdicts. Doc captures the fix, the two falsified adversarial hypotheses (script-body and env staleness — both busted by hash-movement probes), the transport-superset invariant, and the probe protocol. CONCEPTS.md gains the Verdict cache entry (two hash surfaces).
First Actions run failed in both gate and mutation jobs: 'bash -lc' inside nix develop -c re-sources the runner profiles, which re-prepend ~/.cargo/bin ahead of the dev shell PATH. The rustup shim then owned the toolchain: it synced the stable channel mid-build, removed the previous rust-std/rustc components, and every compile died with E0463 (can't find crate for std). cargo-mutants hit the identical failure in its scratch tree. 'bash -c' keeps the dev shell's PATH first; nix develop -c already exports the full toolchain. Comment added at the first use so the flag isn't reintroduced. Verified locally in the exact new form: one-shot gate green (90 tests), mutants cold miss (117 mutants, 113 caught, 4 unviable, 0 survived, 2m36s) then FULL TURBO hit in 9ms with CARGO_BUILD_JOBS=4 matching CI.
Run 2 failed differently from run 1: nix's cargo resolved rustc via PATH, the dev shell never shipped one, and on Actions runners the rustup proxy won the lookup — its stable toolchain then failed to link (std rlibs not applicable/missing). Locally the same shell only worked because ambient PATH satisfied the lookup by accident; 'type -P rustc' inside a clean 'nix develop -c bash -c' proved MISSING. cargo resolves rustc from PATH; a toolchain-provisioning flake that lists cargo without rustc is incomplete. Add rustc beside cargo with a comment naming the failure mode. Verified in a runner simulation (env -i, PATH=nix:/usr/bin:/bin, CI=true, cold pnpm store, no rustup on any path): pnpm install 2.9s; mutants cache miss -> full re-execution, 117 mutants green, 2m37s, exit 0; replay hit. The flake.nix edit itself flipped the turbo input hash, so the gate re-verified under the new toolchain set rather than replaying.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The repo now provisions its entire toolchain from one lockfile (
nix develop), and the mutation gate — the most expensive CI check — replays a content-addressed verdict from the turbo cache when its inputs are unchanged. A fresh machine or runner gets cargo, rustc, clippy, rustfmt, cargo-mutants, node, and pnpm at the exact pinned versions; an unchanged tree answers in milliseconds instead of re-running 117 mutants (~2m36s).What changed and why
turbo.jsonmutants, cache enabled). The cache key covers exactly what determines mutant survival: crate sources and tests, workspace manifests, dependency lockfile, build config, and the toolchain definition + pin. Anything outside that set cannot change the verdict; anything inside it flips the key and forces a genuine re-run. A miss errs toward re-execution, never a stale pass.cacheDirpinned to.turbo/cache— turbo's default in a git worktree silently shares the main checkout's cache; a concurrent writer was observed serving an illegitimate hit mid-session. Pinning gives each worktree its own cache and matches the path CI persists.flake.nix+flake.locklock the toolchain (cargo + rustc 1.97, clippy, rustfmt, cargo-mutants 27.1, gcc, node 24, pnpm 11.21.0 — exactly thepackageManagerpin). rustc is explicit: cargo resolves rustc via PATH, and without it in the shell the runner's rustup proxy won the lookup and failed to link std (fixed in2f32cceed; CI commands also dropped the login shell so runner profiles cannot re-prepend~/.cargo/bin,7dce7b7e0). Replaces rustup/manualcargo install/third-party toolchain steps:nix developis the single source of toolchain truth for local dev and both CI jobs.nix-installer-action; the mutation job runspnpm mutantswith.turbo/cachepersisted byactions/cache(keyed on the gate's input set plus the files turbo self-hashes, prefix restore-keys), a pnpm-store cache, and — restored in this branch — a cargo build-state cache for the miss path. GitHub Actions cache chosen over a remote cache backend to keep the setup credential-free; the tradeoff is hits within re-runs/same-branch rather than across branches.--timeout 90, 100%-kill demand. The CI log surfaces whether a run was a fresh execution or aFULL TURBOreplay — replay transparency is deliberate (see open question below).Validation
nix develop -ctoolchain versionsFULL TURBO; flake.nix edit -> miss 2m36s (the stale-replay hole, closed); revert -> hit 8ms-D warningsclean, 90 tests passActions (run on
2f32cceed, all three jobs green): gate 3m45s incl. nix provisioning + full compile; npm 18s; mutation ran the full gate cold — 117 mutants tested in 7m, 113 caught, 4 unviable, 0 survived. A re-run of the mutation job should now logFULL TURBO(exact-key restore) — that replay is the cache proof's last piece, left visible for the reviewer.Two earlier runs failed and are part of the record:
bash -lclet the runner's rustup shim hijack the toolchain, then the flake's missingrustclet the rustup proxy win the PATH lookup — both diagnosed from Actions logs, both fixed with runner-simulation verification (env -i, no host toolchain on any path).Review
A 4-persona review plus an independent empirical validator ran on this branch before opening (two adversarial P1s were falsified by hash probes — turbo self-hashes the gate script and env values, so those cannot replay stale; one correctness P2 was confirmed — flake.nix outside both keys — and is fixed in
cae08e3bd, along with the dropped cargo build-state cache and the CI-key persistence gap). Residual decision for the reviewer, not the code: AGENTS.md's anti-bypass section says evidence comes from the current run, while this branch's CI deliberately replays content-addressed verdicts. The log makes replay visible per run; amending the AGENTS.md wording to permit content-addressed replay is a locked-surface change left to a maintainer.New concepts
$TURBO_ROOT$— task inputs outside the package directory: turbo hashes task inputs relative to the package;../..globs silently match nothing outside the package (first attempt hashed 1 file instead of 20). The$TURBO_ROOT$prefix re-bases the glob to the repo root so an npm-package task can declare the Rust crate as its input. Use when a task consumes files outside its own package; don't pull the whole repo into a key.Also captured:
docs/solutions/integration-issues/turbo-mutants-flake-nix-cache-key.md(hash-coverage invariants + probe protocol) and theVerdict cacheentry in CONCEPTS.md.