Skip to content

ci: cut wasted minutes from measurement_id and WASM jobs#8776

Open
a10y wants to merge 1 commit into
developfrom
aduffy/ci-quick-wins
Open

ci: cut wasted minutes from measurement_id and WASM jobs#8776
a10y wants to merge 1 commit into
developfrom
aduffy/ci-quick-wins

Conversation

@a10y

@a10y a10y commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Summary

Three targeted fixes from profiling recent CI runs (job/step timings + logs from PR runs of Linters and Tests):

  • measurement_id golden vectors: 6.2 min → ~20 s. The setup-uv action defaults to uv sync --all-extras --dev, which builds the vortex-data PyO3 extension via maturin on an uncached ubuntu-latest runner — ~6 min of the job (verified in job logs: Run uv sync --all-extras --devBuilding vortex-data). The test itself takes ~1 s and uses uv run --no-project, so it needs no workspace packages. Pass sync: false, matching the existing python-cuda-test job.
  • WASM integration smoke test: ~5.2 min → expected ~1–2 min. It was the only Rust-compiling job left on plain ubuntu-latest with no compiler cache; ~4.5 min is a cold cargo build. Moved to the prebuilt runs-on image with S3 sccache, following the same pattern as the other build jobs (fork fallback to ubuntu-latest preserved; rustup target add wasm32-wasip1 mirrors what the wasm32 build job does).
  • Windows sccache observability. Rust tests (windows-x64) is the ci.yml PR critical path (consistently 6.8–7.7 min across the last 8 PR runs, ~5 min of it compile). The sccache server demonstrably starts, but logs give no way to see hit rates. Added an sccache --show-stats step (if: always(), main repo only) so cache effectiveness on Windows is visible per run.

Not included (infra-side observations from the same investigation)

  • Codspeed PR runs take 9–14 min but shards compute for only 0.8–4 min — they queue 5.5–9.8 min for amd64-medium runners while concurrent ci.yml jobs get runners in <1 min. One PR push requests ~26 runs-on VMs at once; this looks like a concurrent-instance/pool cap in .github-private (this repo's runs-on.yml only defines images). Raising capacity there, or consolidating the 8 Codspeed shards to 4, would cut the largest remaining chunk of PR wall clock.
  • The Windows pool uses m8i-flex.2xlarge (8 vCPU); a size bump would directly shrink the PR critical path since compile dominates and linking 53 test binaries is uncacheable.

Checks

  • yamllint --strict -c .yamllint.yaml .github/workflows/ci.yml — passes.
  • Workflow-only change; no Rust/Python code touched, so no cargo/pytest checks were run per repo guidance. The WASM job change is best validated by this PR's own CI run.

🤖 Generated with Claude Code

Two PR-lane jobs spend nearly all their time on avoidable setup work:

- measurement_id golden vectors: setup-uv defaults to a full
  `uv sync --all-extras --dev`, which builds the vortex-data Rust
  extension via maturin on an uncached ubuntu-latest runner (~6 min).
  The test itself takes ~1s and runs with `uv run --no-project`, so
  pass `sync: false` (same as the python-cuda-test job).

- WASM integration smoke test: the only Rust-compiling job still on
  plain ubuntu-latest with no compiler cache; ~4.5 of its ~5 min is a
  cold cargo build. Move it to the prebuilt runs-on image with S3
  sccache like its sibling jobs.

Also add an `sccache --show-stats` step to the Windows test job (the
PR critical path at ~7.5 min, ~5 min of it compile) so cache hit rates
are visible in logs — today there is no way to tell from a run whether
the S3 cache is effective on Windows.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Andrew Duffy <andrew@a10y.dev>
@a10y a10y enabled auto-merge (squash) July 15, 2026 19:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant