ci: gate PR benchmark job behind a benchmark-relevant paths filter - #30
Open
alejoamiras wants to merge 1 commit into
Open
ci: gate PR benchmark job behind a benchmark-relevant paths filter#30alejoamiras wants to merge 1 commit into
alejoamiras wants to merge 1 commit into
Conversation
Benchmark takes ~35 minutes and ran on every PR, docs-only included. A changes job (dorny/paths-filter, SHA-pinned v3.0.2) now gates it on paths that can affect benchmark output: contract sources, Nargo manifests, the benchmark definitions and their shared setup helpers, dependency manifests, and the workflow itself. Verified on a probe PR: with job-level gating a docs-only PR still gets a concluded run and the gated caller reports as a skipped 'benchmark' check. The nested 'benchmark / Benchmark' check is not created when skipped, so that exact context must not be listed as a required status check (see PR description for the required-checks implications). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
alejoamiras
marked this pull request as ready for review
August 18, 2026 16:30
alejoamiras
marked this pull request as draft
August 18, 2026 16:30
alejoamiras
marked this pull request as ready for review
August 18, 2026 16:32
Benchmark Comparison
Contract: escrow
Contract: logic
Contract: multitoken
Contract: nft
Contract: token
Contract: vault
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The
benchmark / Benchmarkjob takes ~35 minutes and runs on every PR — docs-only changes included. It only produces meaningful output when something that affects compiled contracts or the benchmark harness changed.Change
A
changesjob (dorny/paths-filter, SHA-pinned to the verified v3.0.2 commit) gatesbenchmarkat the job level. It runs only when the PR touches:src/**/*.nr**/Nargo.tomlbenchmarks/**src/ts/test/utils.tstsconfig.jsonpackage.json,yarn.lock@aztec/*SDK +config.aztecVersiontoolchain pins.github/workflows/pr-checks.ymlformatandchecks(JS/Noir tests) are deliberately untouched.Pre-merge verification (all done — no reviewer legwork needed)
isRequiredon live PR check contextsChanges: success,benchmark / Benchmarkran.nrcontract change → benchmark runsbenchmark / Benchmarkstartedbenchmark: skipped, run concludes greenbenchmark: skippedNaming asymmetry to be aware of (matters only if checks are ever made required): when the job runs, the check is
benchmark / Benchmark; when skipped, onlybenchmarkexists. If benchmark should ever become a required check, that needs an always-run aggregate job evaluatingneedsresults — happy to do as a follow-up.Accepted trade-offs
benchmark-baseline-<branch>artifact, so a PR stacked on a skipped branch gets no baseline for its comparison (if_no_artifact_found: warndownstream — degrades gracefully, no stale-artifact fallback: the download action doesn't search older runs by default).changesjob itself fails (rare — API outage),benchmarkskips; the failedChangescheck is a loud red X on the PR.src/ts/test/utils.ts, the filter must be updated in the same PR.Review
Adversarially reviewed via a Codex loop (2 rounds): its confirmed finding (missing
utils.ts/tsconfig.json) is incorporated; its stale-baseline concern was checked against the download action's source and retracted; final verdict was ship-as-draft.actionlintpasses.🤖 Generated with Claude Code