Skip to content

ci: cut runner load from drafts, re-pushes, and the mutation canary - #3624

Open
thymikee wants to merge 5 commits into
mainfrom
ci/reduce-runner-load
Open

thymikee wants to merge 5 commits into
mainfrom
ci/reduce-runner-load

Conversation

@thymikee

@thymikee thymikee commented Oct 14, 2026 •

Copy link
Copy Markdown
Member

Summary

Reduces load on GitHub runners. In a 5-hour sample (2026-10-14), the advisory mutation canary used about 16% of runner time, mostly on PRs that only edited a workspace package.json. Agents also re-push often, and each push restarts every lane.

  • Mutation canary: LANE_TOOLING is split into LANE_SOURCES (lane code and config, including vitest.config.ts and scripts/lib/) and LANE_DEPENDENCIES (root and workspace package.json, the lockfile).
    • A diff touching only dependencies runs the self-test.
    • Mutants run only through gh workflow run mutation-affected.yml --ref <branch> [-f base=<parent>], which passes --with-dependencies.
  • Device smoke lanes: they skip drafts, start on ready_for_review, and cancel on converted_to_draft. A gate test enforces this.
  • Agent docs: push once per review round. Allow 2,500 lines for one coherent change with self-contained commits.
    • Two duplicate lines were removed to fit the guidance budget. check:fallow already runs in check:affected. The misrouted-path rule repeats "If selection rules change, plant a failure".

10 files.

Validation

At 2a4c15cb1:

  • pnpm check:affected --run passes.
  • The mutation and gate workflow tests pass (25/25).
  • Removing either draft guard fails the new test, and the new selection test fails on main.
  • A fresh-context review's findings were applied, except the two doc deletions above.

No device runs: workflow and tooling changes only.

🤖 Generated with Claude Code

thymikee and others added 3 commits October 14, 2026 20:07
… lane sources

Device smoke lanes (iOS, macOS, Android, Linux) skip draft PRs and start on
ready_for_review; a gate test holds every PR smoke lane to both.

The mutation canary splits LANE_TOOLING into LANE_SOURCES, which spend mutants
on every PR, and LANE_DEPENDENCIES (root and workspace package.json, the
lockfile), which run the harness self-test and spend mutants only on manual
dispatch with --with-dependencies. Five of the seven PRs that recently ran the
canary touched only workspace package.json files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…atch

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 11 files

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread .github/workflows/mutation-affected.yml
Comment thread .github/workflows/android.yml Outdated
Comment thread .github/workflows/macos.yml Outdated
Comment thread .github/workflows/linux.yml Outdated
Comment thread docs/agents/testing.md Outdated
Comment thread docs/agents/pull-requests.md Outdated
@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Size Report

Metric Base Current Diff
Installed (including dependencies) 5.24 MB 5.24 MB 0 B
Package (unpacked) 5.24 MB 5.24 MB 0 B
Package (download) 1.58 MB 1.58 MB 0 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 27.1 ms 27.8 ms +0.6 ms
CLI --help 78.6 ms 79.5 ms +0.9 ms

thymikee and others added 2 commits October 14, 2026 20:24
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@thymikee

Copy link
Copy Markdown
Member Author

This PR is ready at 2a4c15c, and I found nothing that needs a change. No conflicts. The one red check is Compatibility & Provenance, which hit its 15-minute timeout and was cancelled. The other five jobs on the run are green, and the diff does not touch ci.yml. The only possible overlap is the fallow step, because the diff makes LANE_SOURCES and LANE_DEPENDENCIES unexported in scripts/mutation/run.ts. I could not read the step log, so please re-run that job once before merge. I did not exercise a real draft-to-ready transition or the mutation dispatch path, and I did not rerun the 25/25 tests or pnpm check:affected --run; those rest on reading the workflow YAML and the diff. I also did not trace whether the fixture-artifact wait in setup-fixture-app copes with a ready_for_review run on an unchanged head SHA. Not blocking, and you can take or leave these: https://github.com/callstack/agent-device/blob/2a4c15c/scripts/mutation/run.ts#L660 no longer selects the canary when root package.json, pnpm-lock.yaml or packages/*/package.json change, so a stryker or vitest bump runs only the self-test and the end-to-end run needs a dispatch (you could keep those exact dependency edits in the selection, or name the dispatch step in the checklist for dependency-bump PRs); https://github.com/callstack/agent-device/blob/2a4c15c/docs/agents/pull-requests.md#L19 says drafts skip device lanes but test-app-build-cache.yml still runs on every draft push, so "Drafts skip the platform smoke lanes" would be accurate; and the 2,500-line budget at https://github.com/callstack/agent-device/blob/2a4c15c/docs/agents/pull-requests.md#L34 is a policy change that fits better in its own docs PR.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Oct 14, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant