perf(test): add a per-suite timing option and a wall-clock baseline doc with the Linux column

Parent: #3716 (slice 1 of 3). Runs first; slice 2 waits on it. Slice 3 runs in parallel.
Runs in: lane. Linux only; the Windows column is slice 2.

Problem

#3716 criterion 1 asks for a committed baseline table of per-suite wall clock on Windows and on the Linux CI runner. The Linux numbers partly exist: scripts/suite-seconds.txt holds median seconds per suite, but they are inferred from CI log timestamps (its header comment), because scripts/run-plugin-tests.sh prints only PASS: <suite> and no per-suite time. A Windows host has no way to produce comparable numbers, and the gate scripts the issue names (scripts/check-purged-em-dashes.sh, scripts/check-changed-skills.sh, scripts/check-changelog-parity.sh, plugins/skill-quality/scripts/check-skill.sh, plugins/planning/scripts/check-open-questions.sh; #3717 merge comment) have no recorded time at all.

Fix

Add a timing option to scripts/run-plugin-tests.sh (for example --timings <file>) that writes one TSV row per suite: suite path, wall seconds, exit code. Same option, same format, on Linux and Git Bash. No behavior change without the flag.
Add a way to time the gate scripts above in their typical single-diff invocation into the same TSV format (a flag on an existing script or one small script; name it in the doc).
Commit docs/test-wall-clock-baseline.md: how to produce each column, a Linux CI column filled from a whole-tree CI run with the new option (cite the run), the gate-script rows, and an empty Windows column that slice 2 fills. Link it from the header comment of scripts/affected-tests.sh.

Done when

bash scripts/run-plugin-tests.sh --timings <tmpfile> <one suite> exits 0 and writes one row with three tab-separated fields
test -f docs/test-wall-clock-baseline.md exits 0, and the file cites the CI run behind its Linux column
grep -c 'test-wall-clock-baseline.md' scripts/affected-tests.sh prints 1 or more
The doc has a row for each of the five gate scripts named above
The existing suites pass unchanged (ci-status is green on the PR head)

Split by the backlog HITL session (2026-10-25, owner rule Q93: split large work before a run picks it up). This was generated by AI.

Proposed work class: work-class: scoped
Basis: automated intake triage; the lanes App applied this class.

Size: Small
Complexity: Low
Uncertainty: Known
Flags: none
Blocked by: none

Decision recorded.

PR #11730 delivered the tooling and the doc skeleton. At origin/main ca93260651, scripts/run-plugin-tests.sh takes --timings FILE (:12, :105-109, :216-223) and writes one row per suite, made of path, seconds and exit code (:533).
[quote truncated; the full decision is at the link]

#10200 (comment)

This decision settles the open questions. The tooling and doc skeleton are done. What remains is filling the per-suite Linux CI rows in docs/test-wall-clock-baseline.md from the TIME: lines of the test-bash jobs of the cited whole-tree run. That step needs an authenticated read of the job logs, so it is a host-run step rather than a decision for a person. The Windows column stays with slice 2.

_{Automated triage, applied by the lanes App.}

This records a re-fire of an earlier refusal, not a new refusal. The lanes kill switch stopped run 40797684275, the work-resolve-issue run that agent-ready started, before it did any work; that run ended at 2026-10-29T01:33:05Z. work-unblock has now removed and added agent-ready back to start a fresh run. A kill-switch refusal is no fault in this issue or the lanes, so it never counts against later re-fires; after limits.limit-stops-per-issue of them since a person last added agent-ready or a run last reported, work-unblock waits longer before each one, up to 24 hours.

Decision recorded.

PR #11730 delivered the tooling and the doc skeleton. At origin/main ca93260651, scripts/run-plugin-tests.sh takes --timings FILE (:12, :105-109, :216-223) and writes one row per suite, made of path, seconds and exit code (:533). bash scripts/run-plugin-tests.sh --timings <tmp> scripts/time-gates.test.sh exited 0 and wrote one row with 3 tab-separated fields. scripts/time-gates.sh times the five gate scripts into the same format (:8, :114). docs/test-wall-clock-baseline.md explains how to produce each column (:9) and has a row for each of the five gates (:43-47). scripts/affected-tests.sh:11 links to the doc, and ci-status passed on the PR head (job).

The doc does cite whole-tree run 41211887017 (:52), but only the wording of the second Done-when line is met: the per-suite table that run is meant to fill has no rows (:59-60). Fix step 3 asks for "a Linux CI column filled from a whole-tree CI run", and the issue title asks for "a wall-clock baseline doc with the Linux column". The lane left the table empty because it could not read the run's job logs without authentication (PR #11730, Not verified). The Linux cells of the gate table (:43-47) are also empty, but the issue asks only for the gate rows, so filling them is optional here. The Windows column remains slice 2's. Agents decided this on evidence under standing direction §1.

Acceptance criteria:

The per-suite table in docs/test-wall-clock-baseline.md holds a Linux CI row for each shell suite, taken from the TIME: lines of the test-bash jobs of the whole-tree run the doc cites. That is run 41211887017, or a newer whole-tree run on main that the doc cites in its place.
ci-status is green on the PR head.

Route: host-run. The fill needs an authenticated read of the job logs (gh run view --log --job <id>): an unauthenticated read returned HTTP 403, and the issue lane had no gh credentials in run 41245614213. A host session with authenticated gh reads the logs and opens the pull request.

Claimed by the Backlog HITL host session; work in progress.

Host run. An authenticated host session read the seven test-bash job logs of whole-tree run 41211887017 and filled the Linux CI per-suite table in docs/test-wall-clock-baseline.md.

What ran (WSL2 Linux host, Claude Code 2.1.316, gh 2.103.3):

Listed the run's jobs (actions/runs/41211887017/jobs) and downloaded the logs of test-bash (0) through test-bash (6) with the authenticated jobs-logs API, then kept each TIME: <suite> <n>s line.
Lines per leg: 116, 115, 114, 118, 119, 119, 119. Total 820, each matching the TIME: shape, no suite repeated. 6254 suite-seconds summed across the legs at --jobs 3.
The 820 matches the runner's corpus at the run's commit fd862a49b848: the 10 other tracked *.test.sh files sit under .github/scripts/ and docs/contracts/, outside the runner's discovery roots.
scripts/check-affected.sh --tests on the committed head: 12 passed, 4 skipped, no suite selected for the docs-only diff. markdownlint on the doc: 0 issues. The shared clone's local config and status were unchanged before and after.

Acceptance criteria:

The per-suite table holds a Linux CI row for each shell suite (820), from the TIME: lines of run 41211887017's test-bash jobs, the run the doc cites.
ci-status green on the PR head 0ae492e797. Both review lanes (claude-review, claude-security-review) and Codex reviewed that head with 0 findings and no open threads.

Left out: the Linux cells of the gate table (optional per the decision comment), and the Windows column (slice 2).

PR: #11891

Claim. Backlog HITL host-run worker is taking this now.

Host run. A host session timed the five gate scripts with scripts/time-gates.sh and filled the Linux cells of the gate table in docs/test-wall-clock-baseline.md. That was the last open item on this issue.

What ran (WSL2 Linux host, Claude Code 2.1.316):

From a detached checkout of 52a529d9cf (feat(skill-quality): concise check and check-prerequisites descriptions, measured #11995, a one-commit skill-quality skill change with its changelog fragment), three runs of bash scripts/time-gates.sh --timings <tmp> --base 52a529d9cf~1. Every gate exited 0 on every run, and the spread was under 50 ms per gate.
Median seconds per gate:

Gate Linux (s)

check-purged-em-dashes 1.762

check-changed-skills 1.178

check-changelog-parity 0.039

check-skill 0.627

check-open-questions 0.012
Why that base: the default --base origin/main on an unchanged checkout gives an empty diff, and three diff-scoped gates would then time nothing. The stderr confirms the gates checked real files: check-skill checked 2 skills, and the parity check passed on the version bump.
scripts/check-affected.sh --tests on the committed head: 12 passed, 4 skipped. markdownlint on the doc: 0 issues.

Acceptance criteria (the 2026-11-01 decision comment):

Per-suite Linux CI table: done in docs: fill the Linux CI per-suite wall-clock baseline #11891 (820 rows, run 41211887017).
ci-status green: on docs: fill the Linux CI per-suite wall-clock baseline #11891, and on this PR's head 3d661138f3. claude-review, claude-security-review and Codex each reviewed that head with 0 findings, and no threads are open.
Optional gate Linux cells: filled by this PR.

Blocked on another machine, and owned by #10241 rather than this issue: the Windows column of both tables. Exact step on a Windows 11 Git Bash host, from the repository root:

Run bash scripts/run-plugin-tests.sh --timings <file> at the default --jobs 1.
From a checkout of 52a529d9cf, run bash scripts/time-gates.sh --timings <file> --base 52a529d9cf~1.

PR: #12381 (Closes #10200).

Target labels: add none; remove host-run.

Navigation Menu

Repository navigation

perf(test): add a per-suite timing option and a wall-clock baseline doc with the Linux column #10200

Problem

Fix

Done when

21 remaining items

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Gate	Linux (s)
check-purged-em-dashes	1.762
check-changed-skills	1.178
check-changelog-parity	0.039
check-skill	0.627
check-open-questions	0.012

perf(test): add a per-suite timing option and a wall-clock baseline doc with the Linux column #10200

Description

Problem

Fix

Done when

Activity

github-actions commented on Oct 26, 2026

melodic-automation-lanes commented on Oct 29, 2026

21 remaining items

kyle-sexton commented on Nov 1, 2026

kyle-sexton commented on Nov 1, 2026

kyle-sexton commented on Nov 1, 2026

kyle-sexton commented on Nov 2, 2026

kyle-sexton commented on Nov 2, 2026

Metadata

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Issue actions