Repository navigation
test-bash: two timing flakes dropped #12412 from merge groups (pacing assertion, surface-journey settle ceiling) #12457
Description
Merge groups dropped #12412 twice on test-bash, each time on a different shell suite that passes on main:
- https://github.com/melodic-software/claude-code-plugins/actions/runs/41485537447 (leg 5):
plugins/source-control/skills/babysit-prs/scripts/engine.test.sh - https://github.com/melodic-software/claude-code-plugins/actions/runs/41490445918 (leg 4):
plugins/planning/surface/surface-journey.test.sh
Finding
Both are shared-runner timing failures, not defects in #12412.
engine.test.sh: the known pacing flake. The one failing case is test_calls_are_paced (PerRunLegacyFails). The failing assertion, at plugins/source-control/skills/babysit-prs/scripts/tests/test_babysit_verify_index.py:729, is:
self.assertTrue(all(1.9 < c.args[0] <= 2.0 for c in slept.call_args_list))It requires each computed sleep to be within 0.1 s of 2.0 s of wall clock, so a slow runner fails it. The suite took 234 s on that leg (Ran 3991 tests in 229.022s), over the 120 s per-suite warning threshold. #12429 (issue #12425) moves this assertion to a fake clock. That PR is still open, so main still carries the wall-clock assertion and the flake can hit any merge group that selects engine.test.sh until it lands.
surface-journey.test.sh: a wall-clock settle ceiling in the UI journey. Two checks failed:
ui_journey.1: saving as own stays possible. The detail shows the newest event was still Q4's ask (now carryingdeliveredAt): no save for Q5 had reached the server when the check read state.ui_journey.2: two unseen entries badge Activity and dot the rows they name. The badge read 2, so one of the three dot checks failed. The detail prints only the badge count, so the log cannot say which dot was missing. That it follows from the phase-1 miss is UNVERIFIED.
The code that waits is plugins/planning/surface/tests/ui_journey.js:145, await page.keyboard.press("Escape"); await tap("[data-save]", 700);, followed at :146 by last(). tap waits through settle(ms) (ui_journey.js:34-42), which returns once ms has passed even when the page has not caught up: the until at :37 returns early, and the waitForFunction at :41 swallows its timeout. With a 700 ms ceiling, the check reads state on whatever the clock allows. Two causes fit the evidence, and the log cannot tell them apart: the save's POST was still in flight when last() read state, or a re-render after the delivered-ask state frame swallowed the click. The leg was under load: plugins/planning/shared/tests/ui-b.test.sh on the same leg took 171 s, over the 120 s threshold, with three suites running in parallel.
What the PR covers
The PR (linked below) makes test-bash reprint a failed suite's failing lines beside its error annotation. scripts/run-plugin-tests.sh already replays each suite's whole output block, but in these runs the failing case sat 5,000 to 9,000 lines above the ##[error] line. After each failed suite's annotation the summary now prints its first 20 FAIL and not ok lines, then its last 80 lines, indented so no reprinted line is parsed again as a workflow command or a runner marker. Replayed over both logs above, the excerpt shows test_calls_are_paced with its traceback, and both ui_journey FAIL lines. An 80-line tail alone would have missed the surface-journey lines, which sit about 250 lines before the end of its block.
What remains
- engine.test.sh: the flake stops once test(source-control): assert legacy-fails pacing on a fake clock #12429 merges; nothing more is needed here.
- surface-journey.test.sh: the Q5 own-save check (
ui_journey.js:145-147) should wait for the save itself to land, rather than relying on a 700 ms settle ceiling. Two ways: wait untilevents()holds an own event for Q5, with a bounded timeout, before asserting; or havesettlereport when it hit its ceiling, so the check fails as "page not caught up" rather than as a wrong event. The phase-2 dot check atui_journey.js:160should name the missing dot in its detail.
Activity
github-actions commentedon Nov 2, 2026
Proposed work class: work-class: scoped
Basis: automated intake triage; the lanes App applied this class.
Size: Small
Complexity: Low
Uncertainty: Known
Flags: none
Blocked by: none
Understood: two timing flakes dropped #12412 from merge groups. The engine.test.sh pacing flake is already covered by #12425 (PR #12429). What remains is test-only work in plugins/planning/surface/tests/ui_journey.js. The Q5 own-save check should wait, with a bounded timeout, for the save to land instead of relying on the 700 ms settle ceiling. The phase-2 dot check should name the missing dot in its detail. The refinement comment already lists acceptance criteria. Nothing is missing, and I found no duplicate among open issues.
Automated triage, applied by the lanes App.
melodic-automation-lanes commentedon Nov 2, 2026
Not admitted to the issue lane yet: open pull request #12466 is for it.
melodic-automation-lanes commentedon Nov 2, 2026
The latest triage verdict holds nothing: the issue is at
status: ready.melodic-automation-lanes commentedon Nov 2, 2026
Refinement. A read-only run explored this repository, searched its issues and pull requests, and read the allowed docs to draft what follows. On an issue a listed human filed, it counts as the issue's spec once posted, while unedited; it is still model text, so check it.
Intended outcome: Make the surface journey's Q5 own-save check wait for the save to land with a bounded timeout, and make the phase-2 dot check name the missing dot, as a test-only change.
Acceptance criteria
External state: none
Dependencies
Related work
Sources read
Automated refinement by the lanes App. Intake triages the issue again with it.
github-actions commentedon Nov 2, 2026
The triage sweep dispatched intake triage for this issue (attempt 1 of 2).