ci: give Jest shards CI-only timeout headroom to unblock promotion - #2079
Merged
Conversation
Converts all 57 real-timer `userEvent.setup()` call sites in SearchPicker.test.tsx (47) and HouseholdItemPicker.breadcrumb.test.tsx (10) to the fake-timer idiom via a local `setupUser()` helper, targeting the 11 60-second timeouts that block the v2.15.0 promotion. Applied as an experiment with a verified-safe payload, not as a proven fix. A local unloaded A/B on the full SearchPicker file showed no speedup (1331.6s unconverted vs 1290.7s / 1352.0s converted) and the unconverted file passes locally, so the sandbox never reproduced the CI failure. The ~21s per-dropdown-test cost is CPU-bound and remains unexplained; the "in-file fake-timer control group proves immunity" inference is withdrawn and recorded as withdrawn in agent memory. The dependency bump named in #2076 is ruled out twice: a pre-bump jest/testing-library bisection on #2070 reproduced identical failures on the old versions, and a binary diff of the user-event 14.6.1/14.6.7 tarballs found no hot-path change (utils/misc/wait.js byte-identical, delay:0 unchanged). No production code changed. All 173 assertions and both test counts are byte-identical; `testTimeout: 60000` is unchanged (comment-only edit), since ~592 real-timer call sites remain across the client suite. Fixes #2076 Co-Authored-By: Claude qa-integration-tester <noreply@anthropic.com> Co-Authored-By: Claude dev-team-lead <noreply@anthropic.com>
… tests Quality Gates was failing on the v2.15.0 promotion (#2075) with 60s test timeouts in picker-family suites. Three hypotheses were falsified by measurement before landing on the cause: - The dev-dependencies bump: bisection on #2070 reproduced identical failures on the OLD jest/testing-library versions, and a binary diff of the two user-event tarballs found no hot-path change. - Fake timers: beta already ships SearchPicker.test.tsx and HouseholdItemPicker.test.tsx fully converted, and BOTH still timed out. - Worker contention: in shard 5 the other 77 suites finished at 19:23:29 and SearchPicker.test.tsx then ran alone on the runner until 19:49:23 -- 25m54s with zero contention -- and still blew the ceiling. The actual cause is a machine-speed gap: GitHub-hosted runners are ~1.8x slower single-threaded than the dev boxes the 60s ceiling was calibrated on (SearchPicker 2326s CI vs 1330s local; WorkItemPicker 436s vs 224s). At ~39s/test the heaviest tests cross 60s and the rest do not. - Raise --testTimeout to 240000 in the CI shard invocation only. Verified via --showConfig that the CLI flag lands in globalConfig, dodging the projects[].testTimeout trap. jest.config.ts stays at 60000 so it remains a tight dev-side regression detector. - Record every rejected lever with its evidence in ci.yml so they are not re-proposed, and add a runner-capacity diagnostic step. - Raise scripts/ci-wait.sh defaults (2400s beta / 3600s main). The old 600s/900s were below even a healthy run and reported TIMEOUT ~25min before a verdict existed. - Convert WorkItemPicker.test.tsx (10 sites) and the remaining 6 sites in HouseholdItemPicker.breadcrumb.test.tsx to fake timers. This removes a real wall-clock dependency but is explicitly NOT the fix. - Correct jest.config.ts and agent memory, which asserted a mechanism the measurements refuted. This is a stopgap. The ~21s per-test CPU cost is unexplained and stays open in #2078; next step is node --cpu-prof on a single dropdown test. A green Quality Gates is not evidence the slowness was solved. Refs #2076 Refs #2078 Co-Authored-By: Claude qa-integration-tester <noreply@anthropic.com> Co-Authored-By: Claude product-architect <noreply@anthropic.com> Co-Authored-By: Claude dev-team-lead <noreply@anthropic.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
steilerDev
force-pushed
the
fix/2076-searchpicker-fake-timers
branch
from
September 7, 2026 20:10
36b12c3 to
d3ed005
Compare
steilerDev
changed the title
6 tasks
Contributor
|
🎉 This PR is included in version 2.15.0-beta.7 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
Contributor
|
🎉 This PR is included in version 2.15.0 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Unblocks the v2.15.0 promotion (#2075), whose
Quality Gateshas been failing with 60-second test timeouts in picker-family suites. This is a stopgap, not a root-cause fix — see "What this does not do".Three hypotheses were falsified by measurement before the cause was found. Recording them so they don't get re-proposed:
user-event14.6.1 -> 14.6.7)wait.jsbyte-identical,delay: 0unchanged)betaalready shipsSearchPicker.test.tsxandHouseholdItemPicker.test.tsxfully converted — both still timed outSearchPicker.test.tsxthen ran alone on the runner until 19:49:23 — 25m54s with zero contention — and still blew the ceilingActual cause: a machine-speed gap
GitHub-hosted runners are ~1.8x slower single-threaded than the dev boxes the 60 s ceiling was calibrated on:
SearchPicker.test.tsxWorkItemPicker.test.tsxAt ~39 s/test in CI, the heaviest tests cross 60 s and the rest don't. That also explains why the failing set looked arbitrary — near-identical tests split across the boundary (line 307 failed while line 323 passed), and why it worsened monotonically (8 -> 26 -> 45 min) with no code change: it tracks runner speed, not the tree.
Changes
.github/workflows/ci.yml— append--testTimeout=240000to the shard invocation only. Verified via--showConfigthat the CLI flag lands inglobalConfig, dodging theprojects[].testTimeout-is-silently-ignored trap thatjest.config.tsdocuments.jest.config.tsstays at 60000 so it remains a tight dev-side regression detector. Also adds a runner-capacity diagnostic step and a comment block recording all rejected levers with evidence.scripts/ci-wait.sh— raise defaults to 2400 s (beta) / 3600 s (main). The old 600/900 s were below even a healthy run:Quality Gateson release: promote beta to main #2075 took 40m33s, so the script reportedTIMEOUT~25 min before a verdict existed, reading as a CI fault rather than "still running".WorkItemPicker.test.tsx— convert all 10 real-timeruserEvent.setup()call sites to fake timers.HouseholdItemPicker.breadcrumb.test.tsx— convert the 6 sitesa468c0cfleft behind (it did 4 of 10). Partial conversion is what let failures migrate between files.jest.config.ts+ agent memory — correct text that asserted a mechanism the measurements refuted. The memory file had over-claimed twice already; this stops a third.SearchPicker.test.tsxis not in this diff —a468c0cfconverted it onbetaand that version was kept during the rebase.Verification
WorkItemPicker.test.tsx: 14/14 passing, twice (221.9 s, 220.5 s)HouseholdItemPicker.breadcrumb.test.tsx: 10/10 passing, twiceexpect(...)byte-identical to before; test counts unchanged; no.skip/.only; verified mechanically, not by eyescripts/check-trailers.shpassesWhat this does not do
It does not make anything faster. Shard 5 will still take ~40 min. The ~21 s per-test CPU cost is exactly as unexplained as before — two mechanism guesses were falsified by measurement (
pointerEventsCheck: Neverremoves only ~9%; coverage instrumentation only 0.3%).Do not read a green
Quality Gatesas evidence the slowness was solved. It was only given room. #2078 owns the root cause; next step isnode --cpu-profon a single dropdown test.pointerEventsCheck: Neverwas also rejected on correctness grounds independent of its measurement: it disables user-event'spointer-events: nonecheck, so a test could pass against a control a real user cannot click.Refs #2076
Refs #2078
🤖 Generated with Claude Code