Skip to content

test(spec,lint): cheaper slow-file sweeps and re-measured test budgets - #26089

Merged
objectstack-fleet[bot] merged 4 commits into
mainfrom
claude/project-thread-9tuohg
Oct 26, 2026
Merged

objectstack-fleet[bot] merged 4 commits into
mainfrom
claude/project-thread-9tuohg

Conversation

@objectstack-fleet

Copy link
Copy Markdown
Contributor

Requested by Steedos · project thread

Part of #23180. This PR covers the packages/spec and packages/lint rows of refresh PR #25960's per-file list that are still over 10 s or over their declared budget (12 rows, listed below). #23180 stays open for the other packages' rows.

Clause-②: no
Metadata-protocol: none

Summary

Tests only. Each row is either made cheaper (three files), or given a budget re-measured from #25960's window, or both. Nothing is deleted, no vitest timeout is raised, and no file moves to a slower or skipped project.

  • packages/lint/src/non-record-object-entry.test.ts: the junk-value sweeps ran every rule once per assertion. They now run each rule once per distinct fixture and memoize the outcome (findings count, or the thrown message), and every assertion reads that record. 348/348 tests, 2.11 s to 1.00 s locally.
  • packages/spec/src/ui/app.test.ts: the compiler-API case built its program with the ES5 default lib (DOM declarations) and every ambient @types package. Neither is an input of app.zod.ts, so the program now uses lib.es2022.d.ts and types: []. The guard still reads the full member list off the resolved type. 1.49 s to 1.25 to 1.37 s locally.
  • packages/spec/scripts/build-schemas-check-mode.manifest.test.ts: the [finding] the published bundle json-schema/objectstack.json declares x-schema-count: 1596 but carries 1585 definitions — and a docs page publishes that field as "the total number of definitions" #12588 block ran the generator twice on one fixture for two read-only cases. It now runs it once per block (beforeAll plus a lazy memo, the same shape as openapi-self-consistency.test.ts's shared run). That is one generator spawn (about 7 s) fewer per run. 136.3 s to 125.1 s locally (shared box).

Rows covered, with their source lines

Source: refresh workflow run 40275747937, job 121418010419 (the #25960 refresh). Line numbers are that job log's.

Row Log line Slowest CI reading (run) Declared before Now
spec scripts/build-schemas-check-mode.retired-keys.test.ts 1339 241.05 s (40204927516) 190 budget 310
spec scripts/build-schemas-check-mode.manifest.test.ts 1340 226.64 s (40204927516) 180 one spawn fewer, budget 290
spec scripts/build-schemas-check-mode.anchor.test.ts 1346 144.97 s (40204927516) 140 budget 190
spec src/eager-entry-import.test.ts 1394 41.61 s (40266979145) 30 budget 60
spec scripts/upgrade-guide-docs.test.ts 1408 30.07 s (40214410982) 30 budget 40
lint src/lazy-deps.test.ts 1442 13.91 s (40200279306) none budget 20
lint src/runtime-lazy-deps.test.ts 1452 13.05 s (40200279306) none budget 20
spec scripts/root-entry-type-nameability.pin.test.ts 1453 12.92 s (40214410982) none budget 20
lint src/non-record-object-entry.test.ts 1456 12.49 s (40214410982) none cheaper, about half the local time
spec src/ui/app.test.ts 1459 11.90 s (40204927516) none cheaper, budget 20
spec scripts/pure-schema-construction.test.ts 1460 11.86 s (40214410982) none budget 20
spec scripts/openapi-self-consistency.test.ts 1473 10.48 s (40204927516) none budget 20

The window is the hourly runs 40200279306 to 40272142430. Over the window's executed readings, the declared budgets were exceeded in: retired-keys 6 of 10, manifest 6 of 10, anchor 1 of 10, eager-entry 7 of 8, upgrade-guide 1 of 8.

Re-measure arithmetic

Every budget follows rulings 6181973363 item 2: the slowest CI reading in the window, x 1.25, rounded up to the next 10 s. Each header line names its reading and run, and each was parsed with readTestBudget from scripts/measure-test-shard-timings.mjs.

  • retired-keys: 241.05 x 1.25 = 301.3, so 310
  • manifest: 226.64 x 1.25 = 283.3, so 290
  • anchor: 144.97 x 1.25 = 181.2, so 190
  • eager-entry-import: 41.61 x 1.25 = 52.0, so 60
  • upgrade-guide-docs: 30.07 x 1.25 = 37.6, so 40
  • lazy-deps: 13.91 x 1.25 = 17.4, so 20
  • runtime-lazy-deps: 13.05 x 1.25 = 16.3, so 20
  • root-entry-type-nameability.pin: 12.92 x 1.25 = 16.2, so 20
  • ui/app: 11.90 x 1.25 = 14.9, so 20
  • pure-schema-construction: 11.86 x 1.25 = 14.8, so 20
  • openapi-self-consistency: 10.48 x 1.25 = 13.1, so 20

non-record-object-entry.test.ts gets no budget. Its local time halved, which puts its CI reading at about 6 s by the window's CI-to-local ratio (12.49 / 2.11). Per c91af8e, a later CI reading over 10 s would be the measurement a declaration needs.

Why the bscm, eager-entry and upgrade-guide floors stay

One build-schemas.ts spawn costs about 7.3 s on a profile: zod to JSON Schema 3.14 s, module load and script 2.73 s, writes 0.97 s, GC 0.40 s, child_process 0.24 s. Each bscm case is one spawn on its own single-cause fixture. retired-keys and anchor have no two cases on an identical fixture, so no run can be shared. In manifest, the #12588 pair was the one identical-fixture pair inside a block. Sharing across blocks (for example a #16431 control with another block's run) would couple fixtures that each block seeds itself, so it is left out. Batching causes or redesigning the producer is the option seat ruling 6176284384 declined.

eager-entry-import spawns one fresh tsx child per published entry by design, and upgrade-guide-docs spawns the real producer once per mode or usage error it pins (about 0.7 s each). Both are kept and re-declared.

Measurements

Local runs use the CI shape: CI=true pnpm exec vitest run --project P --maxWorkers=1 with the agent environment variables unset. They ran on a shared box, so ratios carry more weight than absolutes.

File Before After
lint non-record-object-entry 2111 ms 999 ms
lint lazy-deps 2312 ms 2305 ms
lint runtime-lazy-deps 1952 ms 2030 ms
spec ui/app 1490 ms 1250 to 1372 ms
spec manifest (bscm) 136.3 s 125.1 s
spec retired-keys (bscm) 124.0 s header only
spec anchor (bscm) 121.6 s header only
spec eager-entry-import 12.3 s 12.1 s
spec upgrade-guide-docs 6.7 s 7.3 s
spec pure-schema-construction 3.2 to 4.2 s 3.7 s
spec openapi-self-consistency 3.4 s 3.6 s
spec root-entry-type-nameability.pin 3.14 s 3.11 s

Mutation proof (non-record-object-entry)

The memo must not hide a rule regression. I mutated recordsOf in packages/lint/src/object-graph.ts to return v as AnyRec[] with scripts/ablation-replace.mjs, then ran both the new file and the BASE copy against the same mutant. Both failed the identical set, 121 of 348. The restore was proven: the blob equals HEAD's (693c48ac1308) and git diff HEAD was empty.

Acceptance notes

Verification

All runs below are on head 1a44227 (this branch merged with main bbf7061), unless noted.

  • pnpm --filter @objectstack/lint test: 164 files, 7356 tests passed (exit 0).
  • The six changed spec local files, run together after the merge: 6 files, 169 tests passed (exit 0).
  • Before the second merge, on ea742f5: pnpm --filter @objectstack/spec test (723 files, 21902 tests passed, 1 todo), pnpm --filter @objectstack/spec typecheck, pnpm --filter @objectstack/lint typecheck. All exit 0.
  • Derived gates: node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack gives 84 commands. All 84 exit 0, and --ran prints: "84 derived famil(ies) accounted for, 84 run, 0 NOT-MEASURED". One note: check:dts-closure first exited 1. After the merge, @object-ui/plugin-editor could not build because main's new monaco-editor dependency was not installed yet. After pnpm install --frozen-lockfile and a rebuild of the two named UI packages, it exited 0 ("231/231 declared declaration file(s) present").
  • The bscm files belong to the vitest repo project, which CI's Test Core runs. Locally, the manifest file ran after its change, 18/18 passed.

🤖 Generated with Claude Code

https://claude.ai/code/session_01LpCAbWQuprShJmC2GJXTp7

- lint non-record-object-entry: one sweep of the authoring-rule table per
  distinct stack, read by both the throws case and the count case; the
  control stack is swept once per collection.
- spec ui/app: the compiler-API case reads the ES2022 lib and no ambient
  @types, the inputs the NavigationArea type actually has.
- spec bscm manifest: the #12588 pair shares one generation.
- Five declared budgets re-measured over refresh PR #25960's window, and
  five undeclared rows declare their ruled class at their CI reading.

Claude-Session: https://claude.ai/code/session_01LpCAbWQuprShJmC2GJXTp7
Co-authored-by: Claude <noreply@anthropic.com>
The compiler-API case's cheaper program brings the file down about 12%
locally, which still leaves the window's 11.90 s CI reading near the
10 s line. Declare 20 s from that reading (x 1.25, rounded up).

Claude-Session: https://claude.ai/code/session_01LpCAbWQuprShJmC2GJXTp7
Co-authored-by: Claude <noreply@anthropic.com>
@objectstack-fleet objectstack-fleet Bot added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Oct 26, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

Nothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 0 changed package(s)), so this run has no opinion about the docs.

What this run could not see

Coarse fallback — 0 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 8a990d8b4175a3da65c8f7690ac0b0da641654e5 → packageMentionDocs.

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

席位核对

ACCEPT · domain:spec seat 2 · 2026-10-26T11:56Z · head 1a44227437fdc73694b27bd4e602f9d69eb45110

  1. Shape. Draft, base main, Part of #23180 (the packages/spec and packages/lint rows of chore(ci): refresh the Test Core shard-timings dataset #25960's list; test budget (domain:spec): 26 test file(s) over 10 s of wall in the hourly full battery (Part of #23151) #23180 stays open for the other packages). 12 test files, +85/−37, skip-changeset. check-governed-merges --pr 26089: NOT governed.
  2. Review owed. None of the four contract faces is touched: test files only, Clause-②: no, Metadata-protocol: none, no published schema, no governed path, no narrowing declared. Per contract-review.md this is CI plus the seat's own read; no second reviewer.
  3. Gates (check-runs pinned to this head). 53 runs: 40 success, 13 skipped, 0 failed. check-expected-skips --pr 26089: OK.
  4. Seat read.
    • non-record-object-entry.test.ts: one sweep per distinct stack, each rule still given a freshly built stack. A rule that throws still fails the count with its own message instead of counting as zero, so the assertions are as strong as before.
    • build-schemas-check-mode.manifest.test.ts: one generator run shared by the block (beforeAll); both cases only read what that run printed and wrote. Budget 180 → 290, re-measured from the slowest of 10 readings × 1.25.
    • app.test.ts: the compiler program reads lib.es2022.d.ts with types: []; guard ✨ Set up Copilot instructions #2 still reads the full member list off the resolved type.
    • The remaining files move a test-budget: line only, each citing its measured reading.
  5. Landing. Ready plus auto-merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

protocol:ui size/m skip-changeset PR has no user-facing published change; bypasses the changeset gate tests tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants