Skip to content

feat(qa): one shared vitest filter preflight for all eight project-declaring packages - #18121

Merged
os-warren merged 3 commits into
mainfrom
claude/issue-17978-shared-vitest-filter-preflight
Sep 14, 2026
Merged

feat(qa): one shared vitest filter preflight for all eight project-declaring packages#18121
os-warren merged 3 commits into
mainfrom
claude/issue-17978-shared-vitest-filter-preflight

Conversation

@os-warren

Copy link
Copy Markdown
Collaborator

Fixes #17978

One shared vitest filter preflight, serving all eight packages that declare vitest projects. Direction A was ruled by the domain:cli execution seat under 验证策略 (5656963807) and confirmed by triage (5657101795), which also ruled the ownership question:

引用或点名一个包不会把卡拉进它的车道,只有编辑它才会。

What changed

  • New private workspace package packages/qa/vitest-filter-preflight — a domain:cli-owned home, ⛔ not root scripts/ and ⛔ not inside packages/spec. It holds the ONE transcription of vitest's TestProject.filterFiles; packages/qa/refd-timer-testkit is the shape precedent (private, no build, exports straight at src).
  • packages/cli migrates onto it and its local copy is deleted. Triage calls the move "a migration inside the owning lane". Measured behaviour-equivalent (below).
  • Call sites in all eight configs. packages/spec, packages/objectql and packages/core are call sites only — one file each, vitest.config.ts, 32 insertions and 0 deletions, no new export and nothing under src/.
  • Two findings this card carried are folded in: the notice's package name is now a parameter (it was hardcoded @objectstack/cli), and the no---project typo case is covered and pinned.

Three measurements that shaped the design

1. The precedent's shape does not transfer, re-confirmed at a26a114d7. Populations takes concrete paths, so a glob cannot be a member. 82 package roots under packages/, 8 declaring projects — the same eight, no drift. packages/cli is 1 of 8 with two exact-path projects, and only as a by-product of a tier walk it already had (#13504 / #14554). The other 7 each pair one explicit list L with a glob complement of L, so the missing component is identical across all seven. exactAndGlobPopulations is that component.

2. The glob project's population is a deliberate SUPERSET (the ruled shape, and it holds): a plain recursive walk of the *.{test,spec}.?(c|m)[jt]s?(x) family minus node_modules/dist, minus L. Matching nothing in a superset implies matching nothing in the real set, so a false accusation is structurally impossible and drift can only under-report. ⛔ Not vitest's own glob engine. Cost of the walk, measured: 0.3 ms (core) to 5.3 ms (spec, 507 test files) per config load.

3. ⛔ Consumers import the module by RELATIVE PATH, never by its bare package name — and that is a measurement, not a preference. Vite bundles a config's relative imports through esbuild, which transpiles TypeScript; it externalises bare specifiers and leaves Node to load the resolved path, which here is a .ts file. The bare form works on this box and prints no warning — because Node 22.22 strips types by default. Re-run with NODE_OPTIONS=--no-experimental-strip-types:

failed to load config from .../packages/core/vitest.probe-bare.config.ts
TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".ts" for .../src/index.ts

This repo declares engines.node: ">=22.0.0", so on a supported Node the bare form turns a silent-drop defect into a total harness outage for that package. The two escapes are worse: building to dist would make eight harnesses' config load depend on build state, and pnpm --filter PKG exec vitest run FILE — the invocation this card is about — runs no build; authoring it as .mjs would drop the types, and CliParseResultOptions is typed structurally precisely so a vitest upgrade renaming an option is a type error here instead of a silent decline.

Per-package before/after — the preflight now speaks where it was silent

before = origin/main's config, run verbatim via --config vitest.before.config.ts beside the real one so __dirname still resolves to the package root. Every run exits 0 in both legs; the defect is silence, not failure.

package glob side: dropped path named exact side: dropped path named no---project typo named healthy control
packages/types 0 → 4 0 → 4 0 → 2 byte-identical
packages/rest 0 → 4 0 → 4 0 → 2 byte-identical¹
packages/runtime 0 → 4 0 → 4 0 → 2 byte-identical
packages/spec 0 → 4 0 → 4 0 → 2 byte-identical
packages/objectql 0 → 4 0 → 4 0 → 2 byte-identical
packages/core 0 → 4 0 → 4 0 → 2 byte-identical
packages/qa/dogfood 0 → 4 0 → 4 0 → 2 byte-identical¹

¹ the only diff line is the Duration decimal.

"FILTER SELECTED NOTHING present" reads 0 before, 2 after in all three shapes in all seven (twice: once at config load, once from the exit listener). ⭐ This extends the card's reproduction from the 4 packages the first dispatch probed to 7 of 7, including spec, objectql and core, which had never been probed — still zero negative probes, vitest 4.1.11.

packages/qa/dogfood also confirms the shape outside the REPO_TESTS idiom: its exact list is the inline SHARED_SHOWCASE and its glob project is isolated.

packages/cli — migration equivalence, not before/after. Its before already carried a preflight, so the reading asked whether the move changed anything a reader sees. Same notice count (2), same dropped-path count (4), same pnpm --filter @objectstack/cli in the notice (4), healthy control byte-identical. The lost-run output differs in exactly two words, twice: tierproject. Deliberate — "tier" is this package's private vocabulary while "project" is vitest's own and is correct for all eight.

A healthy narrowed run still contributes zero bytes: renderLostFilterNotice returns the empty string and no writer is called and no exit listener is registered.

Tests, and the ablation that proves they can fail

pnpm --filter @objectstack/vitest-filter-preflight test2 files, 73 tests, pass. typecheck green, and tsc --listFiles puts 4 of 4 of the package's files in the program (src/index.ts, both tests, vitest.config.ts) — nothing hidden.

test/config-wiring-sweep.test.ts is the anti-phantom sweep, and its population is derived, not listed: it walks packages/ for package-root vitest configs, masks their comments, and requires every one that declares projects to invoke the preflight — so a ninth package is caught on the PR that adds it. The count is asserted as a floor, so a newcomer fails on its own wiring rather than on a number.

Ablation (fix committed first; mutation proven on disk by occurrence count; trap-restored with absolute paths; restore proven by blob hash, not by an exit code):

leg on-disk proof sweep
delete the runFilterPreflight call from packages/types/vitest.config.ts runFilterPreflight({ count 1 → 0; blob a5d708a1ef829c5c exit 1, 1 failed | 40 passed, fails on that package's row
point packageName at another package own-name count 1 → 0, wrong-name 1; blob → aba7fc56 exit 1, same row
restore blob back to a5d708a1 (= HEAD), git diff HEAD empty

Predicted direction was "turns red" and that is what both legs did. The second leg is the pin for the carried finding: a shared notice bound to one package sends every other package's reader to a command that runs the wrong suite.

Gates

node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack against the actual change set, every command run, exit codes recorded to disk, then reconciled:

Run reconciliation — 91 derived, 91 run, 0 NOT-MEASURED, 0 UNRUN.
✓ dispatch-gates --ran: 91 derived famil(ies) accounted for — 91 run,
  0 NOT-MEASURED (a DERIVED zero — all 91 recorded an exit code and none of them is 3).

Plus this lane's standing full union, which the derivation does not name: pnpm lint exit 0 at cd75179f35 (eslint . --no-inline-config, the whole repo — not a narrowing).

Two gates needed work rather than just passing:

  • check:cross-package-test-inputs went red and is satisfied, not routed around. The sweep reads outside its package by construction. It now declares its radius in scripts/cross-package-test-inputs.mjs — three globs under packages/ plus three scripts/ files, with heldBy witnesses for the two globs built from a loop variable that the detector can see but cannot name — and turbo.json carries the matching @objectstack/vitest-filter-preflight#test inputs with !**/node_modules/**. ⭐ Side effect worth having: the suite's turbo hash now really covers every package's config, so a config change re-runs the sweep.
  • check:type-check-debt first exited 3 (OOM at --max-old-space-size=4096 on a shared box) — recorded as NOT MEASURED, not as a pass, then re-run at 8192 and green: 77/81 packages type-checked, 55 raw errors, none above its recorded number, surplus: none. check:type-check-coverage green with the new package counted (81 packages, +1 vs the record).

check-engine-split-ratio --days 90 also exited 2 first — the gate refusing to measure on a shallow clone — and is green after git fetch --shallow-since, at 97.8%. Neither of those two numbers is about this diff; both are recorded so the zero above is real.

No repo-level sweeper was added, so check:pm-dispatch-gates does not fire: the anti-phantom sweep is a vitest test inside the cli-owned home, which is also what keeps a gate-class file out of root scripts/.

skip-changeset — measured, not assumed

Nothing published moves. The new package is private: true. For each of the seven touched published packages, every files[] path was grepped for the subject symbols (runFilterPreflight, exactAndGlobPopulations, matchesVitestFilter, testFilesUnder): zero hits in all seven, with a positive control proving the grep works (ObjectLogger in packages/core/dist, defineStack in packages/spec/dist). packages/cli's files[] is ["dist","README.md","CHANGELOG.md"] and the moved module lived at the package root, outside all three. packages/spec's files[] includes src/**/*.zod.ts; this diff touches no .zod.ts.

pnpm-lock.yaml moved by 12 insertions and 0 deletions — exactly one new importers: block for the new package, no version change to anything installed — and pnpm install --frozen-lockfile exits 0, which is the proof it is tool-written rather than hand-edited.

Acceptance notes

  • The seven new call sites are in no tsc program. Only packages/cli type-checks its own vitest.config.ts (via tsconfig.test.json); the other seven declare include: ["src/**/*"] and their tsconfig.test.json siblings declare ["src*"], so no package-root config there is compiled by anything. Pre-existing — packages/cli/tsconfig.test.json's header records the same state for cli before Queue-flake anchor: test/vitest-tiers-partition.test.ts #14554 graduated it — and not widened by this change, but it does mean a @ts-expect-error in any of those seven files would be a phantom check. Covered here by measurement instead: the wiring sweep asserts the call shape textually, and every one of the seven configs was loaded and run in both legs above. Noted, not filed — no declared contract requires a config to be type-checked, and the carrier is the check:type-check-coverage ratchet's own graduation work.
  • The wiring sweep is scoped to packages/. A package-root config outside that tree growing projects would not be swept — an under-report, the direction this card resolves uncertainty in. Scoping is what keeps the declared radius to one already-open root instead of opening examples/ and apps/ roots in ci.yml's crosspkg filter.

Fences

Module home is domain:cli-owned, so the stop-clause did not trigger: nothing landed in root scripts/ or inside packages/spec. No gate weakened. packages/rest's OS_REST_LOG: 'silent' from a26a114d7 (#18090) is untouched in all three blocks. No file on #17630's surface was touched — this diff edits no test file in packages/qa/dogfood. No governed surface is in the PR diff.


Generated by Claude Code

@os-warren os-warren added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Sep 14, 2026 — with Claude
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 9 package(s): @objectstack/cli, @objectstack/core, @objectstack/objectql, @objectstack/dogfood, @objectstack/vitest-filter-preflight, @objectstack/rest, @objectstack/runtime, @objectstack/spec, @objectstack/types, touching 18 documentable anchor(s). ⚠️ 11 changed file(s) yielded no anchor (packages/cli/vitest.config.ts, packages/core/vitest.config.ts, packages/objectql/vitest.config.ts, …), so the pages documenting them are NOT COVERED by this run — this is not a clean bill of health for those files.

7 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:

  • content/docs/ai/skills-reference.mdx (via node_modules (literal, a string literal in NEVER_WALKED))
  • content/docs/data-modeling/field-types.mdx (via EMPTY (symbol, a top-level const object))
  • content/docs/deployment/cli.mdx (via node_modules (literal, a string literal in NEVER_WALKED))
  • content/docs/deployment/troubleshooting.mdx (via EMPTY (symbol, a top-level const object))
  • content/docs/kernel/services-checklist.mdx (via EMPTY (symbol, a top-level const object))
  • content/docs/protocol/objectql/types.mdx (via EMPTY (symbol, a top-level const object))
  • content/docs/upgrading.mdx (via node_modules (literal, a string literal in NEVER_WALKED))
What this run could not see
  • 11 changed file(s) yielded no anchor (packages/cli/vitest.config.ts, packages/core/vitest.config.ts, packages/objectql/vitest.config.ts, …) — pages documenting those are invisible to this run
  • 11 name(s) were too generic to anchor anything (single lowercase words)
  • the SDK route bridge reached 60 of 215 client-bound route-ledger rows — the other 155 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 155: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 55 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 100 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 152 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json ca7886047b27667122f9c2c44de3b6e67eb361fepackageMentionDocs.

Which tree this was computed on

This run read content/docs from 147716212017b158f9e0f776e60542c3d91bc2cc — the merge of head cd75179f3519c2c1e7d96f1ac3b564697c4974e9 into base ca7886047b27667122f9c2c44de3b6e67eb361fe, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 147716212017b158f9e0f776e60542c3d91bc2cc && git checkout 147716212017b158f9e0f776e60542c3d91bc2cc
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin ca7886047b27667122f9c2c44de3b6e67eb361fe cd75179f3519c2c1e7d96f1ac3b564697c4974e9 && git checkout -B drift-repro ca7886047b27667122f9c2c44de3b6e67eb361fe && git merge --no-ff cd75179f3519c2c1e7d96f1ac3b564697c4974e9

node scripts/docs-audit/affected-docs.mjs --json ca7886047b27667122f9c2c44de3b6e67eb361fe

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

Advisory only, and a precision-first one (#9192): a page is listed because it names a
symbol, wire route or SDK method this diff touched — not because it mentions a changed
package. Each row says which anchor put it there, so a wrong row is reportable rather than
merely annoying. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs ca7886047b27667122f9c2c44de3b6e67eb361fe → pass the list as
args.docs, on the commit named under Which tree this was computed on.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file size/xl skip-changeset PR has no user-facing published change; bypasses the changeset gate tests tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[finding] the silent partial-filter drop #17853 fixes in packages/cli is live in the seven other packages that declare vitest projects

1 participant