Skip to content

test(driver-sql): name the dialects a local run did not exercise, under the counts - #18649

Merged
huangyiirene merged 3 commits into
mainfrom
claude/issue-18200-driver-sql-skipped-cells-loud
Sep 17, 2026
Merged

huangyiirene merged 3 commits into
mainfrom
claude/issue-18200-driver-sql-skipped-cells-loud

Conversation

@huangyiirene

Copy link
Copy Markdown
Collaborator

Fixes #18200

Clause-②: no

The trap, re-measured on this base (32be735e5b)

pnpm --filter @objectstack/driver-sql test, no servers present:

 Test Files  178 passed | 11 skipped (189)
      Tests  2627 passed | 168 skipped (2795)

The card's arithmetic was 11 of 188, taken 2026-09-14. Re-counted on this base: 11 of 189.
The file total grew by one — sql-driver-17231-multi-value-not-null.test.ts, landed this morning
at 32be735e5b. The skipped-file count itself did not move.

The 11 is the smaller half. On the same run, 168 tests are skipped across 67 of the 189
files: 11 that vitest reports as skipped, and 56 more that it reports as PASSED with a live
cell skipped inside them. A file with a green tick next to it is the least visible skip in the
output, and none of the three recorded instances of this card's failure mode had any way to see it.

The same suite with a live PostgreSQL attached (run in full, below) reports 3315 tests passed,
not 2627
. The local green was blind to 688 tests — 21% of what the suite can run.

What this adds

packages/drivers/driver-sql/src/live-dialect-coverage.reporter.ts, registered in
vitest.config.ts after 'default' so it prints underneath the summary counts — the place a
round actually looks. It names, at the end of every run:

  • which dialects ran and which did not, read from DIALECT_CELLS (this package's own source of
    truth for the driver axis), never inferred from test names;
  • for each one that did not, the env var that would run it;
  • the run's raw skip census, split into the files vitest called skipped and the files it called
    passed;
  • the in-container PostgreSQL recipe, so the signal is an instruction rather than a warning
    (shape 3 as a rider, 7 printed lines).

Declaration-only, as ruled. It reads a run that already happened and writes stdout. It adds no
gate, changes no test's mode, and every hook body is wrapped so a defect in the reporter cannot
redden a run.

Two mechanisms worth naming, both measured rather than assumed:

  • reporters: in the config is spelled conditionally on GITHUB_ACTIONS, because vitest appends
    its github-actions reporter only if (!resolved.reporters.length) (vitest 4.1.11,
    dist/chunks/coverage.*.js). Naming any reporter would silently drop the annotations CI gets
    today; re-adding it under vitest's own condition keeps CI output unchanged.
  • Per-test attribution is deliberately not attempted. In the main process a skipped task
    carries no meta and no skip note (only ctx.skip(note) sets one, and a skipped test never
    reaches its body — probed against 4.1.11), so the only per-test channel left is the test's
    name. Nine files here guard their live cells with a hand-rolled skipIf whose names the
    testkit never wrote, so a name matcher would report a number smaller than the truth — which is
    this card's own disease. The census is therefore reported as the raw skip count it is, and the
    causal sentence is attached to the per-dialect lines, which are exact.

Proof: the two states, run end to end

A signal that fires identically with and without a backend is decoration, so both were run in full.

State A — no servers. pnpm --filter @objectstack/driver-sql test, exit 0:

 Test Files  178 passed | 11 skipped (189)
      Tests  2627 passed | 168 skipped (2795)

  !! driver-sql live-dialect coverage: this run exercised 1 of 3 dialects.

     sqlite         RAN
     live postgres  NOT RUN -- set OS_TEST_POSTGRES_URL to run it
     live mysql     NOT RUN -- set OS_TEST_MYSQL_URL to run it

  The counts above this block are NOT coverage of the dialect(s) marked NOT RUN. This run
  skipped 168 test(s) across 67 of its 189 files: 11 vitest reported as skipped, and 56 more
  it reported as PASSED with skipped tests inside them — the least visible skip in the
  output. Every live postgres and live mysql cell in this package is in that population, and
  a green above says nothing about any of them.

followed by the recipe block and the three-way-zone-skew note.

State B — a live PostgreSQL 16.13, provisioned by the printed recipe. initdb + pg_ctl on
port 54988, server timezone=Asia/Shanghai against process TZ=America/New_York, exit 0:

 Test Files  186 passed | 3 skipped (189)
      Tests  3315 passed | 85 skipped (3400)

  !! driver-sql live-dialect coverage: this run exercised 2 of 3 dialects.

     sqlite         RAN
     live postgres  RAN
     live mysql     NOT RUN -- set OS_TEST_MYSQL_URL to run it

  The counts above this block are NOT coverage of the dialect(s) marked NOT RUN. This run
  skipped 85 test(s) across 60 of its 189 files: 3 vitest reported as skipped, and 57 more
  it reported as PASSED with skipped tests inside them — the least visible skip in the
  output. Every live mysql cell in this package is in that population, and a green above
  says nothing about any of them.

Here the PostgreSQL recipe is suppressed and the MySQL sentence takes its place: provisioning
MySQL means installing a server into the environment, which is not a step to print as a casual
next line.

State C — OS_EXPECT_LIVE_DIALECT_MATRIX=1 with no URLs, the shape CI runs. The testkit
already turns each missing cell into a named failure, so the block does not compete with the red
it would be shouting over:

      Tests  1 failed | 4 passed (5)

  driver-sql live-dialect coverage: OS_EXPECT_LIVE_DIALECT_MATRIX=1, but live postgres
  (OS_TEST_POSTGRES_URL) and live mysql (OS_TEST_MYSQL_URL) were not provisioned — this run
  reported that as a named failure, not as a skip.

The exit code in State C is the pre-existing testkit failure, not the reporter's: States A and B
exit 0 with the block printed, and the summary counts in State A are byte-identical to the same
command run at 32be735e5b before this change.

The recipe was run before it was written down. Teardown was pg_ctl -m fast stop plus removal of
the data directory, verified by observing the port refuse connections rather than by reading an
exit code.

Verification

  • pnpm --filter @objectstack/driver-sql typecheck — exit 0.
  • pnpm --filter @objectstack/driver-sql test — exit 0, twice (States A and B above).
  • pnpm --filter '@objectstack/driver-sql^...' build — exit 0 (dependency closure).
  • Gate families derived from the real changed paths with
    node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands, run with exit
    codes captured to disk before any pipe, then reconciled with --ran on the final head
    7959bdc42f: 45 derived, 43 run, 2 NOT MEASURED, 0 UNRUN.
  • Full repo lint, not narrowed: eslint . --no-inline-config --format json6814 files linted,
    0 findings, exit 0
    , with both changed files present in that population at 0 errors / 0
    warnings. (No narrowing claim is needed, but for the record the config never enables type-aware
    linting for any file, so no diff here can move an untouched file's verdict.)
  • Control-character self-scan over both changed files, beyond check:nul-bytes: clean.

Changeset: none, deliberately

Nothing published moves. @objectstack/driver-sql ships files: ["dist", "README.md", "CHANGELOG.md"] and tsup's entry is src/index.ts alone; the reporter is imported by neither.
Measured after a real build rather than reasoned: LiveDialectCoverageReporter has zero hits
across dist/, README.md and CHANGELOG.md, against a positive control
(UnsupportedDialectEmissionError, from a file that is in the published closure) that hits
dist/index.js, dist/index.mjs and dist/index.d.mts. vitest.config.ts is not published
either. Hence skip-changeset.

Acceptance notes

  • The case for the shape this PR was told not to build. The dispatch reserved "a suite that
    refuses to report green when live cells were skipped" to the maintainer's floor and asked for a
    report if the work made a strong case. It does, and the new number is the argument: the card's
    11 files were never the population — 67 files are, and 56 of them report PASSED. A reader
    who learns to discount the "11 skipped" column still has no signal for the other 56, so a louder
    declaration is doing more work here than the card assumed, and correspondingly more is still
    riding on the reader choosing to look. Recorded for the seat to file to the decision box;
    nothing conditional on it is built here, not even behind a flag.
  • The env-var pair is the only knob, and it has no in-repo discovery path.
    OS_TEST_POSTGRES_URL / OS_TEST_MYSQL_URL appear in the testkit, the globalSetup and
    ci.yml, and now in this block — but packages/drivers/driver-sql/README.md does not mention
    either. Noted, not filed: the block now prints them at the end of every unprovisioned run, which
    is a strictly better discovery path than a README section, so no card is warranted. Successor if
    one is ever wanted: whoever next edits that README.
  • sql-driver-autonumber-cold-race.test.ts names one of its suites
    attemptWithoutPoisoning (skipped) by interpolating the cell's availability into the describe
    title, so with no server the word "skipped" is part of the suite name rather than a state. It
    reads as a quarantined test in the output and is not one. Noted, not filed: cosmetic, single
    site, and the new block makes the real cause legible. Successor: the next PR to touch that file.

Generated by Claude Code

… counts

A local run with no servers ends on `178 passed | 11 skipped` and a round
reads it as coverage. It is not: every live-Postgres and live-MySQL cell is
inside the skipped counts, and 56 further files report PASSED with a live
cell skipped inside them.

Adds a declaration-only reporter, registered after `default` so it lands
under the summary. It names which dialects did not run, the env var that
would run each, and a measured in-container PostgreSQL recipe. It changes no
behaviour, adds no gate and cannot fail a run.

Claude-Session: https://claude.ai/code/session_01CqmCgU5RGDoJYhHUMVp2af
Co-authored-by: Claude <noreply@anthropic.com>
@huangyiirene huangyiirene added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Sep 17, 2026 — with Claude
@github-actions github-actions Bot added the tests label Sep 17, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

7 anchor(s) derived from 1 changed package(s); no hand-written page names any of them. ⚠️ 1 changed file(s) yielded no anchor (packages/drivers/driver-sql/vitest.config.ts), so the pages documenting them are NOT COVERED by this run — this is not a clean bill of health for those files.

What this run could not see
  • 1 changed file(s) yielded no anchor (packages/drivers/driver-sql/vitest.config.ts) — pages documenting those are invisible to this run
  • 4 name(s) were too generic to anchor anything (single lowercase words)
  • the SDK route bridge reached 60 of 215 client-bound route-ledger rows — the other 155 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 155: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 55 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 100 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 11 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json cf39b83c090a5642c1ce260c6bc00972650b9ffapackageMentionDocs.

Which tree this was computed on

This run read content/docs from 07ff3e0a4bb7a9732af44005bf5c8b4e7761b40e — the merge of head 7959bdc42fe54598460cb28823e77ce6dbf71cad into base cf39b83c090a5642c1ce260c6bc00972650b9ffa, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 07ff3e0a4bb7a9732af44005bf5c8b4e7761b40e && git checkout 07ff3e0a4bb7a9732af44005bf5c8b4e7761b40e
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin cf39b83c090a5642c1ce260c6bc00972650b9ffa 7959bdc42fe54598460cb28823e77ce6dbf71cad && git checkout -B drift-repro cf39b83c090a5642c1ce260c6bc00972650b9ffa && git merge --no-ff 7959bdc42fe54598460cb28823e77ce6dbf71cad

node scripts/docs-audit/affected-docs.mjs --json cf39b83c090a5642c1ce260c6bc00972650b9ffa

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/m skip-changeset PR has no user-facing published change; bypasses the changeset gate tests

Projects

None yet

2 participants