From b41bea3927548c744571742e1e830786244e5d5d Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 11 Sep 2026 10:11:43 +0000 Subject: [PATCH 1/2] fix(ci): put the `scripts/__tests__` markdown population on the shard trigger MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `scripts/**/*.test.ts` is in the root Vitest config's `node` project, so the repository's own gate tests run inside `Test (shard N/4)` — the job objectui#8861 taught to ask whether changed markdown is a test's input. Those gate tests read this repository's documentation as data, and they were outside `SCAN_ROOTS`. objectui#8861's stated reason for leaving them out was that `scripts/**` is not on the exclusion list, so a pull request touching one already runs in full. That is true of a pull request touching THE TEST; the hazard is a pull request touching THE MARKDOWN THE TEST READS, and those are different pull requests. Measured rather than argued: all 45 candidate files under `scripts/__tests__` were run under an fs trace and the markdown each opened was recorded. The union is every tracked markdown document in the tree — 1734 of 1734 — so this widening is the "different ruling" the scanner's own header warned it would be, and the header now says so instead of warning about it. Priced in runs rather than documents, over the 513 first-parent commits available: 36 reached the second stage, 21 already ran, 14 of the remaining 15 move from skip to run, and the 15th (`8011852dc`, apps/site-only, no markdown) still skips and should. The rest of the exclusion list keeps its value; its markdown half, on this job only, does not. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01FhBNJcLRZLe8M87VcUgpKr --- .github/workflows/ci.yml | 21 +- content/docs/guide/ci-cd-pipeline.md | 2 +- .../__tests__/markdown-test-inputs.test.ts | 98 +++- scripts/markdown-test-inputs.mjs | 435 +++++++++++++++++- 4 files changed, 527 insertions(+), 29 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 7867baa4c0..0ac761eb7b 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -780,12 +780,27 @@ jobs: # So this job asks one more question of the markdown the exclusions # above dropped: does a test READ it? `markdown-test-inputs.mjs` holds # the derived class -- which documents, and which test reads each -- - # and audits itself against the tree. The widening is deliberately - # narrow: a markdown-only change no test reads still skips, which is - # the whole point of the exclusions. Only THIS job carries the extra + # and audits itself against the tree. Only THIS job carries the extra # stage, because only this job runs the tests that read markdown # (measured: `type-check` reads no document, and no e2e spec does). # + # ⚠️ objectui#9096: that class is now TOTAL over markdown, and this + # comment used to claim the opposite ("the widening is deliberately + # narrow"). `scripts` joined the scan roots, and the gate tests under + # `scripts/__tests__` read the documentation the way the product tests + # read a README -- measured by running all 45 of them under an fs + # trace, 1734 of 1734 tracked markdown files are opened by at least + # one of them. ⇒ for THIS job, the markdown half of the exclusion list + # above is now inert: any markdown-only pull request runs the shards. + # Priced over the 513 first-parent commits available at the time: 36 + # reached this stage, 21 already ran, and the widening moves 14 of the + # remaining 15 from skip to run -- 2.7pp of merges. + # + # ⛔ The rest of the exclusion list is NOT inert and must stay: a + # non-markdown change under `content/**`, `docs/**` or `apps/site/**` + # still skips. `8011852dc` is the live instance -- a site-only change + # carrying no markdown, which skipped then and skips now. + # # Same failure posture as the diff above -- a question that cannot be # answered means RUN. The script needs nothing but the checkout and # the runner's own node, so it is reachable here, before pnpm and diff --git a/content/docs/guide/ci-cd-pipeline.md b/content/docs/guide/ci-cd-pipeline.md index 9913d3e906..74ab0298d6 100644 --- a/content/docs/guide/ci-cd-pipeline.md +++ b/content/docs/guide/ci-cd-pipeline.md @@ -218,7 +218,7 @@ it green — which is how two of `type-check`'s gates came to be missing from th |---|---|---|---| | `changeset-check` | Changeset Fixed Group Check | `scripts/check-changeset-fixed.mjs` — every workspace package must be in the changeset `fixed` group or explicitly ignored. It checks group *membership*; it does **not** check whether the PR added a changeset. | Every run | | `type-check` | Type Check | `scripts/check-type-check-coverage.mjs`, then `pnpm check:phantom-deps`, then `pnpm check:self-import`, then `pnpm check:unreferenced-sources`, then `pnpm check:doc-example-readers`, then `pnpm check:handler-key-reads`, then `pnpm check:published-tsconfig-exclude`, then `pnpm check:side-effects-array`, then `pnpm check:element-data-source-declaration`, then `pnpm check:esm-specifiers`, then `pnpm check:spec-symbols`, then `pnpm check:action-forward-parity`, then `pnpm check:designer-field-key-parity`, then `pnpm check:icon-record-names`, then `pnpm check:i18n-keys`, then `pnpm check:i18n-drift`, then `pnpm check:i18n-designer-parity`, then `pnpm type-check:scripts`, then `pnpm type-check:vitest-config`, then `pnpm type-check`, then `pnpm type-check:vitest-setup`. The coverage guard runs first because turbo silently skips packages that have no `type-check` script, so a package without one would otherwise read as passing (#2911). `pnpm check:phantom-deps` fails when a released package imports a bare specifier its own `package.json` does not declare — a *phantom dependency*, invisible locally because the workspace root's `devDependencies` sit on the upward resolution path from every package directory and on no consumer's, so `require.resolve('react', { paths: ['packages/core/src'] })` succeeds while `@object-ui/core` declares react in no field at all ([#4394](https://github.com/objectstack-ai/objectui/issues/4394)). `pnpm check:self-import` runs next because it reuses that gate's parser: it fails when a file inside a package names its OWN package, a specifier that resolves through the package's `exports` map to `dist/` while `type-check` waits on `^build` — the *dependencies'* builds, never the package's own — so on a cold cache the declarations do not exist yet and the file fails with `TS2307`. Locally it is always green, because every local workflow builds before it type-checks and leaves a `dist/` behind; PR #4789's first run was red on exactly one such line ([#4801](https://github.com/objectstack-ai/objectui/issues/4801)). `pnpm check:unreferenced-sources` runs next, reusing the same parser again: it fails when a covered package ships a source file that nothing reaches — not the package's declared entry, and not its build config. Until [#7515](https://github.com/objectstack-ai/objectui/issues/7515) no gate here could see one: `check-dist-completeness` asks whether `dist/` holds what `tsc` emits, `check-readme-exports` compares documented exports against shipped ones, and a file that is in the tarball while being reachable from nothing is outside both — so the detection mechanism was a human reading unrelated code, which is how both instances found in one week were found ([#7319](https://github.com/objectstack-ai/objectui/issues/7319), [#7397](https://github.com/objectstack-ai/objectui/issues/7397)). The hazard is not the bytes: the file #7319 removed carried the same export name as a live engine one package over and evaluated no predicate, so name-completion alone could have wired a silently wrong renderer into a published package. Reachability has TWO roots, and the second is the whole difficulty — `packages/components` reaches its two `use-sync-external-store` shims only through `vite.config.ts` `resolve.alias` entries whose importer is a bundled dependency no source file names, so a walk that skips that leg reports exactly those two live files as dead on its first run, and a gate that cries wolf gets switched off rather than fixed. Scope is DECLARED per package in `COVERED_PACKAGES` and the uncovered remainder is printed as a count derived from the workspace on every run, because the alias mechanisms differ per package and a gate that covers one package correctly beats one that covers forty with false positives. An alias expression it cannot evaluate is a FINDING rather than a skip, since skipping one would make it accuse whatever file that alias points at. `pnpm check:doc-example-readers` runs next, on the same parser again: it fails when an exported symbol's own JSDoc `@example` hand-spells a resolution that its REAL call sites obtain by calling a shared reader. A doc comment is what the next call site is copied from, so prose that outlives the ruling it encoded re-seeds every later copy — measured at two cards and three copied call sites ([#7627](https://github.com/objectstack-ai/objectui/issues/7627), [#7638](https://github.com/objectstack-ai/objectui/issues/7638)), both closed by pointing the prose at `resolveRecordSourceObjectName`. Nothing here could see either one, and `check-spec-symbol-derivation` was credited with the class twice — in #7638's card body and then in the dispatch that repeated it — while its rule 4 judges `@objectstack/spec` citations at member granularity and says nothing about prose prescribing a LOCAL spelling ([#7652](https://github.com/objectstack-ai/objectui/issues/7652)). It fires on four conditions at once — the example calls the symbol it documents, a real call site fills the same argument slot by calling an exported single-`return` reader, the example does not, and what the example writes there is that reader's own return expression or one of the rungs it resolves between — which is what keeps it off the literals and placeholders an example legitimately carries. It does NOT judge whether a prescribed spelling is correct: on the day either card was filed the prose and every copy of it agreed, and no gate reading only the tree can know a ruling. What it catches is the state right after, when the call sites move and the prose does not. `pnpm check:handler-key-reads` runs next, on the same parser again: it fails when an `on*` handler key that a REGISTERED renderer reads off the authored document is not a declared member of the zod arm for the type it is registered under. `BaseSchema` is `.passthrough()`, so an undeclared key is not refused — it stops being judged and the value is KEPT, then reaches the renderer that reads it; measured on the built dist, `{ type: 'kanban', columns: [], onCardClick: { action: 'toast' } }` went from REFUSED to ACCEPTED with the object surviving into the parsed output ([#7664](https://github.com/objectstack-ai/objectui/issues/7664)). Every gate stayed green, because the [#6124](https://github.com/objectstack-ai/objectui/issues/6124) ledger's population is two hand-written arrays of tuples and that change re-keyed the arm by SUBSTITUTION — so its length assertion held, and a count ratchet would have been green too, which is why [#7753](https://github.com/objectstack-ai/objectui/issues/7753) rejected that option on the instance itself. This gate derives BOTH populations: the arms from every `type: z.literal(…)` in `packages/types/src/zod`, and the read sites from every real `ComponentRegistry.register(…)` call — read off the AST, because one types file NAMES that call in prose eleven times and registers nothing. It follows the document one component at a time rather than every JSX child, because most children are handed a DIFFERENT document (a dashboard's widgets each get their own), and the chain it must reach is four hops long: `register('kanban', ObjectKanbanRenderer)` names a component, that component is an HOC, the document arrives at `ObjectKanban` through a render-prop parameter and at `KanbanRenderer` through an object spread. It says nothing about keys that reach a renderer only through a `{...props}` spread onto a Radix root or a DOM listener slot — there is no read site to derive from — nor about the ledger's `?: never` tombstones, which have no read site by construction; `KNOWN_UNDECLARED_READS` is an exemption list that only shrinks, each row naming the card that owns the fix, and a row whose read site the gate can no longer find fails it. It lives in `scripts/` because the read sites are spread across `@object-ui/plugin-*` and `packages/components`, which `@object-ui/types` may not import — `check:phantom-deps` rejects it and it would close a cycle. `pnpm check:published-tsconfig-exclude` follows, config reads only: it fails when a published package's build `tsconfig.json` excludes tooling by FILE NAME (`*.test.ts`) without also excluding the tooling DIRECTORIES (`**/__tests__/**` and its two siblings, derived from `TOOLING_FILE` rather than retyped). A name-only exclude stops the files that happen to be named that way and nothing else, so the first shared helper added to a `__tests__/` directory becomes a program input and an emitting program writes it into the published `dist` — three times so far, each found by a human and never by a gate ([#4006](https://github.com/objectstack-ai/objectui/issues/4006), [#4836](https://github.com/objectstack-ai/objectui/issues/4836), [#6943](https://github.com/objectstack-ai/objectui/issues/6943), the third in the same package as the first). [#7212](https://github.com/objectstack-ai/objectui/issues/7212) measured the standing exposure — 29 published packages carrying the name form with ZERO offending files, green because nobody had added such a helper yet — and the gate landed together with their conversion so `main` was green on merge. It reads `exclude` arrays and nothing else: no build, no artifact, no emit model, which is the narrower scope that keeps it clear of the modelling [#4846](https://github.com/objectstack-ai/objectui/issues/4846) declined for the artifact-level gate. Six published packages are named carve-outs, each re-proving its own reason on every run: `cli`, `create-plugin` and `data-objectstack` emit from a `tsup` entry graph, `plugin-charts` keeps its tooling exclude in the `dts()` options, and `console` and `runner` are Vite applications with `noEmit: true` and no `dts()` plugin. `pnpm check:side-effects-array` runs next, sources only and no build: it fails when a package's `sideEffects` ARRAY and its module bodies disagree in either direction — a module that registers something at load time and is not named (a bundler drops it, and the registration is gone from a *consumer's* app with no error, no warning and exit 0), or a name whose module no longer registers anything. `@object-ui/app-shell` declares such an array because both simpler answers are measurably wrong for it: omitting the field makes the whole package unshakeable, and `"sideEffects": false` silently drops three live SDUI widget registrations to zero chunks ([#6535](https://github.com/objectstack-ai/objectui/issues/6535), [#6683](https://github.com/objectstack-ai/objectui/issues/6683)). The enumeration is re-derived from the module bodies on every run rather than listed, so there is no second copy to rot. The artifact half of the same contract — do those registrations survive a real bundler — cannot run in this job at all: it needs a built console, so it lives in the SDUI registration pin step of `performance-budget.yml`. `pnpm check:element-data-source-declaration` runs next, sources only and no build: it fails when a source that consumes `ElementDataSourceGate` does not also pass through `elementDataSourceBlock()`, the seam that declares the `dataSource` key the gate reads. A block that wraps the gate off-seam publishes an authoring surface missing the one key its own runtime honours, and the html tier reports that key with the same `unknown-prop` warning it gives the spellings that do nothing ([#6678](https://github.com/objectstack-ai/objectui/issues/6678)). `pnpm check:esm-specifiers` follows it for the same reason — sources only, no build: it fails when a published package whose build preserves import specifiers (a bare emitting `tsc`, which never rewrites them) writes a relative specifier with no file extension. Node's ESM resolver does not extension-search relative specifiers, so such a specifier makes the published entry unloadable outside a bundler; `@object-ui/react`'s entry died with `ERR_MODULE_NOT_FOUND` while every bundler-based consumer, the whole test suite and CI stayed green ([#4538](https://github.com/objectstack-ai/objectui/issues/4538)). The half that actually *imports* each built entry needs a full build and runs in `node-esm-load-gate.yml`. `pnpm check:action-forward-parity` fails when an action renderer's forward whitelist drops a key the action runtime reads — the class that shipped six times one key at a time, each time green, because the key parses and publishes while the payload is dropped one hop before the runner ([#4050](https://github.com/objectstack-ai/objectui/issues/4050)). `pnpm check:designer-field-key-parity` fails when one of the field designers' statically declared payload shapes (`FieldMetadataPayload`, `ServerFieldSchema`, `DesignerFieldDefinition`) declares a key the installed `@objectstack/spec` `FieldSchema` refuses by NAME. Such a key makes `PUT /api/v1/meta/object/:name` return a hard 422 `INVALID_METADATA` that blocks *every subsequent save* of that object, and the author cannot tell from the designer UI which key did it — the class had been filed three times, each closed with a per-key tombstone written after the instance was found in production, with nothing detecting the next one ([#4644](https://github.com/objectstack-ai/objectui/issues/4644) `indexed`, [#4687](https://github.com/objectstack-ai/objectui/issues/4687) `distance_metric`, [#4676](https://github.com/objectstack-ai/objectui/issues/4676) `placeholder`, gated by [#5761](https://github.com/objectstack-ai/objectui/issues/5761)). It reads the accept set off the schema itself rather than from a list, and it covers a deliberately documented *subset* of the write path: a key that reaches the payload only through a `patchDef` spread or an index signature is outside its reach, and the boundary is stated in the script's own docblock. Its draft-I/O half — the `readFields`/`writeFields` round-trip, which has no declared shape to read — runs in the test suite as `object-fields-io.spec-keys.test.ts`. Same placement rationale as the gates around it: it parses the sources with `typescript` and imports the installed spec, so it needs the install and nothing built. `pnpm check:icon-record-names` fails when an authored icon NAME that reaches a resolver reading lucide's runtime `icons` record is not a live key of that record. lucide retires a spelling by dropping it from that record while keeping it as a deprecated named export, so the retired name still imports, still type-checks and still renders wherever it is used as a *component* — `Edit === SquarePen` is true — and resolves to nothing wherever it is used as a *string*: nothing goes red in either direction, which is why the class was repaired twice in two packages before anyone gated it ([#5586](https://github.com/objectstack-ai/objectui/issues/5586), [#5622](https://github.com/objectstack-ai/objectui/issues/5622), [#5633](https://github.com/objectstack-ai/objectui/issues/5633)). It carries no list of retired spellings — the record itself is the judgement — and it re-discovers the resolver population from source on every run, which is how its first pass found four record-reading resolvers nobody had catalogued. It sits here because it parses the sources with `typescript` and reads the installed lucide: the install, and nothing built. The three locale gates sit in the middle because all of them parse the sources with `typescript`: they need the install and nothing built. `pnpm check:i18n-keys` fails when a `t()` call site asks for a key the `en` pack does not define ([#3530](https://github.com/objectstack-ai/objectui/issues/3530)); `pnpm check:i18n-drift` fails when a change to an `en` string is not accompanied by the nine translation packs ([#3650](https://github.com/objectstack-ai/objectui/issues/3650)), and it is why this job's checkout sets `fetch-depth: 0` — it diffs against the merge base, which a depth-1 clone cannot resolve. `pnpm check:i18n-designer-parity` fails when the metadata-admin designer's own module-local string tables come apart — an `en` row with no `zh` row, or a shared row whose two values carry different `{placeholders}` ([#8834](https://github.com/objectstack-ai/objectui/issues/8834)). The two gates before it are blind to that file by construction — the first classifies the module `module-local table` by declaration and skips it, the second read only the ten locale packs and this table is not one of them — so the same card also gave `pnpm check:i18n-drift` that table as a SECOND population, which is the half that catches a changed `en` value whose `zh` row did not follow. `pnpm type-check:scripts` (`tsconfig.scripts.json`) covers `scripts/**/*.ts`, which `pnpm type-check` cannot reach at all — `scripts/` has no package.json, so turbo never walks it, and the coverage guard decides coverage per *package*. Until [#3494](https://github.com/objectstack-ai/objectui/issues/3494) that left the pin tests in `scripts/__tests__/` — including the one pinning this very page — compiled by nothing. `pnpm type-check:vitest-config` runs `apps/console/tsconfig.node.json` directly — the program that already lists `../../vitest.config.mts` and `apps/console/vitest.config.ts` ([#3476](https://github.com/objectstack-ai/objectui/issues/3476)) — because until [#7328](https://github.com/objectstack-ai/objectui/issues/7328) the only thing that ran it was the console's own `type-check` script, reached through the task runner, whose `type-check` task waits on `^build`. (Named in prose rather than as a code span on purpose: the pin below reads this cell as this job's gate list, so spelling that invocation out would credit the job with a command it does not run.) The cheapest compiler that reads the root Vitest config was therefore reachable only through the most expensive job here, and PR #7291 paid for it: a conditionally spread `dist` project whose literal `extends: true` widened to `boolean` degraded the whole `projects` array to `never[]`, every gate its author ran was green, and CI reported three errors, two of them at `../../vitest.config.mts`. It sits in the cheap half beside `pnpm type-check:scripts` for the same measured reason — nothing in its program imports an `@object-ui/*` package, so it needs the install and nothing built. `pnpm type-check:vitest-setup` (`tsconfig.vitest-setup.json`) closes the same gap for the four repo-root `vitest.setup.*` files, uncovered until [#3515](https://github.com/objectstack-ai/objectui/issues/3515); it runs *last*, after `pnpm type-check`, because `vitest.setup.dom.tsx` side-effect-imports four `@object-ui/*` packages and resolves them through the declarations that turbo's `^build` produces. | Every run; on a PR the steps short-circuit when only ignored paths changed | -| `test` | Test (shard N/4) | When a pull request changed nothing outside the exclusion list, the decision step runs `scripts/markdown-test-inputs.mjs` before anything else, and the job runs in full when the answer is yes ([#8861](https://github.com/objectstack-ai/objectui/issues/8861)). The exclusions drop every markdown path, and a markdown document can be a TEST'S INPUT: [#8857](https://github.com/objectstack-ai/objectui/issues/8857) changed one package README, this job reported success in ten seconds having run nothing, and the merge-queue build then failed the same shard in 907 seconds and dequeued it. That script carries the derived class — which documents a test reads, and which test reads each — and audits itself against the tree, so the widening stays narrow: a markdown-only change nothing reads still skips. It is this job's stage only, because this is the job that runs those tests. Then `pnpm test --shard=N/4` across a 4-runner matrix with `fail-fast: false`, so every shard reports its own failures. No coverage instrumentation — v8 adds 40–100% overhead. Then, **on shard 1 only**, `pnpm test:dist` — the built-artifact lane ([#7183](https://github.com/objectstack-ai/objectui/issues/7183)). It delegates to a turbo task scoped to the one package that holds built-artifact pins; that task depends on the package's OWN build (`dependsOn: ["build"]`, not `^build`), so the bundle exists before the pins read it, and then runs the `dist` vitest project, whose pins import a package's BUILT bundle instead of its `src` — a claim the source-aliased suite above is structurally unable to make, since the root config aliases every workspace package to `src`. It is deliberately not sharded and not repeated on the other three runners: the lane is a handful of files, and running it on all four would pay for the same build four times. | Pull requests and merge-queue builds (everything but `push`); steps short-circuit on a PR that changed only ignored paths | +| `test` | Test (shard N/4) | When a pull request changed nothing outside the exclusion list, the decision step runs `scripts/markdown-test-inputs.mjs` before anything else, and the job runs in full when the answer is yes ([#8861](https://github.com/objectstack-ai/objectui/issues/8861)). The exclusions drop every markdown path, and a markdown document can be a TEST'S INPUT: [#8857](https://github.com/objectstack-ai/objectui/issues/8857) changed one package README, this job reported success in ten seconds having run nothing, and the merge-queue build then failed the same shard in 907 seconds and dequeued it. That script carries the derived class — which documents a test reads, and which test reads each — and audits itself against the tree. It is this job's stage only, because this is the job that runs those tests. [#9096](https://github.com/objectstack-ai/objectui/issues/9096) then added `scripts` to that scan, because the gate tests under `scripts/__tests__` read this repository's documentation as data and run in these same shards — measured by running all 45 of them under an fs trace, **every** tracked markdown file in the tree is opened by at least one of them. ⇒ for this job the markdown half of the exclusion list is now inert: **any markdown-only pull request runs the shards**. Priced over the 513 first-parent commits available then: 36 reached this stage, 21 already ran, and the widening moves 14 of the remaining 15 from skip to run. The rest of the exclusion list still pays — a non-markdown change under `content/**`, `docs/**` or `apps/site/**` skips as before. Then `pnpm test --shard=N/4` across a 4-runner matrix with `fail-fast: false`, so every shard reports its own failures. No coverage instrumentation — v8 adds 40–100% overhead. Then, **on shard 1 only**, `pnpm test:dist` — the built-artifact lane ([#7183](https://github.com/objectstack-ai/objectui/issues/7183)). It delegates to a turbo task scoped to the one package that holds built-artifact pins; that task depends on the package's OWN build (`dependsOn: ["build"]`, not `^build`), so the bundle exists before the pins read it, and then runs the `dist` vitest project, whose pins import a package's BUILT bundle instead of its `src` — a claim the source-aliased suite above is structurally unable to make, since the root config aliases every workspace package to `src`. It is deliberately not sharded and not repeated on the other three runners: the lane is a handful of files, and running it on all four would pay for the same build four times. | Pull requests and merge-queue builds (everything but `push`); steps short-circuit on a PR that changed only ignored paths | | `test-coverage` | Test (coverage shard N/4) | `pnpm test:coverage --reporter=blob --shard=N/4` across a 4-runner matrix with `fail-fast: false`. Each shard writes `.vitest-reports/blob-N-4.json` — raw coverage and test results in one file — and uploads it as an artifact even when the shard is red, which is what makes a failing coverage run diagnosable at all (vitest deletes `coverage/` on a red run unless `coverage.reportOnFailure` is set, [#5402](https://github.com/objectstack-ai/objectui/issues/5402)). The configured coverage thresholds are neutralised on the shard legs, because a quarter of the suite judged against a whole-suite threshold is not a defect signal; they are enforced once, on the merged report, by the job below ([#5403](https://github.com/objectstack-ai/objectui/issues/5403)). | **Push only** | | `coverage-report` | Test (coverage) | Downloads the four blob reports, refuses to continue unless all four arrived, merges them with `pnpm test:coverage --merge-reports` into one complete report — which is where the configured coverage thresholds are enforced, over the whole merged map, the shard legs having overridden them to zero — and publishes that report as the `coverage-report` artifact (kept 7 days, the same as the blobs it is derived from). Its last step runs on every path and states the outcome: the job is **red, with an error annotation**, whenever the gate did not run for the commit — before [#5403](https://github.com/objectstack-ai/objectui/issues/5403) the final step carried the implicit `success()` and was silently skipped by 311 of 373 coverage jobs, which is how four days of a 100%-failing coverage job went unnoticed. A breach of the thresholds is reported *separately* from a lane that never delivered, because the two call for opposite actions. ⛔ It never merges a report from fewer than four shards: a wrong coverage number is worse than a missing one. The Codecov upload this job used to carry was retired by [#5436](https://github.com/objectstack-ai/objectui/issues/5436) — `CODECOV_TOKEN` was never set, so it failed on every push; the trend dashboard and PR coverage comments are gone with it, the gate is not. | **Push only** | | `e2e` | Build & E2E | Builds the console with `vite build` (`VITE_BASE_PATH=/console/`), verifies the artifact, then `pnpm test:e2e --project=chromium`. Uploads the Playwright report on failure. | Every run; on a PR the steps short-circuit when only ignored paths changed | diff --git a/scripts/__tests__/markdown-test-inputs.test.ts b/scripts/__tests__/markdown-test-inputs.test.ts index 44364e68ce..7fa3869bb8 100644 --- a/scripts/__tests__/markdown-test-inputs.test.ts +++ b/scripts/__tests__/markdown-test-inputs.test.ts @@ -25,12 +25,21 @@ * is not a reconstruction: `ci.yml` carries three copies of this step * and the other two are untouched by this change, so the "before" * answer is a real one taken from the file rather than from history. - * inert a markdown document nothing reads still yields `should_run=false`, + * inert an excluded change nothing reads still yields `should_run=false`, * through the WIDENED step. Without this leg, deleting the exclusions * entirely would pass every other assertion in this file. * - * The inert leg uses a markdown document the pull request ADDS, which is inert by - * construction: nothing in the tree can read a file that did not exist. + * ## ⚠️ objectui#9096 moved the inert leg off markdown, and that is the ruling + * + * The inert leg used to add a markdown document nothing reads. That leg is gone, + * because after objectui#9096 added `scripts` to the scan roots there is no such + * document: 1734 of 1734 tracked markdown files are opened by at least one test + * under `scripts/__tests__`. ⛔ So this file does NOT pretend the class stayed + * narrow. It pins the opposite — `every markdown path is an input` is asserted + * outright below, so the day that stops being true someone is told — and it + * moves the inert leg to what the exclusion list still buys: a NON-markdown + * change under `apps/site/**`, which is the shape of the one commit in the + * measured 513-merge window that still skips and should. */ import { describe, expect, it, beforeAll, afterAll } from 'vitest'; import { execFileSync } from 'node:child_process'; @@ -41,6 +50,7 @@ import { fileURLToPath } from 'node:url'; import { ADJUDICATED, + SCAN_ROOTS, auditTree, declaredEntries, deriveCandidates, @@ -218,16 +228,36 @@ describe('the objectui#8857 pull request shape', () => { expect(outcome.shouldRun).toBe('false'); }); - it('INERT: a markdown document nothing reads still skips, through the WIDENED step', () => { - // Added by the pull request, so inert by construction. Without this leg a + it('INERT: an excluded NON-markdown change still skips, through the WIDENED step', () => { + // objectui#9096 took markdown off this leg — see the header. What is left + // is the half of the exclusion list that still pays: `apps/site/**` and the + // non-markdown files under `content/**` and `docs/**`. Without this leg a // step that answered `true` unconditionally would pass everything above. + // + // This is not a hypothetical shape: `8011852dc` ("fix(site): drop the + // count-based claims from the homepage") is exactly it, and it is the one + // commit of the 36 that reached this stage in the measured window which + // still skips after the widening. const outcome = runStep(TEST_STEP, { - 'packages/plugin-grid/NOTES.md': '# Notes\n\nNothing reads this.\n', - '.changeset/quiet-owls-tickle.md': '---\n---\n\nNo release.\n', + 'apps/site/src/app/page.tsx': 'export default function Page() { return null; }\n', + 'docs/diagrams/architecture.svg': '\n', }); expect(outcome.shouldRun).toBe('false'); }); + it('FIRES on the objectui#9096 shape: a root document only `scripts/__tests__` reads', () => { + // The card's own named instance, re-run as a fixture: the commit whose + // subject begins `docs(agents): narrow the package-level test claim` + // changed only `AGENTS.md`, three tests under `scripts/__tests__` read it, + // and the shards skipped. `AGENTS.md` is in no product test's class, so + // this leg fires ONLY because `scripts` is a scan root. + const files = { 'AGENTS.md': '# AGENTS\n\nA sentence a gate test reads.\n' }; + expect(runStep(TEST_STEP, files).shouldRun).toBe('true'); + // …and the unwidened copy of the same step, untouched by this change, is + // the "before" reading for it. + expect(runStep(TYPE_CHECK_STEP, files).shouldRun).toBe('false'); + }); + it('CONTROL: the widened step still runs everything for an ordinary source change', () => { // The harness has to be able to say `true` for the boring reason, or the // `false` legs above prove nothing about the harness. @@ -275,10 +305,60 @@ describe('the derived class', () => { expect(findings.map((finding) => finding.kind)).toContain('unadjudicated-document'); }); - it('a declared tree covers a document added inside it, and nothing outside it', () => { + it('a declared tree covers a document added inside it, and never a non-markdown sibling', () => { expect(declaredEntries()).toContain('content/docs/**'); expect(markdownTestInputsAmong(['content/docs/guide/a-page-added-today.md'])).toHaveLength(1); + // The extension is the discriminator that survives objectui#9096: the class + // is total over markdown and empty over everything else, so a JSON file + // beside a declared document is still not an input. expect(markdownTestInputsAmong(['content/docs/guide/a-page-added-today.json'])).toHaveLength(0); - expect(markdownTestInputsAmong(['packages/plugin-grid/README.md'])).toHaveLength(0); + expect(markdownTestInputsAmong(['apps/site/src/app/page.tsx'])).toHaveLength(0); + }); +}); + +/* ── objectui#9096: the ruling, stated as an assertion ───────────────────── */ + +describe('objectui#9096 — `scripts` is a scan root, and the class is now total', () => { + it('scans `scripts`, and the scan is what finds those tests', () => { + expect(SCAN_ROOTS).toContain('scripts'); + const candidates = deriveCandidates(); + const fromScripts = [...candidates.keys()].filter((file) => file.startsWith('scripts/')); + // A floor, not an exact count: the point is that the root is producing + // candidates at all, so a root that silently stopped resolving is caught. + // 45 files the day the root landed. + expect(fromScripts.length).toBeGreaterThanOrEqual(40); + }); + + it('names the card\'s instance: three gate tests read `AGENTS.md`, so it is an input', () => { + const hits = markdownTestInputsAmong(['AGENTS.md']); + expect(hits).toHaveLength(1); + expect(hits[0].readers.length).toBeGreaterThanOrEqual(3); + for (const reader of hits[0].readers) expect(reader.startsWith('scripts/')).toBe(true); + }); + + it('EVERY tracked markdown document is an input — the cost of the ruling, pinned', () => { + // ⚠️ This is not a target, it is a CONFESSION kept honest. objectui#9096 + // widened the class until it covered the whole tree, and the header argues + // why that is acceptable (14 extra runs per 513 merges). An assertion is + // the only form of that statement which cannot quietly stop being true: + // if a future change re-narrows the class, this fails and the person doing + // it has to come here and say so. + const tracked = execFileSync('git', ['ls-files', '--', '*.md', '*.mdx'], { + cwd: REPO_ROOT, + encoding: 'utf8', + maxBuffer: 64 * 1024 * 1024, + }) + .split('\n') + .filter((line) => line.trim() !== ''); + + // Floor under the floor: an empty `git ls-files` would pass the next + // assertion while proving nothing (objectui#8468). + expect(tracked.length).toBeGreaterThanOrEqual(300); + + const uncovered = tracked.filter((file) => markdownTestInputsAmong([file]).length === 0); + expect( + uncovered, + 'the class is declared TOTAL over markdown; a document outside it means the ledger lost a tree', + ).toEqual([]); }); }); diff --git a/scripts/markdown-test-inputs.mjs b/scripts/markdown-test-inputs.mjs index ec49e38560..08dd80d2fb 100644 --- a/scripts/markdown-test-inputs.mjs +++ b/scripts/markdown-test-inputs.mjs @@ -66,20 +66,78 @@ * audit, and a new document dropped into a tree that is already declared * (`content/docs/**`) is covered the moment it lands. * + * ## objectui#9096 -- `scripts` joins the scan, and what that ruling costs + * + * objectui#8861 left `scripts/__tests__/**` out and gave this reason: + * + * ⛔ "`scripts/__tests__/**` is deliberately OUT of the hazard class: + * `scripts/**` is not on the exclusion list, so a PR touching it already + * gets a full run." + * + * ⛔ That sentence is true about a pull request that touches THE TEST. The + * hazard is a pull request that touches THE MARKDOWN THE TEST READS. Those are + * different pull requests, and only the second one is invisible. + * + * ### The measurement, and why the population count is the wrong unit + * + * Every one of the 45 candidate files under `scripts/__tests__` was run under + * an `fs` trace and the markdown it opened was recorded. The union is not + * "nearly every markdown document in the tree" -- it is EVERY one of them: + * 1734 of 1734 tracked `.md` / `.mdx` files are opened by at least one of these + * tests. One file does it alone: `dollar-dialect-alias-census.test.ts` scans + * every tracked path through a helper module, which is limit 2 below in its + * purest form -- the scanner cannot see that read at all, and it is the read + * that makes the class total. + * + * ⇒ so the honest statement of this ruling is: on the `test` job, the decision + * step's markdown exclusions are now INERT. Any markdown-only pull request runs + * the shards. + * + * The reason that is nonetheless the right ruling is that documents are the + * wrong unit to price it in. Runs are. Measured over the 513 first-parent + * commits on `main` available at the time of writing: + * + * 36 reached the second stage at all (everything they changed was excluded) + * 21 of those already run, on the class objectui#8861 declared + * 15 still skipped -- the blind spot this card is about + * 14 of those 15 fire once `scripts` is a scan root + * 1 of those 15 still skips, and SHOULD: `8011852dc` changed only + * `apps/site/**` and carried no markdown at all + * + * ⇒ the price of making the class total is 14 extra full runs per 513 merges + * (2.7pp), because 93% of merges change something outside the exclusions and + * already run. The exclusion list itself does NOT go inert: non-markdown paths + * under `content/**`, `docs/**` and `apps/site/**` still skip, and that last + * commit is the live control for it. + * + * ### ⚠️ What a reviewer should weigh against it + * + * triage on objectui#9096 wrote "⛔ Do not widen the `Test (shard N/4)` trigger + * to 'every markdown file' ... a resolved input list, ⛔ not a glob". The + * MECHANISM that fence prescribes is intact -- the workflow still consults a + * derived, adjudicated, self-auditing list and never a glob. Its EFFECT is not: + * the list's answer is now `true` for every markdown path. That tension is real + * and is left visible on purpose rather than argued away. The two numbers a + * reviewer needs to reverse this are above: 1734 of 1734 documents, 14 of 513 + * merges. + * + * ### Where the trees come from + * + * A `…/**` entry below means "this test walks this tree". Where a test read + * most of a tree, the tree is declared rather than its files, which is the same + * over-produce-rather-than-miss direction the scanner takes: the cost of a + * declared document nothing reads is one extra run, and the cost of a missed + * one is the defect this file exists to close. + * * ## ⛔ What this does NOT answer -- read this before citing it as coverage * - * 1. **Test surfaces outside `SCAN_ROOTS`.** The scan covers the product test - * surface. `scripts/__tests__/**` is deliberately NOT in it, on triage's - * instruction. ⚠️ Measured while deriving this class, that exclusion is - * NOT free: tests under `scripts/__tests__/**` run in the same shards, and - * they read a large population of repository markdown -- `AGENTS.md`, - * `CONTRIBUTING.md`, `QUICK_REFERENCE.md`, `docs/**`, and every package - * README among them. A markdown-only pull request touching one of those - * still walks into the blind spot this file closes for the product - * surface. The number matters to the decision and is recorded on the card - * filed for it, because widening this class to that population would make - * nearly every markdown document in the tree a test input, which is a - * different ruling from the one this file implements. + * 1. **Test surfaces outside `SCAN_ROOTS`.** ⭐ RULED, objectui#9096 -- this + * limit is closed and the section below records what closing it cost. The + * sentence that used to stand here said widening to `scripts/__tests__` + * "would make nearly every markdown document in the tree a test input, + * which is a different ruling". That was right about the population and + * wrong about nothing else: it IS a different ruling, it was measured, and + * it was made. Nothing is excluded from the scan today. * 2. **A read the scanner cannot see.** It resolves string literals and * recognises directory walks. A test that computes a path from parts no * literal carries, or reads markdown through a helper module in another @@ -106,11 +164,13 @@ export const REPO_ROOT = path.resolve(fileURLToPath(import.meta.url), '..', '..' /** * Where `deriveCandidates()` looks for tests. * - * These are the roots the root Vitest config collects from, minus - * `scripts/**` -- see limit 1 in this file's header for what that costs and why - * it is not this file's call to change. + * Every root the root Vitest config collects from, `scripts` included since + * objectui#9096. That config's `node` project collects `.test.ts` files under + * `scripts` too, so those tests run in the same `Test (shard N/4)` job as the + * rest -- see the objectui#9096 section in this file's header for what + * including them costs and why the cost was paid. */ -export const SCAN_ROOTS = ['packages', 'apps', 'examples', 'eslint-rules']; +export const SCAN_ROOTS = ['packages', 'apps', 'examples', 'eslint-rules', 'scripts']; const SKIP_DIRS = new Set(['node_modules', 'dist', '.git', '.turbo', 'coverage', 'build', '.next']); @@ -374,6 +434,349 @@ export const ADJUDICATED = new Map([ reads: ['content/docs/utilities/vscode-extension.mdx', 'packages/vscode-extension/DESIGN.md'], }, ], + + // --------------------------------------------------------------------------- + // objectui#9096 -- the `scripts/__tests__/**` surface. + // + // Added with `scripts` as a scan root. These are repository GATE tests, and + // they read documentation the way the product tests read a README: as data. + // The `reads` column below was not inferred from the literals -- every one of + // these 45 files was run under an `fs` trace and the documents it actually + // opened were recorded, which is also how the three tests that ENUMERATE a + // markdown population without opening it were caught. + // --------------------------------------------------------------------------- + // Reads no markdown: the document literal is an ARGUMENT to a path-matcher assertion, never opened. + [ + 'scripts/__tests__/check-action-forward-parity.test.ts', + { + reads: [], + notRead: ['content/docs/guide/ci-cd-pipeline.md'], + }, + ], + // Reads no markdown: drives the gate against a fixture repository it writes in a temp directory. + [ + 'scripts/__tests__/check-changeset-claims.test.ts', + { + reads: [], + notRead: ['.changeset/README.md', 'README.md'], + }, + ], + [ + 'scripts/__tests__/check-changeset-no-major.test.ts', + { + reads: ['.changeset/**'], + notRead: ['README.md'], + walker: 'markdown-tree', + }, + ], + // Reads no markdown: drives the gate against a fixture repository it writes in a temp directory. + [ + 'scripts/__tests__/check-changeset-overwrite.test.ts', + { + reads: [], + notRead: ['.changeset/README.md', '.changeset/olive-donkeys-smile.md'], + }, + ], + // Reads no markdown: drives the gate against a fixture repository it writes in a temp directory. + [ + 'scripts/__tests__/check-changeset-presence.test.ts', + { + reads: [], + notRead: ['.changeset/README.md', 'CHANGELOG.md', 'README.md', 'apps/console/README.md'], + walker: 'not-markdown: `.github/workflows/*.yml`', + }, + ], + [ + 'scripts/__tests__/check-doc-component-types.test.ts', + { + reads: ['README.md', 'apps/console/docs/UI_IMPROVEMENT_PROPOSAL.md', 'apps/console/docs/deployment.md', 'apps/console/docs/error-tracking.md', 'content/docs/**'], + walker: 'not-markdown: `.github/workflows/*.yml`', + }, + ], + [ + 'scripts/__tests__/check-doc-example-ids.test.ts', + { + reads: ['content/docs/**'], + notRead: ['examples/README.md'], + walker: 'not-markdown: `.github/workflows/*.yml`', + }, + ], + [ + 'scripts/__tests__/check-doc-example-shared-reader.test.ts', + { + reads: ['content/docs/guide/ci-cd-pipeline.md'], + }, + ], + [ + 'scripts/__tests__/check-doc-example-types.test.ts', + { + reads: ['content/docs/guide/ci-cd-pipeline.md'], + walker: 'not-markdown: `.github/workflows/*.yml`', + }, + ], + [ + 'scripts/__tests__/check-doc-expression-carriage.test.ts', + { + reads: ['README.md', 'apps/console/docs/UI_IMPROVEMENT_PROPOSAL.md', 'apps/console/docs/deployment.md', 'apps/console/docs/error-tracking.md', 'content/docs/**'], + walker: 'markdown-tree', + }, + ], + [ + 'scripts/__tests__/check-doc-fence-languages.test.ts', + { + reads: ['apps/console/docs/**', 'content/docs/**', 'docs/**'], + notRead: ['README.md'], + walker: 'markdown-tree', + }, + ], + [ + 'scripts/__tests__/check-doc-links.test.ts', + { + reads: ['AGENTS.md', 'CHANGELOG.md', 'CLAUDE.md', 'CONTRIBUTING.md', 'LICENSE-THIRD-PARTY.md', 'QUICK_REFERENCE.md', 'README.md', 'ROADMAP.md', 'apps/**', 'content/docs/**', 'docs/ARCHITECTURE.md', 'docs/CONSOLE-STREAMLINING-SUMMARY.md', 'docs/adr/**', 'docs/audits/**', 'examples/**', 'packages/**'], + }, + ], + [ + 'scripts/__tests__/check-doc-snippet-emitted-census.test.ts', + { + reads: ['README.md', 'apps/console/docs/UI_IMPROVEMENT_PROPOSAL.md', 'apps/console/docs/deployment.md', 'apps/console/docs/error-tracking.md', 'content/docs/**', 'docs/ARCHITECTURE.md', 'docs/CONSOLE-STREAMLINING-SUMMARY.md', 'docs/adr/**', 'docs/audits/**', 'packages/**'], + }, + ], + [ + 'scripts/__tests__/check-doc-snippet-types.test.ts', + { + reads: ['README.md', 'apps/console/docs/UI_IMPROVEMENT_PROPOSAL.md', 'apps/console/docs/deployment.md', 'apps/console/docs/error-tracking.md', 'content/docs/**', 'docs/ARCHITECTURE.md', 'docs/CONSOLE-STREAMLINING-SUMMARY.md', 'docs/adr/**', 'docs/audits/**', 'packages/**'], + walker: 'markdown-tree', + }, + ], + // Reads no markdown: the root-document literals are fixture INPUTS to a pure path classifier. + [ + 'scripts/__tests__/check-governed-queue-guard.test.ts', + { + reads: [], + notRead: ['AGENTS.md', 'CLAUDE.md'], + }, + ], + [ + 'scripts/__tests__/check-handler-key-read-sites.test.ts', + { + reads: ['content/docs/guide/ci-cd-pipeline.md'], + }, + ], + // Reads no markdown: opens no markdown; the `.md` literal is a message string. + [ + 'scripts/__tests__/check-i18n-dead-keys.test.ts', + { + reads: [], + walker: 'not-markdown: `.github/workflows/*.yml`', + }, + ], + [ + 'scripts/__tests__/check-installed-spec-pin-claims.test.ts', + { + reads: ['.claude/skills/**', '.github/prompts/component.prompt.md', '.github/prompts/engine.prompt.md', '.github/prompts/ui-library.prompt.md', 'AGENTS.md', 'CLAUDE.md', 'CONTRIBUTING.md', 'LICENSE-THIRD-PARTY.md', 'QUICK_REFERENCE.md', 'README.md', 'ROADMAP.md', 'apps/**', 'content/docs/**', 'docs/ARCHITECTURE.md', 'docs/CONSOLE-STREAMLINING-SUMMARY.md', 'docs/adr/**', 'docs/audits/**', 'examples/**', 'packages/**', 'patches/README.md', 'skills/objectui/**'], + notRead: ['.changeset/8897-installed-spec-pin-claims.md', 'CHANGELOG.md'], + }, + ], + [ + 'scripts/__tests__/check-links-workflow.test.ts', + { + reads: ['content/docs/**', 'docs/**'], + notRead: ['README.md'], + walker: 'markdown-tree', + }, + ], + // Reads no markdown: fixture trees only, and the suite asserts markdown is OUT of the ESLint walk. + [ + 'scripts/__tests__/check-lint-rule-coverage.test.ts', + { + reads: [], + walker: 'not-markdown: the files ESLint walks — the suite asserts `.md` is OUT of that walk', + }, + ], + [ + 'scripts/__tests__/check-new-cross-file-line-citations.test.ts', + { + reads: ['content/docs/guide/ci-cd-pipeline.md'], + }, + ], + // Reads no markdown: reads workflow YAML and sources, no markdown. + [ + 'scripts/__tests__/check-pre-install-import-graph.test.ts', + { + reads: [], + walker: 'not-markdown: `.github/workflows/*.yml`', + }, + ], + [ + 'scripts/__tests__/check-prompt-component-keys.test.ts', + { + reads: ['.github/prompts/component.prompt.md', '.github/prompts/engine.prompt.md', '.github/prompts/ui-library.prompt.md'], + walker: 'not-markdown: `.github/workflows/*.yml`', + }, + ], + // Reads no markdown: the `README.md` literals are packed-file names inside a manifest fixture. + [ + 'scripts/__tests__/check-published-dist-tooling.test.ts', + { + reads: [], + notRead: ['README.md'], + walker: 'not-markdown: `.github/workflows/*.yml`', + }, + ], + [ + 'scripts/__tests__/check-readme-exports.test.ts', + { + reads: ['packages/**'], + walker: 'not-markdown: `.github/workflows/*.yml`', + }, + ], + [ + 'scripts/__tests__/check-shell-escape-residue.test.ts', + { + reads: ['.claude/skills/**', 'AGENTS.md', 'CLAUDE.md', 'content/docs/**', 'skills/objectui/**'], + walker: 'not-markdown: `.github/workflows/*.yml`', + }, + ], + [ + 'scripts/__tests__/check-skill-eval-tokens.test.ts', + { + reads: ['content/docs/guide/ci-cd-pipeline.md', 'skills/objectui/**'], + walker: 'not-markdown: `.github/workflows/*.yml`', + }, + ], + [ + 'scripts/__tests__/check-skill-examples.test.ts', + { + reads: ['.claude/skills/**', 'content/docs/guide/ci-cd-pipeline.md', 'skills/objectui/**'], + notRead: ['README.md'], + walker: 'not-markdown: `.github/workflows/*.yml`', + }, + ], + [ + 'scripts/__tests__/check-skills-paths.test.ts', + { + reads: ['.claude/skills/**', 'skills/objectui/**'], + walker: 'not-markdown: `.github/workflows/*.yml`', + }, + ], + [ + 'scripts/__tests__/ci-cd-pipeline-doc.test.ts', + { + reads: ['.github/prompts/component.prompt.md', '.github/prompts/engine.prompt.md', '.github/prompts/ui-library.prompt.md', 'CONTRIBUTING.md', 'content/docs/guide/ci-cd-pipeline.md'], + walker: 'not-markdown: `.github/workflows/*.yml`', + }, + ], + [ + 'scripts/__tests__/component-node-vocabulary-7434.test.ts', + { + reads: ['content/docs/**', 'packages/**'], + walker: 'markdown-tree', + }, + ], + // Reads no markdown: every document literal is fixture text handed to pure functions. + [ + 'scripts/__tests__/cross-file-line-citation-census.test.ts', + { + reads: [], + notRead: ['README.md', 'ROADMAP.md', 'packages/core/README.md', 'packages/plugin-form/CHANGELOG.md', 'packages/plugin-form/README.md'], + }, + ], + [ + 'scripts/__tests__/doc-version-claims.test.ts', + { + reads: ['content/docs/**', 'packages/**', 'skills/objectui/**'], + walker: 'markdown-tree', + }, + ], + // Reads no markdown: reads workspace manifests and turbo inputs; the `README.md` literal is a declared-input fixture. + [ + 'scripts/__tests__/docs-build-trigger.test.ts', + { + reads: [], + notRead: ['README.md'], + walker: 'not-markdown: workspace `package.json` manifests and turbo input globs', + }, + ], + [ + 'scripts/__tests__/dollar-dialect-alias-census.test.ts', + { + reads: ['.changeset/**', '.claude/skills/**', '.github/prompts/component.prompt.md', '.github/prompts/engine.prompt.md', '.github/prompts/ui-library.prompt.md', 'AGENTS.md', 'CHANGELOG.md', 'CLAUDE.md', 'CONTRIBUTING.md', 'LICENSE-THIRD-PARTY.md', 'QUICK_REFERENCE.md', 'README.md', 'ROADMAP.md', 'apps/**', 'content/docs/**', 'docs/ARCHITECTURE.md', 'docs/CONSOLE-STREAMLINING-SUMMARY.md', 'docs/adr/**', 'docs/audits/**', 'examples/**', 'packages/**', 'patches/README.md', 'skills/objectui/**'], + }, + ], + [ + 'scripts/__tests__/extract-mdx-demos.test.ts', + { + reads: ['content/docs/**'], + walker: 'markdown-tree', + }, + ], + // Reads no markdown: the `README.md` literal names a repo-wide labeler rule, not a file it opens. + [ + 'scripts/__tests__/labeler-package-coverage.test.ts', + { + reads: [], + notRead: ['README.md'], + walker: 'not-markdown: package directories under `packages/`', + }, + ], + [ + 'scripts/__tests__/layered-read-declared-path-4016.test.ts', + { + reads: ['skills/objectui/**'], + walker: 'not-markdown: package `src/` directories', + }, + ], + [ + 'scripts/__tests__/lint-workflow.test.ts', + { + reads: ['content/docs/guide/ci-cd-pipeline.md'], + }, + ], + // Reads no markdown: drives the decision step against fixture repositories it + // writes in a temp directory, so every document literal here is a fixture path + // rather than a file in this tree -- `AGENTS.md` included, which is the + // objectui#9096 firing fixture and is WRITTEN by the test, never read from the + // repository root. + [ + 'scripts/__tests__/markdown-test-inputs.test.ts', + { + reads: [], + notRead: ['AGENTS.md', 'README.md', 'ROADMAP.md', 'packages/plugin-dashboard/README.md'], + }, + ], + [ + 'scripts/__tests__/merge-queue-reporting.test.ts', + { + reads: ['content/docs/guide/ci-cd-pipeline.md'], + walker: 'not-markdown: `.github/workflows/*.yml`', + }, + ], + [ + 'scripts/__tests__/quick-reference-commands-4149.test.ts', + { + reads: ['QUICK_REFERENCE.md'], + walker: 'not-markdown: directory listings that test whether a documented path exists', + }, + ], + [ + 'scripts/__tests__/quick-reference-current-release-4143.test.ts', + { + reads: ['QUICK_REFERENCE.md'], + walker: 'not-markdown: package and app directories', + }, + ], + [ + 'scripts/__tests__/sync-quick-reference-release.test.ts', + { + reads: ['QUICK_REFERENCE.md'], + }, + ], + [ + 'scripts/__tests__/unconsumed-widget-option-claim-6186.test.ts', + { + reads: ['content/docs/plugins/plugin-dashboard.mdx'], + }, + ], ]); /** Is `rel` a markdown document at all? The only extensions this tree publishes. */ From 6f03f379dc94c1e2388b006322b3802188283e71 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 12 Sep 2026 15:38:07 +0000 Subject: [PATCH 2/2] fix(ci): re-derive the markdown-input ledger and its counts on the merged tree The merge brought main's drift into the widened scan root, and two things it carries were measured before that drift. 1. `scripts/__tests__/coverage-red-cause-census.test.ts` is new on main and lands inside the `scripts` scan root this card adds, so the ledger did not describe it and `--audit` was red: unadjudicated-test: scripts/__tests__/coverage-red-cause-census.test.ts reads markdown the ledger does not describe (README.md). Measured, not inferred: run under an `fs` probe the suite opens ZERO markdown. Its `README.md` is sample input to a pure path-matcher assertion (`specFilesFrom([... 'README.md'])` asserted to drop it), and the suite's own docblock keeps its inputs literal on purpose -- it is a census OF corpus walkers. Adjudicated `reads: []` + `notRead`. 2. The counts moved with the tree: 45 -> 46 candidate files under `scripts/__tests__`, and 1734 -> 1805 tracked `.md`/`.mdx`. The totality claim was RE-MEASURED on the merged tree rather than scaled: the one test the argument rests on, `dollar-dialect-alias-census.test.ts`, opens 1805 of 1805 tracked documents by itself, zero gaps. The ruling is unchanged; only the numbers are. The commit-pricing figures (513 first-parent commits, 14 of 15) are left as-is -- they are explicitly stamped "available at the time of writing". No behavioural change: `.github/workflows/ci.yml` still carries zero non-comment changes versus main. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01FhBNJcLRZLe8M87VcUgpKr --- .github/workflows/ci.yml | 4 ++-- content/docs/guide/ci-cd-pipeline.md | 2 +- scripts/markdown-test-inputs.mjs | 15 ++++++++++++--- 3 files changed, 15 insertions(+), 6 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index a84d2ab7e4..b70e435521 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -808,8 +808,8 @@ jobs: # comment used to claim the opposite ("the widening is deliberately # narrow"). `scripts` joined the scan roots, and the gate tests under # `scripts/__tests__` read the documentation the way the product tests - # read a README -- measured by running all 45 of them under an fs - # trace, 1734 of 1734 tracked markdown files are opened by at least + # read a README -- measured by running all 46 of them under an fs + # trace, 1805 of 1805 tracked markdown files are opened by at least # one of them. ⇒ for THIS job, the markdown half of the exclusion list # above is now inert: any markdown-only pull request runs the shards. # Priced over the 513 first-parent commits available at the time: 36 diff --git a/content/docs/guide/ci-cd-pipeline.md b/content/docs/guide/ci-cd-pipeline.md index 301ae66a88..026e367835 100644 --- a/content/docs/guide/ci-cd-pipeline.md +++ b/content/docs/guide/ci-cd-pipeline.md @@ -218,7 +218,7 @@ it green — which is how two of `type-check`'s gates came to be missing from th |---|---|---|---| | `changeset-check` | Changeset Fixed Group Check | `scripts/check-changeset-fixed.mjs` — every workspace package must be in the changeset `fixed` group or explicitly ignored. It checks group *membership*; it does **not** check whether the PR added a changeset. | Every run | | `type-check` | Type Check | `scripts/check-type-check-coverage.mjs`, then `pnpm check:phantom-deps`, then `pnpm check:self-import`, then `pnpm check:unreferenced-sources`, then `pnpm check:doc-example-readers`, then `pnpm check:handler-key-reads`, then `pnpm check:metadata-write-doors`, then `pnpm check:published-tsconfig-exclude`, then `pnpm check:side-effects-array`, then `pnpm check:element-data-source-declaration`, then `pnpm check:esm-specifiers`, then `pnpm check:spec-symbols`, then `pnpm check:action-forward-parity`, then `pnpm check:designer-field-key-parity`, then `pnpm check:icon-record-names`, then `pnpm check:i18n-keys`, then `pnpm check:i18n-drift`, then `pnpm check:i18n-designer-parity`, then `pnpm type-check:scripts`, then `pnpm type-check:vitest-config`, then `pnpm type-check`, then `pnpm type-check:vitest-setup`. The coverage guard runs first because turbo silently skips packages that have no `type-check` script, so a package without one would otherwise read as passing (#2911). `pnpm check:phantom-deps` fails when a released package imports a bare specifier its own `package.json` does not declare — a *phantom dependency*, invisible locally because the workspace root's `devDependencies` sit on the upward resolution path from every package directory and on no consumer's, so `require.resolve('react', { paths: ['packages/core/src'] })` succeeds while `@object-ui/core` declares react in no field at all ([#4394](https://github.com/objectstack-ai/objectui/issues/4394)). `pnpm check:self-import` runs next because it reuses that gate's parser: it fails when a file inside a package names its OWN package, a specifier that resolves through the package's `exports` map to `dist/` while `type-check` waits on `^build` — the *dependencies'* builds, never the package's own — so on a cold cache the declarations do not exist yet and the file fails with `TS2307`. Locally it is always green, because every local workflow builds before it type-checks and leaves a `dist/` behind; PR #4789's first run was red on exactly one such line ([#4801](https://github.com/objectstack-ai/objectui/issues/4801)). `pnpm check:unreferenced-sources` runs next, reusing the same parser again: it fails when a covered package ships a source file that nothing reaches — not the package's declared entry, and not its build config. Until [#7515](https://github.com/objectstack-ai/objectui/issues/7515) no gate here could see one: `check-dist-completeness` asks whether `dist/` holds what `tsc` emits, `check-readme-exports` compares documented exports against shipped ones, and a file that is in the tarball while being reachable from nothing is outside both — so the detection mechanism was a human reading unrelated code, which is how both instances found in one week were found ([#7319](https://github.com/objectstack-ai/objectui/issues/7319), [#7397](https://github.com/objectstack-ai/objectui/issues/7397)). The hazard is not the bytes: the file #7319 removed carried the same export name as a live engine one package over and evaluated no predicate, so name-completion alone could have wired a silently wrong renderer into a published package. Reachability has TWO roots, and the second is the whole difficulty — `packages/components` reaches its two `use-sync-external-store` shims only through `vite.config.ts` `resolve.alias` entries whose importer is a bundled dependency no source file names, so a walk that skips that leg reports exactly those two live files as dead on its first run, and a gate that cries wolf gets switched off rather than fixed. Scope is DECLARED per package in `COVERED_PACKAGES` and the uncovered remainder is printed as a count derived from the workspace on every run, because the alias mechanisms differ per package and a gate that covers one package correctly beats one that covers forty with false positives. An alias expression it cannot evaluate is a FINDING rather than a skip, since skipping one would make it accuse whatever file that alias points at. `pnpm check:doc-example-readers` runs next, on the same parser again: it fails when an exported symbol's own JSDoc `@example` hand-spells a resolution that its REAL call sites obtain by calling a shared reader. A doc comment is what the next call site is copied from, so prose that outlives the ruling it encoded re-seeds every later copy — measured at two cards and three copied call sites ([#7627](https://github.com/objectstack-ai/objectui/issues/7627), [#7638](https://github.com/objectstack-ai/objectui/issues/7638)), both closed by pointing the prose at `resolveRecordSourceObjectName`. Nothing here could see either one, and `check-spec-symbol-derivation` was credited with the class twice — in #7638's card body and then in the dispatch that repeated it — while its rule 4 judges `@objectstack/spec` citations at member granularity and says nothing about prose prescribing a LOCAL spelling ([#7652](https://github.com/objectstack-ai/objectui/issues/7652)). It fires on four conditions at once — the example calls the symbol it documents, a real call site fills the same argument slot by calling an exported single-`return` reader, the example does not, and what the example writes there is that reader's own return expression or one of the rungs it resolves between — which is what keeps it off the literals and placeholders an example legitimately carries. It does NOT judge whether a prescribed spelling is correct: on the day either card was filed the prose and every copy of it agreed, and no gate reading only the tree can know a ruling. What it catches is the state right after, when the call sites move and the prose does not. `pnpm check:handler-key-reads` runs next, on the same parser again: it fails when an `on*` handler key that a REGISTERED renderer reads off the authored document is not a declared member of the zod arm for the type it is registered under. `BaseSchema` is `.passthrough()`, so an undeclared key is not refused — it stops being judged and the value is KEPT, then reaches the renderer that reads it; measured on the built dist, `{ type: 'kanban', columns: [], onCardClick: { action: 'toast' } }` went from REFUSED to ACCEPTED with the object surviving into the parsed output ([#7664](https://github.com/objectstack-ai/objectui/issues/7664)). Every gate stayed green, because the [#6124](https://github.com/objectstack-ai/objectui/issues/6124) ledger's population is two hand-written arrays of tuples and that change re-keyed the arm by SUBSTITUTION — so its length assertion held, and a count ratchet would have been green too, which is why [#7753](https://github.com/objectstack-ai/objectui/issues/7753) rejected that option on the instance itself. This gate derives BOTH populations: the arms from every `type: z.literal(…)` in `packages/types/src/zod`, and the read sites from every real `ComponentRegistry.register(…)` call — read off the AST, because one types file NAMES that call in prose eleven times and registers nothing. It follows the document one component at a time rather than every JSX child, because most children are handed a DIFFERENT document (a dashboard's widgets each get their own), and the chain it must reach is four hops long: `register('kanban', ObjectKanbanRenderer)` names a component, that component is an HOC, the document arrives at `ObjectKanban` through a render-prop parameter and at `KanbanRenderer` through an object spread. It says nothing about keys that reach a renderer only through a `{...props}` spread onto a Radix root or a DOM listener slot — there is no read site to derive from — nor about the ledger's `?: never` tombstones, which have no read site by construction; `KNOWN_UNDECLARED_READS` is an exemption list that only shrinks, each row naming the card that owns the fix, and a row whose read site the gate can no longer find fails it. It lives in `scripts/` because the read sites are spread across `@object-ui/plugin-*` and `packages/components`, which `@object-ui/types` may not import — `check:phantom-deps` rejects it and it would close a cycle. `pnpm check:metadata-write-doors` runs next, on the same parser again: it fails when an in-repo DOOR that can PUT an object-metadata document does not apply the object-metadata write guard. [#7714](https://github.com/objectstack-ai/objectui/issues/7714) ruled one client behaviour — a half-filled relationship is held client-side and the PUT body never carries one without a non-empty `reference` — and its PR implemented that ruling by ENUMERATING the writers it knew of, which were two. [#8057](https://github.com/objectstack-ai/objectui/issues/8057) then reproduced the identical defect on a THIRD writer neither guard covered, in that card's own required dogfood, and [#8676](https://github.com/objectstack-ai/objectui/issues/8676) swept and found nine more. The half that outlives the count is that the sweep the question is naturally asked in cannot see its own subject: `git grep 'client\.save('` returns ZERO over the file #8057 is entirely about, because the call is `client.save(type, …)` and the generic argument sits between the name and the paren — and a hand-rolled `fetch` PUT to `/api/v1/meta/object/:name` is not that spelling at all. So this gate enumerates DOORS, never writers: the writer set is OPEN and nobody has to announce a new member, while the transport set is CLOSED and this repository owns it, so guarding the doors covers every writer past and future without a list anywhere. It derives every call carrying a `method: 'PUT'` literal whose URL RESOLVES to a `/meta` path — resolved through templates, fields and helper return values rather than read, because the repo's central door spells its URL three hops from the call and says nothing at the call site — plus every call to the SDK's `meta.saveItem`, which lives in a package this repo does not own and so cannot be guarded from the inside. A door whose type is a string literal other than `object` is exempt; a door whose type is a runtime value is judged CAPABLE, which is the fail-closed direction. It answers COVERAGE only — whether a guarded door's body is correct is the guard's own pins — and it refuses to report OK unless it found at least one door of each kind and at least one guarded door, so a renamed transport turns it red rather than green. Same placement rationale as the gates around it: it parses the sources with `typescript`, so it needs the install and nothing built. `pnpm check:published-tsconfig-exclude` follows, config reads only: it fails when a published package's build `tsconfig.json` excludes tooling by FILE NAME (`*.test.ts`) without also excluding the tooling DIRECTORIES (`**/__tests__/**` and its two siblings, derived from `TOOLING_FILE` rather than retyped). A name-only exclude stops the files that happen to be named that way and nothing else, so the first shared helper added to a `__tests__/` directory becomes a program input and an emitting program writes it into the published `dist` — three times so far, each found by a human and never by a gate ([#4006](https://github.com/objectstack-ai/objectui/issues/4006), [#4836](https://github.com/objectstack-ai/objectui/issues/4836), [#6943](https://github.com/objectstack-ai/objectui/issues/6943), the third in the same package as the first). [#7212](https://github.com/objectstack-ai/objectui/issues/7212) measured the standing exposure — 29 published packages carrying the name form with ZERO offending files, green because nobody had added such a helper yet — and the gate landed together with their conversion so `main` was green on merge. It reads `exclude` arrays and nothing else: no build, no artifact, no emit model, which is the narrower scope that keeps it clear of the modelling [#4846](https://github.com/objectstack-ai/objectui/issues/4846) declined for the artifact-level gate. Six published packages are named carve-outs, each re-proving its own reason on every run: `cli`, `create-plugin` and `data-objectstack` emit from a `tsup` entry graph, `plugin-charts` keeps its tooling exclude in the `dts()` options, and `console` and `runner` are Vite applications with `noEmit: true` and no `dts()` plugin. `pnpm check:side-effects-array` runs next, sources only and no build: it fails when a package's `sideEffects` ARRAY and its module bodies disagree in either direction — a module that registers something at load time and is not named (a bundler drops it, and the registration is gone from a *consumer's* app with no error, no warning and exit 0), or a name whose module no longer registers anything. `@object-ui/app-shell` declares such an array because both simpler answers are measurably wrong for it: omitting the field makes the whole package unshakeable, and `"sideEffects": false` silently drops three live SDUI widget registrations to zero chunks ([#6535](https://github.com/objectstack-ai/objectui/issues/6535), [#6683](https://github.com/objectstack-ai/objectui/issues/6683)). The enumeration is re-derived from the module bodies on every run rather than listed, so there is no second copy to rot. The artifact half of the same contract — do those registrations survive a real bundler — cannot run in this job at all: it needs a built console, so it lives in the SDUI registration pin step of `performance-budget.yml`. `pnpm check:element-data-source-declaration` runs next, sources only and no build: it fails when a source that consumes `ElementDataSourceGate` does not also pass through `elementDataSourceBlock()`, the seam that declares the `dataSource` key the gate reads. A block that wraps the gate off-seam publishes an authoring surface missing the one key its own runtime honours, and the html tier reports that key with the same `unknown-prop` warning it gives the spellings that do nothing ([#6678](https://github.com/objectstack-ai/objectui/issues/6678)). `pnpm check:esm-specifiers` follows it for the same reason — sources only, no build: it fails when a published package whose build preserves import specifiers (a bare emitting `tsc`, which never rewrites them) writes a relative specifier with no file extension. Node's ESM resolver does not extension-search relative specifiers, so such a specifier makes the published entry unloadable outside a bundler; `@object-ui/react`'s entry died with `ERR_MODULE_NOT_FOUND` while every bundler-based consumer, the whole test suite and CI stayed green ([#4538](https://github.com/objectstack-ai/objectui/issues/4538)). The half that actually *imports* each built entry needs a full build and runs in `node-esm-load-gate.yml`. `pnpm check:action-forward-parity` fails when an action renderer's forward whitelist drops a key the action runtime reads — the class that shipped six times one key at a time, each time green, because the key parses and publishes while the payload is dropped one hop before the runner ([#4050](https://github.com/objectstack-ai/objectui/issues/4050)). `pnpm check:designer-field-key-parity` fails when one of the field designers' statically declared payload shapes (`FieldMetadataPayload`, `ServerFieldSchema`, `DesignerFieldDefinition`) declares a key the installed `@objectstack/spec` `FieldSchema` refuses by NAME. Such a key makes `PUT /api/v1/meta/object/:name` return a hard 422 `INVALID_METADATA` that blocks *every subsequent save* of that object, and the author cannot tell from the designer UI which key did it — the class had been filed three times, each closed with a per-key tombstone written after the instance was found in production, with nothing detecting the next one ([#4644](https://github.com/objectstack-ai/objectui/issues/4644) `indexed`, [#4687](https://github.com/objectstack-ai/objectui/issues/4687) `distance_metric`, [#4676](https://github.com/objectstack-ai/objectui/issues/4676) `placeholder`, gated by [#5761](https://github.com/objectstack-ai/objectui/issues/5761)). It reads the accept set off the schema itself rather than from a list, and it covers a deliberately documented *subset* of the write path: a key that reaches the payload only through a `patchDef` spread or an index signature is outside its reach, and the boundary is stated in the script's own docblock. Its draft-I/O half — the `readFields`/`writeFields` round-trip, which has no declared shape to read — runs in the test suite as `object-fields-io.spec-keys.test.ts`. Same placement rationale as the gates around it: it parses the sources with `typescript` and imports the installed spec, so it needs the install and nothing built. `pnpm check:icon-record-names` fails when an authored icon NAME that reaches a resolver reading lucide's runtime `icons` record is not a live key of that record. lucide retires a spelling by dropping it from that record while keeping it as a deprecated named export, so the retired name still imports, still type-checks and still renders wherever it is used as a *component* — `Edit === SquarePen` is true — and resolves to nothing wherever it is used as a *string*: nothing goes red in either direction, which is why the class was repaired twice in two packages before anyone gated it ([#5586](https://github.com/objectstack-ai/objectui/issues/5586), [#5622](https://github.com/objectstack-ai/objectui/issues/5622), [#5633](https://github.com/objectstack-ai/objectui/issues/5633)). It carries no list of retired spellings — the record itself is the judgement — and it re-discovers the resolver population from source on every run, which is how its first pass found four record-reading resolvers nobody had catalogued. It sits here because it parses the sources with `typescript` and reads the installed lucide: the install, and nothing built. The three locale gates sit in the middle because all of them parse the sources with `typescript`: they need the install and nothing built. `pnpm check:i18n-keys` fails when a `t()` call site asks for a key the `en` pack does not define ([#3530](https://github.com/objectstack-ai/objectui/issues/3530)); `pnpm check:i18n-drift` fails when a change to an `en` string is not accompanied by the nine translation packs ([#3650](https://github.com/objectstack-ai/objectui/issues/3650)), and it is why this job's checkout sets `fetch-depth: 0` — it diffs against the merge base, which a depth-1 clone cannot resolve. `pnpm check:i18n-designer-parity` fails when the metadata-admin designer's own module-local string tables come apart — an `en` row with no `zh` row, or a shared row whose two values carry different `{placeholders}` ([#8834](https://github.com/objectstack-ai/objectui/issues/8834)). The two gates before it are blind to that file by construction — the first classifies the module `module-local table` by declaration and skips it, the second read only the ten locale packs and this table is not one of them — so the same card also gave `pnpm check:i18n-drift` that table as a SECOND population, which is the half that catches a changed `en` value whose `zh` row did not follow. `pnpm type-check:scripts` (`tsconfig.scripts.json`) covers `scripts/**/*.ts`, which `pnpm type-check` cannot reach at all — `scripts/` has no package.json, so turbo never walks it, and the coverage guard decides coverage per *package*. Until [#3494](https://github.com/objectstack-ai/objectui/issues/3494) that left the pin tests in `scripts/__tests__/` — including the one pinning this very page — compiled by nothing. `pnpm type-check:vitest-config` runs `apps/console/tsconfig.node.json` directly — the program that already lists `../../vitest.config.mts` and `apps/console/vitest.config.ts` ([#3476](https://github.com/objectstack-ai/objectui/issues/3476)) — because until [#7328](https://github.com/objectstack-ai/objectui/issues/7328) the only thing that ran it was the console's own `type-check` script, reached through the task runner, whose `type-check` task waits on `^build`. (Named in prose rather than as a code span on purpose: the pin below reads this cell as this job's gate list, so spelling that invocation out would credit the job with a command it does not run.) The cheapest compiler that reads the root Vitest config was therefore reachable only through the most expensive job here, and PR #7291 paid for it: a conditionally spread `dist` project whose literal `extends: true` widened to `boolean` degraded the whole `projects` array to `never[]`, every gate its author ran was green, and CI reported three errors, two of them at `../../vitest.config.mts`. It sits in the cheap half beside `pnpm type-check:scripts` for the same measured reason — nothing in its program imports an `@object-ui/*` package, so it needs the install and nothing built. `pnpm type-check:vitest-setup` (`tsconfig.vitest-setup.json`) closes the same gap for the four repo-root `vitest.setup.*` files, uncovered until [#3515](https://github.com/objectstack-ai/objectui/issues/3515); it runs *last*, after `pnpm type-check`, because `vitest.setup.dom.tsx` side-effect-imports four `@object-ui/*` packages and resolves them through the declarations that turbo's `^build` produces. | Every run; on a PR the steps short-circuit when only ignored paths changed | -| `test` | Test (shard N/4) | When a pull request changed nothing outside the exclusion list, the decision step runs `scripts/markdown-test-inputs.mjs` before anything else, and the job runs in full when the answer is yes ([#8861](https://github.com/objectstack-ai/objectui/issues/8861)). The exclusions drop every markdown path, and a markdown document can be a TEST'S INPUT: [#8857](https://github.com/objectstack-ai/objectui/issues/8857) changed one package README, this job reported success in ten seconds having run nothing, and the merge-queue build then failed the same shard in 907 seconds and dequeued it. That script carries the derived class — which documents a test reads, and which test reads each — and audits itself against the tree. It is this job's stage only, because this is the job that runs those tests. [#9096](https://github.com/objectstack-ai/objectui/issues/9096) then added `scripts` to that scan, because the gate tests under `scripts/__tests__` read this repository's documentation as data and run in these same shards — measured by running all 45 of them under an fs trace, **every** tracked markdown file in the tree is opened by at least one of them. ⇒ for this job the markdown half of the exclusion list is now inert: **any markdown-only pull request runs the shards**. Priced over the 513 first-parent commits available then: 36 reached this stage, 21 already ran, and the widening moves 14 of the remaining 15 from skip to run. The rest of the exclusion list still pays — a non-markdown change under `content/**`, `docs/**` or `apps/site/**` skips as before. Then `pnpm test --shard=N/4` across a 4-runner matrix with `fail-fast: false`, so every shard reports its own failures. No coverage instrumentation — v8 adds 40–100% overhead. Then, **on shard 1 only**, `pnpm test:dist` — the built-artifact lane ([#7183](https://github.com/objectstack-ai/objectui/issues/7183)). It delegates to a turbo task scoped to the one package that holds built-artifact pins; that task depends on the package's OWN build (`dependsOn: ["build"]`, not `^build`), so the bundle exists before the pins read it, and then runs the `dist` vitest project, whose pins import a package's BUILT bundle instead of its `src` — a claim the source-aliased suite above is structurally unable to make, since the root config aliases every workspace package to `src`. It is deliberately not sharded and not repeated on the other three runners: the lane is a handful of files, and running it on all four would pay for the same build four times. | Pull requests and merge-queue builds (everything but `push`); steps short-circuit on a PR that changed only ignored paths | +| `test` | Test (shard N/4) | When a pull request changed nothing outside the exclusion list, the decision step runs `scripts/markdown-test-inputs.mjs` before anything else, and the job runs in full when the answer is yes ([#8861](https://github.com/objectstack-ai/objectui/issues/8861)). The exclusions drop every markdown path, and a markdown document can be a TEST'S INPUT: [#8857](https://github.com/objectstack-ai/objectui/issues/8857) changed one package README, this job reported success in ten seconds having run nothing, and the merge-queue build then failed the same shard in 907 seconds and dequeued it. That script carries the derived class — which documents a test reads, and which test reads each — and audits itself against the tree. It is this job's stage only, because this is the job that runs those tests. [#9096](https://github.com/objectstack-ai/objectui/issues/9096) then added `scripts` to that scan, because the gate tests under `scripts/__tests__` read this repository's documentation as data and run in these same shards — measured by running all 46 of them under an fs trace, **every** tracked markdown file in the tree is opened by at least one of them. ⇒ for this job the markdown half of the exclusion list is now inert: **any markdown-only pull request runs the shards**. Priced over the 513 first-parent commits available then: 36 reached this stage, 21 already ran, and the widening moves 14 of the remaining 15 from skip to run. The rest of the exclusion list still pays — a non-markdown change under `content/**`, `docs/**` or `apps/site/**` skips as before. Then `pnpm test --shard=N/4` across a 4-runner matrix with `fail-fast: false`, so every shard reports its own failures. No coverage instrumentation — v8 adds 40–100% overhead. Then, **on shard 1 only**, `pnpm test:dist` — the built-artifact lane ([#7183](https://github.com/objectstack-ai/objectui/issues/7183)). It delegates to a turbo task scoped to the one package that holds built-artifact pins; that task depends on the package's OWN build (`dependsOn: ["build"]`, not `^build`), so the bundle exists before the pins read it, and then runs the `dist` vitest project, whose pins import a package's BUILT bundle instead of its `src` — a claim the source-aliased suite above is structurally unable to make, since the root config aliases every workspace package to `src`. It is deliberately not sharded and not repeated on the other three runners: the lane is a handful of files, and running it on all four would pay for the same build four times. | Pull requests and merge-queue builds (everything but `push`); steps short-circuit on a PR that changed only ignored paths | | `test-coverage` | Test (coverage shard N/4) | `pnpm test:coverage --reporter=blob --reporter=default --reporter=github-actions --shard=N/4` across a 4-runner matrix with `fail-fast: false`. Each shard writes `.vitest-reports/blob-N-4.json` — raw coverage and test results in one file — and uploads it as an artifact even when the shard is red (vitest deletes `coverage/` on a red run unless `coverage.reportOnFailure` is set, [#5402](https://github.com/objectstack-ai/objectui/issues/5402)). The two reporters after the blob are what keep a red shard *readable*: a CLI `--reporter` replaces the default reporter set rather than adding to it, so `--reporter=blob` on its own ended a failing shard's log at `blob report written to …` with no test name and no assertion text, leaving the download-only artifact as the only copy of the failure ([#9177](https://github.com/objectstack-ai/objectui/issues/9177)); `--reporter=default` restores the log and `--reporter=github-actions` restores the per-test `::error` annotation. The flags are pinned by `scripts/__tests__/coverage-shard-reporter-readability.test.ts`. The configured coverage thresholds are neutralised on the shard legs, because a quarter of the suite judged against a whole-suite threshold is not a defect signal; they are enforced once, on the merged report, by the job below ([#5403](https://github.com/objectstack-ai/objectui/issues/5403)). | **Push only** | | `coverage-report` | Test (coverage) | Downloads the four blob reports, refuses to continue unless all four arrived, merges them with `pnpm test:coverage --merge-reports` into one complete report — which is where the configured coverage thresholds are enforced, over the whole merged map, the shard legs having overridden them to zero — and publishes that report as the `coverage-report` artifact (kept 7 days, the same as the blobs it is derived from). Its last step runs on every path and states the outcome: the job is **red, with an error annotation**, whenever the gate did not run for the commit — before [#5403](https://github.com/objectstack-ai/objectui/issues/5403) the final step carried the implicit `success()` and was silently skipped by 311 of 373 coverage jobs, which is how four days of a 100%-failing coverage job went unnoticed. A breach of the thresholds is reported *separately* from a lane that never delivered, because the two call for opposite actions. ⛔ It never merges a report from fewer than four shards: a wrong coverage number is worse than a missing one. The Codecov upload this job used to carry was retired by [#5436](https://github.com/objectstack-ai/objectui/issues/5436) — `CODECOV_TOKEN` was never set, so it failed on every push; the trend dashboard and PR coverage comments are gone with it, the gate is not. | **Push only** | | `e2e` | Build & E2E | Builds the console with `vite build` (`VITE_BASE_PATH=/console/`), verifies the artifact, then `pnpm test:e2e --project=chromium`. On failure it uploads `test-results/` — the screenshots, traces and `error-context.md` Playwright writes for failing specs — as the `e2e-failure-artifacts` upload; the `github` reporter this lane runs on CI writes annotations, so there is no HTML report in it. | Every run; on a PR the steps short-circuit when only ignored paths changed | diff --git a/scripts/markdown-test-inputs.mjs b/scripts/markdown-test-inputs.mjs index 1e219190ac..c1e44783bd 100644 --- a/scripts/markdown-test-inputs.mjs +++ b/scripts/markdown-test-inputs.mjs @@ -80,10 +80,10 @@ * * ### The measurement, and why the population count is the wrong unit * - * Every one of the 45 candidate files under `scripts/__tests__` was run under + * Every one of the 46 candidate files under `scripts/__tests__` was run under * an `fs` trace and the markdown it opened was recorded. The union is not * "nearly every markdown document in the tree" -- it is EVERY one of them: - * 1734 of 1734 tracked `.md` / `.mdx` files are opened by at least one of these + * 1805 of 1805 tracked `.md` / `.mdx` files are opened by at least one of these * tests. One file does it alone: `dollar-dialect-alias-census.test.ts` scans * every tracked path through a helper module, which is limit 2 below in its * purest form -- the scanner cannot see that read at all, and it is the read @@ -118,7 +118,7 @@ * derived, adjudicated, self-auditing list and never a glob. Its EFFECT is not: * the list's answer is now `true` for every markdown path. That tension is real * and is left visible on purpose rather than argued away. The two numbers a - * reviewer needs to reverse this are above: 1734 of 1734 documents, 14 of 513 + * reviewer needs to reverse this are above: 1805 of 1805 documents, 14 of 513 * merges. * * ### Where the trees come from @@ -694,6 +694,15 @@ export const ADJUDICATED = new Map([ walker: 'markdown-tree', }, ], + // Reads no markdown: its `README.md` literal is sample input to a pure path-matcher + // assertion -- the suite is a census OF corpus walkers and keeps its own inputs literal. + [ + 'scripts/__tests__/coverage-red-cause-census.test.ts', + { + reads: [], + notRead: ['README.md'], + }, + ], // Reads no markdown: every document literal is fixture text handed to pure functions. [ 'scripts/__tests__/cross-file-line-citation-census.test.ts',