diff --git a/.changeset/18133-liveness-governance-denominator.md b/.changeset/18133-liveness-governance-denominator.md new file mode 100644 index 00000000000..86801d73049 --- /dev/null +++ b/.changeset/18133-liveness-governance-denominator.md @@ -0,0 +1,37 @@ +--- +'@objectstack/spec': patch +--- + +The liveness ledger's published README no longer claims the metadata-type registry is "exactly the set of authorable metadata types" — the governance denominator is now that set, and every run prints it + +`check-liveness.mts` built its coverage denominator from +`listMetadataTypeSchemaTypes()` under a comment stating that function returns +"exactly the set of *authorable* metadata types", and the ledger README carried +the same sentence. It is false in a specific, load-bearing way: that function +deliberately does not enumerate `UNREGISTERED_KIND_SCHEMAS` — enrolling those +entries there "would claim a status this change is careful not to grant" — while +the kinds bound in that map are authored on every boot through their stack +collections (`connectors:`, `sharingRules:`, `analyticsCubes:`, `webhooks:`) and +on every write through `PUT /api/v1/meta/:type/:name`, whose `resolveOverlaySchema` +resolves them through `getMetadataTypeSchema()`. + +So `connector`, `sharing_rule` and `analytics_cube` sat in **neither** `GOVERNED` +**nor** `PENDING_GOVERNANCE`, and a type in no bucket produces no row in any of +this gate's lists. The blindness was therefore invisible in the gate's own +output: `ungoverned: []` read exactly the same whether the gate had looked and +found nothing or had never looked at all. + +The denominator is now `authorableTypes()` — the registered kinds UNION +`listUnregisteredKindSchemaTypes()`, the enumeration helper that exists so a check +can read that map and which grants nothing by listing a name. The registry itself +is untouched: no kind is registered, no enum grows, no create seed is demanded and +no accept set moves, and the same split already landed one gate over as +`reachabilityRootTypes()` in `build-schemas.ts`. The three newly visible types are +recorded as declared debts with a reason and an issue number apiece, which is what +the ratchet asks for and what the README now says; the direction of travel is out +of that map and into `GOVERNED`. + +Every run also prints the denominator and its composition unconditionally. That +line used to appear only when `PENDING_GOVERNANCE` was non-empty, so the one state +worth reporting — "N authorable types looked at, none unaccounted for" — rendered +as nothing at all, which is the same silence an unseen type produces. diff --git a/packages/spec/liveness/README.md b/packages/spec/liveness/README.md index b2203fd9f71..d3691cad368 100644 --- a/packages/spec/liveness/README.md +++ b/packages/spec/liveness/README.md @@ -28,12 +28,26 @@ route decides the disposition — see `scripts/liveness/orphans.mts` and The gate reads `BUILTIN_METADATA_TYPE_SCHEMAS` (`packages/spec/src/kernel/metadata-type-schemas.ts`) via `listMetadataTypeSchemaTypes()` / `getMetadataTypeSchema()` — **the same registry the runtime `/api/v1/meta/types/:type` endpoint and the Studio metadata-admin forms use**, -i.e. exactly the set of *authorable* metadata types. It walks each type's Zod schema +i.e. the set of *registered* metadata KINDS. It walks each type's Zod schema directly (not `z.toJSONSchema`, which throws on `object`/`action`). This matters: the older gate read the generated `json-schema/` directory, which omits most top-level authorable types (object/field/flow/action/...) — so it was blind to the -core surface. The registry is complete. +core surface. The registry is complete *as a registry*. + +⛔ **Registered is not the same set as authorable, and this page used to say it was** — +"i.e. exactly the set of *authorable* metadata types" was the sentence #17356 measured +false for the reachability gate and #18133 for this one. `listMetadataTypeSchemaTypes()` +deliberately does not enumerate `UNREGISTERED_KIND_SCHEMAS` (#6245: enrolling those +entries there "would claim a status this change is careful not to grant"), yet the kinds +bound in that map are authored on every boot through their stack collections +(`connectors:` / `sharingRules:` / `analyticsCubes:` / `webhooks:`) and on every write +through `PUT /api/v1/meta/:type/:name`. So the WALK is registry-rooted, as above, while +the **governance denominator** — whom a ledger must exist for — is the registered kinds +UNION `listUnregisteredKindSchemaTypes()` (#6931), computed by `authorableTypes()` in +`check-liveness.mts`. Every run prints that denominator and how it is composed, because +a type in no bucket produces no row anywhere: without the printed count, "nothing +ungoverned here" and "never looked" are the same output. **Spec-only exception (`SPEC_ONLY_SCHEMAS`).** A type can be authorable yet deliberately *not* registered — `webhook` is the case: its schema is authored on a Stack/connector but @@ -929,12 +943,21 @@ misleading entry carries `authorWarn` so authors hear about it at compile time (governed types with warn entries must also be registered in the CLI lint's `TYPE_COLLECTIONS` — see lint-liveness-properties.ts). -**Coverage is complete as of #4488**: every type in the metadata-type registry -is governed, and `PENDING_GOVERNANCE` in `check-liveness.mts` is empty. The map -itself stays, because the ratchet is the point — registering a new type without -a ledger fails CI with instructions to govern it or record the debt (reason + -issue number). The paragraph that used to sit here, listing nine ungoverned -types as prose, is precisely how the gap survived for a year: prose cannot fail -a build. Now the gate compares `GOVERNED` against the registry in both -directions (an ungoverned registered type fails; so does a stale pending row -whose debt is already paid). +**Every registered type has been governed since #4488**, which emptied +`PENDING_GOVERNANCE` of all nine debts the map opened with. The map itself stays, +because the ratchet is the point — registering a new type without a ledger fails +CI with instructions to govern it or record the debt (reason + issue number). The +paragraph that used to sit here, listing nine ungoverned types as prose, is +precisely how the gap survived for a year: prose cannot fail a build. Now the gate +compares `GOVERNED` against the denominator in both directions (an ungoverned +authorable type fails; so does a stale pending row whose debt is already paid). + +⚠️ **The map is no longer empty, and that is #18133's finding rather than a +regression.** Widening the denominator from the registered kinds to the authorable +set (see the ⛔ note under *Source of truth* above) made three types visible that +had been in **neither** `GOVERNED` **nor** `PENDING_GOVERNANCE` — `connector`, +`sharing_rule` and `analytics_cube` — and therefore produced no row in any of the +gate's lists while the report read complete. They are now declared debts with a +reason and an issue number apiece, which is the state this ratchet exists to +produce; the direction of travel is out of that map and into `GOVERNED`, exactly +as it was for the nine. ⛔ Their presence is not a licence to leave them there. diff --git a/packages/spec/scripts/liveness/check-liveness.mts b/packages/spec/scripts/liveness/check-liveness.mts index e634fab18e1..b3c8aded6e3 100644 --- a/packages/spec/scripts/liveness/check-liveness.mts +++ b/packages/spec/scripts/liveness/check-liveness.mts @@ -6,13 +6,29 @@ // consumer is a silent no-op (worst case, a *security* no-op — false compliance). // // SOURCE OF TRUTH: the metadata-type registry (BUILTIN_METADATA_TYPE_SCHEMAS via -// listMetadataTypeSchemaTypes/getMetadataTypeSchema). This is the same registry the +// listMetadataTypeSchemaTypes/getMetadataTypeSchema). That is the same registry the // runtime `/api/v1/meta/types/:type` endpoint and the Studio metadata-admin forms -// use — i.e. exactly the set of *authorable* metadata types. (We walk the Zod schema +// use — i.e. the set of REGISTERED metadata KINDS. (We walk the Zod schema // directly rather than z.toJSONSchema, because a couple of schemas — object, action — // throw in the JSON-schema converter, which is precisely why the old json-schema/-based // gate was blind to them.) // +// ⛔ REGISTERED IS NOT THE SAME SET AS AUTHORABLE, and this comment used to say it +// was — "i.e. exactly the set of *authorable* metadata types", the sentence #17356 +// measured false one gate over and #18133 filed here. `listMetadataTypeSchemaTypes()` +// deliberately does not enumerate `UNREGISTERED_KIND_SCHEMAS` (#6245: enrolling those +// entries there "would claim a status this change is careful not to grant"), yet the +// four kinds bound in that map are authored through real doors — `stack.connectors[]` +// / `stack.sharingRules[]` / `stack.analyticsCubes[]` / `stack.webhooks[]` on every +// boot, and `PUT /api/v1/meta/:type/:name`, whose `resolveOverlaySchema` resolves them +// through `getMetadataTypeSchema()`'s third fallback. So the two questions are split +// here the way `reachabilityRootTypes()` in build-schemas.ts splits them: the WALK +// resolves a governed type's schema through the registry (plus SPEC_ONLY_SCHEMAS), +// while the governance DENOMINATOR — "whom must a ledger exist for?" — is +// `authorableTypes()` below, the registered kinds UNION the unregistered-kind stack +// collections. See that function for why the union lives in this gate and not in the +// registry. +// // Governed types must declare every authorable property's liveness status with // evidence in packages/spec/liveness/.json, or CI fails (the ratchet — no new // undeclared surface). Property granularity follows the LEDGER: a container property @@ -153,7 +169,11 @@ process.env.OS_EAGER_SCHEMAS = '1'; import { readFileSync, existsSync, readdirSync } from 'node:fs'; import { fileURLToPath } from 'node:url'; import { dirname, join, resolve } from 'node:path'; -import { getMetadataTypeSchema, listMetadataTypeSchemaTypes } from '../../src/kernel/metadata-type-schemas'; +import { + getMetadataTypeSchema, + listMetadataTypeSchemaTypes, + listUnregisteredKindSchemaTypes, +} from '../../src/kernel/metadata-type-schemas'; import { WebhookSchema } from '../../src/automation/webhook.zod'; import { QuerySchema } from '../../src/data/query.zod'; import { ValidationRuleSchema } from '../../src/data/validation.zod'; @@ -245,7 +265,7 @@ const ledgerRoot = ledgerRootArg // (`query` is not a metadata type — see SPEC_ONLY_SCHEMAS below.) const GOVERNED = ['object', 'field', 'flow', 'action', 'hook', 'permission', 'position', 'agent', 'tool', 'skill', 'dataset', 'page', 'view', 'report', 'dashboard', 'webhook', 'query', 'datasource', 'app', 'book', 'doc', 'email_template', 'job', 'mapping', 'seed', 'translation', 'validation', 'api', 'capability', 'qa', 'manifest', 'crud_endpoints', 'metadata_endpoints', 'batch_endpoints', 'route_generation', 'realtime_subscription']; -// Registered metadata types that are NOT yet governed — the coverage ratchet. +// Authorable metadata types that are NOT yet governed — the coverage ratchet. // // WHY THIS EXISTS. `GOVERNED` was hand-maintained and nothing compared it // against the registry it claims to cover. So a type could be registered — @@ -258,8 +278,9 @@ const GOVERNED = ['object', 'field', 'flow', 'action', 'hook', 'permission', 'po // type was not on the list — and a list that governs 15 of 25 registered types // while reporting itself complete is worse than one that admits the gap. // -// So the list is now answerable to the registry: every registered type must be -// governed OR appear here with a reason. Registering a type and forgetting the +// So the list is now answerable to the denominator: every AUTHORABLE type must +// be governed OR appear here with a reason. Registering a type — or binding a +// new unregistered kind into `UNREGISTERED_KIND_SCHEMAS` — and forgetting the // ledger fails CI with the entry to write. // // This is a RATCHET, not an allowlist to grow. An entry is a debt with an issue @@ -267,13 +288,37 @@ const GOVERNED = ['object', 'field', 'flow', 'action', 'hook', 'permission', 'po // add one to silence the gate on a type you just registered — that is exactly // the failure this map exists to make visible, and an entry with no issue is // indistinguishable from never having looked. +// +// [#18133] The map went from EMPTY to three rows without a single type changing +// status, and that is the finding, not a regression: #4488 paid off all nine +// debts the map opened with (app, book, doc, email_template, job, mapping, seed, +// translation, validation) and the map then read empty under a denominator that +// could not see the unregistered kinds at all. `connector`, `sharing_rule` and +// `analytics_cube` were in neither `GOVERNED` nor here, so they produced no row +// anywhere and the report read complete — a reader could not tell "nothing +// ungoverned here" from "never looked". Widening the denominator (see +// `authorableTypes()`) is what converts that silence into three declared debts. +// ⛔ These three rows are NOT a licence to leave them: the direction of travel +// is out of this map, exactly as it was for the nine. const PENDING_GOVERNANCE: Record = { - // EMPTY since #4488 paid off all nine debts the map opened with (app, book, - // doc, email_template, job, mapping, seed, translation, validation) — every - // registered type is governed. The map stays because the ratchet is the - // point, not the entries: registering a NEW type without a ledger fails CI - // with instructions to either govern it or record the debt here (reason + - // issue number). Do not add an entry just to silence the gate. + connector: + 'Authored via `stack.connectors[]` and `PUT /api/v1/meta/connector/:name` ' + + '(#6245 bound DeclarativeConnectorEntrySchema for exactly those two doors), and ' + + 'enrolled into this gate\'s denominator by #18133 — before which it was invisible ' + + 'to the ratchet rather than covered by it. No ledger walked it yet: seed ' + + 'packages/spec/liveness/connector.json (#18133).', + sharing_rule: + 'Authored via `stack.sharingRules[]` and `PUT /api/v1/meta/sharing_rule/:name` ' + + '(#6245). Security-shaped surface — a sharing rule grants read scope — which is ' + + 'the class the #4410/#4465/#4481 datasource keys came from, and enrolling it here ' + + 'is #18133. No ledger walked it yet: seed packages/spec/liveness/sharing_rule.json ' + + '(#18133).', + analytics_cube: + 'Authored via `stack.analyticsCubes[]` and `PUT /api/v1/meta/analytics_cube/:name` ' + + '(#10194). Whether that authoring is live end-to-end is its own measurement ' + + '(#10238) and this row does not prejudge it — the debt recorded here is only that ' + + 'no ledger asks who reads the cube\'s properties. Enrolled by #18133; seed ' + + 'packages/spec/liveness/analytics_cube.json.', }; // Spec-only override: governed types whose canonical schema is NOT (yet) in the @@ -638,8 +683,11 @@ const report: any = { proofMissing: [] as string[], // a bound high-risk `live` entry with no proof at all orphanProofs: [] as string[], // a dogfood `@proof:` tag not registered in proof-registry.mts orphanEntries: [] as string[], // a ledger row whose property is gone from the schema (the reverse direction) - ungoverned: [] as string[], // a REGISTERED metadata type absent from both GOVERNED and PENDING_GOVERNANCE - stalePending: [] as string[], // a PENDING_GOVERNANCE row for a type that is now governed / no longer registered + authorable: [] as string[], // the governance DENOMINATOR itself — printed and emitted so "never looked" cannot pass for "nothing to report" (#18133) + authorableRegistered: 0, // how many of it are registered KINDS (listMetadataTypeSchemaTypes) + authorableUnregisteredKinds: [] as string[], // …and which are unregistered-kind stack collections (#6245/#6931) + ungoverned: [] as string[], // an AUTHORABLE metadata type absent from both GOVERNED and PENDING_GOVERNANCE + stalePending: [] as string[], // a PENDING_GOVERNANCE row for a type that is now governed / no longer authorable undrilledNew: [] as string[], // a container riding on inheritance that the baseline does not record (see drill.mts) undrilledStale: [] as string[], // a baseline row whose container now drills / is no longer a container undrilled: [] as Array<{ key: string; childKeys: string[] }>, // the recorded inheritance population — a worklist, not a failure @@ -1188,20 +1236,80 @@ report.verification = buildVerificationReport(verificationEntries, { staleDays } const showProducerGap = args.includes('--producer-gap'); report.producers = buildProducerReport(producerEntries); -// ── coverage: is every REGISTERED metadata type accounted for? ── +// ── coverage: is every AUTHORABLE metadata type accounted for? ── // The gate's own blind spot until #4487. Everything above asks "is every // property of a governed type classified?" — nothing asked "is every authorable // type governed?", so a type absent from GOVERNED was never in the denominator // and its silence read as success. +// +// #4487 closed that for REGISTERED types and left the same hole one set over, +// which is #18133: the denominator was `listMetadataTypeSchemaTypes()` under a +// comment claiming it was "exactly the set of authorable metadata types". It is +// not, and the difference is not a rounding error — it is every entry of +// `UNREGISTERED_KIND_SCHEMAS`. A type in neither `GOVERNED` nor +// `PENDING_GOVERNANCE` produces no row in ANY of this gate's lists, so the +// blindness was invisible in the gate's own output: `connector`, +// `sharing_rule` and `analytics_cube` read exactly like "nothing to report". + +/** + * The governance denominator: every type somebody AUTHORS a document against. + * + * ⛔ Deliberately NOT `listMetadataTypeSchemaTypes()`, and ⛔ not to be + * "simplified" back into it. The two answer different questions, and #6245 drew + * that line on purpose: + * + * - "is this a REGISTERED metadata KIND?" — `listMetadataTypeSchemaTypes()`. + * Membership there carries KIND obligations (a `MetadataTypeSchema` enum + * member, a `DEFAULT_METADATA_TYPE_REGISTRY` entry, a create seed, a place + * in the #4001 campaign count), which is why enrolling the unregistered + * kinds there "would claim a status this change is careful not to grant". + * That function answers its own question correctly and this gate does not + * touch it — the repair for #18133 is HERE, in the consumer that was asking + * the wrong question, not in the registry that was answering the right one. + * - "must a ledger exist for this type?" — the only question a governance + * denominator asks, and for THAT question an unregistered kind is an + * authored document like any other: `stack.connectors[]` parses one on every + * boot and `PUT /api/v1/meta/connector/:name` parses one per write, both + * against `UNREGISTERED_KIND_SCHEMAS['connector']`. + * + * `listUnregisteredKindSchemaTypes()` exists (#6931) so a check can ENUMERATE + * that map and for nothing else, and its own docblock is explicit that "being + * listed by this function grants NOTHING" — which is exactly why reading it here + * costs #6245 nothing: no kind is registered, no enum grows, no create seed is + * demanded, no accept set moves. The gate gains a row to fill in, that is all. + * + * SAME SHAPE, SECOND GATE. `reachabilityRootTypes()` in scripts/build-schemas.ts + * is this union under another name, landed for #17356 against the identical + * false sentence. The two are not yet ONE spelling because that file is held by + * in-flight work; folding them together is a follow-up, and until then this + * docblock and that one are each other's cross-reference. + * + * NOT in the denominator, on purpose: the `SPEC_ONLY_SCHEMAS` types that are not + * metadata types at all (`query`, `qa`, `manifest`, the four `RestServerConfig` + * sub-objects, `realtime_subscription`, `validation`). Those are governed BY the + * override — the override IS their governance — so asking "is it governed?" of + * them is a question the override already answered. They are counted separately + * in the coverage line so the two populations never read as one number. + */ +function authorableTypes(): string[] { + const types = new Set(listMetadataTypeSchemaTypes()); + for (const kind of listUnregisteredKindSchemaTypes()) types.add(kind); + return [...types].sort(); +} + const governedSet = new Set(GOVERNED); -report.ungoverned = listMetadataTypeSchemaTypes() +const denominator = authorableTypes(); +report.authorable = denominator; +report.authorableRegistered = listMetadataTypeSchemaTypes().length; +report.authorableUnregisteredKinds = listUnregisteredKindSchemaTypes(); +report.ungoverned = denominator .filter((t) => !governedSet.has(t) && !(t in PENDING_GOVERNANCE)) .sort(); // A PENDING_GOVERNANCE row for a type that is now governed (or no longer -// registered) is the same rot as an orphan ledger row: it claims a debt that +// authorable) is the same rot as an orphan ledger row: it claims a debt that // does not exist, and it makes the map's length a lie about how much is left. report.stalePending = Object.keys(PENDING_GOVERNANCE) - .filter((t) => governedSet.has(t) || !listMetadataTypeSchemaTypes().includes(t)) + .filter((t) => governedSet.has(t) || !denominator.includes(t)) .sort(); const totalUnclassified = report.unclassified.length; @@ -1506,13 +1614,15 @@ if (asJson) { ); } if (report.ungoverned.length) { - console.log(`\n✗ ${report.ungoverned.length} REGISTERED metadata type(s) governed by nothing:`); + console.log(`\n✗ ${report.ungoverned.length} AUTHORABLE metadata type(s) governed by nothing:`); report.ungoverned.forEach((t: string) => console.log(` ${t}`)); console.log( - '\n These are authorable — `/api/v1/meta/types/:type` serves them and Studio edits\n' + - ' them — but no ledger asks who reads their properties, so an inert key on one is\n' + - ' invisible to CI. `datasource` sat here for its whole life and cost six inert keys\n' + - ' found by hand, two of them security-shaped (#4410, #4465, #4481).\n\n' + + '\n These are authorable — a REGISTERED kind is served by `/api/v1/meta/types/:type`\n' + + ' and edited in Studio; an UNREGISTERED kind (#6245) is authored through its stack\n' + + ' collection and `PUT /api/v1/meta/:type/:name` — but no ledger asks who reads their\n' + + ' properties, so an inert key on one is invisible to CI. `datasource` sat here for its\n' + + ' whole life and cost six inert keys found by hand, two of them security-shaped\n' + + ' (#4410, #4465, #4481).\n\n' + ' Either govern the type (add it to GOVERNED and seed packages/spec/liveness/.json\n' + ' — see the seeding aid: `tsx check-liveness.mts --dump `), or record the debt in\n' + " PENDING_GOVERNANCE with a reason AND an issue number. Do not pick the second option\n" + @@ -1712,13 +1822,30 @@ if (asJson) { } else if (report.undrilled.length) { console.log(' run with --undrilled for the worklist.'); } + // The DENOMINATOR, printed unconditionally (#18133). It used to print only when + // `PENDING_GOVERNANCE` was non-empty, which made the one state worth reporting — + // "this gate looked at N types and none of them is unaccounted for" — render as + // nothing at all, the same silence a type in no bucket produces. Now the run + // always says what it counted and how the count is composed, so a reader can + // tell "nothing ungoverned here" from "never looked" without reading the source. const pendingCount = Object.keys(PENDING_GOVERNANCE).length; - if (pendingCount) { - console.log( - `\ncoverage: ${GOVERNED.length} type(s) governed, ${pendingCount} registered type(s) awaiting a ledger ` + - `(${Object.keys(PENDING_GOVERNANCE).sort().join(', ')}) — a worklist, not a merge gate.`, - ); - } + const unregisteredKinds = report.authorableUnregisteredKinds as string[]; + const inDenominator = (t: string) => report.authorable.includes(t); + const governedInDenominator = GOVERNED.filter(inDenominator).length; + const governedOutside = GOVERNED.length - governedInDenominator; + console.log( + `\ngovernance denominator: ${report.authorable.length} authorable type(s) — ` + + `${report.authorableRegistered} registered kind(s) + ${unregisteredKinds.length} ` + + `unregistered-kind stack collection(s) (${unregisteredKinds.join(', ')}); ` + + `${governedInDenominator} governed, ${pendingCount} awaiting a ledger` + + (pendingCount ? ` (${Object.keys(PENDING_GOVERNANCE).sort().join(', ')})` : '') + + ' — a worklist, not a merge gate.', + ); + console.log( + ` (+ ${governedOutside} type(s) governed from OUTSIDE the denominator via SPEC_ONLY_SCHEMAS — ` + + 'not metadata types, so the override IS their governance; ' + + `${GOVERNED.length} governed in total.)`, + ); if (!failed) { // Deliberately qualified. The old wording — "all governed-type properties // are classified" — was the instrument's own false claim: it counted a @@ -1730,7 +1857,8 @@ if (asJson) { // a success line that overstates its own reach is the defect it qualifies. console.log( '\n✓ every governed-type property, at every depth the ledger drills, is classified, every ' + - 'registered type is governed or explicitly pending, no ledger row outlives its property, ' + + 'authorable type — registered kind or unregistered-kind stack collection — is governed or ' + + 'explicitly pending, no ledger row outlives its property, ' + `every container inheritance is declared, every ${EVIDENCE_SCANNED_LABEL} entry's repo-local evidence path ` + 'resolves, every `path:NNN` citation names a line that file actually has, every ' + '`path#symbol` anchor names a symbol its file contains, and every cited ' + diff --git a/packages/spec/scripts/liveness/check-liveness.test.ts b/packages/spec/scripts/liveness/check-liveness.test.ts index e4a52508db8..921df73e9d7 100644 --- a/packages/spec/scripts/liveness/check-liveness.test.ts +++ b/packages/spec/scripts/liveness/check-liveness.test.ts @@ -25,6 +25,12 @@ import { cpSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:f import { tmpdir } from 'node:os'; import path from 'node:path'; import { fileURLToPath } from 'node:url'; +// The registry itself, so the denominator block at the bottom of this file can +// hold the gate's output answerable to it rather than to a copied list (#18133). +import { + listMetadataTypeSchemaTypes, + listUnregisteredKindSchemaTypes, +} from '../../src/kernel/metadata-type-schemas'; const HERE = path.dirname(fileURLToPath(import.meta.url)); const SPEC = path.resolve(HERE, '../..'); @@ -1120,3 +1126,105 @@ describe('check:liveness — the drill recurses past one level (#17424)', () => expect(status, output).toBe(1); }); }); + +// The DENOMINATOR the coverage ratchet divides by (#18133). +// +// Everything in the block above asks whether the gate judges what it walks +// correctly. This asks the prior question — WHOM does it walk for? — and it is +// the one question a gate cannot ask about itself, because the failure mode has +// no output: a type in neither `GOVERNED` nor `PENDING_GOVERNANCE` produces no +// row in any bucket, so `ungoverned: []` reads identically whether the gate +// looked and found nothing or never looked at all. +// +// WHAT WAS WRONG. The denominator was `listMetadataTypeSchemaTypes()` under a +// comment claiming it was "exactly the set of authorable metadata types". That +// function deliberately does NOT enumerate `UNREGISTERED_KIND_SCHEMAS` (#6245 — +// enrolling those entries there "would claim a status this change is careful +// not to grant"), while the four kinds bound in that map are authored on every +// boot through their stack collections and on every write through +// `PUT /api/v1/meta/:type/:name`. So `connector`, `sharing_rule` and +// `analytics_cube` were structurally unnameable by `report.ungoverned` — the +// same sentence #17356 measured false for the reachability gate, one gate over. +// +// WHY THESE ASSERT AGAINST THE LIVE REGISTRY rather than against a literal list: +// a hard-coded expectation would pass unchanged if the gate stopped reading the +// registry at all, which is the regression class this whole block exists for. +// The registry is imported here and the gate is spawned; the two have to agree. +describe('check:liveness — the governance denominator is the AUTHORABLE set (#18133)', () => { + function jsonReport(extraArgs: readonly string[] = []): any { + const { output } = runGate(undefined, ['--json', ...extraArgs]); + const start = output.indexOf('{'); + expect(start, output).toBeGreaterThanOrEqual(0); + return JSON.parse(output.slice(start)); + } + + // The control for every assertion below. Without it, "the denominator omits + // nothing" is also satisfied by a registry that enumerates nothing. + it('has a non-empty registry on BOTH sides of the union', () => { + expect(listMetadataTypeSchemaTypes().length).toBeGreaterThan(20); + expect(listUnregisteredKindSchemaTypes().length).toBeGreaterThan(0); + // The two sets are disjoint — that disjointness IS #6245, and it is why the + // union is not a no-op. If this ever fails, the fix below has become moot + // and this whole block needs re-reading, not re-pinning. + const registered = new Set(listMetadataTypeSchemaTypes()); + expect(listUnregisteredKindSchemaTypes().filter((t) => registered.has(t))).toEqual([]); + }); + + it('counts every unregistered kind, which the registered set alone cannot', () => { + const report = jsonReport(); + for (const kind of listUnregisteredKindSchemaTypes()) { + expect(report.authorable, `'${kind}' is authored through its stack collection and through ` + + 'PUT /api/v1/meta/:type/:name, so a governance denominator that omits it cannot report ' + + 'on it — which is exactly the state #18133 found').toContain(kind); + } + // …and the denominator is STRICTLY larger than the registered set, which is + // the assertion that goes red the moment somebody "simplifies" the union + // back into `listMetadataTypeSchemaTypes()`. + expect(report.authorable.length).toBeGreaterThan(listMetadataTypeSchemaTypes().length); + expect(report.authorable).toEqual( + [...new Set([...listMetadataTypeSchemaTypes(), ...listUnregisteredKindSchemaTypes()])].sort(), + ); + }); + + it('accounts for every member of it — governed or explicitly pending, never silent', () => { + const report = jsonReport(); + expect(report.ungoverned).toEqual([]); + // An empty `ungoverned` is only meaningful next to a denominator that could + // have populated it, so assert the population too — this is the pair the + // old output could not print. + expect(report.authorable.length).toBeGreaterThan(0); + // And no pending row claims a debt for a type the denominator does not hold: + // before the union landed, recording one of the unregistered kinds here would + // have been reported STALE rather than pending. + expect(report.stalePending).toEqual([]); + }); + + it('prints the denominator and its composition on EVERY run, green included', () => { + const { status, output } = runGate(); + expect(status, output).toBe(0); + const line = output.split('\n').find((l) => l.startsWith('governance denominator:')) ?? ''; + // The line used to print only when `PENDING_GOVERNANCE` was non-empty, so the + // one state worth reporting — "N types looked at, none unaccounted for" — + // rendered as nothing at all: the same silence an unseen type produces. + expect(line, output).not.toBe(''); + expect(line).toMatch(/^governance denominator: \d+ authorable type\(s\) — \d+ registered kind\(s\) \+ \d+ unregistered-kind stack collection\(s\)/); + for (const kind of listUnregisteredKindSchemaTypes()) expect(line).toContain(kind); + }); + + // #6245's guarantee, asserted from the gate that had the motive to break it. + // The repair for #18133 belongs in this gate's own denominator; enrolling the + // unregistered kinds in the registry instead would have granted them a KIND + // status (`MetadataTypeSchema` enum membership, a `DEFAULT_METADATA_TYPE_REGISTRY` + // entry, a create seed, a place in the #4001 campaign count) that #6245 and + // #2657's still-open B/C decision deliberately withhold. + it('reads the unregistered kinds WITHOUT registering them', () => { + const registered = listMetadataTypeSchemaTypes(); + for (const kind of listUnregisteredKindSchemaTypes()) { + expect(registered, `#6245: '${kind}' must not become a registered KIND just because a ` + + 'check needs to enumerate it — listUnregisteredKindSchemaTypes() (#6931) exists so ' + + 'that enumeration costs nothing').not.toContain(kind); + } + const src = readFileSync(GATE, 'utf8'); + expect(src).toContain('listUnregisteredKindSchemaTypes'); + }); +});