Skip to content

qa/spec: the ADR-0058 D7 expression-conformance ratchet is blind to a roster schema used as a union member or behind a local alias — 5 declaring positions unclassified while the test stays green #17630

Description

@os-bill

Found while producing the measured census for #15811 (that card is not addressed here — it stays open for the per-slot narrowing decisions). Filed by the os-dev seat, session session_01MkQhmuuJAVDjmeWNixwDDH. No assignee; domain:*, type and priority are triage's.

Measured on origin/main 0918c44.

The declared contract

packages/qa/dogfood/test/expression-conformance.ledger.ts states it in its own header:

The companion test (expression-conformance.test.ts) RE-DISCOVERS every expression-declaring field in packages/spec/src (plus the RLS using/check string predicates) and asserts each is covers-ed by exactly one row. A NEW expression surface that nobody classified — the #1887 class of "declared-but-unwired predicate" — breaks the build.

The header names ONE surviving limit: "discovery still cannot see a slot typed with a schema nobody registered (the hazard is structural, not spent)".

What is actually true

Discovery is a line regex anchored to the HEAD of the declaration (expression-conformance.test.ts):

const DECLARES_EXPRESSION = new RegExp(
  String.raw`^\s*([a-zA-Z_][a-zA-Z0-9_]*)\s*:\s*(?:${EXPRESSION_INPUT_SCHEMAS.join('|')})\b`,
);

So a roster schema mounted anywhere but at the head of field: is invisible — including when the roster name is literally on the line. Two mechanisms, only the second of which the header names:

  • (A) roster schema as a UNION MEMBERfield: z.union([ ..., ExpressionInputSchema ]). Unnamed in the header, and the worse of the two: a reader sees the roster name on the line and assumes discovery saw it.
  • (B) roster schema behind a LOCAL ALIAS constconst ActionConditionInputSchema = z.union([z.boolean(), ExpressionInputSchema]) at ui/action.zod.ts:825, then field: ActionConditionInputSchema. The header's named hazard, live today.

Measurement

Replaying the test's own discovery regex over packages/spec/src, subtracting it from an identity-matched scan of the roster names (negative lookbehind on identifier characters, so the Cron and Template siblings do not match the bare name), and excluding imports, prose and the schemas' own definition lines, leaves exactly four declaration lines mounting five declaring positions:

position line mechanism
system/metrics.zod.ts:ServiceLevelIndicatorSchema.successCriteria system/metrics.zod.ts:436 A — union member
system/tracing.zod.ts:TraceSamplingConfigSchema.composite[].condition system/tracing.zod.ts:349 A — union member
ui/component.zod.ts:RecordAlertProps.visible ui/component.zod.ts:1593 A — union member
ui/action.zod.ts:ActionSchema.visible ui/action.zod.ts:1306 (alias at :825) B — local alias
ui/action.zod.ts:ActionSchema.disabled ui/action.zod.ts:1321 (alias at :825) B — local alias

Reconciling the ledger against its own discovery, run mechanically at the same commit:

discovered positions    : 39
ledger cover keys       : 39
discovered NOT covered  : 0
covered NOT discovered  : 0

The ledger is perfectly consistent with what discovery can see, which is exactly why the ratchet is green over the five it cannot:

pnpm --filter @objectstack/dogfood exec vitest run --maxWorkers=2 test/expression-conformance.test.ts
  Test Files  1 passed (1)
       Tests  5 passed (5)

All five parse both non-evaluable envelopes today (safeParse at the mounted slot, never a reading of the schema source):

key                                                     good  ast-only  blank-source  blank-string
ui/action.zod.ts:ActionSchema.visible                   true  true      true          true
ui/action.zod.ts:ActionSchema.disabled                  true  true      true          true
ui/component.zod.ts:RecordAlertProps.visible            true  true      true          true
system/metrics.zod.ts:...successCriteria                true  true      true          true
system/tracing.zod.ts:...composite[].condition          true  true      true          true
CONTROL automation/flow.zod.ts:FlowEdgeSchema.condition true  false     false         false
CONTROL builtin-node-config.zod.ts:AssignmentExpr...    true  false     false         false

The two control rows are what make the trues a reading rather than a probe that could only ever have answered one way.

Why this is more than bookkeeping

Two of the five have zero consumers anywhere outside packages/spec/src — measured: the only non-spec hits for successCriteria, ServiceLevelIndicatorSchema and TraceSamplingConfigSchema are generated artifacts under packages/spec/, content/docs/references/**, and one skills/** prose mention. That is the ledger's unevaluated / PARSE-ONLY tier: a published, documented, author-facing predicate that nothing evaluates. The ratchet exists to surface exactly that class, and it did not.

A latent third vector, measured

shared/expression.zod.ts:398 declares export const PredicateInputSchema = ExpressionInputSchema;. It has zero slot users today (identity hits are the definition and the index.ts barrel re-export only), so nothing is hidden by it right now — but a slot typed with it would be invisible both to the roster regex and to any grep for the bare name.

What a fix would be

Not a bigger roster alone — a roster cannot enumerate inline unions. Directions, cheapest first:

  1. Drop the head anchor: discover on any line where a roster identifier appears by identity, attributing it to the nearest preceding field: and the enclosing top-level const. Catches mechanism A outright.
  2. Resolve aliases: a file-local const X = ...RosterSchema... registers X for the rest of that file. Catches mechanism B and PredicateInputSchema.
  3. Replace the text scan with a runtime walk of the exported schema objects plus a behavioural probe at each reachable slot (how the five above were found: parse a healthy envelope and a bad dialect at every slot, classify by the answers). Structurally immune to both mechanisms; costs a built packages/spec.

Whichever lands should restate the surviving limit honestly in the header, the way the current one does.

Back-links: #15811 (the census that surfaced this), #15500 (the key-collision half of the same ratchet), #15027 (the dialect-roster half), ADR-0058 D7.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions