Retire check-reference-carrier-shape; refuse an unreadable reference carrier at the reader

.changeset/18095-retire-reference-carrier-shape-gate.md

@@ -0,0 +1,26 @@
+---
+"@objectstack/spec": minor
+"@objectstack/lint": minor
+---
+
+A `reference` carrier that no reader can read is now **REFUSED** where it is read, instead of coming back as `undefined`. The source-level gate that guarded the same shape (`check:reference-carrier-shape`) is retired in the same change (#18095, executing a maintainer ruling).
+
+`FieldSchema.reference` is `z.string().optional()`, so `ObjectSchema.safeParse` already refuses an object- or array-valued carrier at the contract door with a located `invalid_type` issue. Measured on the pre-change tree:
+
+```
+ObjectSchema.safeParse({ fields: { invoice: { type: 'lookup',
+                                              reference: { object: 'shop_invoice' } } } })
+  -> success = false, issue invalid_type at path ["fields","invoice","reference"]
+control: the same object with reference: 'shop_invoice'
+  -> success = true      (so the refusal is about the carrier's SHAPE)
+```
+
+What was missing was the other door — the one a value reaches only when it never went through parse at all. #13053's fixture spelled `reference: { object: … }` inside `fields:`, and the rule reading it answered `undefined`: refused where it was written, read as absent where it was consumed, reported nowhere. The fixture passed, and would have kept passing.
+
+**New export — `referenceCarrierOf(def, reader?)` in `@objectstack/spec/data`.** It answers the carrier as the string the contract declares, and throws a `TypeError` naming the shape and the fix when the key is present in any other shape. `null`, `undefined` and `''` are ABSENCE, not a wrong shape, and still answer `undefined` — a field is allowed to name no target.
+
+**`referenceTargetOf` reads through it**, so the single arbiter of "what does this field expand into" refuses rather than answering "no target". Every consumer that already asks the arbiter — `$expand`, the record-title deriver, the dangling-reference audit, the analytics dimension labeller — inherits the refusal with no edit.
+
+**`@objectstack/lint`** routes its own target readers through the same accessor: `refOf` in `validate-security-posture.ts` (the reader in the #13053 incident) and in `data-model-rules.ts`, plus the object-graph slice every other rule downstream reads.
+
+Upgrading: nothing conformant changes. A non-string `reference` could not be authored, stored or parsed before this release either; what changes is that a hand-built fixture or a raw registry entry carrying one now fails loudly at the read instead of being silently treated as targetless. If a test asserted the old silence, assert the refusal instead — `packages/cli/test/data-model-rules.test.ts` is the worked example.

.github/workflows/lint.yml

@@ -4630,38 +4630,6 @@ jobs:
       - name: workspace manifest dependency graph has no cycle
         run: pnpm check:workspace-manifest-cycles
 
-      # A relationship carrier is spelled as the target object's NAME (#13103).
-      # `FieldSchema.reference` is `z.string()`, and #13053 measured what a
-      # non-string one costs: a fixture spelling `reference: { object: '...' }`
-      # inside `fields:` is refused by `ObjectSchema.safeParse` AND read as
-      # `undefined` by the rule resolving it, so the suite was blind in both
-      # directions and the test still passed.
-      #
-      # ⛔ Deliberately NOT the broad guard that first suggests itself. Running
-      # every object fixture through `ObjectSchema.safeParse` was measured at
-      # 838 fixtures across 303 files going red, and those fixtures are invalid
-      # ON PURPOSE — the same carve-out `check:query-options-erasure` prints for
-      # itself: a rejection test must be able to build off-contract input. So
-      # the population judged here is the CARRIER, never the fixture.
-      #
-      # Lands as a ZERO-BASELINE ratchet: re-measured before landing at 443
-      # string carriers and 0 object/array ones at a field-def position, so
-      # there is no grandfathering file and nowhere to park the next defect.
-      # Two words keep it narrow, and the self-test is where both are observed:
-      # only a LITERAL is judged (identifiers, pass-throughs, Zod declarations
-      # and `reference: null` stay unjudged), and only at a field-def carrier
-      # POSITION (the generated i18n bundles write `reference:` as a field NAME,
-      # and are excluded BY SHAPE, never by a path ignore that would rot).
-      # A position it cannot resolve REFUSES with exit 3 rather than counting as
-      # "not a field def", and so does a run that measures zero carriers —
-      # a green whose success condition equals its total-failure condition.
-      #
-      # Reads source with scripts/ts-parse.mjs; no build, no spawns, ~5s.
-      - name: Relationship carriers are spelled as the string the spec declares
-        run: |
-          node packages/lint/scripts/check-reference-carrier-shape.mjs --self-test
-          node packages/lint/scripts/check-reference-carrier-shape.mjs
-
       # #15149. Grep-level guard for the defect this very repo just shipped: an
       # unquoted `- name:` step name containing ` #` is silently truncated by
       # YAML at that point (a space + hash starts a comment inside a plain

packages/cli/test/data-model-rules.test.ts

@@ -780,32 +780,51 @@ describe('lintDataModel — `reference_to` is a rejected alias, not a tolerated
     ).toBe(false);
   });
 
-  it('a non-string `reference` is not a target either', () => {
-    // `refOf` is declared `string | undefined`; the old `||` chain returned
-    // whatever truthy value was there, so this shape used to be reported as a
-    // resolved target named `[object Object]`.
+  it('a non-string `reference` is REFUSED, loudly, rather than read as no target', () => {
+    // #13053's whole defect, and the assertion this case exists for, INVERTED on
+    // purpose. `refOf` used to answer `undefined` for a non-string carrier, so a
+    // fixture spelling `reference: { object: … }` was invisible in both
+    // directions at once: refused by `ObjectSchema.safeParse` where it was
+    // written, read as absent where it was consumed, and therefore reported
+    // nowhere. This case used to pin the SECOND half of that silence — it
+    // asserted the shape resolved to nothing and produced `missing-reference`,
+    // which is a finding about the wrong thing: the target is not missing, it is
+    // unreadable, and the two want different fixes from the author.
     //
-    // ⚠️ The non-string value is BOUND THROUGH A VARIABLE rather than written
-    // inline, and the binding is a legibility device — not a way past a gate.
-    // `packages/lint/scripts/check-reference-carrier-shape.mjs` refuses a
-    // non-string LITERAL at a `reference` carrier position, and it is right to:
-    // an AUTHORED site of that shape is invisible in both directions at once —
-    // refused by `ObjectSchema.safeParse` and read as `undefined` by every rule
-    // that resolves it (#13053), so it reports nothing either way.
+    // The reader now throws (`referenceCarrierOf`, `@objectstack/spec/data`), so
+    // a fixture that cannot be read fails its test instead of passing it. The
+    // gate that used to guard this shape at the source level
+    // (`check:reference-carrier-shape`) is retired: it caught zero in its
+    // lifetime, its coverage was partial by its own header, and the protocol
+    // already refuses the shape at the contract door.
     //
-    // That shape is exactly what this case must keep driving, because the whole
-    // assertion is that such a value resolves to NOTHING: it is a deliberate
-    // counter-example, not an authored carrier. The gate judges literals and
-    // leaves a non-literal unjudged, so the binding keeps its authored-site
-    // sweep honest while the assertion goes on testing the identical shape.
-    // ⛔ Do not inline this value again; ⛔ do not weaken the gate or add a path
-    // ignore there — it has no baseline and wants none, by design.
+    // ⛔ Not a bare `toThrow()`: an unrepaired reader that threw some other
+    // `Error` on some other input would satisfy that. The class and the sentence
+    // the author actually reads are both asserted.
     const nonStringReference = { object: 'project' };
+    const lintWithObjectCarrier = () =>
+      lintDataModel([
+        { name: 'task', fields: { project: { type: 'lookup', reference: nonStringReference } } },
+      ]);
+    expect(lintWithObjectCarrier).toThrow(TypeError);
+    expect(lintWithObjectCarrier).toThrow(/`reference` is an object/);
+    expect(lintWithObjectCarrier).toThrow(/FieldSchema declares it as an optional STRING/);
+
+    // CONTROL — the identical object with a STRING carrier reads fine and reaches
+    // the ordinary rules, so the throw above is about the carrier's SHAPE and not
+    // about this fixture, this field type, or `lintDataModel` refusing to run.
+    const stringCarrier = lintDataModel([
+      { name: 'project', fields: { name: { type: 'text', label: 'Name' } } },
+      { name: 'task', fields: { project: { type: 'lookup', reference: 'project' } } },
+    ]);
+    expect(has(stringCarrier, 'relationship/missing-reference')).toBe(false);
+
+    // CONTROL — an ABSENT carrier is still the ordinary `missing-reference`
+    // finding, not a throw. Absence and unreadability are different answers and
+    // the reader must keep telling them apart.
     expect(
       has(
-        lintDataModel([
-          { name: 'task', fields: { project: { type: 'lookup', reference: nonStringReference } } },
-        ]),
+        lintDataModel([{ name: 'task', fields: { project: { type: 'lookup' } } }]),
         'relationship/missing-reference',
       ),
     ).toBe(true);