Skip to content

feat(objectql): serve the nested-relation filter in where — lowered at the engine seam, the related object read as the caller, a loud cap, drivers untouched (#20802) - #20872

Merged
objectstack-fleet[bot] merged 8 commits into
mainfrom
claude/issue-20802-relation-filter-lowering
Sep 30, 2026
Merged

objectstack-fleet[bot] merged 8 commits into
mainfrom
claude/issue-20802-relation-filter-lowering

Conversation

@objectstack-fleet

Copy link
Copy Markdown
Contributor

Part of #20802
Clause-②: yes (widening)

The engine half of ruling 5907789183 (letter A). The analytics cube read and the analytics read-scope face are the second half, after #5930 step 3 (#20810); skills/objectstack-query (#20782) belongs to the skills seat, who is told after this lands. So this PR does not complete the card.

What this does

The nested-relation form { relation: { field: value } } is served in where. It is lowered at the engine's filter seam that #5930 step 2 built (cfa931535). The drivers receive $in / $contains and are not changed (ADR-0053 D-D1 item 5, D4 (b)).

  • Where the lowering runs. Stage 2 of where admission now has three steps: resolve the placeholders, then lower each nested-relation condition, then run the shared lowerFilterCondition. ObjectQL.resolveRelateThenLowerWhere holds that order, so a verb cannot resolve without it. resolveWhereTokens and withResolvedWhere became async to carry it.
  • How the lowering works. lowerRelationConditions reads the related object with the engine's own find: fields: ['id'], limit: RELATION_FILTER_ID_CAP + 1, and the caller's execution context. The ids it returns become { relation: { $in: ids } } on a single-valued relation. On a multi-valued one they become an $or of one $contains per id, which matches on any member. That is the spec's own any-of spelling, and the SQL family refuses $in over the JSON column. No related record matching gives $in: [] or $or: []: FALSE, never an absent predicate.
  • No second walk. A condition is found by the one filter walk the engine already runs with each column's declaration in hand, walkCondition in number-comparand-declared-type-door.ts.
    • At the door (stage 1), that walk admits a condition structurally, from declarations alone (admitRelationCondition), instead of refusing it.
    • The lowering calls the same walk (mapRelationConditions) twice: once to collect the conditions, and once, after the reads, to replace them in the same order.
    • The door and the lowering therefore find a condition at the same boundaries by construction. No driver-local guard exists.
  • As the caller. The related read goes through the middleware chain like any read. The related object's CRUD gate, row scope and field permissions apply, and its own doors judge the condition's comparands. A refusal from any of them is the answer, loudly.
  • Bounded. The cap is RELATION_FILTER_ID_CAP = 1000, one named constant, exported from @objectstack/objectql. Past it the filter is refused with INVALID_FILTER / 400. The refusal names the cap, the related object and the two-step route. The filter is never run over a cut-off list.

The accept set that widens. It is the engine's where (find, findOne, count, aggregate, update, delete) plus judgeFilter, and the REST query doors that reach findData (POST /api/v1/data/:object/query and the filter / $filter spellings). Per relation kind:

  • lookup, master_detail, user, tree, single-valued: refused, now served ($in).
  • the same kinds with multiple: true: refused, now served (any member).

Nothing served today narrows.

  • The admission runs only where the arm refused before: a no-operator object beneath a relation column at where.
  • The cap refusal is new, but it applies to a form that was refused.
  • Every filter without a nested-relation condition reaches the drivers by reference, byte-identical to before (pinned).

What stays refused. Each is refused in the engine's words, before any read:

Text.

The dotted-path words, made true again (a bounded in-place fix, named here).

  • Both dotted relation refusals said "a filter reaches only columns of '…' itself". This change makes that false.
  • The two refusals are the engine's (filter-comparand-shape.ts) and the query-parameter door's (metadata-protocol protocol.ts, outside the claim's declared file surface).
  • Both now name the nested form to write instead ({ "owner": { "region": VALUE } }), in the same words, and keep the shared denormalise remedy.
  • A conformance pin holds the two routes equal.

Measured

On this branch at 56da9b6d50 through POST /api/v1/data/:object/query. Owner u1 is region NA on d1 and d3, and d4 has no owner. Before, on origin/main after PR #20781, every relation row answered INVALID_FILTER / 400 on every driver.

where SQLite PostgreSQL 16.13 (live, local) InMemoryDriver (measured, not pinned)
{ owner: { region: 'NA' } } (lookup), boss (master_detail) d1, d3 d1, d3 d1, d3
{ owners: { region: 'NA' } } (multiple lookup) d1, d3 d1, d3 d1, d3 (see the note)
{ parent: { title: 'a' } } (tree) d2, d3 d2, d3 d2, d3
{ $not: { owner: { region: 'NA' } } } d2, d4 d2, d4 d2, d4
{ $or: [{ owner: { region: 'EU' } }, { title: 'a' }] } d1, d2 d1, d2 d1, d2
{ owner: { region: 'APAC' } } (no match) none none none
a condition matching 1001 related records 400 INVALID_FILTER, the cap words same not measured
  • The memory note. The in-memory driver matches $contains over a stored array by substring per element. That is the gap FILTER_OPERATORS' $contains docblock records for that driver. So with ids u1 and u10, a multi-valued condition meaning u1 also matches the row holding ['u10']: measured memory d1, d3, d5, against SQL d1, d3. Single-valued relations are exact everywhere.
  • Why memory is not pinned. check:driver-memory-census refuses a new test consumer of that driver without a ruling.
  • H7 (RLS), measured. A policy record.owner.region == 'NA' is refused at compile: "cross-object/nested field path … is not pushdown-able". The policy is dropped to the deny sentinel, so it answers zero rows. The RLS compile seam is untouched, and no async read was added there.

Mechanism hypotheses: which held

  • H1: held, refined. lowerFilterCondition is pure and synchronous, and the relation step needs the engine. So the step lives in objectql, between token resolution and the shared lowering, and each engine filter position reaches it at most once.
    • Served: where on find, findOne, count, aggregate, update and delete (the multi and by-predicate paths alike), and the judge.
    • Not served (refused, named): aggregations[i].filter and having.
  • H2: held, with one widening of the kind list. The door admits the form under every REFERENCE_VALUE_TYPES kind, not only lookup / master_detail.
  • H3: measured "yes": the one check exists and is reused.
    • A direct filter on a field the caller cannot read is refused today: 403 PERMISSION_DENIED, the security layer's filter-oracle guard assertReadableQueryFields.
    • The related read reaches that same check, so the nested form answers the same 403, naming the field.
    • There is no second copy of the rule. ⚠️ This is not the ruling's literal INVALID_FILTER; see the open question in the report.
  • H4: $in does not mean "any member" everywhere; $or of $contains does on SQL.
    • $in over a multi-valued lookup is refused on SQL (JSON column) and is any-member on memory.
    • $contains is membership on SQLite and PostgreSQL, and substring-per-element on memory (the note above).
    • The $or of $contains is the spec's declared any-of spelling, so it is the lowered form.
  • H5: held, and no bound to reuse. expand's batch loader bounds nothing: it deliberately forwards no limit. The cap is a new named constant.
  • H6: held. $and / $or compose as written. $not over a relation condition takes the shared lowering's NULL-safe negation, so a row with no relation satisfies it. The driver input is pinned equal to the hand-written two-step route's, and $not + no match gives every row. An empty inner result is FALSE, never "no filter". $nor is not in the vocabulary.
  • H7: held; out of scope. Measured above.

Tests (all on 56da9b6d50)

  • @objectstack/objectql test: 345 files / 6786 passed. typecheck exit 0, check:test-typecheck OK.
  • @objectstack/rest test, with OS_TEST_POSTGRES_URL set to a local PostgreSQL 16.13: 237 files / 4706 passed / 35 skipped (MySQL cells and suites with no URL). typecheck exit 0.
  • @objectstack/metadata-protocol test: 191 files passed, 3 skipped / 2801 passed, 19 skipped. typecheck exit 0.
  • @objectstack/spec test: 578 files / 17066 passed / 1 todo. typecheck exit 0. check:generated: all 15 artifacts up to date (no regeneration needed; docblock only).
  • Downstream: @objectstack/plugin-security 149 files / 3227 passed, 23 skipped. driver-memory 65 / 1470 passed. driver-sql 201 files passed, 11 skipped / 3254 passed, 188 skipped. The other ...@objectstack/objectql consumers are declared to CI.
  • New pins:
    • packages/objectql/src/engine-nested-relation-lowering.test.ts (13 tests, recording driver). It covers:
      • every relation type;
      • any-member on a multi-valued relation;
      • the empty id set;
      • $and / $or / $not / sugar, pinned equal to the two-step route's driver input;
      • every verb and the judge;
      • placeholder resolution;
      • the related read as the caller, and a middleware refusal surfacing with no outer read;
      • the cap at 1000 and at 1001;
      • the kept refusals, with the judge answering execution's words verbatim;
      • the related object's own doors;
      • the aggregation filter / having refusals;
      • a pass-through control.
    • packages/rest/src/data-nested-object-door.test.ts (rewritten): [finding] a no-operator object under a lookup, master_detail or json field answers per driver: the declared nested-relation filter returns no rows on memory and a 400 on SQL, and a json object comparand deep-equals on memory and is refused on SQL #20745's table turned into rows on SQLite and live PostgreSQL, plus the two-step equivalence, the kept refusals with the route inside the 500-character REST bound, the cap pin (1001 related records refused, 1000 served) and the controls.
    • packages/rest/src/data-nested-relation-permission.test.ts: the permission pin, with the real SecurityPlugin on a real engine and SQLite, through the REST door. An unreadable related field is refused 403, never emptied, while a system read can filter by it. A hidden related record matches nothing.
  • Fixture triage for the removed refusal branch:
    • engine-nested-object-door.test.ts keeps only what still refuses.
    • query-expression-conformance.test.ts: its nested-form control now pins the served rows, and a new pin checks that both doors' dotted refusals name the same route.
    • protocol-explicit-filter-field-gate.test.ts: its GUARD still proves the name gate never descends.

Ablations, each from the committed fix. Each ran through scripts/ablation-replace.mjs in WRAP mode, trap-restored. After each mutation objectql was rebuilt, and ablation-dist-preflight found the marker in 4 built files.

  • A, the lowering disabled. if (sites.length === 0) return where; became an unconditional return where (marker __ablated_20802_lowering__). Blob 9237c3dfc995 → f9994599356f. Predicted red, observed red:
    • objectql pins: 10 failed / 235 passed;
    • rest pins: 11 failed / 4 passed / 6 skipped.
    • The structural refusals and controls stayed green.
  • B, the related read as the system. ...(execCtx ? { context: execCtx } : {}), became isSystem: true (marker __ablated_20802_caller__). Blob → b0c1748c5b70. Predicted red, observed red:
    • objectql 1 failed / 244 passed (the as-the-caller pin);
    • rest 2 failed / 13 passed / 6 skipped: the permission pin answered rows instead of 403, and the row-scope pin returned d4.
    • (A first run of B used a mutation that left execCtx unused, and its DTS step failed on TS6133; the JS carried the marker. It was re-run type-clean, and those numbers are the ones above.)
  • Restore. Blob equals HEAD 9237c3dfc995, and git diff HEAD is empty. After a rebuild, the --absent preflight found both markers absent from all 14 built files, and the whole tree was clean. The pins were green again: 245 passed; 15 passed / 6 skipped.

Gates

node scripts/pm/dispatch-gates.mjs --commands at 56da9b6d50 (fresh, not stale) derived 120 commands, and all 120 were run. --ran with each exit code recorded: 120 derived, 120 run, 0 NOT-MEASURED, 0 UNRUN, all exit 0.

Three gates refused first with PREREQUISITE NOT MET (exit 3), and none of the three is counted as a failure:

  • check:skill-examples
  • check:dual-build-cjs-loads
  • check:type-check-debt

They were re-run green after turbo run build --filter='./packages/*' --filter='./packages/*/*'.

Lint, narrowed and proven: eslint --no-inline-config --format json over the 15 changed .ts files gave 15 files, 0 errors, 0 warnings.

  • isPathIgnored is false for all 15.
  • The population is eslint.config.mjs's **/*.{ts,…} block.
  • parserOptions.project / projectService are unset for every file. Type-aware linting is off, so this diff cannot move a verdict on an untouched file.

Changesets

  • .changeset/20802-nested-relation-filter-served.md: @objectstack/objectql minor, Clause-②: yes (widening). It says it supersedes the relation-field paragraph of the pending 20745-nested-object-door entry, and it states the in-memory $contains substring caveat.
  • .changeset/20802-nested-relation-prose.md: @objectstack/spec patch (shipped JSDoc).
  • .changeset/20802-dotted-relation-route.md: @objectstack/metadata-protocol patch (refusal words).
  • ADR anchor: scripts/adr-anchors/packages__objectql__src__relation-filter-lowering.ts.json → ADR-0053.

Acceptance notes

  • The analytics half. The cube read and the analytics read-scope face are not touched here (#5930 step 3: the shared filter lowering at the analytics seams (the analytics where / preview door, the read scope) and the memory cube face's door, with the F5 / F11 output vocabulary #20810 first). Until then the analytics face still flattens the nested form to cube members, and the read scope still refuses it.
  • The permission refusal's envelope. It is PERMISSION_DENIED / 403, the one existing check, reused. It is not the ruling's parenthetical INVALID_FILTER, and it is raised to the PM as an open question.
  • Read order. The related read runs at stage 2, before the outer verb's own middleware. A caller with no read access to the queried object still gets the outer 403, but the related read has already run as that caller. It reads nothing the caller could not read directly.
  • Lint face. @objectstack/lint's list-view dotted-path hint still says "Filter on a column of … itself". That is an instruction rather than a claim this change made false. Carrier: none; not changed here.
  • The in-memory substring gap. On the in-memory driver, the multi-valued any-member lowering inherits that driver's substring-per-element $contains. It is reported, not fixed here (no driver file).

Generated by Claude Code

…als name the nested route

Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY
Co-authored-by: Claude <noreply@anthropic.com>
…esets, ADR anchor

Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY
Co-authored-by: Claude <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 3 package(s): @objectstack/metadata-protocol, @objectstack/objectql, @objectstack/spec, touching 50 documentable anchor(s). ⚠️ 1 changed file(s) yielded no anchor (packages/objectql/src/index.ts), so the pages documenting them are NOT COVERED by this run — this is not a clean bill of health for those files.

25 hand-written doc(s) name something this change touched — list omitted above 15 rows. Re-derive on the tree named below: node scripts/docs-audit/affected-docs.mjs --json 72f8c3820154c69cdf9faa6f887dc544e4c4b823.

⛔ 6 release-owned page(s) also affected — read-only, see AGENTS.md Documentation Guardrails.

What this run could not see
  • 1 changed file(s) yielded no anchor (packages/objectql/src/index.ts) — pages documenting those are invisible to this run
  • 1 anchor(s) matched too much of the corpus to be a work list: ObjectQL (symbol, 70 pages)
  • 11 name(s) were too generic to anchor anything (single lowercase words)
  • the SDK route bridge reached 54 of 206 client-bound route-ledger rows — the other 152 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 152: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 55 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 97 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.
  • a key NAME is not a key, so the hand re-read the line above prescribes can land on the wrong schema. The same spelling is authorable on one governed type and a [REMOVED] tombstone on another for each of active, aria, joins, objects, template, tools and version (censused on [finding] tools is a key on BOTH AgentSchema (tombstoned, dead) and SkillSchema (live, cloud-attested), so a name-based search attributes skill examples to the agent key — it produced a false stop-the-line alarm on PR #19059 #19093 over the liveness ledger's governed types, top-level keys); nothing in a search result distinguishes the two, so a grep hit on a LIVE example reads as evidence about the DEAD key. Measured on fix(spec): the agent.tools liveness row says dead — it claimed live on a key the schema tombstoned #19059: content/docs/ai/agents.mdx was reported as contradicting the agent.tools tombstone over its tools: example at :161, which is inside the defineSkill({ block opened at :155 — the page was already correct. Settle ownership by PARSING the value against both schemas, never by the name: that literal PASSES SkillSchema, and as an AgentSchema it FAILS at tools with the tombstone prescription. ⛔ These names are not the whole class — a key retired through a .strict() guidance map leaves no tombstone in the walked shape and none of them here (tool.category, live as AIToolDefinition.category).

Coarse fallback — 139 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 72f8c3820154c69cdf9faa6f887dc544e4c4b823 → packageMentionDocs.

Which tree this was computed on

This run read content/docs from 6dfbe15fcd231284b26b0f0ca86f981072ef980c — the merge of head 56da9b6d50340f2cbc1674236f8957cab1e5c2ff into base 72f8c3820154c69cdf9faa6f887dc544e4c4b823, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 6dfbe15fcd231284b26b0f0ca86f981072ef980c && git checkout 6dfbe15fcd231284b26b0f0ca86f981072ef980c
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 72f8c3820154c69cdf9faa6f887dc544e4c4b823 56da9b6d50340f2cbc1674236f8957cab1e5c2ff && git checkout -B drift-repro 72f8c3820154c69cdf9faa6f887dc544e4c4b823 && git merge --no-ff 56da9b6d50340f2cbc1674236f8957cab1e5c2ff

node scripts/docs-audit/affected-docs.mjs --json 72f8c3820154c69cdf9faa6f887dc544e4c4b823

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

Advisory only, and a precision-first one (#9192): a page is listed because it names a
symbol, wire route or SDK method this diff touched — not because it mentions a changed
package. Each row says which anchor put it there, so a wrong row is reportable rather than
merely annoying. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs 72f8c3820154c69cdf9faa6f887dc544e4c4b823 → pass the list as
args.docs, on the commit named under Which tree this was computed on.

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

Contract review

Served-tier: CONTRACT_REVIEW_TIER
Head-sha: 56da9b6d50340f2cbc1674236f8957cab1e5c2ff
Local-runs: none

At-tier review of PR #20872, the engine half of card #20802 (ruling 5907789183, letter A), rendered at 2026-09-30T14:19Z. Read only: the PR body, its 20-file list and the net diff against main (git diff origin/main..., merge-base 660a9b247e), the card's body and every comment (5907789183, 5907891818, 5910636932, 5912874377, 5912908152), #20745 with PR #20781 (today's refusal and its table), PR #20794 (cfa931535, the seam), and the head's check-runs. Not the dispatch order, not the seat's conclusions. The head is still 56da9b6d50340f2cbc1674236f8957cab1e5c2ff (a draft, Part of #20802, the declaration on its second line). Nothing was built, run, re-run or ablated; mergeability was read with git merge-tree (an object-database read, no worktree).

Check-runs on this head, as read at the stamp above: 32 runs. 17 success (Auto Label, Build Core, Build Docs, Check Changeset, Check Documentation Links, Check PR Size, Dogfood Verify CLI, filter, Flag docs affected, Governed Surface Queue Guard, the two single-writer/claim guards, Part-of PR must not also close its card, Spec property liveness, The card this PR closes must claim this branch, Type Check · debt ledger, Type Check · source gates); 2 skipped (Console Pin Gate, Packed-tarball smoke, both opt-in); 13 in_progress (Test Core 1 to 6, Dogfood Regression Gate 1 to 3, Lint & Repo Gates, Temporal Conformance, Type Check · consumer gates, Type Check · workspace). The in-progress runs are not conclusions and this record does not wait for them; the landing does (AGENTS.md: every check green). Nothing read is red.

① Derived judgments

Each accept-set and public-surface change the diff implies, named right or wrong against the ruling's execution parameters.

  • Where: the [finding] 仓内存在 5 个独立的过滤器→谓词编译器,每次语义裁决成本 ×5 —— 值得立「谓词编译收敛」调查程序(#5298 成本清单副产品) #5930 seam, drivers untouched, no second walk — right. Stage 2 of where admission is now resolveWhereFilterTokens, then ObjectQL.lowerRelationConditions, then lowerFilterCondition (resolveRelateThenLowerWhere, engine.ts). Every lowerWhereFilterArray call on the branch (find, findOne, update, delete, count, aggregate, and the judge) passes this.relatedSchemaOf, and every one of those verbs awaits resolveWhereTokens / withResolvedWhere with the relation step, so no position admits the form at the door without lowering it before a driver. The only remaining callers of the older resolveThenLowerWhere are having (guarded by position === 'having') and the per-aggregation filter's token pass, neither of which admits the form (their walk contexts carry servesRelations: false). No file under packages/drivers/** changes. The door's admission (admitRelationCondition), the collecting pass and the replacing pass all run through the one walkCondition; the two passes walk the same resolved object in Object.entries order and replace by index, so they cannot disagree about where a condition sits.
  • First cut: one level, forward, any member — right. admitRelationCondition refuses a dotted inner key and a relation-typed inner key holding a no-operator object (second-level), so recursion in the judge is bounded at one; the reverse form is not touched. A multi-valued relation lowers to an $or of one $contains per id (lowerRelationSite), the spec's any-of spelling; the $or is AND-ed into the node (withClauses, appending to an existing list $and, wrapping a non-list one). A single-valued relation lowers to $in.
  • As the caller — right, verified from the code, not the ablation. The inner read is this.find(site.target, { where, fields: ['id'], limit: RELATION_FILTER_ID_CAP + 1, ...(execCtx ? { context: execCtx } : {}) }). execCtx is the verb's own opCtx.context (mergeReadContext(query.context, options.context) on the reads; options.context on update/delete), the same identity the outer middleware later sees; context, fields and limit are in ENGINE_FIND_OPTION_KEYS. Nothing sets isSystem; an absent caller context stays absent, which is what a direct read with no context gets. The inner find enters executeWithMiddleware, where the security plugin's read branch guards opCtx.ast with assertReadableQueryFields (security-plugin.ts, the filter-oracle guard) on the related object and injects that object's RLS. The objectql pin asserts the middleware sees operation: 'find', the caller's userId, and isSystem not true; the rest pin runs the real SecurityPlugin over SQLite through the public door.
  • Bounded, refused, never truncated — right. limit is RELATION_FILTER_ID_CAP + 1, and a matched.length above RELATION_FILTER_ID_CAP throws relationFilterCapError before any outer read. No clamp sits between the engine and the driver: engine.ts, packages/plugins/*/src and packages/drivers/*/src carry no Math.min on limit, no MAX_LIMIT or page ceiling (grepped on the branch). The rest pin inserts 1001 matching owners over SQLite and reads the 400, and 1000 served whole; the objectql pin does the same with the recording driver at cap and cap+1.
  • Empty id set is FALSE, never an absent predicate — right. $in: [] and $or: [] are the 空组合子在同仓有两个对立答案:五个后端归约成布尔单位元,service-analytics 的两个编译器 fail-closed 抛错 —— #5239 的一致性表四条因此进不了表 #5322 identities pinned for every backend in filter-logic-conformance.ts (empty $or matches nothing); SqlDriver.applyFalseConstant renders 1 = 0. The rest pin reads no rows for the single- and multi-valued no-match cases and all four rows for $not over a no-match.
  • Accept set that widens — right, and within "a relation field". where on find / findOne / count / aggregate / update / delete and judgeFilter (ok: true where it answered ok: false), reached from the REST query doors through findData, under every REFERENCE_VALUE_TYPES kind: lookup, master_detail, user, tree, single or multiple. The widening to user and tree is the ruling's "a relation field" as the fleet already reads it: [finding] a no-operator object under a lookup, master_detail or json field answers per driver: the declared nested-relation filter returns no rows on memory and a 400 on SQL, and a json object comparand deep-equals on memory and is refused on SQL #20745's seat answer 5904634845 ruled that one door judges the spec's class REFERENCE_VALUE_TYPES, and serving the same class is the same one-class-one-answer; referenceTargetOf resolves user to sys_user (IMPLICIT_REFERENCE_TARGETS) and refuses loudly where that object is not registered. Public surface: one new root export, RELATION_FILTER_ID_CAP, from @objectstack/objectql (objectql has no api-surface baseline to regenerate). The Clause-② line reads yes (widening) on the PR body's second line and in the objectql changeset; under scripts/pm/clause2-line.mjs that is the widening arm, at least minor, and it answers the grammar's question (the accept set widens from refused to served, the public surface grows by one export). The ruling's own no reading of that line was about the type not moving; the claim 5910636932 named the deviation and left it to the director's veto. The line is right.
  • Nothing narrows — right. The admission runs only where the arm refused (a no-operator object beneath a relation column at where); {} beneath a relation, an unregistered related object, a user field with no sys_user, a second level, a dotted inner key and an undeclared inner key were all INVALID_FILTER / 400 before and stay so, in new words and before any read; the structured-JSON, scalar and provisioned-id verdicts are untouched; the lowering pass (lowerOnly) re-asks only the relation question and skips the number arm, and tokens resolve only to scalars ({current_user_id}, date macros), so a filter stage 1 admitted cannot be refused at stage 2 by the re-walk. A filter with no nested condition returns from lowerRelationConditions by reference (sites.length === 0), and the control pin reads the driver input deep-equal to the input.
  • Kept refusals list — right, and the dotted-path words are true again. The kept refusals are the six named above plus the form at aggregations[i].filter and having (their relationWords now say the engine serves it in where, and a multi-valued column there names no $contains route because the in-process evaluator has no member test), the json object comparand, and the dotted path (INVALID_FIELD). Both dotted relation refusals (filter-comparand-shape.ts and metadata-protocol protocol.ts) drop "a filter reaches only columns of X itself", which this diff makes false, and name the nested route in the same words; no test anywhere on the branch still asserts the old words; the conformance pin holds the two doors' routes equal.
  • Files beyond the claim, each with its reason. number-comparand-declared-type-door.ts (+164 −21) had to change because the ruling forbids a second walk: the one walk that already had each column's declaration in hand gained the relation arm at where (admission instead of refusal), the lowerOnly pass, withClauses and mapRelationConditions; no other form's answer moved there (the no-operator-object arm for scalar and JSON columns, the number arm, the three positions and the boundaries are the same code paths). filter-comparand-shape.ts had to change because its relation-head refusal made a claim this change falsified; only the relation head's words and a nestedRelationRoute helper change. metadata-protocol/src/protocol.ts is the same sentence at the query-parameter door, outside the claim's declared surface and named in the PR body and report; it carries its own patch changeset. The ADR anchor JSON pins ADR-0053 to the new module, as PR feat(spec,objectql,plugin-security): one shared filter lowering, run once at the engine and RLS seams (#5930 step 2) #20794 did for the seam. index.ts is the one export. The rest pins are where [finding] a no-operator object under a lookup, master_detail or json field answers per driver: the declared nested-relation filter returns no rows on memory and a 400 on SQL, and a json object comparand deep-equals on memory and is refused on SQL #20745's table lives.
  • Security, adversarially: no path found where the nested form yields what a direct filter on the related object would refuse or hide. Row scope: the inner read carries the caller's context into the related object's RLS, so a hidden related record contributes no id (the rest pin: HIDDEN owner matches nothing for the member, d4 for the system). Field permissions on the inner key: the inner ast.where is the flat related condition, guarded by assertReadableQueryFields on the related object; a refusal propagates from the awaited find and the outer read never runs (both pins). A related object the caller cannot read at all: the inner find meets that object's CRUD gate as a direct read would. The inner read before the outer verb's middleware: it runs as the caller through the same engine entry a direct read uses, so it can return or refuse nothing a direct read would not; a caller without read access to the queried object still gets the outer 403, and the only observable difference is that the related object's hooks and audit see a read the caller could have made directly. Error text: describeKeys names keys, never values; the cap words name the cap, the relation and the related object; the related object's own doors echo the caller's comparand, not stored data; the count the cap discloses is a count of rows the caller can see. Two disclosures were weighed and judged not leaks of data: the structural refusals name whether an inner key is declared or is itself a relation of the related object (schema shape, already readable through the dotted-path door's head classification and the metadata routes). $not over a scoped inner read negates only the ids the caller may see, which is exactly what the hand-written two-step route answers.
  • Open question A (5912908152) — keeps the ruling's operative guarantees; no review face claims otherwise. Loud (403), names the field (toContain('secret') on both the direct and nested refusals), never empty (records undefined beside the refusal), and it is the one check a direct filter meets. The changeset, the PR body and data-engine.mdx say PERMISSION_DENIED / 403; the spec docblock says "refused, never answered empty" and names no code. No face writes INVALID_FILTER for that case. The veto path (the security layer's envelope for all filter-oracle refusals, its own card) is stated on the seat's answer.
  • Review faces, sentence by sentence. The three changesets: true (the objectql entry's supersession claim holds, 20745-nested-object-door.md is still pending on main; the memory $contains caveat is stated). The PR body: true on every sentence checked against the diff, the seam and the ruling, including the H1 to H7 readings and the measured tables it labels as local (the PostgreSQL column and the InMemoryDriver column are measured, not CI-pinned, and the body says so). filter.zod.ts docblock item 4, the QueryFilter example and the Filter nested-arm comment: true. data-engine.mdx: true. The refusal texts (the cap, the kept refusals, both dotted-path doors): true. One precision nit, not a false sentence: the docblock, the mdx and the changeset say every inner key must be one the related object declares, while admitRelationCondition also admits the platform-provisioned id / created_at / updated_at when the declared map omits them (the objectql pin uses { owner: { id: '{current_user_id}' } } on an owner object declaring no id); the fleet's own reading (5904634845) counts those three as declared on every object, and the served set is wider than the words only in that direction.

② Semver level

  • @objectstack/objectql minor — right. An additive widening takes at least minor (the WHICH LEVEL ruling finding(changeset): two independent contract reviews read the repo's own history to opposite bumps for "add an exported symbol to a published index" #15294), and the package gains one root export. No BREAKING banner is due: nothing served today narrows, so no ADR-0087 disposition is owed.
  • @objectstack/spec patch — right. filter.zod.ts changes are prose only (the docblock, the example, one comment in checkFilterConditionComparands, one comment on Filter); the type and the schema are unchanged; Build Docs and Spec property liveness are green on the head.
  • @objectstack/metadata-protocol patch — right. The words of an INVALID_FIELD / 400 refusal change; its envelope and the door's accept set do not.
  • The declaration reads yes (widening) on the PR body and the objectql changeset, the arm clause2-line.mjs reads as a widening at minor or above; it matches what the diff publishes.

③ Boundary flags

Every dev flag and every open_questions entry, answered or escalated.

Implemented-by: claude/issue-20802-relation-filter-lowering
Reviewed-by: session_01DEvba2nBuD4tWzfq8r8NFY

VERDICT: PASS


Generated by Claude Code

@objectstack-fleet
objectstack-fleet Bot marked this pull request as ready for review September 30, 2026 14:31
@objectstack-fleet
objectstack-fleet Bot added this pull request to the merge queue Sep 30, 2026
Merged via the queue into main with commit ca5408c Sep 30, 2026
37 checks passed
@objectstack-fleet
objectstack-fleet Bot deleted the claude/issue-20802-relation-filter-lowering branch September 30, 2026 15:19
akarma-synetal pushed a commit to akarma-synetal/framework that referenced this pull request Oct 7, 2026
…nested-relation form (objectstack-ai#20906)

Fixes objectstack-ai#20876
Clause-②: no

## What changed

`content/docs/protocol/objectql/query-syntax.mdx`, section "Filtering
Across Relationships" only. The old callout said relation traversal in
`where` is "not supported" and blamed `SqlDriver.applyFilters()` for
compiling a nested object and emitting a dotted key to Knex. Both were
false: the headline since PR objectstack-ai#20872 (`ca5408c62`), the mechanism
sentences since the engine refuses the nested form before any driver.
The section now states the served form, per triage 5914865117:

- `{ relation: { field: value } }` in `where`, one level, forward only,
evaluated as the caller; a multi-valued relation matches any member;
refused past 1000 ids; an unreadable related field answers `403`.
- Verbs and REST doors that serve it; second level and reverse direction
refused; dotted path still `INVALID_FIELD` / 400; an aggregation's
`filter` and `having` refuse the form.
- The two-step example is kept, now framed as the route past the cap and
for the reverse direction (a reverse snippet is added).
- No driver-mechanism prose.

## Code anchors (all at `origin/main` 33b6e8b)

- Lowering seam: `packages/objectql/src/engine.ts:11268`
(`resolveRelateThenLowerWhere`, calls `lowerRelationConditions` at
:11276); the related read is the engine's own `find` with the caller's
context, `:11322`.
- `$in` vs `$contains`:
`packages/objectql/src/relation-filter-lowering.ts:213-218`
(`lowerRelationSite`): `$in` for single-valued, an `$or` of one
`$contains` per id for `multiple: true`.
- Cap: `RELATION_FILTER_ID_CAP = 1000` at
`relation-filter-lowering.ts:84`; read asks for cap+1
(`engine.ts:11325`), refuses at `:11330` via `relationFilterCapError`
(`relation-filter-lowering.ts:297`) as `INVALID_FILTER` / 400
(`filter-comparand-shape.ts:78-85`).
- 403: the related read goes through the security layer's
`assertReadableQueryFields`
(`packages/plugins/plugin-security/src/predicate-guard.ts:106`, called
at `security-plugin.ts:3652`), `PERMISSION_DENIED` / 403. Pinned:
`engine-nested-relation-lowering.test.ts:227`,
`packages/rest/src/data-nested-relation-permission.test.ts`.
- One level: `admitRelationCondition`,
`relation-filter-lowering.ts:162`; dotted key inside the condition
`:189`, relation key inside `:197` (`second-level`), both
`INVALID_FILTER` / 400. Pinned:
`engine-nested-relation-lowering.test.ts:284`,
`packages/rest/src/data-nested-object-door.test.ts` (REFUSED table).
- Reverse direction: not served by the lowering
(`relation-filter-lowering.ts:16-17`); over REST the parent has no such
field, so `assertFilterFieldsExist` answers `INVALID_FIELD` / 400
(`packages/metadata-protocol/src/protocol.ts:10057`, called at
`:11510`). The direct `engine.find` has no field-name door
(`query-syntax.mdx` section 9, "Unknown Fields Are Tolerated"), so the
page states the reverse refusal for the REST doors only.
- Dotted path: `classifyDottedFilterHead`
(`packages/spec/src/data/filter-dotted-head.ts:119`); engine door
`assertFilterIsMaterializable` (`filter-comparand-shape.ts:204`, code
set at `:270`); REST ingress `protocol.ts:10114`.
- Verbs: `find` `engine.ts:11662`, `findOne` `:11935`, `update`
`:13558`, `delete` `:16227`, `count` `:16760`, `aggregate` `:17138`;
pinned for all six plus `judgeFilter` at
`engine-nested-relation-lowering.test.ts:190`. REST: `GET /data/:object`
and `POST /data/:object/query` (`packages/rest/src/rest-server.ts:8635`,
`:8818`) both call `findData` (`protocol.ts:11183`); the POST query door
is the one the REST test drives.
- `aggregations[i].filter` and `having` refuse the form:
`no-operator-object-door.ts:307`; pinned
`engine-nested-relation-lowering.test.ts:329`, REST
`data-nested-object-door.test.ts:256`.

## Shared wording with objectstack-ai#20888

The skill half is PR objectstack-ai#20902 (open, not merged). Two passages are quoted
verbatim from its `skills/objectstack-query/rules/filters.md`: the
served-form sentence ("A condition on a related record's fields beneath
a relation field ... any member when `multiple: true`).") and the
"Limits — one level: ..." sentence. The code sentence is identical; if
objectstack-ai#20902 changes those sentences before landing, this page must be
re-quoted.

## Census of hand-written `content/docs/**`

Searched: "Relation traversal", "not supported" near where/relation,
`applyFilters`, dotted `'account.` examples, "two queries" / "$in its
ids" / "nested form".

- Fixed: `protocol/objectql/query-syntax.mdx` (the section above), the
only hit.
- Already correct, no change: `kernel/contracts/data-engine.mdx:187-215`
states the served form, the 1000 cap, one level, `403`, aggregation
`filter`/`having` and the dotted refusal (landed with objectstack-ai#20872).
- Not hits: `permissions/*.mdx` dotted `'account.annual_revenue'` keys
are field-permission keys, not filters;
`protocol/objectql/types.mdx:1035` `'metadata.color'` is the deliberate
structured/JSON carve-out; `query-syntax.mdx` :1135/:1167 and
`data-modeling/queries.mdx:442,519`, `schema-design.mdx:111`,
`api/data-api.mdx:177` concern the search axis and `expand`/`fields`
paths, not `where`.
- Generated reference pages (`content/docs/references/**`) not touched.
Also fixed (contract review, second commit `a2a66881e2`): the
`query-syntax.mdx` line 4 frontmatter `description` listed "joins", a
tombstoned key (`packages/spec/src/data/query.zod.ts:562`; the page says
so at its Joins section); it now reads "expand", which the page's
section 4 (Relationships (Expand)) covers.

## Gates

`node scripts/pm/dispatch-gates.mjs --commands` derived 40 commands at
head `a2a66881e2` (re-run after the second commit; the first pass was at
`bd718b653a`); 39 ran green (exit 0) after `pnpm install` and a
lint-closure build. `--ran` reconciliation: 39 of 40 run, 1 UNRUN: `pnpm
--filter @objectstack/spec run check:skill-examples` exited 3
(prerequisite: the client-SDK surface has no built output). NOT
MEASURED: it type-checks marked TypeScript examples in skills and docs,
and this diff adds no marked block. CI owns it. No changeset (docs only,
nothing published).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01VDtqoecgES7ScQYGbFVDRv

---------

Co-authored-by: Claude <noreply@anthropic.com>
akarma-synetal pushed a commit to akarma-synetal/framework that referenced this pull request Oct 7, 2026
…ter in where, its limits, and the two-step route past them (objectstack-ai#20902)

Fixes objectstack-ai#20888
Clause-②: no

## What this changes

Since PR objectstack-ai#20872 (`ca5408c62`, the engine half of ruling 5907789183 on
objectstack-ai#20802) the engine serves `{ relation: { field: value } }` in `where`,
lowered at its filter seam
(`packages/objectql/src/relation-filter-lowering.ts`, `engine.ts`
`lowerRelationConditions`): the related object is read through the
engine's own `find` with `fields: ['id']` and `limit:
RELATION_FILTER_ID_CAP + 1` in the caller's execution context, and the
ids become `{ relation: { $in: ids } }` on a single-valued relation or
an `$or` of one `$contains` per id on a `multiple: true` one. The
published skill `skills/objectstack-query`, rewritten by PR objectstack-ai#20811
(`1d202453`) the same day and before that landing, still told AI authors
the form is refused (`INVALID_FILTER` / 400) and that the two-step `$in`
route is the only way. This PR states the served form, its limits and
the two-step route past them, at every site PR objectstack-ai#20811 rewrote — and
nothing about analytics, which objectstack-ai#20887 carries.

Every limit is stated from the code and its pins, read at `origin/main`
`00a92e18d`:

| Limit | Where it lives | Code / status |
|:--|:--|:--|
| One level: every key a field the related object declares; a relation
condition or a dotted key inside is refused before any read |
`admitRelationCondition` (`second-level`, `dotted-key`,
`undeclared-key`, `empty`, `unregistered-target`), pinned in
`engine-nested-relation-lowering.test.ts` | `INVALID_FILTER` / 400, "one
level only" |
| Forward only: the condition sits beneath a relation field of the
queried object; a parent by its children is not served |
`relation-filter-lowering.ts` header ("The reverse form (a parent
filtered by its children) is not served here at all") | the two-step
route stays |
| `where` only: an aggregation's own `filter` and `having` refuse the
form | `no-operator-object-door.ts` `relationWords` ("which the engine
serves in 'where' and not in an aggregation's 'filter'" / "not in
'having'") | `INVALID_FILTER` / 400 |
| At most 1000 related ids; past that refused, never truncated |
`RELATION_FILTER_ID_CAP = 1000`, `relationFilterCapError` ("a cut-off id
list would silently drop matching rows") | `INVALID_FILTER` / 400 |
| As the caller: the related object's row scope and field permissions
apply; a field the caller cannot read is refused, never an empty result
| `packages/rest/src/data-nested-relation-permission.test.ts` (the
security layer's filter-oracle guard) | `PERMISSION_DENIED` / 403 |

Landing sites, all in `skills/objectstack-query`, before (`00a92e18d`)
and after (`b73023a34`):

| Site | Before | After |
|:--|:--|:--|
| `SKILL.md:76`, Removed-key row `query.joins` | "`expand` (display), or
filter the related object and `$in` its ids" | "`expand` (display), or
`{ relation: { field: value } }` in `where` (filter)" |
| `SKILL.md:189-:191`, "Filtering by a related record" | the refusal
(`INVALID_FILTER` / 400) and the two-step route | served in `where`,
read as the caller; the five limits by name and the two-step route past
them, by pointer to the rule (now `:189-:192`) |
| `SKILL.md:331`, the `search` paragraph's last sentence | "To *filter*
by a related record's column, `$in` ids from its own query" | "`where: {
relation: { column: value } }`" (now `:332`) |
| `SKILL.md:340`, Cross-Object row "Filter rows by their lookup target's
column" | query the target, then `{ lookup: { $in: ids } }` | the served
form in `where` up to 1000 related ids; past the cap the two-step route,
`$contains` per id when `multiple` (now `:341`) |
| `rules/filters.md:98-:115`, `## Relation Filters` | the refusal, one
two-step example, the multi-valued spelling, the reverse direction | the
served form with one ✅ example, one limits paragraph, the two-step route
past the cap, the multi-valued spelling and the reverse direction
unchanged |

Re-read against the code and left as they are: `SKILL.md:88` (the rules
index, "filtering by a related record"); `SKILL.md:326-:327` and `:342`
("`search` never traverses": `searchFields` names are judged exactly at
the ingress, `protocol.ts` "Names are judged EXACTLY (no dotted-head
tolerance)", so a dotted name is still refused and the mirror-field
route still holds); `SKILL.md:341` "Filter parent by child conditions"
(the reverse form, not served, stays two-step). The same-day rows `:26`
and `:38-:44` (PR objectstack-ai#20776) are untouched.

**Census.** `grep -rn -i -E 'nested.relation|two steps|two-step|\$in its
ids|never traverses|relation.*INVALID_FILTER|filter the related object'
skills/` at `00a92e18d` finds the sites above plus
`evals/filters-pagination-search.json:37` (a `search` case: the mirror
field, still true) and, outside this skill,
`objectstack-ui/rules/list-views.md:235` and
`objectstack-data/SKILL.md:120` (searching by a related record's title —
still true). No other sentence in `skills/**` states the refusal or the
two-step route as the only way. The evals carry no relation-filter case,
so nothing there asserts the refusal; no block in the skill carries an
`os:check` marker, so `check:skill-examples` does not apply.

## Sentences for objectstack-ai#20876 to quote

`rules/filters.md` `## Relation Filters`, verbatim at `b73023a34`:

A condition on a related record's fields beneath a relation field
(`lookup`, `master_detail`, `user`, `tree`) is served in `where`: the
engine reads the related object with it **as the caller**, then matches
the field against the ids it returns (`$in`; any member when `multiple:
true`).

```typescript
// ✅ Orders whose customer is in the US
where: { customer: { country: 'US' } }
```

Limits — one level: every key a field the related object declares, no
relation or dotted key inside; forward only: never a parent by its
children; `where` only: an aggregation's `filter` and `having` refuse
it, `INVALID_FILTER` / 400; at most 1000 related ids, refused past that,
`INVALID_FILTER` / 400, never truncated; as the caller: the related
object's row scope and field permissions apply, so a field the caller
cannot read is refused, `PERMISSION_DENIED` / 403, never an empty
result. Past the cap, run the two steps yourself — filter the related
object, then `$in` its ids:

```typescript
const us = await engine.find('customer', { where: { country: 'US' }, fields: ['id'] });
where: { customer: { $in: us.map((c) => c.id) } }
```

On a `multiple: true` lookup match each id with `$contains` (an `$or` of
those for several); the SQL driver refuses `$in` there. A parent by its
children's fields is the same two steps reversed: query the child with
the condition and `fields: [the lookup]`, then `{ id: { $in: … } }` on
the parent.

These sentences agree with the first landed wording of the fact —
`content/docs/kernel/contracts/data-engine.mdx` (PR objectstack-ai#20872), the
`FilterCondition` docblock form 4, and the changesets
`20802-nested-relation-filter-served.md` /
`20802-dotted-relation-route.md` / `20802-nested-relation-prose.md` —
each read against the code; none contradicts it, and the skill
contradicts none of them. The `where` dotted path (`{
'account.industry': 'tech' }`) stays refused `INVALID_FIELD` / 400 and
its words now name the nested form; the skill teaches no dotted `where`
path, so no sentence changes for it.

## Paying for it inside `rules/filters.md`

The file sat at 2149 / 2149 tokens (headroom 0). The section rewrite
(881 → 1463 bytes) is paid for by deleting the `## Logical Operators`
section (579 bytes, 41 lines), whose two rules already have a home in
the same package; the file lands at 2148 / 2149, ceiling untouched.

| Deleted from `rules/filters.md` | Home |
|:--|:--|
| `### AND (implicit)` — sibling keys are AND-combined; explicit `$and`
| `SKILL.md` Logical Operators (`:173-:181`, the `$and` example) and
this file's first Common Mistake ("sibling keys always are" AND) |
| `### OR` — `$or`, and `$in` as the one-field equivalent | `SKILL.md`
Logical Operators (`:170-:171`, the `$or` example) and `$in` (`:123`,
`:128`); this file's Operator Reference `$in` row |

The Common Mistake's pointer "(see **Logical Operators** above)" now
reads "(SKILL.md, **Logical Operators**)". The five `role: …` literals
in the deleted examples moved `check:role-word`'s count for the file 7 →
2; the gate prescribes the ratchet-down ("run `--update` and commit the
baseline"), and commit `b73023a34` is that `--update` output: one row of
`scripts/role-word-baseline.json`. That file is outside the claim's
declared surface; declared here and in the report.

## `skills/**` readings (lines and tokens; tokens are the ratchet's
`ceil(utf8 bytes / 4)`)

| | Before (`00a92e18d`) | After (`b73023a34`) | Δ |
|:--|--:|--:|--:|
| `SKILL.md` | 399 lines · 4055 tokens (ceiling 5552) | 400 lines · 4109
tokens | +1 line · +54 tokens |
| `rules/filters.md` | 214 lines · 2149 tokens (ceiling 2149) | 185
lines · 2148 tokens | −29 lines · −1 token |
| Whole package `skills/objectstack-query` (6 files) | 1145 lines ·
10527 tokens | 1117 lines · 10580 tokens | −28 lines · +53 tokens |
| Whole catalog `skills/**` (every file) | 13375 lines | 13347 lines |
−28 lines |

Line budget (PM-set, net +4 at most): −28. No untouched line was
re-wrapped; no ceiling row moved.

**Changeset.** `skills/**` ships in no package's `files[]`: of the 82
tracked `package.json` files, 69 declare `files[]` and 0 entries name
`skills` (measured at `b73023a34`); the catalog ships from `main` via
`npx skills add`. Nothing published moves, so `skip-changeset` is the
seat's to apply; this PR writes no label.

## Verification

Gates at head `b73023a34`, each exit code captured by redirect and the
verdict line quoted from its log: `node scripts/pm/dispatch-gates.mjs
--commands --repo objectstack-ai/objectstack` (no paths; "change set
derived from git — 3 path(s) vs merge base 00a92e1") derives 31
commands, six families more than the dispatch's list because the
baseline row adds `scripts/**`; reconciled with `--ran`: "31 derived
famil(ies) accounted for — 31 run, 0 NOT-MEASURED (a DERIVED zero — all
31 recorded an exit code and none of them is 3)". Every one exit 0:
`check-skills-token-ratchet` ("54 authored bundle file(s) within their
ceilings") and its `--self-test` (65 cases), `check:role-word` after the
ratchet-down ("Ledger: 44 baselined file(s) still carrying it (117
occurrence(s))"), `check:corpus-claim-drift`,
`check:skill-identifier-liveness` ("Leg 1: 457 citation(s) over 53
published file(s)"), `check:doc-authoring`, `check:nul-bytes`,
`check:skill-compatibility`, `check:skill-frame-sync`,
`check:agent-test-spelling`, `check:cross-package-test-inputs`,
`check:driver-memory-census`, `check:gitlink-declared`,
`check:pm-governed-merges` (the `--self-test`, as the script spells it),
`check:refd-timer-probe`, `check:watch-hint-literal`,
`check-ci-filter-parity` (+ `--self-test`),
`check-closing-keyword-parity` (+ `--self-test`),
`check-comment-mask-corpus` ("7664 files, 0 disagree"),
`check-doc-route-spelling --advisory` (+ `--self-test`),
`check-scripts-symbol-anchors` (+ `--self-test`), `check:bash32-floor`,
`check:cli-command-ids`, `check:entry-guard`, `check:parse-guard`,
`check:pnpm-filter-targets`, `@objectstack/lint`
`check:doc-formula-expressions` ("22 record-scoped formula example(s)
across 458 files / 1380 TS blocks judged clean", after building
`@objectstack/lint...` under `os-verify-lock.sh`, `VERDICT command-exit
0`), spec `check:skill-docs` ("Skill docs in sync") and spec
`check:skill-refs` ("9 generated files in sync"). The two ratchet
families were re-run after the final commit at `b73023a34` (both exit 0,
verdicts as quoted).

Pins of the served form, run first-hand under `os-verify-lock.sh` after
building each package's dependency closure (`VERDICT command-exit 0` on
both): `pnpm --filter @objectstack/objectql exec vitest run
--maxWorkers=2 src/engine-nested-relation-lowering.test.ts` → "Test
Files 1 passed (1), Tests 13 passed (13)" (the served rows on every
relation type, the any-member `$or` of `$contains`, the cap refusal with
the two-step route, the one-level / dotted / undeclared refusals, the
`aggregations[i].filter` and `having` refusals naming `where`, the
caller's context on the related read); `pnpm --filter @objectstack/rest
exec vitest run --maxWorkers=2
src/data-nested-relation-permission.test.ts
src/data-nested-object-door.test.ts` → "Test Files 2 passed (2), Tests 9
passed | 12 skipped (21)" — the 12 are the PostgreSQL and MySQL cells of
the door test, named skips (`OS_TEST_POSTGRES_URL` / `OS_TEST_MYSQL_URL`
unset here); the memory and SQLite cells and the 403 pin ran. No package
is touched, so no package build or test suite is owed beyond that.

## Acceptance notes

- `SKILL.md`'s `compatibility: Requires @objectstack/spec 17.x` is left
as is: the catalog states `main`'s behaviour (as PR objectstack-ai#20811 did) and
`main` is 17.5.0 with the served form; the published 17.5.0 engine still
refuses it, and which release carries `ca5408c62` is the ruling's
sequencing, not this card's. Noted, not filed.
- The skill says nothing about analytics (the cube read and the read
scope): objectstack-ai#20887 is open, and the sentences above name `where` on the
engine and the data door only.
- `content/docs/protocol/objectql/query-syntax.mdx` is read-only here;
objectstack-ai#20876 carries it and quotes the section above.
- The report's `api_writes` lists every relay write of this run.

## 维护者速读(草稿)

**改了什么。** 发布的查询技能包 `skills/objectstack-query` 今天早些时候(PR
objectstack-ai#20811)被改成「引擎拒绝在关联字段下直接写条件」;同一天晚些时候引擎侧 PR objectstack-ai#20872 落地,引擎已经支持这种写法(`where: {
customer: { country: 'US' } }`)。本 PR
把同样的几处句子改成引擎现在的真实行为:支持,以及四条边界(只到一层、只能正向、只在 `where` 里、关联 id 最多 1000
条超出即拒绝),并且以调用者身份读关联对象(读不到的字段报 403,不会静默给空结果);超过上限或反向(用子记录条件筛父记录)仍走原来的两步
`$in`。规则文件里删掉了与 SKILL.md 重复的一节逻辑运算符示例,用来支付这段改写;整个包净减 28 行,每个文件的 token
上限都没动。

**为什么改。** 技能包是客户项目里 AI 的教材。裁决 5907789183(「20802 同意」)明确写了技能包在同一轮更新;不改,AI
作者会照着旧句子手写两步查询,正是裁决要去掉的模式。措辞与已落地的文档页(`data-engine.mdx`)、三份 changeset 和
`FilterCondition` 的注释逐句核对过,一致;objectstack-ai#20876 改 `query-syntax.mdx` 时引用本 PR
的句子,两处不会分叉。

**风险与代价(含回滚)。** 只改两份 markdown 与一行门禁基线(`role-word` 计数 7 →
2,门禁自己要求的下调),不改任何代码或发布包。风险在两点:一是措辞若与后续分析(objectstack-ai#20887)落地后的行为有出入,分析那一半另改;二是已发布的
17.5.0 引擎仍拒绝这种写法,技能包描述的是 `main`。回滚即 revert 本 PR 的两个提交。

**席位意见。**

**你要做的。** 复核五条边界的措辞与「逻辑运算符」一节的删除归宿;批准后由席位落地(Tier H)。

---
_Generated by [Claude
Code](https://claude.ai/code/session_01KTZmMfzVzjNvyaLyQ8mHvg)_

---------

Co-authored-by: objectstack-fleet[bot] <332303061+objectstack-fleet[bot]@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
akarma-synetal pushed a commit to akarma-synetal/framework that referenced this pull request Oct 7, 2026
… answer on every analytics face — the related object read as the caller, capped (objectstack-ai#20887) (objectstack-ai#20916)

Fixes objectstack-ai#20887
Clause-②: yes (narrowing)

The analytics half of ruling 5907789183, whose parent card is objectstack-ai#20802
(its engine half landed as objectstack-ai#20872, `ca5408c62`). The nested-relation
filter `{ relation: { field: value } }` now gets ONE answer on every
analytics face, and it is the engine's: the related object read as the
caller (its row scope and field permissions), capped at
`RELATION_FILTER_ID_CAP`, a multi-valued relation matching on any
member. The analytics layer holds no copy of that rule. The native-SQL
strategy declines a query carrying the form, and the engine-aggregate
strategy hands the form to the engine as written.

## Per face

The engine's answer for the same filter, computed in the same test over
the same rows, is the reference for every cell. Fixture: a ledger with
`owner` (lookup) and `owners` (multiple lookup) to an owner object; the
member cannot read `owner.secret`, and its row scope hides owners in
region `HIDDEN`. Past the cap means 1,001 matching owners. Measured with
the real `SecurityPlugin`, `ObjectQL` and `SqlDriver` (SQLite).

| face | strategy | engine rows equal | caller permissions | cap |
|---|---|---|---|---|
| cube read, `POST /api/v1/analytics/query` (`AnalyticsService.query`) |
NativeSQL composition (declines, so the engine answers) | yes: single,
multi, related row scope, `$not`, `$or` (was 500 `DATABASE_ERROR`: the
join named a table `owner` that does not exist) | 403
`PERMISSION_DENIED`, as the engine (was 500) | 400 `INVALID_FILTER`, as
the engine (was 500) |
| cube read | ObjectQL | yes (was 400 `INVALID_FIELD`, "cannot evaluate
a cross-object filter") | 403 (was 400) | 400 (was 400 cross-object) |
| dataset door, `POST /api/v1/analytics/dataset/query`, the dataset
`include`s `owner` | NativeSQL composition | yes (was: single-valued
rows via the JOIN; multi-valued 400 `DATASET_INVALID`; under `$not` the
member got `b` where the engine answers `b, d`) | 403 (was **200 with
rows a, c**: filtered by a field the caller cannot read) | 400 (was 200
with no rows) |
| dataset door | ObjectQL | yes (was 400) | 403 (was 400) | 400 (was
400) |
| a measure's own `filter` carrying the form | both | refused 400
`INVALID_FILTER`, as the engine refuses it at an aggregation's `filter`
(was: native counted it through the JOIN; ObjectQL 400 `INVALID_FIELD`)
| n/a | n/a |
| SQL echo, `POST /api/v1/analytics/sql` | both | refused 400
`INVALID_FILTER`, naming the served route (was: native printed the JOIN;
ObjectQL 400) | n/a | n/a |
| a read scope carrying the form (a host `getReadScope`) | NativeSQL |
refused 500 `READ_SCOPE_COMPILE_FAILED`, policy withheld, words now
naming the route (outcome unchanged) | n/a | n/a |
| a read scope carrying the form | ObjectQL | served, the engine's rows
(unchanged: the scope reaches the engine as written) | as the caller |
the engine's |

## Mechanism assumptions, measured

- **B1 held.** The engine's answer for the fixture, as the member: `{
owner: { region: 'NA' } }` is d1, d3; the multi-valued form is d1, d3;
`{ owner: { secret: 's1' } }` is 403 `PERMISSION_DENIED` naming `secret`
(a system caller gets d1, d3); region `HIDDEN` gives no rows (a system
caller gets d4); past the cap is 400 `INVALID_FILTER` for both
spellings; `$not` gives d2, d4; the `$or` gives d1, d2, d3; `{ owner: {}
}` and a second level are 400. `RELATION_FILTER_ID_CAP` is exported
(`packages/objectql/src/index.ts:148`), and nothing here imports it: the
analytics layer never counts ids, the engine does.
- **B2: no on every axis**, per the table. The native path joined the
related table itself: the related row scope rode in as a `WHERE`
conjunct, the field permissions did not, nothing bounded the match, and
a multi-valued relation or an undeclared join failed. The ObjectQL path
refused the form outright.
- **B3: call the engine.** `@objectstack/objectql` exports only the cap.
The lowering (`admitRelationCondition`, `lowerRelationSite`) is
module-internal, and this package has `@objectstack/objectql` as a dev
dependency only. The route that needs no export: the engine-aggregate
strategy hands the condition to `engine.aggregate` through
`executeAggregate`, with the caller's context. No export was needed, and
there is no second permission rule.
- **B4: the read scope keeps its refusal, and the words name the served
route.** `compileScopedFilterToSql` is a synchronous string builder. It
holds the caller's `ExecutionContext` for placeholders only, and no data
engine, so its compile cannot run the inner read as the caller. Routing
a read scope carrying the form to the engine instead was built and
measured, then withdrawn: on a native-only host it traded the declared
`READ_SCOPE_COMPILE_FAILED` (policy withheld) for a generic no-strategy
fault (`packages/rest/src/analytics-read-scope-refusal-envelope.test.ts`
went red). No in-repo producer emits the form in a scope: the RLS
compiler refuses a relation traversal when it compiles the policy. On
the ObjectQL path the scope reaches the engine as before, and the engine
serves it as the caller.
- **B5: `yes (narrowing)`.** Widening: the cube read (both strategies),
the ObjectQL dataset door, a multi-valued relation and a dataset without
the declared join on the native path, and a dataset's own `filter` on
the ObjectQL path all now serve the form (they answered 500 or 400).
Narrowing: on the native path the dataset door now refuses a condition
on a related field the caller cannot read (was rows), a match past the
cap (was an empty 200), and a measure filter carrying the form (was a
count). The SQL echo refuses the form. And a query combining the form
with something only the native strategy serves (a cross-object measure,
a multi-hop dimension) is refused by the engine-aggregate path.
`@objectstack/service-analytics` ships `minor` with the BREAKING banner
and an ADR-0087 `not-required (no-migration-prescription)` disposition;
`check-adr-0087-registration` and `check-changeset-no-major` pass.
- **B6: no page to update.** No hand-written `content/docs/**` page
states how the analytics read or the read scope treats the nested form.
`data-engine.mdx`, and `query-syntax.mdx` (objectstack-ai#20906, which landed during
this work), describe the engine only.

## What changed

- `strategies/filter-normalizer.ts`: a nested-relation condition becomes
a `relation` node carrying the condition as written. It is no longer
flattened to the dotted member. `shieldNestedRelations` holds it out of
the shared lowering, because under `$not` the lowering guarded the
relation column, and this package's engine hand-off spells that guard
`$ne: null`, which `driver-sql` refuses over a multi-valued JSON column.
Measured: the multi-valued `$not` pin went red before the shield, and
the engine guards what it lowers the condition to itself.
`findNestedRelationCondition` is the routing detector.
- `strategies/native-sql-strategy.ts`: `canHandle` declines when the
`where`, the dataset's own `filter` or a requested measure's `filter`
carries the form. This is the mechanism of the cross-field decline
(maintainer ruling 2026-08-12, Q1 = B). Its compiler refuses a
`relation` node bare, as routing drift.
- `strategies/objectql-strategy.ts`: the condition goes to the engine as
its own conjunct, under the key the author wrote. The display-SQL echo
declines it.
- `read-scope-sql.ts`: the nested-relation form's refusal has its own
words, naming the route. An empty or mixed value object keeps the old
words.
- `analytics-service.ts`: the no-strategy error names the
nested-relation decline.
- The mixed-wrapper refusal no longer says a nested member "compiles to
the dotted member".

## Pins, red first (`568727629`)

- `packages/rest/src/analytics-nested-relation-filter.test.ts`: both
compositions, the cube read and the dataset door through its route,
against the engine's answer. It was red 10 of 10 on the base, and is 10
of 10 green now.
-
`packages/services/service-analytics/src/__tests__/nested-relation-engine-handoff.test.ts`:
the native decline per producer, the ObjectQL hand-off as written, the
compile backstop and the read-scope words. It was 7 red with 2 controls
green on the base, and is 9 of 9 green now.

## Ablations, predicted before running, at `5bb764181`

Each ablation mutated the committed file through
`scripts/ablation-replace.mjs` (anchor hit once, blob moved), rebuilt
`@objectstack/service-analytics`, and passed `ablation-dist-preflight`
(the marker present in 2 built files). It then ran both pin files, plus
`where-door-shared-lowering-seam.test.ts` in the unit run. The restore
leg proved the blob equal to HEAD and `git diff HEAD` empty, rebuilt,
and found the marker absent from all 6 built files. Every observed count
equals its prediction.

| ablation | face it guards | unit (27) | route pins (10) |
|---|---|---|---|
| A1 the native decline removed | cube read and dataset door, native | 4
red | 4 red (native: rows, refusals, measure filter, echo) |
| A2 the hand-off drops the condition | both strategies' rows,
permission, cap | 2 red | 6 red |
| A3 the aggregate call forwards no caller context | caller permissions
| 0 | 4 red: the member then saw `d` (a hidden owner's row), and the
unreadable field answered rows |
| A4 the read-scope route words | read scope, native | 1 red | 1 red |
| A6 the lowering shield removed | multi-valued `$not` | 1 red | 2 red |
| A7 the echo refusal removed | SQL echo | 0 | 2 red |

A first round at `dca1af7cb` matched its own predictions too, including
A5, the read-scope decline arm, which B4's correction removed from the
code.

## Pins re-judged

These pins recorded the flattening this change removes, so each was
re-judged:

- respelled to the dotted cube member where the pin was about the
traversal: `filter-normalizer-not-null-safe`,
`icontains-text-comparand-refusal`;
- re-expected as a `relation` node where the pin was about acceptance:
`where-equality-slot-list-refusal`, `where-face-arms-refusal`,
`where-type-face-refusal`, `filter-normalizer-mixed-wrapper`'s
pure-shape block;
- replaced where the pin held the removed branch: mixed-wrapper row 6
and its `guardFieldEntry` recursion row (the engine refuses that inner
wrapper, `INVALID_FILTER` / 400, measured),
`where-door-shared-lowering-seam`, `infer-cube-relation-traversal`,
`infer-cube-where-spelling-parity`, and `where-source-field-gate`, which
now judges the relation field `owner` as a column of the queried object;
- re-worded to the read-scope refusal's new words: `read-scope-sql`,
`read-scope-not-null-safe`, `read-scope-undefined-comparand`, and
`read-scope-refusal-envelope`, which gains row 17 because the
nested-relation form now has a throw site of its own.

## Verification

- `@objectstack/service-analytics`: `test` 146 files, 3334 passed;
`typecheck` exit 0, with 146 of 146 test files in the tsc program
(`--listFiles`). Both at `4d383dac0`, after merging `main`.
- `@objectstack/rest`: the full `local` project, 239 files, 4662 passed
and 106 skipped, at `5bb764181`. The merge brought no rest or analytics
change. At `4d383dac0`, the new pin, the read-scope envelope pin and the
engine half's permission pin: 3 files, 21 passed. `typecheck` passes,
including the test layer (`check:test-typecheck` OK).
- Consumer sweep, narrowed to the files that load this package:
`@objectstack/runtime` `analytics-*` plus
`cross-field-refusal-operand-withhold`, 5 files, 38 passed and 4
skipped; `@objectstack/client` `analytics-automation-json-erasure`, 7
passed.
- Gates at `4d383dac0`: `dispatch-gates --commands` derived 62. All 62
were run, plus the 4 roster families (`check-changeset-fixed`,
`check:authz-resolver`, `check:error-code-casing`,
`check:filter-alias-parity`), all exit 0. `dispatch-gates --ran`: 62
derived, 62 run, 0 NOT-MEASURED, 0 UNRUN. `check:dual-build-cjs-loads`
and `check:type-check-debt` first exited 3 (prerequisite not met) and
were re-run green after `turbo run build` over `./packages/*`.
- Lint, narrowed and proved at `4d383dac0`. The population is the 21
changed `.ts` files, none ignored by eslint's own config
(`isPathIgnored` false for all 21). `eslint --no-inline-config --format
json` over them gives 21 files, 0 errors, 0 warnings.
`parserOptions.project` and `projectService` are unset for all 21, so no
type-aware lint runs and no untouched file's verdict can move.
- NOT MEASURED: a live PostgreSQL cell (the dialect axis of the lowering
is the engine's, pinned by objectstack-ai#20872's `data-nested-object-door.test.ts`;
this file's axis is the analytics faces), the dogfood and integration
lanes, and the whole-workspace typecheck. All are left to CI.

## Acceptance notes

- The ObjectQL path's cross-object refusal still says "Run this query on
a native-SQL driver". A query the form routed away from the native
strategy can meet those words on a SQL deployment.
- The engine's cap refusal names the position the engine received. The
strategy ANDs the condition in, so the words read `where.$and[0].owner`
where the author wrote `where.owner`.
- `packages/types/src/error-leak.test.ts` keeps a hand-written stand-in
of the read-scope refusal shapes. Its nested-relation line is the old
wording. It is a heuristic fixture, not a pin of this module, and it
stays green.
- `MemoryAnalyticsService` (driver-memory's cube face, objectstack-ai#20859's
position) is not touched, and its answer for the form is not measured
here. The draft preview refuses the form as an operator it cannot
evaluate, unchanged.

## Patch rounds (the seat's append from the dev's reports `5917211340`,
`5917807320` and `5918233769`; the dev writes a body only once)

### Patch round 1

`Test Core (3/6)` went red on `4d383dac0`, in `packages/client`'s
`envelope-caller-census.test.ts`: 2 of its tests failed. Reproduced
locally: the client suite fails 1 file and 2 tests at `4d383dac0`, and
passes 50 files and 641 tests at the merge base `9509ea106`.

**Root cause.** The census walks the whole workspace for call sites of
`analytics.query(` and requires a hand-ledger row for each one. This
PR's new pin,
`packages/rest/src/analytics-nested-relation-filter.test.ts`, calls the
real `AnalyticsService`'s `analytics.query` five times. Those are
producer reads, the census's `NOT_SDK` class, and the ledger has no row
for them. The failing assertions are the §3 key comparison (the one
extra key is that file, `analytics.query`, `service`, 5) and the §2
producer-receiver count (1 expected, 6 found).

**What it is not.** It is not a product defect. It is not a client pin
of the nested-relation form or of the read-scope wording either.

**The fix, pending the seat.** It lives in
`packages/client/src/envelope-caller-census.test.ts`, which is outside
this card's claim surface. It adds one `NOT_SDK` ledger row with a count
of 5, and moves the two producer-read counts from 1 to 6. Measured on a
scratch copy of that file: 20 of 20 tests passed. The copy was restored
byte-identical, and nothing was committed.

### Patch round 2

The seat authorised the census remedy, round 1's option A, for one file:
`packages/client/src/envelope-caller-census.test.ts`.

- **main moved.** `975b2481c` (objectstack-ai#20808) touched
`packages/services/service-analytics`, so `origin/main` was merged into
the branch as `1fdaff7e5` (no rebase). The merge was clean, with no
regeneration pending.
- **The census change is its own commit, `7eb2ecf20`.**
- It adds one `NOT_SDK` ledger row for
`packages/rest/src/analytics-nested-relation-filter.test.ts`
(`analytics.query`, `service`, count 5).
- The producer-receiver count goes from 1 to 6, and the set of two files
is asserted.
- `verdictTotal('NOT_SDK')` goes from 1 to 6, and its test title changes
with it.
  - Nothing else in that file changed.
- **Measured at `7eb2ecf20`.** Each exit code was captured before any
pipe.
- `pnpm --filter @objectstack/client test`: exit 0, 50 files and 641
tests passed. The census file run alone passed 20 of 20.
- `pnpm --filter @objectstack/client typecheck`: exit 0. `tsc --noEmit`
passed, and `check:test-typecheck` answered OK.
- `@objectstack/service-analytics` test: exit 0, 146 files and 3335
tests passed. Its typecheck: exit 0.
- The three rest files (`analytics-nested-relation-filter`,
`analytics-read-scope-refusal-envelope`,
`data-nested-relation-permission`): exit 0, 3 files and 21 tests passed.
- ESLint over the 22 changed `.ts` files: 0 errors and 0 warnings. The
config ignores none of them and lints none type-aware, so this diff
cannot move a verdict on an untouched file.
- Gates, re-derived: 63 derived and 63 run, 0 not measured, plus the 4
roster families. All exit 0 except one.
- **The one red is `check:cross-package-test-inputs`.**
- Cause: the new ledger row spells the rest pin's path as a literal, and
`@objectstack/client`'s declared cross-package input globs do not cover
it. The gate is green at `1fdaff7e5`, the commit before.
- The gate's own remedy: declare that one file in
`scripts/cross-package-test-inputs.mjs`, and mirror it in `turbo.json`'s
`@objectstack/client#test` inputs.
- Measured on the working tree: that gate and `check-ci-filter-parity`
both exit 0. The two files were then restored byte-identical.
- Both files lie outside the authorised surface, so the remedy waits for
the seat.

### Patch round 3

The seat authorised the gate's own remedy for
`check:cross-package-test-inputs`, in two files.

- **main.** No commit since `975b2481c` touched this card's surface, the
census or either of the two files, so there was no merge. The commits
checked were `def279a39`, `4d0b9cd54` and `d78a0bda0`.
- **The declaration is its own commit, `2881f478c`.**
- `scripts/cross-package-test-inputs.mjs`: in `@objectstack/client`'s
entry, one per-file glob,
`packages/rest/src/analytics-nested-relation-filter.test.ts`, with a
3-line comment.
- `turbo.json`:
`$TURBO_ROOT$/packages/rest/src/analytics-nested-relation-filter.test.ts`
in `@objectstack/client#test`'s inputs. The line before it gains the
comma JSON requires.
  - Nothing else changed in either file.
- **Measured at `2881f478c`.**
- Gates, re-derived: 81 derived and 81 run, 0 not measured. Also run:
the 11 roster families whose roster lies under a path this diff touches,
`check-ci-filter-parity --self-test` and `check:select-shard-packages`.
All 93 commands exit 0.
- `check:cross-package-test-inputs` (with `--self-test`) is green: "OK:
29 package(s) read outside themselves, all declared".
- `check-ci-filter-parity` is green: "all 188 declared cross-package
glob(s) (135 unique) are covered".
    - `check:turbo-task-graph` is green.
- `pnpm --filter @objectstack/client test`, as the control: exit 0, 50
files and 641 tests passed, the same as at `7eb2ecf20`. The declaration
moved no verdict.
- Layer A at work: `--union-into`, given a diff of the rest pin alone,
now pulls `@objectstack/client` into the run (8 packages). At
`7eb2ecf20` it did not (7 packages). No other package changed.
- Turbo hashes, from `--dry=json` before and after, over build, test,
test:repo and typecheck (303 tasks):
- The global hash is unchanged, and no build or typecheck hash moved.
- 7 test hashes moved. `@objectstack/client#test` moved through
`turbo.json`: its task definition changed, and the rest pin is a new
input.
- The other 6 moved only because the content of
`scripts/cross-package-test-inputs.mjs`, an input they declare, changed.
They are `cli#test`, `plugin-auth#test`, `vitest-filter-preflight#test`,
`objectql#test:repo`, `runtime#test:repo` and `spec#test:repo`.
- ESLint over the 23 changed `.ts` and `.mjs` files: 0 errors and 0
warnings. The config ignores none of them and lints none type-aware.

### Patch round 4 (the seat's append from the dev's report `5922062971`)

`main` was merged (no rebase) to take in four landings in
`service-analytics`:

- PR objectstack-ai#20931: the field-read gate at the door.
- PR objectstack-ai#20955: the queryable-field gate.
- PR objectstack-ai#20954: `plugin-security`'s comparand guard.
- PR objectstack-ai#20962: relationship-path objects in the admitted and scoped set.

**The merge, `6b6bffb3e`.** It is clean at the text level, in
`analytics-service.ts` and in `native-sql-strategy.ts`. Every line
either side added is present in the merged files, checked line by line.

**What the landed gate and object set do with the `relation` node.**
This was measured on the merged tree, in the shipped composition (the
real `SecurityPlugin` over `ObjectQL` on SQLite), under both strategies.

- **The gate judges the relation field.** `collectFilterLeaves` yields
the nested form's relation field as its member: `{ owner: { region: 'NA'
} }` gives `owner`, with operator `relation`. It does the same under
`$not` and inside `$or`. So the field gate judges the relation field on
the base object.
- A caller who may not read `owner` is refused by the gate: 403
`PERMISSION_DENIED`, in the engine's own words, with no engine call
made.
- **The related object does not enter `queryObjects`.** The security
service is asked only about the base object.
- **The engine guards the related object instead.** The nested form is
served on the ObjectQL path, where the engine reads the related object
as the caller. Each of these is refused with the same code, status and
words as `engine.find`, and never answered:
  - a related field the caller cannot read: 403;
  - a masked related field (objectstack-ai#20935): 403;
  - a related object the caller cannot read at all: 403.
- **Before this branch, the answer was a refusal.** On `main` alone,
even a readable nested condition was refused 403, "reading "owner" is
not permitted", because the flattened `owner.region` named the relation
field as an object to admit. With this branch, the answer is the
engine's.

**Measured at `6b6bffb3e`.** Every run was under the shared lock, with
each exit code captured before any pipe.

- `@objectstack/service-analytics`: tests exit 0 (149 files, 3434
tests), and typecheck exits 0.
- The five rest route pins pass 61 of 61:
  - `analytics-nested-relation-filter`: 10
  - `analytics-read-scope-refusal-envelope`: 8
  - `data-nested-relation-permission`: 3
  - `analytics-field-permission-gate`: 12
  - `analytics-relationship-path-admission`: 28
- The client census passes 20 of 20.
- Gates, re-derived and run as one locked sequential script: 81 derived,
81 run, 0 not measured. Also run: the 11 roster families and 2 extras.
All 93 commands exit 0.
- ESLint over the 23 changed `.ts` and `.mjs` files: 0 errors and 0
warnings.

**Acceptance notes.**

- **A dotted path on an inferred cube is still refused.** `{
'owner.region': 'NA' }` is refused 403, "reading "owner" is not
permitted". The hop object is taken from the alias, because an inferred
cube declares no join. This is the same on `origin/main`, and it is
outside this card; it went to the seat as a finding.
- **A host read scope does not reach the related object.** The related
object is not in `queryObjects`, so a host-supplied `getReadScope` is
not asked about it. In the shipped composition that provider is the
security service's `getReadFilter`, the same row scope the engine
applies when it reads the related object as the caller. That case is
pinned in `analytics-nested-relation-filter` ("the related row scope").

---
_Generated by [Claude
Code](https://claude.ai/code/session_01XY5uCwTjZj7884yYtyur4H)_

---------

Co-authored-by: Claude <noreply@anthropic.com>
akarma-synetal pushed a commit to akarma-synetal/framework that referenced this pull request Oct 7, 2026
…is membership, on every face (objectstack-ai#20984)

Fixes objectstack-ai#20874
Clause-②: yes (widening)

## What changes

`driver-memory` now answers `$contains` and `$notContains` on a
**declared JSON-stored field** by whole-element membership, the reading
`driver-sql` compiles on all three dialects. A field counts as
JSON-stored when it is `multiple: true`, a `multiselect` / `checkboxes`
/ `tags` field, or a `STRUCTURED_JSON_TYPES` member such as `json`. A
scalar text column keeps the case-exact substring test. The change
covers every face of the package:

- the query path's `$`-spelling (`normalizeFieldOperators`) and its AST
spelling (`convertConditionToMongo`): `find`, `count`, and every verb
that goes through `convertToMongoQuery`;
- the analytics (cube) face's mingo `$match`;
- the analytics face's SQL echo (`generateSql`), which now renders
SQLite's `json_each` membership construct for such a column, so the
echoed statement still returns the rows the chart was drawn from.

One rule backs all of them: `InMemoryDriver.filterContainsTest(object,
field, value)`. On a JSON-stored field it returns `{ $elemMatch: { $in:
members, $not: { $type: 'array' } } }`, and on any other field `{
$regex: filterSubstringPattern(value) }`. `$notContains` is `$not` over
that test, on both faces. The members come from the comparand's text,
read the way `driver-sql`'s `jsonMembershipCandidates` reads it: `'1'`
names the string or the number 1, `'1.50'` names 1.5, and `'true'` /
`'false'` / `'null'` name the string or the JSON literal.

The `$contains` docblock in `packages/spec/src/data/filter.zod.ts` lost
its stale "driver-memory DOES NOT ANSWER IT YET" bullet. Its dead
tracker link went with it (that card answers 404). The replacement text
states the measured status, and states that the other text operators
over a stored array are not ruled by that section.

## Measured on `origin/main` `f6ccca4a` before the change, and after
(HEAD `10656601`)

Fixture: `driver-sql`'s objectstack-ai#17590 fixture plus a multi-valued lookup
`owners` (`['u1','u2']`, `['u10']`, `['u3','u1']`, `[]`).

| `where` | memory before | SQLite (`driver-sql`) | memory after |
|:--|:--|:--|:--|
| `{ owners: { $contains: 'u1' } }` | 1, **2**, 3 | 1, 3 | 1, 3 |
| `{ owners: { $notContains: 'u1' } }` | 4 (**2 dropped**) | 2, 4 | 2, 4
|
| `{ tags_: { $contains: 'red' } }` | 1, **2** | 1 | 1 |
| `{ nums: { $contains: '1' } }` (members are numbers) | **none** | 1 |
1 |
| `{ label: { $contains: 'red' } }` (scalar control) | 1, 2 | 1, 2 | 1,
2 |
| the cube face, the same filters | the query path's rows | n/a | the
query path's rows |

`engine.find` on a real `InMemoryDriver` with the nested-relation filter
`{ owners: { region: 'NA' } }` (owner `u1` NA, `u10` EU; this is PR
objectstack-ai#20872's lowering): `d1, d3, d5` before and `d1, d3` after. Its `$not`
gave `d2, d4` before and `d2, d4, d5` after, which matches the rest
suite's SQLite rows. That was a one-off run (a throwaway probe file, not
committed): this package may not import the engine and the engine's
packages may not import this driver (`check:driver-memory-census`). What
is pinned instead is the driver input the engine sends, an `$or` of one
`$contains` per related id, which objectql's
`engine-nested-relation-lowering.test.ts` asserts.

## Mechanism hypotheses (order Zone 2)

- **H1 confirmed.** The `$contains` arm lowered to an escaped `$regex`,
and mingo applies a `$regex` to each element of an array value.
Reproduced on the fixture above before any edit.
- **H2: the fork is by the DECLARED field, not the row's runtime
shape.** Memory has the declaration (`valueShapes`, recorded by
`syncSchema` since the `$empty` work), and SQL forks on its JSON-column
registry, which is also filled from the declaration. The population is
the spec's JSON-stored classes: `STRUCTURED_JSON_TYPES` plus
`isMultiValueField`, the two halves `driver-sql`'s registry is built
from. The two readings DO give different rows on one fixture, so the
choice goes to the PM in the report's `open_questions`: a declared
`json` field holding the scalar string `'u1'`. SQLite answers no member,
because its constructs are array-only. A per-row-shape fork would answer
it by substring. The declared fork matches SQLite, and the contract text
says the question is "selected by the COLUMN", declared metadata.
Undeclared fields (an object never passed through `syncSchema`) keep the
substring reading, as `SqlDriver.isJsonColumn` answers `false` for a
table it was never told about.
- **H3 confirmed and pinned.** Number members answered nothing; `'1'`,
`'2'`, `'10'`, `'0'` and `'1.50'` now give SQLite's rows.
- **H4 confirmed: `$notContains` diverged.** It is the mirror arm of the
same defect, changed under os-dev rule 3's bounded in-place exemption.
All four conditions hold:
  1. Same defect class.
2. Mechanical, with its shape pinned by SQL's `col IS NULL OR NOT (…)`:
`$not` over the test admits null and missing rows, pinned on both
fixtures.
3. `memory-driver.ts` is held by no other claim; the sibling `$exists`
card is kept off it by its own claim.
  4. Same tests, no new gate.

  **The claim's surface should be amended to include it.**
- **H5 confirmed.** The cube face borrowed `filterSubstringPattern` and
wrapped it in its own `$regex`, so it had the same defect. It now takes
the driver's whole test. The echo reads its member set off that same
test, so the chart and its echo cannot name different sets.
- **H6 confirmed.** See the `engine.find` run above.

## Compile-surface conclusions

| # | face | conclusion |
|---|---|---|
| 1 | `driver-sql` `applyFilterCondition` (`driver-sqlite-wasm`,
`driver-turso` local inherit it) | **already compliant**:
`applyJsonMembership` emits membership on JSON columns. Evidence: the
objectstack-ai#17590 suite now carries a multi-valued lookup column and the `u1`/`u10`
case, green on SQLite here. Its live PostgreSQL/MySQL cells are
unprovisioned locally; the required live job runs the whole driver-sql
suite. The two inheriting drivers were not separately run. |
| 2 | turso `RemoteTransport.buildWhereSQL` | **out of scope** (another
package). By reading, its `$contains` arm emits `pushLike` (a GLOB
substring) on every column, JSON columns included, so remotely `u1`
would match `["u10"]` while the local transport answers membership. Not
measured. Reported as a finding. |
| 3 | service-analytics `read-scope-sql` `compileScopedFilterToSql` |
**out of scope** (another package). **Measured**: `{ owners: {
$contains: 'u1' } }` on a declared `lookup` + `multiple: true` field
compiles to `instr("t"."owners", ?) > 0` on SQLite and `LIKE '%u1%'` on
PostgreSQL/MySQL, a substring over the stored JSON text. This is an RLS
read-scope face, so it is reported as a security-relevant finding. |
| 4 | service-analytics `filter-normalizer` `lowerAnalyticsWhere` |
**out of scope**. By reading, it lowers `$contains` to the cube
`contains` operator, which the native SQL strategy renders as `LIKE`, a
substring. Reported. |
| 5 | `formula` `matchesFilterCondition` | **out of scope**. By reading,
the arm is `typeof actual === 'string' && actual.includes(v)`, so a
stored array never matches (fail-closed). Reported. |
| half | objectql `having-filter` | **out of scope by the claim**
(serial behind another card). Untouched. |
| unfrozen | `driver-memory` query path and cube face
(`memory-analytics.ts`), behind `filter-refusal.ts` | **changed** (this
PR). `filter-refusal.ts` and the `$exists` arm are untouched (sibling
card). |
| unfrozen | `driver-mongodb` `translateFieldOperators` | **out of
scope** (another package). By reading, it lowers `$contains` to a native
`$regex`, which MongoDB applies per array element: the same defect. Not
measured. Reported. |

## Why the shared pin is a mirrored literal table, not
`FILTER_TEXT_CASES` (order Zone 3, not taken)

`FILTER_TEXT_CASES` has no array column, so adding one changes the
fixture of all five enrolled drivers. `driver-mongodb` imports every row
of it and still carries the per-element defect, so its suite would go
red. The table's own rule 2 says rows join a driver's suite in the PR
that ends that driver's gap. Doing it here would also widen the spec
touch beyond the one declared docblock. A new sibling case-set would add
DEBT rows to a ledger that only goes down. So the new memory file
mirrors `driver-sql`'s objectstack-ai#17590 fixture row for row and asserts the same
literal row sets. That is how the objectstack-ai#17590 file already holds its three
dialect cells to one answer.

## Files

- `packages/drivers/driver-memory/src/memory-driver.ts`: the population
(`isJsonStoredField`), the members, the one test (`filterContainsTest`),
both query-path spellings.
- `packages/drivers/driver-memory/src/memory-analytics.ts`: the `$match`
rows take the driver's test; the echo renders membership
(`sqliteMembershipPredicate`).
-
`packages/drivers/driver-memory/src/memory-20874-contains-membership.test.ts`
(new): two fixtures. The first is the objectstack-ai#17590 fixture plus `owners`. The
second holds the stored shapes SQLite decides: a scalar, an object, a
nested array, `[null]`, `[[null]]`, `[true, 1.5]`, `['']` and a NULL
row, each answer measured on `driver-sql`/SQLite first. Every case runs
on `find` in both spellings, on `count`, and on the cube's `$match`. The
cube's echo is EXECUTED on sql.js over the same rows. The file also
covers the nested-relation driver input and the fork.
-
`packages/drivers/driver-sql/src/sql-driver-17590-json-column-membership.test.ts`:
an `owners` multi-valued lookup column and the `u1`/`u10` case (the
claim's SQLite pin).
- `packages/spec/src/data/filter.zod.ts`: the one declared docblock.
- `.changeset/20874-memory-contains-membership.md`: **`minor`**, not the
`patch` the order suggested. `filterContainsTest` is a new public method
on the exported `InMemoryDriver`. It ships in `dist/index.d.ts` (the
existing `filterSubstringPattern` appears there as the positive
control), and the level ruling quoted in `pr-automation.yml` (WHICH
LEVEL) grades an additive widening of a published package's public
surface at least `minor`. `Clause-②: yes (widening)`: the new public
method widens the published surface; no filter key or operator is added.
- **Sweep, beyond the claim's listed surface.** The order's pin sweep
asks for every same-semantics statement to be flipped in one round. Each
item below is an edit to a statement this change makes false:
- `packages/rest/src/data-nested-object-door.test.ts` (a header comment
that cited the gap and the removed docblock clause);
- `.changeset/20802-nested-relation-filter-served.md`, one sentence of a
**pending** release note that said the in-memory driver still matches
per element.

## Confirmation requested: a pending release note is corrected

This PR changes `.changeset/20802-nested-relation-filter-served.md`, a
pending release note this PR did not add. This is the DELIBERATE
CORRECTION class `check-empty-changeset.mjs` names, so `Check Changeset`
stays **red** on purpose. It is not a required context. `skip-changeset`
must not be applied. The note's last sentence said:

> On the in-memory driver, a multi-valued relation's `$contains` still
matches a stored id by substring per element, so there an id that is a
substring of another stored id (`u1` inside `u10`) also matches; SQLite
and PostgreSQL match the element.

This PR makes that false, so it now reads:

> SQLite, PostgreSQL and the in-memory driver match the element of a
multi-valued relation, so an id that is a substring of another stored id
(`u1` inside `u10`) does not match it.

Please confirm the correction on this PR. If the release that consumes
that note ships before this PR lands, the edit should be dropped on
rebase.

## Tests (HEAD `10656601`)

- `pnpm --filter @objectstack/driver-memory typecheck` passes, and
`vitest run` gives **68 files / 1553 tests passed**. Both typecheck
programs include the new test file (`--listFiles`).
- The new file alone gives **113 passed**.
- `pnpm --filter @objectstack/driver-sql typecheck` passes (its program
includes the edited test).
`sql-driver-17590-json-column-membership.test.ts` gives **22 passed, 2
skipped**; the skips are the live PostgreSQL/MySQL cells, unprovisioned
here, NOT MEASURED locally.
- `pnpm --filter @objectstack/spec typecheck` passes. `spec build` and
then `check:generated` report "All 15 generated artifacts are up to
date".

**Reverse verification.** The code change was committed first. Then
`memory-driver.ts` and `memory-analytics.ts` were checked out at BASE
under an EXIT/INT/TERM restore trap. The landed mutation was verified on
disk before the run: `filterContainsTest` count 0, two
`escapeRegex(val)` arms back, and both files byte-equal to BASE.

The predicted direction was that the cases where per-element substring
and membership disagree go red and the agreeing ones stay green.
Measured: **69 failed / 44 passed**. The failures were every disagreeing
case, on all four faces, plus the executed echo, the relation-lowering
input and the fork pin. The `label` controls and the `redwood`, `ab`,
`u10` and `'0'` cases stayed green. The restore was proven by blob
hashes equal to HEAD (`d376dadb…` / `9de9198b…`), an empty `git diff
HEAD` and clean porcelain.

**Guard ablation** (`scripts/ablation-replace.mjs`, anchor 1 to 0).
Dropping `$not: { $type: 'array' }` turned exactly the nested-array
cases red: 12, the json `'u1'`, its complement and `'null'`, on all four
faces. The executed echo stayed green, since SQLite is the oracle there.
The file was restored to the HEAD blob with an empty `git diff HEAD`.

**Lint (narrowed, a measurement).**
1. Population: `eslint.config.mjs` lints
`**/*.{ts,tsx,mts,cts,js,jsx,mjs,cjs}` minus `NEVER_LINTED`. The six
changed TS files are all in it; the two changesets are not.
2. `eslint --no-inline-config --format json` over those six files
reports 6 files, 0 errors, 0 warnings at `10656601`.
3. The config never enables type-aware linting (no
`parserOptions.project`, no typed rules; it says so itself), so this
diff cannot move any untouched file's verdict.

**Driver conformance ledger**: `node
scripts/check-driver-conformance.mjs` reads "50 covered cell(s), 0 in
the DEBT ledger, 0 exempt" both at `f6ccca4a` (before) and at `10656601`
(after).

**Gates**: `dispatch-gates --commands` was re-derived at `10656601`.
That gives 84 commands, six more than the dispatch list:
`engine-double-contract`, `objectql-double-limit`,
`query-options-erasure`, `type-check-coverage`, `type-check-debt` and
`where-matcher`. The run also covered the eight roster gates flagged as
sharing a directory with these paths. All exit 0 except:
- `check-empty-changeset`: exit 1, the deliberate correction above.
- `check:dual-build-cjs-loads` and `check:type-check-debt`: exit 3,
`PREREQUISITE NOT MET` (they need the whole workspace built). **NOT
MEASURED**; CI's Build/Lint jobs own those.
- `check:lean-entry-closure`: exit 3 at first. It passed after building
objectql.

## Acceptance notes (observed, not filed by this PR)

- The other text operators over a stored array are not ruled by the
membership section and still differ: `{ owners: { $startsWith: 'u1' } }`
gives `d1, d3, d5` on memory (per element) and no rows on SQLite (GLOB
over the serialized text). The spec docblock now says so.
- objectql `engine.ts`'s delete-probe docblock (the `$contains`
pushdown) still says every backend answers a substring SUPERSET. That is
stale for `driver-sql` since objectstack-ai#17590 and for memory after this PR. The
code stays correct because it narrows exactly through
`storedReferenceIncludes`. Comment only, no carrier.
- A field `driver-memory` holds no declaration for keeps the substring
reading, and mingo still applies that `$regex` per element of an array
value. This is the deliberate counterpart of `driver-sql`'s
`isJsonColumn` returning false for an unknown table.

---
_Generated by [Claude
Code](https://claude.ai/code/session_01Ujdtvqs7ree7WyQmEDwEnG)_

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation protocol:data size/xl tests tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants