Skip to content

fix(objectql): a per-aggregation filter counts $contains on a multi-valued field by membership, as its where twin does - #21004

Merged
objectstack-fleet[bot] merged 4 commits into
mainfrom
claude/issue-20873-aggregation-filter-array-membership
Oct 1, 2026
Merged

objectstack-fleet[bot] merged 4 commits into
mainfrom
claude/issue-20873-aggregation-filter-array-membership

Conversation

@objectstack-fleet

@objectstack-fleet objectstack-fleet Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #20873
Clause-②: no

What changes

The per-aggregation filter (engine.aggregate({ aggregations: [{ …, filter }] }), and so POST /api/v1/data/:object/query) is evaluated by the engine's own walker, matchesAggregationFilter in packages/objectql/src/having-filter.ts. Its $contains arm failed every value that was not a string, so a stored array never matched, and its $notContains arm passed every such value, members included.

On a DECLARED JSON-stored field both arms now ask MEMBERSHIP, the reading FILTER_OPERATORS' $contains docblock (@objectstack/spec) declares and where already gives on every SQL dialect (SqlDriver.applyJsonMembership):

Files: having-filter.ts (the two arms, storedArrayHasMember, declaredJsonStoredFields, and an optional jsonStored set threaded through matchesHaving / matchesAggregationFilter), plus in-memory-aggregation.ts. That file is the one place the engine hands the object's declared field map to the per-aggregation filter, so it reads the declared set once per call beside declaredFieldClasses. having-filter.ts is not on objectql's published entry points. in-memory-aggregation.ts is (applyInMemoryAggregation and bucketDateValue, from index.ts and core.ts), and neither exported signature changes; the declared set is threaded through the internal aggregateBucket only.

The card's table, through the REST door, before and after

Measured with a real SqlDriver on SQLite and on a live PostgreSQL 16.14. Rows: d1 ['u1','u2'], d2 ['u2'], d3 ['u3','u1'], d4 [], d5 ['u10'], d6 null. Base 212d613c, head 90ba78d9.

query where twin per-aggregation m, base m, head
owners $contains 'u1' (the card) 2 0 2
owners $contains 'u10' 1 0 1
owners $notContains 'u1' 4 6 4
tags $contains 'red' (d3 holds ['redwood']) 2 0 2
tags $notContains 'red' 4 6 4
$or of $contains u1 / u3 (the any-of spelling #7398's refusal prescribes) 2 0 2
$not over owners $contains 'u1' 4 6 4
title $contains 'u1' (text, the control) 3 3 3

On the in-memory driver the per-aggregation m is the same evaluator's answer, also 2 now. Memory's own where answers 3 for the card at this base (d5 too, by a per-element substring). That face belongs to #20874 (in flight), whose branch (10656601) moves it to membership and pins d1, d3.

The fork: by the DECLARED column (Zone 2 H2)

The fork reads the declaration (STRUCTURED_JSON_TYPES or isMultiValueField), never the row. That is the contract's sentence: "One operator, two questions, selected by the COLUMN rather than by the caller". It is also SqlDriver.isJsonColumn's population (built from the same two spec sets) and #20874's isJsonStoredField, character for character.

Measured: on every fixture reachable through the public doors, the declared reading and a value-shape reading select the same rows.

  • $contains / $notContains on a declared structured-JSON field is refused before any row is read: the engine's text-operator declared-type door, INVALID_FILTER 400, in where and in the per-aggregation filter alike, on all three backends.
  • A multi-valued field's find() value is an array or null on memory, SQLite and PostgreSQL alike. The write door wraps a scalar: 'u1' is stored as ['u1'] on memory and SQLite.

The two readings differ only on rows a direct caller hands the walker: a declared multi-valued column holding a scalar string, or an undeclared column holding an array. There the declared reading gives what SQL where gives (no member; the substring reading), and a value-shape reading would not. Both cases are pinned. No open_questions fork results.

Zone 2 hypotheses, measured

Compile-surface conclusions

# face conclusion
1 driver-sql applyFilterCondition already compliant (evidence) — $contains / $notContains on a JSON column go through applyJsonMembership; the where twin numbers in the table above are this face, measured on SQLite and PostgreSQL 16.14. driver-sqlite-wasm and driver-turso local inherit it (not measured separately).
2 turso RemoteTransport.buildWhereSQL out of scope (reason) — an independent compiler this card does not touch. Read at 212d613c: its $contains / $notContains arms go pushLike (substring over the stored text) with no JSON-column fork. Not measured (no remote libsql here). In the out-of-scope finding below.
3 service-analytics compileScopedFilterToSql out of scope (reason) — a different compiler. Measured function-level at 212d613c: { owners: { $contains: 'u1' } } compiles to instr("t"."owners", ?) > 0 on SQLite, which admits a row holding ["u10"]. On PostgreSQL it compiles to "t"."owners" LIKE ? ESCAPE ? over a json column. In the out-of-scope finding below (an RLS read scope).
4 service-analytics lowerAnalyticsWhere out of scope (reason) — it lowers the analytics where to a FilterCondition and adds no $contains reading of its own. The ObjectQL strategy hands that to the driver (face 1). The native SQL strategy maps contains to the substring LIKE shape (native-sql-strategy.ts, read, not measured), in the same finding as face 3.
5 formula matchesFilterCondition out of scope (reason: fenced; PR #20972 landed on this file during this run) — its arm is typeof actual === 'string' && typeof v === 'string' && actual.includes(v) (unchanged by #20972). Measured: ['u1','u2'] → false, ['u10'] → false, 'u1 memo' → true. In the out-of-scope finding below.
half objectql having-filter changed — the per-aggregation filter, as above; applyHaving / matchesHaving without a declared set unchanged (H4).
unfrozen driver-memory / driver-mongodb out of scope (reason: fenced, #20874 / #20897 in flight). Memory measured above, and #20874's fork matches this one. Mongo's translateFieldOperators compiles $contains to a bare $regex, which MongoDB applies per array element (per-element substring). Read, not measured.

Tests

  • pnpm --filter @objectstack/objectql exec vitest run --project local --maxWorkers=2 src/engine-aggregate-filter-array-membership.test.ts — 30 passed. The card's rows with the rows themselves, empty table, per group, having control, the member-text reading (number / exponent / boolean / null / non-JSON-number spellings / nested / object / scalar), the declared fork, the declared population.
  • OS_TEST_POSTGRES_URL=… pnpm --filter @objectstack/rest exec vitest run --project local --maxWorkers=2 src/aggregation-filter-array-membership.test.ts — 18 passed (9 SQLite, 9 live PostgreSQL 16.14), 9 named skips (MySQL). Each row runs beside its live where twin, populated and empty.
  • objectql whole local project on the merged head 90ba78d9: 349 files, 6851 tests passed; repo project 1 file / 5 passed.
  • REST aggregation-adjacent files on 90ba78d9 with the PostgreSQL cell live: 9 files, 106 passed, 34 skipped.
  • pnpm --filter @objectstack/objectql typecheck and pnpm --filter @objectstack/rest typecheck: green. Both new test files are in their package's test program (tsc -p tsconfig.test.json --listFiles).

Reverse verification

Each leg ran through scripts/ablation-replace.mjs (anchor hits proven on disk; restore proven blob == HEAD and git diff HEAD empty), from the committed change.

  • A1 — the $contains arm put back to the substring test: 15 of 30 red (every membership $contains row, the member-text rows, the declared-fork row); the $notContains rows and controls green, as predicted.
  • A2 — the $notContains arm put back: 3 red (its two rows and the complement row).
  • B — the REST suite reads objectql through dist/. declaredJsonStoredFields was emptied, objectql rebuilt, and ablation-dist-preflight.mjs found the marker in 4 built files. 12 red: the 6 membership rows on each of SQLite and PostgreSQL. Text control, having and empty-table rows stayed green. Restore leg: rebuilt, marker absent from all 14 built files, tree clean, 18 passed.

Driver conformance ledger

node scripts/check-driver-conformance.mjs: before (212d613c) "50 covered cell(s), 0 in the DEBT ledger, 0 exempt"; after (90ba78d9) the same.

Gates

  • node scripts/pm/dispatch-gates.mjs --commands re-derived with no paths at 90ba78d9 gives 63 commands, all run, exit codes recorded to disk. --ran reconciliation: 63 derived, 61 run (all exit 0), 2 NOT MEASURED, 0 unrun.
    • NOT MEASURED: check:dual-build-cjs-loads and check:type-check-debt. Both are PREREQUISITE NOT MET (exit 3): they need the whole-workspace build lint.yml performs first, and 42 / 5 packages have no dist/ in this worktree.
    • check-engine-split-ratio first refused on the shallow checkout. It was green after a deepen to its window (git fetch --shallow-since=2026-06-26 origin main).
  • Lint, a declared narrowing over the four touched TS files at 90ba78d9: eslint --no-inline-config --format json reports 4 files, 0 errors, 0 warnings.
    • Each file maps to a config (--print-config).
    • eslint.config.mjs never enables type-aware linting (no parserOptions.project, no typed rules; its own lines 327-328 say so), so this diff cannot move an untouched file's verdict.

Acceptance notes


Generated by Claude Code

claude added 4 commits October 1, 2026 00:24
…ed multi-valued field by membership

The engine's aggregation evaluator failed `$contains` on every value that
was not a string, so a stored array never matched and `$notContains`
matched every row. On a declared JSON-stored field (STRUCTURED_JSON_TYPES
or isMultiValueField) both now ask membership, the reading `where` gives
on every SQL dialect; a scalar string column keeps the substring test.

Claude-Session: https://claude.ai/code/session_01Ujdtvqs7ree7WyQmEDwEnG
Co-authored-by: Claude <noreply@anthropic.com>
…lti-valued field beside its where twin

Engine-level over the read shape find() presents on memory, SQLite and
PostgreSQL; REST door on a real SqlDriver, SQLite always and the live
PostgreSQL / MySQL cells where their URL is set, each row beside its
where twin. having keeps the substring reading on a text projection.

Claude-Session: https://claude.ai/code/session_01Ujdtvqs7ree7WyQmEDwEnG
Co-authored-by: Claude <noreply@anthropic.com>
@github-actions github-actions Bot added size/l documentation Improvements or additions to documentation tests tooling labels Oct 1, 2026
@github-actions

github-actions Bot commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

8 anchor(s) derived from 1 changed package(s); no hand-written page names any of them, so this run has nothing to list — not a clean bill of health. This check sees only pages that NAME a derived anchor: one that documents this change in prose, or enumerates it in an authoring dialect, names none and stays invisible to it on every run.

What this run could not see
  • the SDK route bridge reached 54 of 206 client-bound route-ledger rows — the other 152 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 152: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 55 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 97 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.
  • a key NAME is not a key, so the hand re-read the line above prescribes can land on the wrong schema. The same spelling is authorable on one governed type and a [REMOVED] tombstone on another for each of active, aria, joins, objects, template, tools and version (censused on [finding] tools is a key on BOTH AgentSchema (tombstoned, dead) and SkillSchema (live, cloud-attested), so a name-based search attributes skill examples to the agent key — it produced a false stop-the-line alarm on PR #19059 #19093 over the liveness ledger's governed types, top-level keys); nothing in a search result distinguishes the two, so a grep hit on a LIVE example reads as evidence about the DEAD key. Measured on fix(spec): the agent.tools liveness row says dead — it claimed live on a key the schema tombstoned #19059: content/docs/ai/agents.mdx was reported as contradicting the agent.tools tombstone over its tools: example at :161, which is inside the defineSkill({ block opened at :155 — the page was already correct. Settle ownership by PARSING the value against both schemas, never by the name: that literal PASSES SkillSchema, and as an AgentSchema it FAILS at tools with the tombstone prescription. ⛔ These names are not the whole class — a key retired through a .strict() guidance map leaves no tombstone in the walked shape and none of them here (tool.category, live as AIToolDefinition.category).

Coarse fallback — 17 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json a5bce408883b81a6e2382ebe709a03cc3a5b40b4 → packageMentionDocs.

Which tree this was computed on

This run read content/docs from 9b6a0192b0271d3310efbabdeca28e2801694fd8 — the merge of head 90ba78d9b150faf40e31f68841e9ac94aa1e13a1 into base a5bce408883b81a6e2382ebe709a03cc3a5b40b4, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 9b6a0192b0271d3310efbabdeca28e2801694fd8 && git checkout 9b6a0192b0271d3310efbabdeca28e2801694fd8
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin a5bce408883b81a6e2382ebe709a03cc3a5b40b4 90ba78d9b150faf40e31f68841e9ac94aa1e13a1 && git checkout -B drift-repro a5bce408883b81a6e2382ebe709a03cc3a5b40b4 && git merge --no-ff 90ba78d9b150faf40e31f68841e9ac94aa1e13a1

node scripts/docs-audit/affected-docs.mjs --json a5bce408883b81a6e2382ebe709a03cc3a5b40b4

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

Contract review

Served-tier: CONTRACT_REVIEW_TIER
Head-sha: 90ba78d9b150faf40e31f68841e9ac94aa1e13a1
Local-runs: none

Inputs: card #20873 (body and all five comments: triage 5914962540, the blocked transition 5921472356, claim 5921990075, report 5922812043, claim amendment 5922857157); PR #21004 body, its five-file list, its four commits and the net diff against the merge base with origin/main (7fa67dada, the head's second parent: 5 files, 596 insertions, 14 deletions); the 34 check-runs on this head, read once at 2026-10-01T01:51Z. The head is a merge commit (e9d4728c9 plus origin/main at 7fa67dada); origin/main has since moved to e161ad358 (PR #20984 f8178ffec included) without touching either changed source file. driver-sql, driver-memory, spec and objectql sources were read with git show at the merge base and at current origin/main to judge the claims against them. Nothing built, run or re-run.

① Derived judgments

  1. Accept set unchanged in both directions — right. At the merge base the evaluator's $contains arm was typeof value !== 'string' ⇒ false (a stored array never matched) and its $notContains arm was typeof value === 'string' && includes ⇒ false (a stored array always passed). The diff changes only the boolean each arm returns when jsonStored is true; it adds no throw and removes none. $in / $nin (listHolds), $startsWith, $icontains, the flag arms and the F8 whole-day arms are untouched. The changeset's sentence "No query that was refused now answers, and none that answered is refused" holds against the diff.
  2. Published surface unmoved — right in substance; one sentence of the PR body is wrong. having-filter.ts gains one module export, declaredJsonStoredFields, and matchesHaving / matchesAggregationFilter gain a trailing optional jsonStored; checkCondition and storedArrayHasMember are private. None reaches the package entry: index.ts and core.ts have no export *, name none of these symbols, and engine.ts only imports from having-filter.js. But the body's "Neither module is on objectql's published entry points" is wrong for in-memory-aggregation.ts: index.ts line 366 and core.ts line 78 both export applyInMemoryAggregation and bucketDateValue from it. Consequence nil: applyInMemoryAggregation(rows, ast, timezone?, fields?) keeps its signature at the head, the new set is read from the fields it already receives, and aggregateBucket is module-private. The amendment's narrower statement (having-filter named nowhere in index.ts) is right, and the changeset's "no export" claim holds for the published surface.
  3. The member rule is driver-sql's, read as a predicate — right. storedArrayHasMember and jsonMembershipCandidates (sql-driver.ts 3821 at the base) carry the same JSON-number regex literal; a string element by text equality; a number element by Number(text) only when the text is a JSON number and finite (so '1.50' names 1.5, '1e2' names 100, '01' / ' 1' / '0x10' name nothing); true / false / null by literal text; array-only, with an object or nested-array element no member. The driver compares JSON texts and the evaluator JS values, and the same rows result on every case the new suite pins.
  4. The fork population — right for the authorable population; the body's "SqlDriver.isJsonColumn ... character for character" is over-stated. declaredJsonStoredFields is STRUCTURED_JSON_TYPES.has(type) || isMultiValueField({ type, multiple }), the identical expression to isJsonStoredField that fix(driver-memory): $contains on a multi-valued or JSON-stored field is membership, on every face #20984 landed on main. driver-sql's isJsonField is JSON_COLUMN_TYPES (the same two spec sets plus the driver-internal object / array aliases) || isMultiValueField, with a per-instance mediaColumnIsJson for a single-value media column on a deployment that has not moved those columns. The aliases are not authorable types, so the only member where the driver's registry is deployment-dependent and the evaluator's is not is single-value media; fix(driver-memory): $contains on a multi-valued or JSON-stored field is membership, on every face #20984's docblock excludes it for the same reason, it is unmeasured here, and it rides the consolidation flag in ③.
  5. Threading — right. in-memory-aggregation.ts reads the set once per call and only when some aggregation carries a filter, beside declaredFieldClasses; engine.ts 17297 is the one call that passes declaredFields. applyHaving passes none, so having keeps the substring reading (H4), which the two suites pin on the groupBy text projection.
  6. $in / $nin left as they are — right under the direction's own conditional. Triage asked for $in per element "where the operator's declaration says so". SET_MEMBER_DESCRIPTION says nothing about a stored array, and the spec's $contains docblock records membership as "the one operator driver-sql: a declared operator on a multiple: true (JSON array) column silently answers wrong — $in/$eq always zero rows, $nin returns the rows it was asked to EXCLUDE #7398 left working on a JSON column after refusing the equality family there". There is no declared per-element reading to implement; the per-aggregation 200 where where is a 400 is a residual the report files (③).
  7. Pins against triage's set — met at the evaluator and on SQLite; PostgreSQL and memory are qualified (③). The card's table (m: 2), the u10 prefix row, the $or any-of spelling, the $not complement and the text control are pinned in both suites; the REST suite runs each row beside its live where twin and the SQLite cell runs in CI (Test Core success). The PostgreSQL cell is a named skip in CI; the memory cell is engine-level over a rows-mock driver.
  8. The merge and its trailers — right. A true merge (parent 2 7fa67dada), no rebase; all four commits carry Claude-Session plus Co-authored-by: Claude.

② Semver level

Clause-②: no

  • The arm is right. No input is refused or newly accepted (① 1), no published export, type or signature moves (① 2), packages/spec/src is untouched. What moves is the count a per-aggregation filter returns on a declared multi-valued field, to the count the spec's $contains docblock declares and where already gives on every SQL dialect: a fix toward the declared contract, not a narrowing or widening of it. The claim, the amendment, the PR body line 2 and the changeset carry the same bare no.
  • patch for @objectstack/objectql is right. No BREAKING banner, no ADR-0087 marker and no migration are owed; the changeset states the FROM/TO (0 and every-row, to the where count), the member-text reading, and that a scalar text field and having are unchanged. Check Changeset is success on this head. The changeset's "where gives on every SQL dialect" is scoped to the SQL family and does not claim driver-memory (which at this merge base still answered per-element substring; fix(driver-memory): $contains on a multi-valued or JSON-stored field is membership, on every face #20984 moved it after).

③ Boundary flags

  • Triage's clause, "If a shared membership predicate exists (the reference semantics the drivers pin), call it. ⛔ No third copy." — antecedent false on both bases; the prohibition cannot be honoured inside the claimed surface; escalated. On the merge base 7fa67dada and on current origin/main e161ad358 alike, @objectstack/spec/data exports the population halves (STRUCTURED_JSON_TYPES, isMultiValueField) but no member-candidate rule; driver-sql's jsonMembershipCandidates is module-private; driver-memory's containsMemberCandidates (memory-driver.ts 369, landed as f8178ffec after this merge base) is module-private; objectql depends on neither driver (core, formula, metadata, metadata-core, metadata-protocol, spec, types). So there was nothing to call, and once this lands storedArrayHasMember is the third module-private copy on main, which is what the clause forbids. The diff is not silent about it: the function's docblock says "A second copy of one rule, not the shared one" and names the home; the PR's Acceptance notes and the report's finding [4] (which counts three readers) say the same; and claim 5921990075, the contract the dev worked under, required exactly that ("a second JS copy is named in the report, ⛔ never silent"). The remedy is a packages/spec/src/** addition beside asciiCaseInsensitiveContains plus re-pointing driver-sql (fenced for #5930 step 4 (domain:engine): the engine-fed faces delete their hand-copied filter meaning (driver-sql, turso remote, memory query, mongodb, formula, having); the memory reference matcher retires (D6) #20822 group 2) and driver-memory (just landed): outside this claim's surface on every leg. ESCALATED to the seat as a consolidation card, carrier none today, with the single-value-media population edge (① 4) listed on it. The verdict is not withheld on it: the copy is verbatim in rule (① 3), disclosed in three places, and the clause's call-it branch had no callee.
  • open_questions: none in either report. The four additions the dev flagged beyond the original surface (in-memory-aggregation.ts, the $notContains arm, the two test files) were taken into the amended claim 5922857157 before this review; answered.
  • H5, the $notContains arm under the bounded exemption — accepted. It is the exact complement of the $contains arm in the same case pair, the null and absent rows satisfy it as SQL's col IS NULL OR NOT (...) does, the base counted 6 where where counted 4, and the amended claim names it.
  • Deviation: the memory pin is engine-level over a rows-mock driver, not InMemoryDriver — accepted, gap named. check:driver-memory-census bars a new test consumer of that driver, so triage's "on memory" pin is met at the evaluator over the find() read shape; the real-driver m: 2 in the body is the dev's measurement, a claim. A real-driver pin is a census-ledger entry, the seat's own act if wanted.
  • Deviation: PostgreSQL / MySQL REST cells are named skips in CI — accepted. The Test Core verdict covers the SQLite cell; the live PostgreSQL 16.14 run (18 passed) is local evidence only, the same note data-group-by-json-door.test.ts carries. The private PG instance outside the scratchpad, the shallow-since deepen that advanced the shared origin/main ref, and the --maxWorkers spelling: mechanics, no bearing on the diff.
  • out_of_scope_findings [0]–[3], for the seat to file and route; no action on this PR. [0] per-aggregation $in / $nin on a multi-valued field answer 200 ($nin fail-open, counting the rows it was asked to exclude) where where is a 400 on the SQL family: the same evaluator, the same file, the natural next card there. [1] the $contains membership contract unanswered on the turso remote transport, the service-analytics SQL compilers (an RLS read scope over-reaches), driver-mongodb and formula. [2] min / max over a multi-valued field answered three ways. [3] $startsWith / $icontains on a multi-valued field, PostgreSQL 500. [4] is escalated above; [5] (no shared conformance kit for stored-array membership) is a routing note.
  • PR body: one wrong sentence (① 2) and one stale count. The body says "a second JS copy"; the report's "three readers" is the right count on current main. Named for the seat; prose on the PR, no bearing on the verdict.
  • Check-runs on this head, read once at 2026-10-01T01:51Z: 34 — 31 success, 3 skipped by design (Build Docs, Console Pin Gate, Packed-tarball smoke), 0 failure, 0 in progress. Check Changeset, Lint & Repo Gates, Test Core 1–6 and the roll-up, all four Type Check jobs, Build Core, Dogfood Regression Gate 1–3 and Dogfood Verify CLI, Temporal Conformance, the four claim and single-writer guards: success. Vercel commit status: success. Every gate verdict is in.

Implemented-by: claude/issue-20873-aggregation-filter-array-membership
Reviewed-by: session_01Ujdtvqs7ree7WyQmEDwEnG

VERDICT: PASS

@objectstack-fleet
objectstack-fleet Bot marked this pull request as ready for review October 1, 2026 02:07
@objectstack-fleet
objectstack-fleet Bot enabled auto-merge October 1, 2026 02:08
@objectstack-fleet
objectstack-fleet Bot added this pull request to the merge queue Oct 1, 2026
Merged via the queue into main with commit d67b942 Oct 1, 2026
50 checks passed
@objectstack-fleet
objectstack-fleet Bot deleted the claude/issue-20873-aggregation-filter-array-membership branch October 1, 2026 03:27
akarma-synetal pushed a commit to akarma-synetal/framework that referenced this pull request Oct 7, 2026
…on a declared JSON-stored field, in where's words (objectstack-ai#21097)

Fixes objectstack-ai#21007
Clause-②: yes (widening)

A per-aggregation `filter` now refuses a scalar comparison on a declared
JSON-stored field (`$eq`, `$ne`, `$gt`, `$gte`, `$lt`, `$lte`,
`$between`, `$in`, `$nin`, implicit equality) with `INVALID_FILTER` /
400, in the words `where` refuses the same filter in. It no longer
counts rows the stored arrays cannot support. The operator set and the
refusal text move from `driver-sql` to `@objectstack/core`, byte for
byte, so both faces read one set and one sentence.

**Clause-② has two halves.** It is `yes (widening)` because
`@objectstack/core`'s root gains three exports
(`JSON_COLUMN_INCOMPATIBLE_OPERATORS`, `jsonColumnOperatorRefusalText`
and its return type `JsonColumnOperatorRefusalText`), and
`applyInMemoryAggregation` gains an optional trailing `reportWithheld`
parameter. It also narrows: `@objectstack/objectql` refuses queries it
used to answer 200, at `engine.aggregate` and at the published
`applyInMemoryAggregation` given a field map. The changeset therefore
carries `minor` for objectql with a BREAKING banner and one ADR-0087
marker, `minor` for core, and `patch` for driver-sql, whose output is
unchanged. The seat answer on the card (`## Seat answer — objectstack-ai#21007`,
comment 5924546829) amended the claim to this surface and this
`Clause-②`.

## What was wrong (measured before this change, `d1f8ce865`)

`POST /api/v1/data/:object/query` on SQLite and a live PostgreSQL 16.14,
over the card's six rows (`owners` is a `multiple: true` lookup, and
`d1` and `d3` hold `u1`). Both dialects answered identically:

| filter | `where` twin | per-aggregation `m` before | now |
|:--|:--|:--|:--|
| `owners $in ['u1','u9']` (the card) | 400 `INVALID_FILTER` | 0 | 400,
same body |
| `owners $nin ['u1','u9']` (the card) | 400 | 6, with `d1` and `d3`
counted | 400, same body |
| `owners $eq 'u1'` / `{ owners: 'u1' }` | 400 | 0 | 400 |
| `owners $ne` / `$gt` / `$lte` / `$between` | 400 | 6 / 4 / 1 / 5 | 400
|
| `tags $eq 'red'` | 400 | 1 (`['red']` loosely `==` `'red'`) | 400 |
| `meta` (json) `$eq` / `$in` | 400 | 0 / 0 | 400 |
| `owners $contains 'u1'` (the prescribed spelling) | 2 | 2 | 2
(unchanged) |
| `title $in` / `$nin` / `$eq` (controls) | 2 / 4 / 1 | 2 / 4 / 1 |
unchanged |

## What changed

- **`@objectstack/core`:** a new
`src/utils/json-column-operator-refusal.ts`, exported from the root
beside `temporal-storage-form.js`. The `export *` publishes three names:
`JSON_COLUMN_INCOMPATIBLE_OPERATORS` (driver-sql's 22 spellings, member
for member), `jsonColumnOperatorRefusalText(field, op, bare)`, and its
return type `JsonColumnOperatorRefusalText` (`{ message, diagnostic }`).
These are the two strings `jsonColumnOperatorError` built, and nothing
else. Each face keeps its own error constructor: driver-sql keeps its
objectstack-ai#8220 provenance seam, and objectql keeps its ADR-0112 envelope.
- **`driver-sql`** (`sql-driver.ts`): the module-private set and the two
template strings are gone. `jsonColumnOperatorError` keeps its name and
signature, and now calls the core builder (hunk at `:3298`). One import
line (with its comment) sits at `:153`–`:156`, after the top import
block. It is outside the declared `:3376`–`:3460` region on purpose, so
as not to touch the `@objectstack/core` import block that objectstack-ai#20988 and
objectstack-ai#20987 edit. `assertOperatorAppliesToColumn` (in objectstack-ai#20988's former region)
is untouched: it reads the imported set under the same name.
- **`objectql`** (`having-filter.ts`):
`assertAggregationFilterIsEvaluable` gains
`assertAggregationFilterSparesJsonStoredFields`. It runs once on the
filter after the reference rule, against
`declaredJsonStoredFields(declared.fields)`, and before any driver is
asked for a row, so an empty table refuses too (objectstack-ai#20122's rule). It walks
`$and` / `$or` / `$not` and refuses implicit equality (reported as `=`,
bare, as driver-sql does) and every operator in the shared set, whatever
the comparand (`null` and `[]` included). The withheld message is
thrown, and the diagnostic, with the aggregation position, goes to
`reportWithheld`, as objectstack-ai#20148 does. `$contains` / `$notContains` /
`$exists` / `$null` / `$empty` keep answering. `checkCondition` carries
the same refusal above its no-value exit, but only as a backstop for a
row that reaches the arm. Its reach is the row's: an empty row set never
gets there, and spec lowering rule 3 puts a `$null` arm ahead of every
negation that the walker's `$or` short-circuit takes first. The gate is
the complete door, and the docblocks now say so (round 2, from the
review's ①.5).
- **`objectql`** (`in-memory-aggregation.ts`, round 2, the review's F10
(b)): the published `applyInMemoryAggregation(rows, ast, timezone,
fields, reportWithheld?)` now calls the same
`assertAggregationFilterSparesJsonStoredFields` (exported from
`having-filter.ts`, not from the package root) once per
`aggregations[i].filter` when it is handed `fields`, before any row is
judged. Before this, a direct caller reached only the backstop and got a
row-dependent answer. This entry point holds no logger, so the closest
seam is a new optional trailing `reportWithheld(diagnostic)`, which
receives the field, operator and position; without it the diagnostic is
dropped and the 400 is unchanged. `engine.aggregate` passes none,
because it has already judged and logged the same filter. The gate's
declaration parameter is narrowed to just the `fields` and
`reportWithheld` members of `AggregationFilterDeclaration`, since
`object` is the reference rule's.
- **`objectql`** (`engine.ts`): one comment block and the
`reportWithheld` log line at `assertAggregationFilterIsEvaluable`'s call
site. The log line now reads "as it is for the same refusal in a where"
instead of "…cross-field comparison…", since it carries two refusals
now.

## Measured findings behind the shape (H1–H5)

- **H1, the premise, holds.** `SET_MEMBER_DESCRIPTION`, the `$in` /
`$nin` entries of `FILTER_OPERATORS` and the `$contains` docblock give
no per-element reading. driver-sql's `where` refuses (it does not answer
membership), so triage's "membership, as driver-sql does" misread it.
- **H2, the set.** All ten operators, plus null and empty-list
comparands, were answered with a wrong count; none was already refused.
The bare infix spellings (`in`, `=`, `nin`) are refused earlier at both
positions by the nested-relation door, so the evaluator only meets the
`$` forms.
- **H3, the home.** None existed; per the seat answer, the home is
`@objectstack/core`.
- **H4, where it fires.** The engine's in-memory lowering is the only
evaluator of `aggregations[i].filter`: every driver's aggregate face
refuses a per-aggregation filter 501, and the analytics ObjectQL
strategy hands measure filters to `engine.aggregate`.
- **H5, `having`.** After objectstack-ai#21037 (landed, merged here), `min` / `max`
over a multi-valued field is refused `INVALID_FIELD` at the aggregate
door. Measured on InMemoryDriver at the merged head: `max(owners)` with
or without `having` gives 400 `INVALID_FIELD`. So `having` cannot meet a
JSON-stored column, and it is left alone.

## Tests

Round 1 numbers were read at `6e541b101`; round 2 numbers are marked
with the head `f66bed950` (main merged).

- `@objectstack/core` `json-column-operator-refusal.test.ts`: 6 passed.
It pins the set member for member, and the message and three diagnostics
by SHA-256 and length against what driver-sql printed at `8f784959c`.
Hashes avoid a third literal copy of the sentence.
- `driver-sql` `sql-driver-json-column-refusal-shared-text.test.ts`, run
beside the existing JSON-column, compile-refusal-seam and provenance
suites: 248 passed. It checks every `FILTER_OPERATORS` member against
the shared set: the driver's thrown message and withheld diagnostic
equal the core builder's output.
- **Byte identity of the move.** A scratch capture through the built
driver-sql (`SqlDriver` over SQLite) covered all 22 spellings plus bare
equality, unmarked and author-marked, message and diagnostic, 46
entries. Before (`8f784959c`) and after: `cmp` identical, sha256
`dbcf32f5…534b2` on both. driver-sql's `dist` no longer contains the
sentence.
- `objectql` `engine-aggregate-filter-json-column-refusal.test.ts`
(engine-level cell over the `find()` read shape): 74 passed. It covers
18 family cases on each of `owners`, `tags` and `meta` (`code`,
`status`, the `$contains` / `$or` prescription, the field absent from
the message, field and operator in the logged diagnostic, and the driver
never asked for a row), an empty table (pure and grouped), the logged
position, 11 answered cases (membership, null predicates, `title`
controls) and the per-row floor.
- `rest` `aggregation-filter-json-column-refusal.test.ts`: 52 per cell.
SQLite passes and a live PostgreSQL 16.14 passes locally; MySQL is a
named skip. Every family case asserts that the per-aggregation 400
body's `error` is **the same string** as its `where` twin's. objectstack-ai#21004's
`aggregation-filter-array-membership.test.ts` still passes beside it.
- **Full suites.** Read at merge `1a226419e`: objectql local 6948
passed, rest local 5072 passed / 247 skipped, core 1809 passed. Read
before the first merge: driver-sql 3285 passed / 188 skipped.
`typecheck` passed for core, driver-sql, objectql and rest. At head
`6e541b101`: core, driver-sql (refusal suites), objectql
`engine-aggregate*` (571 passed) and rest `aggregation-filter*` (150
passed with PostgreSQL) re-ran green.
- **Ablation A: the engine gate call deleted** (`ablation-replace`, plus
a rebuilt objectql `dist`, plus `ablation-dist-preflight --absent`):
- objectql suite: 56 of 74 red. The 54 family cases, the empty table and
the logged position failed; the 11 answered cases and the 7 floor cases
stayed green.
- rest suite: 92 of 104 red (46 per dialect). Populated `owners` /
`tags` cases still got a 400 from the per-row floor, without the logged
diagnostic. `meta` negations (`$ne`, `$nin`, `$nin []`, `$not $in`,
where `meta` is null on every row) answered 200 `{ n: 6, m: 6 }`; the
mechanism was not traced. The empty table answered 200.
- Restored: blob equals HEAD, `git diff HEAD` empty, objectql rebuilt,
preflight shows the marker present in 4 dist files with a clean tree,
and both suites green again (74 and 104).
- **Ablation B: the per-row operator floor replaced by a no-op** (src,
engine suite): 5 red, the 4 operator floor cases and the no-value row;
restored blob equals HEAD.
- **Round 2: the direct-caller pins**
(`engine-aggregate-filter-json-column-refusal.test.ts`, 92 passed). For
`meta` (json) `$ne`, `$nin` and `$not $in`, each on four cells: an empty
row set, an empty grouped row set, `meta` null in every row with the
filter as spec `lowerFilterCondition` lowers it (the shape that carries
the `$null` arm), and the same rows with the filter as written. Each
must refuse 400 `INVALID_FILTER` with exactly `engine.aggregate`'s
message, and hand the diagnostic (field, operator, `At
aggregations[1].filter.…`) to `reportWithheld`. Also pinned: no reporter
means the same refusal; no field map means nothing judged (`m: 0`, as
before); and `$contains` still answers.
- **Ablation C: the new `applyInMemoryAggregation` call deleted**
(`ablation-replace`, anchor 1 to 0, blob `c65412761a90` to
`f530761c0559`): 13 of 92 red.
- The 9 empty, empty-grouped and lowered-null cells, plus the
no-reporter case, answered instead of refusing. That is the backstop's
200.
- The 3 as-written null-row cells were refused by the backstop but with
no diagnostic reported.
- Restored: blob equals HEAD `c65412761a90`, `git diff HEAD` empty, 92
passed again.
- **Round 2 at `f66bed950`:** objectql local full suite 7008 passed (356
files), rest `aggregation-filter*` 150 passed / 61 skipped with a live
PostgreSQL 16.14, core refusal pin 6 passed, driver-sql refusal pins 141
passed.
- **Driver conformance ledger:** 50 covered cells, 0 in the DEBT ledger,
0 exempt, both before and after, in both rounds.

## Gates

- `node scripts/pm/dispatch-gates.mjs --commands` (no paths) derived 70
families at `6e541b101`.
- **68 ran, exit 0.** Among them: `check:adr-0087-registration`,
`check:changeset-no-major`, `check:engine-double-contract`,
`check:nul-bytes`, `check:doc-authoring`, `check:driver-conformance`,
`check:driver-memory-census`, `check:query-options-erasure` and
`check:test-source-alias`.
- **2 NOT MEASURED (exit 3, prerequisite not met):**
`check:dual-build-cjs-loads` and `check:type-check-debt`. Both need the
whole workspace built; two attempts at that build timed out in the
shared verify-lock queue. CI's lint job builds first.
- `--ran` reconciliation: 70 derived, 68 run, 2 NOT MEASURED, 0 unrun.
- **Round 2 at `f66bed950`:** re-derived with no paths, the same 70
families; 68 ran with exit 0 and the same 2 were NOT MEASURED (exit 3).
`--ran`: 70 derived, 68 run, 2 NOT MEASURED, 0 unrun. `typecheck` passed
for core and objectql. Narrowed lint: 10 changed `.ts` files, 0 errors,
0 warnings.
- An earlier run caught one real finding, fixed in `e7bd7f667` ("type
the shared-text pin's find options"): `check:query-options-erasure`'s
test surface grew 236 to 237 because of an `as any` on a `find` options
bag in the new driver-sql test.
- **Lint, narrowed and declared:** `eslint --no-inline-config --format
json` over the 9 changed `.ts` files reports 9 files, 0 errors, 0
warnings. `eslint.config.mjs` sets no `parserOptions.project` and
registers no typed rule, so linting is not type-aware and this diff
cannot move a verdict on an untouched file. The full `pnpm lint` is
CI's.

## Acceptance notes

- **Round 2, from the at-tier review (5926186339).**
  - F10 (b): `applyInMemoryAggregation` is gated (above).
  - F10 (a): the backstop docblocks are corrected.
- F12 and ①.6: the export count is three, and the changeset's "Who is
affected" names `applyInMemoryAggregation` direct callers and the new
optional `reportWithheld`.

- **The "flip the `m: 0` / `m: 6` pins" step had nothing to flip.** No
suite on `main` pinned a per-aggregation `$in` / `$nin` count on a
JSON-stored field (objectstack-ai#21004's two suites pin only `$contains` /
`$notContains`). The full objectql, rest and driver-sql runs found no
other pin that this change turns. The refusal pins are new files beside
objectstack-ai#21004's.
- **A `{ $field }` comparand on a JSON-stored field** (`{ owners: { $eq:
{ $field: 'title' } } }`) is refused by this gate in the JSON-column
words. driver-sql's `where` refuses it through its cross-field class
rule, in that rule's words. Both answers are `INVALID_FILTER` / 400 with
the field withheld, so the two faces disagree only on which sentence
they print.
- **The REST envelope truncates the shared message at 500 characters**
on both faces, so it ends "…because the answ…". That is unchanged here
by direction, and filed separately by the seat.
- **Findings for the seat, not filed here:**
- driver-memory's `where` answers the family per element on a
multi-valued field, while the SQL family refuses it (engine-level
measurement). The seat files it.
- service-analytics' native measure-filter compiler has no JSON-column
gate (read at source, not measured). Carrier: objectstack-ai#20987.

---
_Generated by [Claude
Code](https://claude.ai/code/session_01Ujdtvqs7ree7WyQmEDwEnG)_

---------

Co-authored-by: Claude <noreply@anthropic.com>
akarma-synetal pushed a commit to akarma-synetal/framework that referenced this pull request Oct 7, 2026
…lared JSON-stored field, in the SQL family's words (objectstack-ai#21066) (objectstack-ai#21159)

Fixes objectstack-ai#21066
Clause-②: yes (narrowing)

On a field the object declares JSON-stored (a `multiple: true` field,
`tags` / `multiselect` / `checkboxes`, or a structured-JSON type such as
`json`), `driver-memory` now refuses the scalar-comparison family that
`driver-sql`'s `where` refuses: `$eq`, `$ne`, `$gt`, `$gte`, `$lt`,
`$lte`, `$between`, `$in`, `$nin` and implicit equality, whatever the
comparand, at any depth. The answer is `INVALID_FILTER` / 400 with the
same message. The operator set and the sentence are read from
`@objectstack/core` (`JSON_COLUMN_INCOMPATIBLE_OPERATORS`,
`jsonColumnOperatorRefusalText`, homed by PR objectstack-ai#21097). There is no third
copy. `$contains` / `$notContains` (membership), `$null`, `$exists` and
`$empty` keep answering.

## What was wrong (H1, measured at `origin/main` `670680e93` through
`engine.find`)

A real `ObjectQL` over `InMemoryDriver`, objectstack-ai#21004's six rows (`owners` is
a `multiple: true` lookup, `tags` is a `tags` field). Every row
reproduces the card:

| `where` | before | now |
|:--|:--|:--|
| `owners` `$eq 'u1'` | `d1`, `d3` (per element) | 400 `INVALID_FILTER`
|
| `owners` `$in ['u1','u9']` | `d1`, `d3` | 400 |
| `owners` `$nin ['u1','u9']` | `d2`, `d4`, `d5`, `d6` | 400 |
| `owners` `$gt 'u1'` | `d1`, `d2`, `d3`, `d5` | 400 |
| `tags` `$gt 'red'` | `d3` | 400 |
| also: bare `{ owners: 'u1' }`, `$ne`, `$gte`, `$lt`, `$lte`,
`$between`, `tags $eq`, `{ owners: null }`, `$eq null`, `$ne null`, `$in
[]`, `$nin []` | rows, per element | 400 |
| controls: `owners $contains 'u1'` / `$notContains` / `$null` /
`$exists` / `$empty`, `title $in` | rows | unchanged rows |

The engine hands the driver the operators as written, except `$ne` /
`$nin`. Those arrive inside the spec's null-safe lowering (`$and` of
`$or` of `$null: true` and the operator). The gate walks `$and` / `$or`
/ `$not`, so that shape is refused too.

The analytics face (`MemoryAnalyticsService`) answered the same
per-element rows. Its SQL echo rendered `owners = 'u1'`, which matches
no row over the JSON text the SQL family stores. It now refuses in
`query()` and `generateSql()` alike.

## What changed

- `filter-refusal.ts`: the shape gate (`assertFilterConditionShape`)
takes an optional `FilterFieldDeclarations` (`isJsonStoredField`,
`reportWithheld`). It has two arms. Implicit equality on a declared
JSON-stored field is refused as `=`, bare. Any operator in the shared
set is refused AFTER the existing comparand-shape rules, which is
`driver-sql`'s order (comparand gate, then column-type gate). So an
array under `$eq` or a one-element `$between` still gets its own refusal
first. `jsonStoredFieldOperatorError` builds the error from the shared
text: this package's `unsupportedFilterError` envelope, with the
withheld diagnostic handed to `reportWithheld` (prefixed `At PATH:`)
before the throw.
- `memory-driver.ts`: `convertToMongoQuery` passes
`this.filterFieldDeclarations(object)`. The population is
`isJsonStoredField`, the predicate `$contains` already forks on
(`STRUCTURED_JSON_TYPES` or `isMultiValueField`). So the fields where
`$contains` asks membership are exactly the fields where the family is
refused. The diagnostic goes to the driver's logger at `warn`, the level
`driver-sql` uses for its withheld filter diagnostics. That keeps the
message's "the full diagnostic is in the server log" true here.
- `memory-analytics.ts`: `normalizeFilters` takes the cube. It judges a
`where` key (a cube member) by the field it maps to on the cube's table,
the same (table, field path) pair `filterContainsTest` reads. Its
diagnostic goes to the analytics service's own logger.
- `.changeset/21066-memory-json-column-family-refusal.md`:
`@objectstack/driver-memory` `minor`, BREAKING banner, `Clause-②: yes
(narrowing)`, one ADR-0087 marker `not-required
(no-migration-prescription)`. No registered id covers a filter operator
on a JSON-stored column. The one migration-registry entry that mentions
json columns (`cel-predicate-one-value-comparand-refused`) is the CEL
list-comparand surface, not this one.

## Hypotheses, measured

- **H1** holds: the table above.
- **H2.** The shared home is `@objectstack/core`'s
`json-column-operator-refusal.ts`, and both names are read. Before this
change `driver-memory` had NO withheld-diagnostic seam: every refusal it
raises (the `$null` / `$exists` non-boolean refusals included) names the
field in the message, and nothing in the package logged a diagnostic.
This change keeps the shared posture: the message names neither field
nor operator, and the diagnostic goes to the server log.
- **H3.** `driver-sql` decides a JSON column from `jsonFields`, filled
from `JSON_COLUMN_TYPES.has(type) or isMultiValueField(field)`.
`JSON_COLUMN_TYPES` is `STRUCTURED_JSON_TYPES` plus `MULTI_OPTION_TYPES`
plus the driver-internal `object` / `array` aliases. Memory's population
is the same predicate less those aliases and less a single-value media
field on an unmoved deployment (both recorded on `isJsonStoredField`).
On a schemaless direct call (an object never passed through
`syncSchema`), nothing is judged. Every operator answers per element as
before, as `SqlDriver.isJsonColumn` answers `false` for a table it was
never told about. Pinned. A field declared SCALAR (`text`) that holds an
array is not judged either.
- **H4.** `@objectstack/formula`'s `ORDERING_OPERATORS` docblock does
NOT declare a per-element reading for the query plane. It records a
non-alignment ("driver-memory's read, a frozen test driver, compares a
stored list element by element and keeps returning those rows ...
declared on objectstack-ai#15104"). objectstack-ai#15104 is the `$field` cross-field reference card,
shut as `not_planned` under the driver-memory investment freeze. It
rules nothing about the equality or ordering family on a stored list. So
this is a formula-plane record of observed behaviour, not a query-plane
contract, and no contract conflict stops the card. That docblock
sentence goes stale on declared fields once this lands (see Acceptance
notes).
- **H5.** objectstack-ai#21009 widens the same shared set to the text operators. Both
gates here read the set live, and the new suite iterates
`JSON_COLUMN_INCOMPATIBLE_OPERATORS` intersected with this driver's
vocabulary, with a floor of the nine `$`-spellings. So once both land,
memory refuses `$startsWith` / `$endsWith` / `$icontains` on these
fields with no edit here, and the suite pins them. Whichever of the two
lands second merges `main` and checks the other's members on its face.
The suite's `$contains` control is outside objectstack-ai#21009's scope.

## Pin sweep

- The ONE per-element pin the package carried on a declared field
flipped: `memory-20444-empty-operator.test.ts` had `{ tags: { $empty:
true, $ne: null } }` giving `r2`. It is now a refusal pin (`code` +
`status` + the shared message). The composition (`$empty` beside a
has-a-value sibling on one multi-value field) is kept through `$null:
false`, which answers `r2`. `driver-sql`/SQLite answers that row too,
and refuses the `$ne: null` spelling with the same body (measured on the
built driver).
- `memory-matcher-scalar-comparand-array-value.test.ts` pins per-element
answers on a column declared `text`. Those cells still hold, and a
header note now says the population there is a scalar-declared column.
- Repo-wide: only two tests outside this package bind the real driver
(`packages/runtime`'s two ruled consumers). Neither filters a
JSON-stored field. No other `INVALID_FILTER` pin moves.

## Tests (final head `13407b76f`)

- `pnpm --filter @objectstack/driver-memory exec vitest run
--maxWorkers=2`: **70 files, 1703 passed**. The first run after the
implementation, before any test edit: 69 files, 1 red of 1613, the
per-element pin flipped above.
- `pnpm --filter @objectstack/driver-memory run typecheck`: exit 0. `tsc
--listFiles` includes both edited test files.
- New `memory-21066-json-column-family-refusal.test.ts` (89 tests):
  - the card's five rows;
  - every family member on `owners` / `tags` / `meta` (`json`);
- ten shapes per field (bare, bare null, `$eq null`, `$ne null`, the
engine's `$ne` lowering, `$in []`, `$nin []`, under `$not`, an `$or`
branch after a holding one, `$eq` beside `$contains`);
- `count` / `findOne` / `updateMany` / `deleteMany` refusing with the
table untouched;
- the withheld message plus the logged `At filter.$or[1].owners.$gte:`
diagnostic;
  - comparand-first ordering;
  - eleven answered controls;
- the declaration boundary (undeclared object, scalar-declared column,
the gate with and without declarations);
- the analytics face, `query()` and `generateSql()`, including the
`cube.member` spelling, plus its log line and a `$contains` control.
- Ablations (`node scripts/ablation-replace.mjs`, wrap mode, run at
`de1fef341`; the two later merges touched no file in this package; each
restore proven blob == HEAD with `git diff HEAD` empty). The subjects
are this package's `src`, imported relatively, so no `dist` is involved:
- **A** `filterFieldDeclarations`' predicate forced to `() => false`:
**73 red / 37 green** of 110 across the new file and 20444. The 17 green
in the new file are exactly the answered controls, the
declaration-boundary trio, the premise, the comparand-first case and the
analytics `$contains` control.
- **B** the implicit-equality arm disabled: **7 red**, exactly the six
bare cases and the analytics bare case.
- **C** the operator arm disabled: **67 red**, every operator-based
refusal pin, the direct-gate test and the 20444 flip.
- Gate union, derived with `node scripts/pm/dispatch-gates.mjs
--commands --repo objectstack-ai/objectstack` (no paths; 7 changed
paths, working tree clean) at `13407b76f`: **60 derived, 60 run, every
one exit 0**. Reconciled with `--ran`: "60 derived famil(ies) accounted
for — 60 run, 0 NOT-MEASURED (a DERIVED zero — all 60 recorded an exit
code and none of them is 3)". The same 60 also ran all-zero at the
previous merge head `5b75fe461`. At `de1fef341`,
`check:dual-build-cjs-loads` exited 3 (PREREQUISITE NOT MET, no dist
yet); it measured on both later heads.
- Driver conformance ledger (`node
scripts/check-driver-conformance.mjs`), before and after:
byte-identical. 50 covered cells, 0 DEBT, 0 exempt. The shared matrix
has no JSON-column or multi-value case-set, so this invariant is held by
the per-package pins, not the matrix.

## Acceptance notes

- **The class gains one method.**
`InMemoryDriver.filterFieldDeclarations` is tagged `@internal`. It is
not private only because the analytics face is another class.
`FilterFieldDeclarations` is not exported from the package root, but the
method does appear in the published `.d.ts`. objectstack-ai#20984 graded the analogous
public `filterContainsTest` as a surface widening (`Clause-②: yes
(widening)`). The seat graded it so (5929927010): the line is `yes
(narrowing)`, with the semver (`minor`) and the ADR-0087 marker
unchanged.
- **Surface beyond the claim's list.** `memory-driver.ts` and
`memory-analytics.ts` are edited. The gate cannot see a declaration on
its own, so the plumbing is the minimum the direction needs, and the
analytics face calls the same gate. No open PR touched either file when
read before the first edit.
- **The AST comparison-node door** (`{ type: 'comparison', field:
'owners', operator: '=', value: 'u1' }`) still answers per element on a
declared field: `d1`, `d3`, measured on the built driver. No seam emits
that form (the engine and the protocol hand a driver a FilterCondition),
so it is reachable only by a direct driver call. Left alone.
- **The shared sentence's mechanism clause** ("a field this driver
stores as a JSON TEXT column", "$in/$eq matched nothing") is
`driver-sql`'s, and is literally untrue of this driver and of the
engine's per-aggregation face. The prescription (`$contains`, an `$or`
of `$contains`) is right on all three. Inherited as objectstack-ai#21007 shipped it.
objectstack-ai#21009 is the PR that next edits the shared home.
- `@objectstack/formula`'s `ORDERING_OPERATORS` docblock
("driver-memory's read ... keeps returning those rows") is now true only
of undeclared objects. It is a comment, and no claim holds that file.
- **Tooling.** With the `turbo` 2.10.10 to 2.11.5 bump now on `main`,
every repo-scoped turbo run in an agent session appends a managed
"turborepo-agent-rules" block (an HTML-comment-delimited section) to
`AGENTS.md`. These include `pnpm exec turbo run build`, `pnpm
check:type-check-debt`, `check:query-options-erasure` and
`check:slot-lookup`. It happened repeatedly in this worktree and was
restored each time, and every gate derivation above was taken on a clean
tree; this PR does not touch `AGENTS.md`. Tracked as objectstack-ai#21146 (PR objectstack-ai#21151).

---
_Generated by [Claude
Code](https://claude.ai/code/session_01Ujdtvqs7ree7WyQmEDwEnG)_

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/l tests tooling

Projects

None yet

2 participants