fix(driver-memory,driver-mongodb): refuse a non-boolean $exists comparand with INVALID_FILTER / 400, as $null's is refused (#20897) - #20979
Conversation
…rand as $null's is refused Claude-Session: https://claude.ai/code/session_01Ujdtvqs7ree7WyQmEDwEnG Co-authored-by: Claude <noreply@anthropic.com>
…slator walk Claude-Session: https://claude.ai/code/session_01Ujdtvqs7ree7WyQmEDwEnG Co-authored-by: Claude <noreply@anthropic.com>
…boolean $exists refusal Claude-Session: https://claude.ai/code/session_01Ujdtvqs7ree7WyQmEDwEnG Co-authored-by: Claude <noreply@anthropic.com>
📓 Docs Drift Check4 anchor(s) derived from 2 changed package(s); no hand-written page names any of them, so this run has nothing to list — not a clean bill of health. This check sees only pages that NAME a derived anchor: one that documents this change in prose, or enumerates it in an authoring dialect, names none and stays invisible to it on every run. What this run could not see
Coarse fallback — 9 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin e99b1d782f4040516528da55a5c9d4d321be4f5c && git checkout e99b1d782f4040516528da55a5c9d4d321be4f5c
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin f8178ffeceba053667353f86ec193f2152ad557c 94daed863bd7ef3069c572f65ab3bc333ed1e35c && git checkout -B drift-repro f8178ffeceba053667353f86ec193f2152ad557c && git merge --no-ff 94daed863bd7ef3069c572f65ab3bc333ed1e35c
node scripts/docs-audit/affected-docs.mjs --json f8178ffeceba053667353f86ec193f2152ad557c |
…ites in the envelope caller census Claude-Session: https://claude.ai/code/session_01Ujdtvqs7ree7WyQmEDwEnG Co-authored-by: Claude <noreply@anthropic.com>
…t-set narrowing (minor, breaking) Claude-Session: https://claude.ai/code/session_01Ujdtvqs7ree7WyQmEDwEnG Co-authored-by: Claude <noreply@anthropic.com>
… census ledgers as a client test input Claude-Session: https://claude.ai/code/session_01Ujdtvqs7ree7WyQmEDwEnG Co-authored-by: Claude <noreply@anthropic.com>
Contract reviewServed-tier: Inputs: card #20897 (body and all seven comments: triage 5916839636, claim 5921382209, reports 5921891523 and 5922406027, claim amendment 5922434933, correction 5922450072, id fix 5922467874); PR #20979 body, its nine-file list and the net diff against merge base ① Derived judgments
No derived judgment in the PR body or the changeset was found wrong. ② Semver levelClause-②: no (narrowing)
③ Boundary flags
Implemented-by: VERDICT: PASS |
Conflicts resolved so both census declarations survive: the rest nested-relation pin's NOT_SDK row and this branch's driver-memory row, the service-receiver control at 8 sites over three files, the NOT_SDK split at 8 (recomputed from the merged ledger), and both per-file client test inputs in scripts/cross-package-test-inputs.mjs and turbo.json. Claude-Session: https://claude.ai/code/session_01Ujdtvqs7ree7WyQmEDwEnG Co-authored-by: Claude <noreply@anthropic.com>
Contract reviewServed-tier: Delta review. The at-tier record for the previous head ① Derived judgments
No derived judgment in the PR body, the changeset or the merge-round report was found wrong at this head. ② Semver levelClause-②: no (narrowing)
③ Boundary flags
Implemented-by: VERDICT: PASS |
Fixes #20897
Clause-②: no (narrowing)
A non-boolean
$existscomparand ("yes",1,"false",0,null,undefined, an object) is now refused withINVALID_FILTER/ 400 on driver-memory and driver-mongodb, which were the two faces still answering it. The words are driver-sql'snonBooleanExistsComparandError, and the refusal takes the same place and form as each driver's existing$nullrefusal.trueandfalseanswer exactly as before.Why the declaration is no (narrowing):
FieldOperatorsSchemadeclares$exists: z.boolean(), so this pulls two faces back to the declared contract. It reuses the existingINVALID_FILTERcode and driver-sql's existing wording, and it adds no key, code or accepted shape. It does narrow what driver-memory and driver-mongodb accept: a filter they used to answer is now refused. The changeset declares it BREAKING,minorfor both packages under the launch-window convention for accept-set narrowings, with the ADR-0087 dispositionnot-required (already-registered filter-query-face-comparands-refused-at-save). That registered entry's reason already states that every query face refuses a non-boolean flag, and its replacement is this change's migration.What was wrong
Measured on
origin/mainf6ccca4awith an objectpxholding rowa(name: "won") and rowb(name: null):"yes"1"false"0/nulltruefalseengine.findover driver-memory[b][b][b][b][a][b]aaababtranslateFilter$eq: null$eq: null$eq: null$eq: null$ne: null$eq: nullengine.findover driver-sql (better-sqlite3)[a][b]driver-memory's query path and driver-mongodb's emitter both asked
value === trueand sent every other value to the no-value side, so$exists: "yes"returned the rows with NO value. That is the author's intent inverted. driver-memory's own cube face read the same flag by truthiness (Boolean(raw[0])) and answered the opposite rows for"yes",1and"false". So one package gave two answers to one filter.The landing: per-face gates, not an engine door
The PM's H1 held.
$null's refusal lives per face, and no engine-level door judges a flag. The spec's shared comparand faces (assertListComparandShapes,normalizeFilterComparandTypes) deliberately leave$null/$exists/$emptyto the faces, asfilter.zod.tsrecords. Each driver already runs one validating walk ahead of its lowering, and$nullis refused there. This PR puts$existsbeside it:filter-refusal.ts:assertFilterConditionShaperefuses a non-boolean$existsthrough a newnonBooleanExistsComparandError, next to the$null/$emptychecks. Every memory entry runs this gate before it lowers anything, so the one edit coversfind,findOne,count,aggregate,updateMany,deleteManyand the analytics face (query()andgenerateSql()). All of these were measured.memory-driver.tsis not touched: its$existsarm now sees only booleans.mongodb-filter.ts:reduceFilterKey(the walk) refuses it next to the$nullgate, and the emitter's$existsarm keeps a local check with the same constructor, as the$nullarm does.I did not add an engine-level door. The faces that still accept a non-boolean
$existsafter this PR are the engine's own in-process evaluators: objectql's aggregationfilterandhaving, plus@objectstack/formula. Their evaluability doors (assertHavingIsEvaluable,assertAggregationFilterIsEvaluable, andassertConditionIsEvaluable, which already refuses a non-boolean$empty) live inhaving-filter.ts. That file and formula'smatches-filter.tsare fenced for this card. A door anywhere else would be a second copy of the rule that still does not reach them. They are measured and named below for routing.Every compile face
driver-sqlapplyFilterCondition(anddriver-sqlite-wasm, Turso local, which inherit it)engine.findon better-sqlite3 anddriver-sqlite-wasmfindrefuse"yes",1,"false",0andnullwithINVALID_FILTER/ 400, andtrue/falsegive[a]/[b]. Pinned insql-driver-out-of-contract-filter-input.test.ts(the[#5369]block:"yes",1,0,null,undefined,{},"false", plus atrue/falsecontrol). Not edited (#20822 group 2 in flight).RemoteTransportbuildWhereSQLremote-transport-null-comparand-refusal.test.ts(block d) andturso-local-remote-null-parity.test.ts("both transports REFUSE a non-boolean$exists"): 90 tests green at this head. Not edited.read-scope-sqlcompileScopedFilterToSqlassertBooleanFlagComparandsrefuses$null/$exists/$empty, fail-closed.read-scope-boolean-flag-comparand.test.tsis green.filter-normalizerlowerAnalyticsWhereassertBooleanNullFlagsrefuses a non-boolean$null/$existsbefore any lowering.where-boolean-flag-refusal.test.tsis green (the two suites, 104 tests).formulamatchesFilterCondition"yes",1,"false",0andnullall match rowb, becausev === true ? actual != null : actual == null. Its$nullarm is unrefused the same way. Named in the report for the PM to route.having-filter(applyHaving,matchesHaving,matchesAggregationFilter)engine.aggregateon memory AND SQLite: both the aggregationfilterandhavingread$existsas!!target."yes",1and"false"select the valued rows and groups, and0/nullselect the no-value ones. The engine evaluates these in-process, so no driver refuses them. For context,$null: "yes"on these two faces drops the constraint (every row or group comes back). Named in the report.driver-memory(query path and cube face)driver-mongodbtranslateFieldOperatorsPins
memory-null-comparand-refusal.test.ts: the case "$existsis deliberately NOT tightened here" pinned the inverted answer['2']for$exists: "yes". It now pins the refusal (codeINVALID_FILTER,status400, driver-sql's first sentence, the position) for"yes",1and"false", through the live path and the gate alike, withtrue/falseas the control. The case was flipped, not deleted.memory-exists-non-boolean-refusal.test.ts(new) is the multi-face invariant: every entry of the package (the eight listed above) refuses the seven non-booleans. Fortrue/false, every read entry, the cube face included, answers exactlyfind()'s rows. The same row set asfind(), orINVALID_FILTER, and never a third answer. On the cube face,undefinedand a plain object are refused first by that face's comparand-type check, which is its documented precedence. It is the same envelope and the same position, with that face's own sentence. The file also pins the refusal at every depth, including behind a TRUE identity ($or: [{}, …]), and checks that a refusedupdateMany/deleteManyleaves the store untouched.mongodb-exists-non-boolean-refusal.test.ts(new): the seven non-booleans refused on the translator, with the position inside combinators and behind a boolean identity that would otherwise settle the node before the emitter runs.true/falsestill translate to$ne: null/$eq: nulland equal their$nullmirror."yes",1and"false"refused with atrue/falsecontrol. driver-sql is fenced, so they were cited, not duplicated.Pin sweep. A repo-wide grep for a non-boolean
$existsliteral and for the$existsrefusal's words outside the faces that already refuse found one pin that asserted the old answer, the memory case above. The CHANGELOG entries that say "$existsis deliberately NOT tightened" are released text and are not edited.Verification (driver changes at
da4feaf8; final head378effc8)pnpm --filter @objectstack/driver-memory test: 68 files, 1500 tests passed.pnpm --filter @objectstack/driver-mongodb test: 30 files passed, 5 skipped, 671 passed and 172 skipped. The skipped files need a realmongod, which is opt-in (OS_TEST_MONGODB_MEMORY_SERVER_ENABLED). The new pins do not need it: they run on the translator.typecheckfor both packages exit 0. The new memory test is intsc --noEmit's program (--listFiles), and the mongodb one is intsconfig.test.json's (check:test-typecheck).scripts/ablation-replace.mjs, which asserts each mutation on disk (anchor count 1 to 0, blob changed) and proves each restore (blob equals HEAD,git diff HEADempty). Both packages' tests importsrc/directly, so nodist/was involved.$nullcases of the older file, the twotrue/falsecontrols and the four cube-face cells the type face refuses first.true/falsecontrol stayed green.node scripts/check-driver-conformance.mjs): 50 covered, 0 DEBT, 0 exempt before and after.node scripts/pm/dispatch-gates.mjs --commandsafter the last commit (378effc8) derived 79 families.--ranreconciles them as 79 accounted for: 77 run and exit 0, and 2 NOT MEASURED. Those two arecheck:dual-build-cjs-loads, which reads every package'sdist/, andcheck:type-check-debt, whose re-measure needs the whole build closure. Both exit 3 (PREREQUISITE NOT MET) on this partial build and are declared to CI.check:skill-examplesfirst exited 3 for want of theclient-reactbuild, and ran green once that closure was built..tsfiles andscripts/cross-package-test-inputs.mjs) with--no-inline-config: 7 files, 0 errors, 0 warnings (counted from--format json). Each file resolves a config (--print-config).eslint.config.mjsenables no type-aware linting (noparserOptions.project), so this diff cannot move the verdict on an untouched file.Outside the drivers: the envelope caller census
The new memory suite calls
analytics.query(twice on its ownMemoryAnalyticsService.@objectstack/client'senvelope-caller-census.test.tswalks the whole repo for that call shape, so CI went red atda4feaf8(Test Core (5/6)). The census prescribes classifying every counted site, and its receiver split already has the class for a producer call. SoLEDGERgains oneNOT_SDKrow (count 2, receiverservice), and its two exact-count controls move with it: the service-receiver control now expects 3 sites and pins their files, and the verdict split reads 3 not-SDK. Nothing is exempted or loosened. The census as it stood atda4feaf8goes red locally on the same two cases CI named. At378effc8it is 20/20 green, and the client package's tests (641) and typecheck pass.The census reads that driver-memory file, so
pnpm check:cross-package-test-inputsrequires it declared.scripts/cross-package-test-inputs.mjsnames the one file in@objectstack/client's globs, andturbo.jsonmirrors it into@objectstack/client#testinputs, so a change to that suite re-runs the census. It is per-file, notpackages/**, for the price the census header records.check:ci-filter-paritypasses, becausecorealready covers the path.Acceptance notes
memory-driver.ts's$existsarm has no local totality check. The$nullarm has one. After the gate it is unreachable with a non-boolean, so the asymmetry is dead code. That file belongs to [finding] driver-memory answers$containson a stored array by substring per element (u1matches a row storingu10), where the SQL drivers answer membership; the spec docblock records the gap against a card that answers 404 #20874 this round and was not touched.filter.zod.ts(the save-door docblock, "Every query face refuses a non-boolean$null/$existsflag") andfilter-save-door-refusals.ts("refused on every query face") now hold for every driver. They still overstate the two engine-side evaluators and formula, measured above.FILTER_LOGIC_CASESasserts rows only, andFILTER_COMPARAND_TYPE_CASES' refusal verdict is the upstreamparseFilterASTdoor. So the flag refusal is held per driver (sql, sqlite-wasm, the Turso parity suite, mongodb, and now memory), not by one table.Generated by Claude Code