fix(spec)!: a filter carrying a comparand the query faces refuse is refused when it is saved (#20116) - #20247
Conversation
… the query faces refuse The save door asks the comparand-shape face about each slot of a field entry (read-only) and refuses what it refuses, in the schema door's words, plus a non-boolean $null / $exists flag, which every query face refuses. Claude-Session: https://claude.ai/code/session_01Rjy9MeetSfq34PKn81CRiN Co-authored-by: Claude <noreply@anthropic.com>
… the ADR-0087 entry and changeset Claude-Session: https://claude.ai/code/session_01Rjy9MeetSfq34PKn81CRiN Co-authored-by: Claude <noreply@anthropic.com>
…lter-save-door-face-parity
…comparands-refused-at-save Claude-Session: https://claude.ai/code/session_01Rjy9MeetSfq34PKn81CRiN Co-authored-by: Claude <noreply@anthropic.com>
…lter-save-door-face-parity
… refuse every face-refused slot inside a nested relation The judge moves to a non-barrel module so the dataset carriers' nested-relation walk asks the same function FilterConditionSchema's walk asks. Claude-Session: https://claude.ai/code/session_01Rjy9MeetSfq34PKn81CRiN Co-authored-by: Claude <noreply@anthropic.com>
…n reach Claude-Session: https://claude.ai/code/session_01Rjy9MeetSfq34PKn81CRiN Co-authored-by: Claude <noreply@anthropic.com>
…lter-save-door-face-parity
📓 Docs Drift CheckThis PR changes 1 package(s): 6 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:
⛔ 2 release-owned page(s) also name something this change touched. These are read-only:
What this run could not see
Coarse fallback — 136 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin ab42804770462b9d577290acabe020b81457c056 && git checkout ab42804770462b9d577290acabe020b81457c056
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 08c8484a191f39684a0ace3565f84a7313684f5e ab146a9a3f4e3aa077594ba66b86880bb4ff933e && git checkout -B drift-repro 08c8484a191f39684a0ace3565f84a7313684f5e && git merge --no-ff ab146a9a3f4e3aa077594ba66b86880bb4ff933e
node scripts/docs-audit/affected-docs.mjs --json 08c8484a191f39684a0ace3565f84a7313684f5e
|
…lter-save-door-face-parity
…s schema door now The row moves from the normalizer-refusal table to the #17551 door table, asserting 400 VALIDATION_FAILED, the member path and the face's sentence; the changeset gains the HTTP-door FROM -> TO rows. Claude-Session: https://claude.ai/code/session_01Rjy9MeetSfq34PKn81CRiN Co-authored-by: Claude <noreply@anthropic.com>
Contract reviewServed-tier: ① Derived judgments
② Semver level
③ Boundary flags
Implemented-by: VERDICT: FAIL
|
…n null-member rows Claude-Session: https://claude.ai/code/session_01Rjy9MeetSfq34PKn81CRiN Co-authored-by: Claude <noreply@anthropic.com>
Contract reviewServed-tier: ① Derived judgmentsDelta review of
② Semver level
③ Boundary flags
Implemented-by: VERDICT: PASS |
Part of #20116
Clause-②: no
Summary
FilterConditionSchema, the save door behind every stored filter, now refuses every comparand slot the query faces refuse. The datasetfilterand measurefilteralso refuse those slots inside a nested-relation condition, through #20207's walk. This closes every member the collector lists. The remainder named below is new members this run found, so this PR saysPart of, notFixes.The judge is the shared comparand-shape face itself (
assertListComparandShapes), called read-only per slot. So the save door refuses exactly what the face refuses on every query, and passes what it passes. The two boolean flags, which that face does not judge, are refused on the predicate every flag face uses: the comparand is not a boolean.Measured before (
origin/mainaf32cf9a)For every member,
FilterConditionSchema,DatasetSchema.filter, a dataset measurefilter, a dashboard widgetfilterand a reportruntimeFilterall answeredsuccess: true. The query faces answered as below. Probe run from the worktree; "face" isassertListComparandShapes, "analytics" isnormalizeWhereComparands.$and)$and/ nested){ stage: { $null: 'x' } },$exists: 'false',$null: null,$exists: 1INVALID_FILTER/ 400, every position{ amount: { $gt: null } },$lte: null{ stage: { $in: 'won' } },$nin: 'won'{ stage: { $in: ['won', null] } },$nin: [null]{ amount: { $between: [null, 5] } },5,[1],[1, 2, 3],['', 5],[{ $field: 'a' }, 5]{ stage: { $ne: ['won', 'lost'] } },$ne: []Controls answered accept on every door and face:
$null: true,$exists: false,$ne: null,$eq: null,$gt: { $field },$ne: { $field },$in: [],$nin: [],$between: [1, 5],$between: [' ', 'M'],$in: [{ $field }],$gt: true.What changes
packages/spec/src/data/filter-save-door-refusals.ts(new, not in thedatabarrel). It holdsreportQueryFaceRefusals, the one function both walks call. It asks the face about one slot,{ [field]: comparand }or{ [field]: { [op]: comparand } }, and reads only anINVALID_FILTERthrow as a verdict; anything else is rethrown. Then it picks the words (below). It applies the flag rule to$null/$exists. It is a module of its own becausedata/index.tsre-exportsfilter.zod.tswhole, so an export there would be published API. It cannot importfilter.zod.tswithout a cycle, so the enforced operator slots (FieldOperatorsSchema) are passed in.FilterConditionSchema's walk (checkFilterConditionComparands) asks that function about every implicit comparand and every operator of a field entry, at depth 0: this node's own field entries, and every$and/$or/$notmember through the schema's own re-parse. That is the face's reach. The walk's hand-written equality-slot checks ([finding]FilterConditionSchemastill PARSES{ field: [...] }and{ field: { $eq: [...] } }, so a stored dataset or widget filter publishes clean and is refused at query time on every backend #19889) are now two of the face's arms, with the same sentence and the same path.dataset.zod.ts, renamedrefuseNestedRelationEqualityLists→refuseNestedRelationComparands) asks the same function about every entry INSIDE a nested relation. That is the analytics door's reach, which flattens a relation to dotted members. The walk still decides only where; the verdict and the words are the shared function's. Its equality-list refusals keep their sentence and path.at PATHmid-sentence in seven different shapes (at P.,(at P),(at P[i]),at P[i] of, …). So the face is the JUDGE and not the text source; the text is chosen per arm as listed next. The [finding]FilterConditionSchemastill PARSES{ field: [...] }and{ field: { $eq: [...] } }, so a stored dataset or widget filter publishes clean and is refused at query time on every backend #19889 / fix(spec)!: refuse an array under $ne at the shared comparand-shape face and at FieldOperatorsSchema.$ne #20204 precedent moved the text into a shared module first, which would edit the face (⛔ in this order).The words, per arm (every new or changed refusal text, quoted)
$eqstage,stage.$eqarrayEqualityComparandMessage, shared with the face$ne(route A)stage.$nearrayInequalityComparandMessagewith the field, the face's sentence less its locationnullordering comparandamount.$gtFieldOperatorsSchema's slot for the same comparandnull$in/$ninmemberstage.$in.1null, blank or{ $field }$betweenendpointamount.$between.0$in/$ninstage.$innonListComparandErrorsentence lessat PATH; the slot has only zod's generic wording$betweenthat is not a pairamount.$betweenmalformedRangeComparandErrorsentence lessat PATH$null/$existsstage.$nulldriver-sql's first sentence word for word, then the analytics door's reason and prescriptionThe texts as printed (
FilterConditionSchema.safeParse):A nested member on a dataset carrier prints the same sentence as its top-level form, with the leaf field named, at its own path, for example
filter.acct.stage.$in.1.The changeset also carries a FROM → TO table for the HTTP doors.
POST /analytics/dataset/queryanswers a one-bound$betweeninselection.runtimeFilterwith400 VALIDATION_FAILED, located on the member, instead of400 INVALID_FILTERfrom the normalizer, andPOST /analytics/queryrefuses the same shape inwhereat its request schema. It also carries rows for the producer's two spellings,$in: [null, ""]and$nin: [null, ""].Changed docblocks
checkFilterConditionComparands: a new section, "Every slot the query faces refuse is refused on save (spec: FilterConditionSchema (the save-time door) admits a non-boolean $null / $exists flag, so a stored dataset or widget filter carrying one saves clean and is refused 400 on every query face #20116)". The old bullet "⛔$neis not judged by this walk" is replaced; it was made false.inequalityComparandSchema: the scope note "its own walk … does not judge$ne" now says the walk refuses the same shape through the face.refuseNestedRelationComparands: the "⛔ Not judged here:$ne… and the face's OTHER arms inside a nested relation" paragraph is replaced by "Every slot the door refuses inside a relation (spec: FilterConditionSchema (the save-time door) admits a non-boolean $null / $exists flag, so a stored dataset or widget filter carrying one saves clean and is refused 400 on every query face #20116)".What stays accepted (pinned, both doors)
The null predicate (
$eq: null,$ne: null),$null/$existstrueandfalse, and a{ $field }reference as the whole comparand of$eq/$ne/$gt/$gte/$lt/$lte. Also a column-to-column range as two bounds,$in: []/$nin: [],$inwith a{ $field }member (the face does not judge members), a whitespace or falsy$betweenendpoint, and relation traversal with no operator. On the shared schema (every carrier but the two dataset ones), any member shape INSIDE a nested relation stays accepted: neither the face nor the drivers' flag checks descend one.Remainder, named (why
Part of)Every member the collector lists (
5854575239,5854887743, and this card's own) is closed. This run measured two more positions of the same family:filterand a reportruntimeFilter. Both reach the analyticswheredoor, which refuses every member inside a relation:dataset-executor.tssendscombineFilters(compiled.filter, selection.runtimeFilter)to it. Both still save the shape, because fix(spec)!: a dataset or measure filter with a list inside a nested relation is refused on save (#20080) #20207's carrier refinement sits on the two dataset carriers only (triage record5825670610). The same gap holds for A list inside a nested-relation condition in a dataset or measure filter ({ account: { region: ['a'] } }) passes the save-time schema door and is refused only when the chart runs #20080's own equality-list shape on these two carriers. Closing it means applying the same refinement toDashboardWidgetSchema.filterand the reportruntimeFilters, which are outside this order's file surface.normalizeFilterComparandTypesrefuses a plain-object orMapcomparand, for example{ stage: { $eq: { a: 1 } } }or{ stage: { $in: [{ a: 1 }] } }. The analytics door refuses both, in every position, and every save door accepts both. The collector's enumeration names the shape face's tables and the flag arm, not the type face, so this PR does not move it.Producer census (Zone 2 answer 4)
A literal-comparand
git grep -Pfor each member shape, with a lit control per shape, over non-test, non-doc files:$null/$existsboolean,$in: [,$between: [)examples/**ataf32cf9apackages/**ataf32cf9af8a9d0fb05packages/fields/src/widgets/FilterConditionField.tsx:240-241main48d70663abThe producer is objectui's filter-condition widget.
condToMongowrites "is empty" as{ [field]: { $in: [null, ''] } }and "is not empty" as{ [field]: { $nin: [null, ''] } }. Both are offered by default for text, number, date, select and lookup fields.field.form.tsputs this widget onrelatedListFilterand on a rollup'ssummaryOperations.filter, bothFilterConditionSchemacarriers, and it also editssys_sharing_rule.criteria_json. The face has refused anulllist member on every query since the 2026-08-31 ruling, so such a filter already fails its related list or rollup. After this PR, the Studio save is refused instead, at the slot, with the$or/$nullprescription. No D2 conversion: the 2026-08-31 ruling declined to give anulllist member any meaning, so a conversion would have to invent one. The producer fix is objectui's and is reported to the PM for routing. Blind spot: a multi-line literal or a runtime-built comparand is not matched by a line grep.Tests
All heavy runs went through
scripts/pm/os-verify-lock.sh. Each reading names the tree it was taken on.pnpm --filter @objectstack/spec build,check:generated,typecheckc300e80e(final)@objectstack/specfull suite (vitest run --maxWorkers=2)c300e80e(final)@objectstack/specfull suite70e9a610plus the regenerated registry (31ddfa40)filter-save-door-face-parity.test.ts,dataset-filter-nested-relation-list.test.ts)70e9a610, after the ablations were restored@objectstack/service-analyticsfull suite (thewheredoor)70e9a610@objectstack/lintfull suite (validate-chart-bindingsand the dataset readers)70e9a610dispatch-gates.mjs --commands, 87 lines)c300e80echeck:dual-build-cjs-loads,check:type-check-debt, both need the whole-repo build);--ranreconciliation: 87 accounted, 0 unrunPatch round (review 5856977110, CI
Test Core (4/6)atc300e80e).packages/rest/src/analytics-filter-refusal-envelope.test.tsasserted the one-bound$betweenas400 INVALID_FILTERfrom the normalizer.DatasetSelectionSchema.runtimeFilterandAnalyticsQueryRequestSchema.whereareFilterConditionSchema, so the route's schema door now answers first. Readings onab146a9a(the tree batch F ran on; the batch header readsdfca00f0because the last commit, a test and changeset edit, landed before its spec step): rest full suite (--project local) 201 files, 3576 passed, 1 skipped; the flipped file 31 passed; service-analytics full suite 129 files, 3041 passed; spec full suite 581 files, 16770 passed, 1 todo (was 2: the$ne§5 todo is now a pin); speccheck:generatedexit 0; derived gates 87 derived, 85 exit 0, 2 exit 3 PREREQUISITE NOT MET (check:dual-build-cjs-loads,check:type-check-debt),--ran87 accounted. Consumer sweep: agit grep -Pfor every refused member shape over all non-spec test files found 62 files. Only this one drives a request or schema door (DatasetSchemaorDatasetSelectionSchemaon/analytics/dataset/query,AnalyticsQueryRequestSchemaon/analytics/query,ObjectSchemaat registration) with a refused shape. Every other hit is an enginewhere, an RLSscope, a driver input or a comment.Pin sweep. Two published-behaviour pins flipped and were rewritten to assert the new semantics with their substance:
filter.test.ts"is not judged by the loose FilterConditionSchema — and neither is the [spec]FieldReferenceSchemais declared in the$betweenendpoints but NO backend resolves a$fieldinside a list #7596 shape" became "is refused by FilterConditionSchema too, at the endpoint, in the operator slot's words". It asserts the pathage.$between.1and equality withFieldOperatorsSchema's message for the same pair.fix(spec)!: a dataset or measure filter with a list inside a nested relation is refused on save (#20080) #20207's §4 control row "
$necarrying a list … not yet at this save door (spec: FilterConditionSchema (the save-time door) admits a non-boolean $null / $exists flag, so a stored dataset or widget filter carrying one saves clean and is refused 400 on every query face #20116)" became a refusal test on both carriers. It assertsINVALID_FILTER/ 400 at the analytics door, then the carrier's message equal to the door's less its location, and the$ninremedy.Patch round:
packages/rest/src/analytics-filter-refusal-envelope.test.ts. "a $between with one bound → 400 INVALID_FILTER" moved from the normalizer table to the[#17551]door table: it now asserts 400,VALIDATION_FAILED, exactly onedetails.fields[]entry atselection.runtimeFilter.amount.$between, and the face's sentence withoutat where.. The sibling-schema control now asserts the refusal atwhere.amount.$betweenin the same sentence. The pass-list control is three spellings, not four. Its service-analytics: object-form analyticswhereskips the shared comparand-shape face's other arms ($innull member,$gt: null, null/blank$betweenbound, scalar$in) that the FilterArray spelling refuses 400 #20010 note moved with the row and still holds: the sentence is still the face's.Patch round:
packages/spec/src/data/filter-ne-array-schema-door.test.ts§5. Itsit.todo("the carrier still saves the shape") is fulfilled by this PR's$nearm. It is now a real pin: pathstage.$ne, message equal to the face's less its location, the$ninremedy, a dataset carrier atfilter.$or.0.stage.$ne, and a null / scalar control.Ablation (reverse verification), one-off, no permanent file. Each leg went through
scripts/ablation-replace.mjsWRAP mode from the committed tree70e9a610. The anchor hit exactly 1 time (x1 → x0, replacement x0 → x1), and the restore was proven by blob hash equal to HEAD with an emptygit diff HEAD:if (face && false))depth === 0removed)$eq/ §5 rows)Acceptance notes
nullOrderingComparandMessage(./filter.zod.ts)" and so on. Those builders did not move; the new module reads the same sentences offFieldOperatorsSchema's slots. No edit to the face was needed or made.FieldOperatorsSchema.$in/$nin/$between/$null/$existsstill print zod's generic wording for a non-list, a malformed range and a non-boolean flag.FilterConditionSchemaprints the pointed sentences above. Pointing the operator slots too is polish, and is not done here.Generated by Claude Code