feat(query): existence tests — scoped predicates, M2M, error surfaces & docs (slices 2–4)#323
Merged
Merged
Conversation
…s-scope guard The existence test's optional inner lambda is a full ferro predicate over the child model, resolved through the same validating QueryProxy a root where() receives: every operator, &/|/~ composition, forward-FK traversal, and nested existence tests to arbitrary depth. The explicit lambda scope is what makes multi-condition grouping unambiguous — exists(lambda l: A & B) (one child row matches both) vs exists(A-test) & exists(B-test) (some child row each) — the Django-style implicit-traversal ambiguity ADR-0007 rejects. - Forward traversal inside the test renders its joins INSIDE the EXISTS subquery under unchanged ADR-0006 semantics (INNER at every hop, narrowing). The hop facts ride the exists node's own `joins` section (QueryJoin entries, always "inner"), serialized only when non-empty — bare-test wire bytes are unchanged from the tracer bullet. - Nesting is recursion, not a second mechanism: a nested test's correlation hop resolves against the enclosing SUBQUERY's alias through the ExistsScope qualifier, pinned by a Rust render test and depth-2 e2e coverage. - Cross-scope references are rejected at build time pointing at the deferred capability (#309): a leaf built from any proxy other than the inner lambda's parameter (detected via a compile-side `owner` scope tag the proxies now stamp on comparison nodes), a FieldProxy as comparison RHS (column-to-column), and a nested test built from another scope's proxy. Silent misrendering is the failure mode this guard exists to prevent. - Typed binds inside the subquery resolve against the leaf's OWNING table: the SELECT walkers now populate registrations and enum catalogs for every table an existence test reaches (correlation hops, inner join hops, nested tests). - Hand-authored golden vectors pin the scoped shape (inner tree + inner-traversal joins) and the nested exists-in-exists shape, asserted from the Python emitter and the Rust decoder. - End-to-end on both backends: the full #308 line-aware category filter (three matching lines → one root row; child-less roots survive the OR; keyset order_by + limit compose), the explicit grouping contrast, forward traversal inside the subquery, depth-2 nesting, negated and counted scoped tests, and every pinned cross-scope error. Refs #311, #315
t.tags.exists(...) works on many-to-many relations, bare, scoped, and ~-negated, spelled and behaving exactly like the reverse-FK form (ADR-0007: one verb at every cardinality and relation kind). The same exists node carries a two-hop correlation path — join table first, correlated to the enclosing scope, then the target — and the existing Rust render loop covers it unchanged: no new render mechanism, proving the design's claim that M2M is test surface, not a second mechanism. - build_reverse_specs maps M2M descriptors to join-table specs; each side's descriptor already orients source_col/target_col for that side, so both the declaring and the related_name side spell identically. - The proxy builds hops from the spec: join_table.source_col against the root PK, then target on join_table.target_col = target.pk. The inner lambda resolves over the M2M target with full predicate power (operators, combinators, negation, cross-scope guard) — the #315 machinery, untouched. - Hand-authored golden vector pins the 2-hop shape, asserted from the Python emitter and the Rust decoder; a Rust render test pins join-table-FROM + inner-join-target + last-hop-alias qualification. - e2e on both backends: bare (each root exactly once regardless of how many join rows match), scoped, negated, both sides of the relation, root-predicate/count composition, and the proxy's single-verb error surfaces. Refs #311, #316
…dges Every remaining surface where a reverse or M2M relation can be named now answers with the supported spelling, and the docs teach existence tests end to end. Error surfaces beyond the proxy's own (those shipped in the tracer bullet): - join()/left_join() on a reverse/M2M edge stays rejected — the pinned "a join never multiplies root rows" property is preserved by rejection, not implicit DISTINCT — with the error naming .exists() (previously an opaque "must return a relation path" TypeError). - in_() with a query RHS stays a TypeError whose message names the existence test when the RHS is a query (the #307 repro's second guess); a non-query RHS keeps the plain message. - include() rejection verified unchanged for M2M edges too — population remains a separate future mechanism. Docs: - New guide section "Existence Tests on Reverse & Many-to-Many Relations": real models (Assignment + Annotated tabs), lambda-style predicates throughout, rendered SQL for the #307/#308 shapes, the explicit grouping contrast, negation, traversal-inside-the-test, nesting, M2M, the loud dead ends, and the exists-test vs query-terminal disambiguation. Backed by two new runnable example scripts exercised by the docs harness. - Operators table gains the .exists() row; the forward existence/absence section and the reverse-relation filtering section cross-link the reverse direction. - Reference (api/queries.md) states the one-verb rule and its rejections. - Roadmap and "Not Yet Supported" now distinguish reverse predicates (shipped — the existence test) from reverse population (still future; the include() rejection stays), and track cross-scope correlation as #309. Refs #311, #317
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #315, closes #316, closes #317, closes #311, closes #307, closes #308
Slices 2–4 of the existence-tests PRD (#311, ADR-0007), rebased onto main after the tracer bullet (#319) merged. Three commits, one per slice, structured for a rebase merge — each is independently green (tests, both backends, wire vectors) and matches its sub-issue's acceptance criteria. Supersedes the stacked PRs #320, #321, #322 (closed; identical content, tree byte-identical to the verified stack tip).
Commit 1 — scoped inner predicates (#315)
The optional inner lambda is a full ferro predicate over the child model: every operator,
&/|/~, forward traversal, nested tests to arbitrary depth.joinssection, present only when non-empty — bare-test wire bytes unchanged, no version bump.featCross-scope correlation in existence tests (deferred from ADR-0007) #309): a leaf built from any proxy other than the inner lambda's parameter (via a compile-sideownerscope tag — same-named columns can't silently rescope), aFieldProxyas comparison RHS, and a foreign nested test all raise at build time namingfeatCross-scope correlation in existence tests (deferred from ADR-0007) #309.featOR-composition across a left-joined reverse child table (PRD from Pinch) #308 shape (multi-line match → one root row; child-less root survives the OR; keysetorder_by+limitcompose), the explicit grouping contrast (exists(A & B)vsexists(A) & exists(B)— different, correct row sets), golden vectors for scoped + nested shapes.Commit 2 — M2M two-hop path (#316)
u.tags.exists(...)bare, scoped, negated — same verb, same node, a two-hop correlation path (join table correlated to the enclosing scope, then the target). The reverse-spec map learns M2M entries from each side's descriptor (which orients the join-table triple per side, so both sides spell identically). The existing Rust render loop covered it unchanged — zero new mechanism, as the design predicted. Golden vector + Rust render test pin the 2-hop shape; e2e covers exactly-once semantics regardless of matching join-row count.Commit 3 — error surfaces & docs (#317)
join()/left_join()on a reverse/M2M edge rejects naming.exists()(row multiplication stays forbidden by rejection, not implicit DISTINCT).in_()with a query RHS names the existence test (thefeatReverse-relation membership predicates: EXISTS from the root query (PRD from Pinch) #307 repro's second guess); non-query RHS keeps the plain message.in_(subquery)stays declined, not deferred.include()rejection verified unchanged for M2M too — population remains a separate future mechanism.featReverse-relation membership predicates: EXISTS from the root query (PRD from Pinch) #307/featOR-composition across a left-joined reverse child table (PRD from Pinch) #308 shapes, grouping contrast, negation, traversal-inside, nesting, M2M, the loud dead ends), two runnable example scripts under the docs harness, operators-table row, reference update, and roadmap/"Not Yet Supported" now distinguishing reverse predicates (shipped) from reverse population (future), with cross-scope correlation tracked asfeatCross-scope correlation in existence tests (deferred from ADR-0007) #309.Testing (at the tip)
pytest --db-backends=sqlite,postgres: 1679 passed (full matrix, local Postgres)cargo test: 196 + 28 (ferro-schema-ir) passedty check/ruff check: clean; docs example harness greenMust-pass demos from the PRD, all covered e2e on both backends: both #307
is_transferbranches (membership via either FK column, each root exactly once), the #308 shape, the grouping contrast, depth-2 nesting, M2M bare + scoped + negated.