Skip to content

feat(query): scoped existence tests — inner predicates, nesting, cross-scope guard#320

Closed
0x054 wants to merge 2 commits into
mainfrom
feat/existence-tests-315-scoped-exists
Closed

feat(query): scoped existence tests — inner predicates, nesting, cross-scope guard#320
0x054 wants to merge 2 commits into
mainfrom
feat/existence-tests-315-scoped-exists

Conversation

@0x054

@0x054 0x054 commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

Closes #315

Stacked on #319 (slice 1/4); base retargets to main when that merges.

Slice 2/4 of the existence-tests PRD (#311, ADR-0007): the optional inner lambda, resolved as a full ferro predicate over the child model through the same validating proxy a root where() receives.

What ships

# 308: line-aware category filter; child-less roots survive the OR
Txn.where(lambda t: t.category_id.in_(ids)
                  | t.lines.exists(lambda line: line.category_id.in_(ids)))

renders

SELECT ... FROM "txn"
WHERE "txn"."category_id" IN (...)
   OR EXISTS (SELECT 1 FROM "line" AS "x1_lines"
              WHERE "x1_lines"."txn_id" = "txn"."id"
                AND "x1_lines"."category_id" IN (...))
  • Full predicate power inside the test. Every operator, &/|/~ composition, forward-FK traversal, and nested existence tests to arbitrary depth. The inner lambda's explicit scope is what makes grouping unambiguous: exists(lambda l: A & B) (one child row matches both) and exists(A-test) & exists(B-test) (some child row each) are different, correct row sets — covered e2e.
  • Traversal joins render INSIDE the subquery under unchanged ADR-0006 semantics (INNER at every hop, narrowing — pinned e2e by a NULL-FK child that never matches a traversed inner predicate). On the wire, the hop facts ride the exists node's own joins section (always "inner"), present only when non-empty — the bare-test bytes from slice 1 are unchanged, so no version bump.
  • Nesting is recursion. A nested test's correlation hop resolves against the enclosing subquery's alias through the ExistsScope qualifier — pinned by a Rust render test ("x2_transactions"."account_id" = "x1_accounts"."id"), depth-2 e2e, and a golden vector.
  • Cross-scope guard (feat Cross-scope correlation in existence tests (deferred from ADR-0007) #309). The moment inner lambdas exist, three build-time rejections land, each pointing at the deferred capability: a leaf built from any proxy other than the inner lambda's parameter (detected via a compile-side owner scope tag the proxies now stamp on comparison nodes — same-named columns can't silently rescope), a FieldProxy as comparison RHS (column-to-column), and a nested test built from another scope's proxy. Messages pinned by tests.
  • Typed binds inside the subquery resolve against the leaf's owning table: the SELECT walkers populate registrations + enum catalogs for every table an existence test reaches (correlation hops, inner-join hops, nested tests, any depth).

Acceptance criteria (#315)

  • Inner lambda supports every operator and &/|/~ over the child model
  • Forward traversal inside the subquery renders joins within the EXISTS under ADR-0006 semantics (covered e2e)
  • Nested exists-in-exists works; depth 2 pinned e2e and by a golden vector
  • The feat OR-composition across a left-joined reverse child table (PRD from Pinch) #308 shape passes e2e on both backends: multi-line match → one root row; child-less root survives the OR; keyset composition
  • Explicit grouping semantics covered: exists(lambda l: A & B) vs exists(A) & exists(B) return different, correct row sets
  • Cross-scope reference and field-proxy-as-RHS raise build-time errors naming feat Cross-scope correlation in existence tests (deferred from ADR-0007) #309 (messages pinned by tests)
  • Hand-authored golden vectors: scoped 1-hop exists, nested exists

Testing

  • cargo test: 196 + 27 (ferro-schema-ir) passed, including scoped/nested render tests and both new vector round-trips
  • pytest --db-backends=sqlite,postgres: 1653 passed (full matrix, local Postgres)
  • ty check and ruff check: clean

0x054 added 2 commits July 18, 2026 10:58
A reverse (BackRef) relation now appears in a predicate in exactly one
form: the existence test t.rel.exists(), rendered as a correlated
EXISTS (SELECT 1 FROM child WHERE child.fk = root.pk) at every
cardinality — one-to-one BackRefs included, no LEFT JOIN
specialization — and negated with the uniform ~ (NOT node, ADR-0008).
The result stays root-shaped, so it composes with any other predicate,
root order_by, and limit (ADR-0007: reverse relations are tested, not
traversed).

- A reverse-spec map (ReverseSpec) is derived at the compile choke
  point beside __ferro_relation_specs__, from the facts
  resolve_relationships already computes on RelationshipDescriptor
  (child model, child FK column, one-to-one flag). M2M entries are
  skipped until the two-hop slice.
- QueryProxy consults it ahead of the column fallback and returns a
  ReverseRelationProxy exposing .exists() only: column access,
  comparisons (including != None / == None), in_, like, and a bare
  proxy in where() all raise at build time naming .exists().
- Wire: QueryIR v7 adds the recursive `exists` node kind
  {hops, where} beside leaf/compound/not — hops reuses the
  QueryJoinHop vocabulary (one hop for a reverse FK), where is an
  ordinary inner condition tree (empty in this slice). Hand-authored
  golden vectors pin the bare 1-hop and not-of-exists shapes, asserted
  from the Python emitter and the Rust decoder; existing query vectors
  move to v7.
- Rust: one render loop — the first hop's table is the subquery FROM
  correlated to the enclosing scope's alias, remaining hops render as
  inner joins inside the subquery (M2M-ready), the inner tree recurses
  through the existing condition builder under an ExistsScope
  qualifier. Statement-wide x{n}_{relation} aliases keep sibling and
  nested tests collision-free.
- Mutating verbs reject existence tests at build time (single-table
  write shapes), with Rust boundary defense behind the Python guard.
- include() population rejection (#287) is preserved now that the
  proxy resolves reverse names before the AttributeError path.
- End-to-end on both backends: both #307 is_transfer branches
  (membership via either FK column, each root exactly once), to-many
  exactly-once semantics, &/| and order_by/limit composition, count(),
  and every pinned error surface.

Refs #311, #314
…s-scope guard

The existence test's optional inner lambda is a full ferro predicate
over the child model, resolved through the same validating QueryProxy a
root where() receives: every operator, &/|/~ composition, forward-FK
traversal, and nested existence tests to arbitrary depth. The explicit
lambda scope is what makes multi-condition grouping unambiguous —
exists(lambda l: A & B) (one child row matches both) vs
exists(A-test) & exists(B-test) (some child row each) — the
Django-style implicit-traversal ambiguity ADR-0007 rejects.

- Forward traversal inside the test renders its joins INSIDE the EXISTS
  subquery under unchanged ADR-0006 semantics (INNER at every hop,
  narrowing). The hop facts ride the exists node's own `joins` section
  (QueryJoin entries, always "inner"), serialized only when non-empty —
  bare-test wire bytes are unchanged from the tracer bullet.
- Nesting is recursion, not a second mechanism: a nested test's
  correlation hop resolves against the enclosing SUBQUERY's alias
  through the ExistsScope qualifier, pinned by a Rust render test and
  depth-2 e2e coverage.
- Cross-scope references are rejected at build time pointing at the
  deferred capability (#309): a leaf built from any proxy other than
  the inner lambda's parameter (detected via a compile-side `owner`
  scope tag the proxies now stamp on comparison nodes), a FieldProxy as
  comparison RHS (column-to-column), and a nested test built from
  another scope's proxy. Silent misrendering is the failure mode this
  guard exists to prevent.
- Typed binds inside the subquery resolve against the leaf's OWNING
  table: the SELECT walkers now populate registrations and enum
  catalogs for every table an existence test reaches (correlation hops,
  inner join hops, nested tests).
- Hand-authored golden vectors pin the scoped shape (inner tree +
  inner-traversal joins) and the nested exists-in-exists shape,
  asserted from the Python emitter and the Rust decoder.
- End-to-end on both backends: the full #308 line-aware category filter
  (three matching lines → one root row; child-less roots survive the
  OR; keyset order_by + limit compose), the explicit grouping contrast,
  forward traversal inside the subquery, depth-2 nesting, negated and
  counted scoped tests, and every pinned cross-scope error.

Refs #311, #315
Base automatically changed from feat/existence-tests-314-bare-exists to main July 18, 2026 15:39
@0x054

0x054 commented Jul 18, 2026

Copy link
Copy Markdown
Contributor Author

Superseded by #323 (slices 2–4 combined into one rebase-mergeable PR per maintainer preference; content identical).

@0x054 0x054 closed this Jul 18, 2026
@0x054
0x054 deleted the feat/existence-tests-315-scoped-exists branch July 18, 2026 15:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat Existence tests 2/4: scoped inner predicates — forward traversal, nesting, cross-scope guard

1 participant