Skip to content

Match type filters through re-export spellings via canonical code refs - #5800

Merged
lukemelia merged 2 commits into
mainfrom
cs-12168-type-filtered-search-expands-the-adoption-chain-for-card
Aug 20, 2026
Merged

Match type filters through re-export spellings via canonical code refs#5800
lukemelia merged 2 commits into
mainfrom
cs-12168-type-filtered-search-expands-the-adoption-chain-for-card

Conversation

@lukemelia

Copy link
Copy Markdown
Contributor

What this does

A filter.type/on code ref that names a type through a re-exporting module — the canonical example being base FileDef addressed via its documented file-api module, which re-exports it from card-api — silently matched nothing. Index rows stamp types with the canonical (defining-module) key from identifyCard, so only the canonical spelling ever joined the membership keys: filtering on base FileDef returned zero rows while concrete subtype filters matched, making the base-type filter a silent dead end.

Two halves, kept in agreement:

  • Query engine: each type condition's ref now resolves through the definition lookup, and the compiled types membership predicates union the canonical ref's spelling-tolerant keys (RRI / real-URL / virtual-alias) with the as-given ref's. Because the definition lookup is async, the type condition becomes a deferred expression node resolved in pass 1, alongside the existing deferred kinds. A ref whose definition doesn't resolve keeps only its spelling-based keys and matches nothing, exactly as before.
  • Host search resource: the same rewrite is applied (via the loader + identifyCard) before client-side matching — the matcher compares refs against identifyCard of an instance's class, so an uncanonicalized re-export spelling would strip the server's correct results during live reconciliation. Until the rewrite settles for the active filter, or when a ref doesn't resolve, the search stays a server-only passthrough so the two evaluations can't disagree.

The shared walker (runtime-common/query-canonicalization.ts) rewrites type/on refs through any/every/not without mutating the input tree, and single-flights duplicate ref spellings by memoizing the in-flight promise — sibling nodes are walked concurrently, so a value-memo would double-resolve.

Test plan

  • packages/realm-server/tests/search-entries-engine-test.ts: a pure base-FileDef filter matches every file row across subtypes, and the file-api re-export spelling matches the same rows as the canonical card-api spelling (36/36 pass locally).
  • packages/host/tests/unit/query-canonicalization-test.ts: rewrite of top-level and nested refs, input-tree immutability, unresolvable refs marking the result incomplete, and duplicate refs resolving once (8/8 pass locally via the dev test page).
  • Typechecks pass across runtime-common, realm-server, and host.

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Preview deployments

Host Test Results

    1 files      1 suites   1h 13m 58s ⏱️
2 172 tests 2 164 ✅ 8 💤 0 ❌
2 181 runs  2 173 ✅ 8 💤 0 ❌

Results for commit b1ea516.

Realm Server Test Results

    1 files  ±0      1 suites  ±0   15m 22s ⏱️ +5s
2 176 tests ±0  2 176 ✅ ±0  0 💤 ±0  0 ❌ ±0 
2 256 runs  ±0  2 256 ✅ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit 2a58eb1. ± Comparison against earlier commit 7921aea.

@lukemelia
lukemelia force-pushed the cs-12168-type-filtered-search-expands-the-adoption-chain-for-card branch from bcbbc1f to 7921aea Compare August 18, 2026 20:58

@lukemelia lukemelia left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Claude Code 🤖] Review focus: the two-sided canonical-ref contract — that the server's types-membership compile and the host's client-side matcher agree on what a re-export spelling matches, and that the new deferred type-condition node composes exactly as the inline any() it replaces.

Bottom line: the design is correct — I found no correctness defect in the compiled SQL or in the server/matcher agreement. Two non-blocking findings: a stale-source race in the host's async canonicalization, and a swallow-all error path in typeKeysFor. Details inline. No blocking issues.

What lands right.

  • The mechanism checks out end-to-end. Index rows stamp types via identifyCard (the defining-module ref — makeDefinition in host/app/routes/module.ts sets codeRef = identifyCard(cardOrFieldDef)). The matcher's instanceIsType walks the instance's prototype chain comparing identifyCard(klass) through codeRefEquals, which tolerates URL-spelling variance (RRI / real-URL / virtual-alias) but not module-identity variance. So a raw file-api/FileDef filter makes instanceIsType return false and the reconciler strips the server's correct rows. Canonicalizing the filter's refs to the card-api defining-module ref on both sides is exactly the right fix, and doing it upstream in search.ts rather than inside the matcher keeps that single call site as the only consumer needing it (verified: search.ts is the only non-test caller of matchInstanceAgainstFilter / isClientEvaluable).
  • The deferred type-condition node is structurally equivalent to the old inline path. filterCondition still wraps this.typeCondition(ref) (now [typeConditionNode]) through every/any, whose addExplicitParens parenthesizes the single-element array; pass 1 splices handleTypeCondition's any(keys.map(typesContains)) — the same token shape the old code emitted — inside those parens. All three type-key sites (handleTypeCondition, hasFileType, hasInstanceType) route through typeKeysFor, so no twin is left on the old inline path.
  • typeKeysFor mutating internalKeysFor's return with .push is safe — internalKeysFor returns a fresh .map(...) array per call.
  • The canonicalization walker's single-flight memo is correct: canonicalRef runs memo.has / memo.set synchronously before its first await, so concurrent sibling walks can't both miss (the duplicate refs resolve once test pins it), and the per-node { ...node } shallow clone keeps the input tree immutable (also pinned).

Recommendations.

  1. Guard the host .then against a stale filter so an out-of-order loader.import resolution can't stickily disable client reconciliation — see the loadCanonicalizedFilter thread. (non-blocking)
  2. Consider distinguishing "definition not found" from a transient lookup failure in typeKeysFor, so a valid re-export ref doesn't silently under-match under load — see the typeKeysFor thread. (non-blocking)

Adjacent, out of scope. The realm test-results status comment reports realm-server tests as ±0 against the base even though this PR adds two tests to search-entries-engine-test.ts — most likely a status-comment/base-comparison artifact rather than the tests not running, but worth a glance to confirm they executed in CI.

Comment thread packages/host/app/resources/search.ts
Comment thread packages/runtime-common/index-query-engine.ts Outdated
@lukemelia
lukemelia force-pushed the cs-12168-type-filtered-search-expands-the-adoption-chain-for-card branch from 2a58eb1 to 6ddc33d Compare August 19, 2026 15:56
@lukemelia
lukemelia marked this pull request as ready for review August 19, 2026 16:52
@lukemelia
lukemelia requested a review from a team August 19, 2026 16:52
lukemelia and others added 2 commits August 19, 2026 13:34
A filter whose `type`/`on` ref named a type through a re-exporting module
(e.g. file-api's FileDef, re-exported from card-api) silently matched
nothing: index rows stamp `types` with the canonical (defining-module) key
from identifyCard, so only the canonical spelling joined the membership
keys. Filtering on base FileDef via its documented file-api module returned
zero rows while subtype filters matched.

The query engine now resolves each type condition's ref through the
definition lookup and unions the canonical ref's spelling-tolerant keys with
the as-given ref's, so both spellings compile to the same `types` membership
predicates. The type condition becomes a deferred expression node
(pass-1-resolved) since the definition lookup is async.

The host's search resource applies the same rewrite before client-side
matching — the matcher compares against identifyCard of the instance's
class, so an uncanonicalized re-export spelling would strip the server's
correct results during reconciliation. Until the rewrite settles for the
active filter (or when a ref doesn't resolve), the search stays a
server-only passthrough.

The shared canonicalization walker single-flights duplicate ref spellings
by memoizing the in-flight promise: sibling filter nodes are walked
concurrently, and a value-memo would let both siblings miss.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… errors

Two robustness fixes from review of the canonical-code-ref type matching:

- SearchResource.loadCanonicalizedFilter guards its `.then` against a stale
  filter. Sibling canonicalizations race and a cache-warm loader import can
  resolve ahead of a cold one, so an earlier filter's resolution could land
  last and overwrite `canonicalizedFilter` with a `source` no longer matching
  the active query. That left `isClientFilterEligible` false and the search
  stuck in server-only passthrough until the next query change. The guard drops
  a resolution whose filter is no longer the active one.

- IndexQueryEngine.typeKeysFor no longer swallows every lookup error. A ref
  that resolves to no known type still falls through to the spelling-based keys
  (matching nothing unless rows were stamped under that spelling), matching how
  the engine's top-level catch treats a nonexistent type as an empty result;
  any other, unexpected error now propagates instead of silently narrowing the
  match.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@lukemelia
lukemelia force-pushed the cs-12168-type-filtered-search-expands-the-adoption-chain-for-card branch from 6ddc33d to b1ea516 Compare August 19, 2026 17:35
@lukemelia
lukemelia merged commit e3a71a2 into main Aug 20, 2026
83 of 102 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants