Skip to content

perf(index): norm-addend cache and slim top-k heap for FTS scoring#7629

Draft
BubbleCal wants to merge 1 commit into
yang/lan2-88-fts-simd-merge-kernelsfrom
yang/lan2-88-fts-norm-cache
Draft

perf(index): norm-addend cache and slim top-k heap for FTS scoring#7629
BubbleCal wants to merge 1 commit into
yang/lan2-88-fts-simd-merge-kernelsfrom
yang/lan2-88-fts-norm-cache

Conversation

@BubbleCal

@BubbleCal BubbleCal commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

Stacked on #7625; last of the series. (Re-opened as a new PR after #7626 was consolidated into #7466 and its branch deleted, which auto-closed the original #7627.)

What

Two hot-loop changes, both score-identical (only tie ordering among equal scores can shift with the slimmer heap tuple):

  • Per-search norm cache (Lucene's norm cache): Scorer::doc_norm exposes the BM25 doc-length denominator addend, and quantized (V3) searches bake the 256 possible addends once per partition-search. The OR streaming loop (collect_window_scores), the OR drain/single-essential completion paths, and the bulk conjunction scoring pass then score with one byte-norm load plus a cached addend instead of recomputing k1*(1-b+b*dl/avgdl) per doc. The factored expressions are evaluated identically, so scores are bit-equal to the uncached path.
  • Slim top-k heap: heap entries shrink to a Copy tuple; the per-candidate (term, freq) pairs go to an append-only side log (FreqLog) and only the final top-k materialize a Vec — no more per-insert allocation, and heap sifts move 24 bytes instead of a Vec-carrying tuple.

Measured vs #7625 (its base)

Per-branch-tip wheels, 1000 queries × 8 concurrent. OR and AND on the
200M-doc v3-256 index (warm, prewarmed); phrase on the 50M-doc v3-256
positions index (steady-state passes, no prewarm):

query #7625 this PR
OR 3w k10 0.0343s / 230 qps 0.0250s / 316 qps (1.37×)
OR 3w k100 0.0629s / 127 qps 0.0470s / 169 qps (1.34×)
AND 3w k10 0.0459s / 172 qps 0.0447s / 177 qps (+3%)
AND 3w k100 0.0924s / 86 qps 0.0892s / 89 qps (+3.5%)
phrase 3w k10 @50m 0.2256s 0.2187s (+3%)

The OR win comes from the streaming loop: recomputing the BM25 denominator
per (clause × doc) was its largest single cost.

Verification

  • A/B vs the previous stack tip: score_diff=0 across AND (bulk and classic) and phrase query sets; 34 tie-reorders among equal scores from the heap-tuple change, as expected.
  • cargo test -p lance-index, clippy -D warnings, fmt --check clean.

🤖 Generated with Claude Code

Two hot-loop changes, both result-identical (scores are bit-equal; only
tie ordering among equal scores can shift with the slimmer heap tuple):

- Scorer::doc_norm exposes the BM25 doc-length denominator addend, and
  quantized (v3) searches bake a per-norm-code addend cache once per
  partition search (Lucene's norm cache). The OR streaming/drain loops
  and the bulk conjunction pass score with one byte-norm load plus the
  cached addend instead of recomputing the denominator per doc — the
  factored expressions are evaluated identically, so scores don't move.
- Top-k heap entries shrink to a Copy tuple; the per-candidate
  (term, freq) pairs go to an append-only side log and only the final
  top-k materialize a Vec, removing the per-insert allocation.

Warm mmlb, 8 concurrent: OR@200M k10 0.0343->0.0249s (318qps), k100
0.0628->0.0467s (170qps) — both now ahead of Lucene 10.4's sliced
searcher; AND@200M k10 0.0456->0.0443s, k100 0.0916->0.0883s; phrase@50M
2w 0.0392->0.0370s.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions github-actions Bot added A-index Vector index, linalg, tokenizer performance and removed A-index Vector index, linalg, tokenizer labels Jul 4, 2026
@BubbleCal BubbleCal marked this pull request as draft July 5, 2026 17:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant