perf(index): norm-addend cache and slim top-k heap for FTS scoring#7629
Draft
BubbleCal wants to merge 1 commit into
Draft
perf(index): norm-addend cache and slim top-k heap for FTS scoring#7629BubbleCal wants to merge 1 commit into
BubbleCal wants to merge 1 commit into
Conversation
Two hot-loop changes, both result-identical (scores are bit-equal; only tie ordering among equal scores can shift with the slimmer heap tuple): - Scorer::doc_norm exposes the BM25 doc-length denominator addend, and quantized (v3) searches bake a per-norm-code addend cache once per partition search (Lucene's norm cache). The OR streaming/drain loops and the bulk conjunction pass score with one byte-norm load plus the cached addend instead of recomputing the denominator per doc — the factored expressions are evaluated identically, so scores don't move. - Top-k heap entries shrink to a Copy tuple; the per-candidate (term, freq) pairs go to an append-only side log and only the final top-k materialize a Vec, removing the per-insert allocation. Warm mmlb, 8 concurrent: OR@200M k10 0.0343->0.0249s (318qps), k100 0.0628->0.0467s (170qps) — both now ahead of Lucene 10.4's sliced searcher; AND@200M k10 0.0456->0.0443s, k100 0.0916->0.0883s; phrase@50M 2w 0.0392->0.0370s. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #7625; last of the series. (Re-opened as a new PR after #7626 was consolidated into #7466 and its branch deleted, which auto-closed the original #7627.)
What
Two hot-loop changes, both score-identical (only tie ordering among equal scores can shift with the slimmer heap tuple):
Scorer::doc_normexposes the BM25 doc-length denominator addend, and quantized (V3) searches bake the 256 possible addends once per partition-search. The OR streaming loop (collect_window_scores), the OR drain/single-essential completion paths, and the bulk conjunction scoring pass then score with one byte-norm load plus a cached addend instead of recomputingk1*(1-b+b*dl/avgdl)per doc. The factored expressions are evaluated identically, so scores are bit-equal to the uncached path.FreqLog) and only the final top-k materialize aVec— no more per-insert allocation, and heap sifts move 24 bytes instead of a Vec-carrying tuple.Measured vs #7625 (its base)
Per-branch-tip wheels, 1000 queries × 8 concurrent. OR and AND on the
200M-doc v3-256 index (warm, prewarmed); phrase on the 50M-doc v3-256
positions index (steady-state passes, no prewarm):
The OR win comes from the streaming loop: recomputing the BM25 denominator
per (clause × doc) was its largest single cost.
Verification
cargo test -p lance-index,clippy -D warnings,fmt --checkclean.🤖 Generated with Claude Code