perf(fts): cache the num_tokens-only DocSet on LazyDocSet#7600
Merged
Conversation
ensure_num_tokens_loaded rebuilt the DocSet from the cached arrow column on every call, copying the whole num_tokens array (tens of MB per partition) once per query per partition — profiled at ~50% of single-term query time and a large slice of every warm query. Materialize it once in a OnceCell like the full DocSet, and account for it in deep_size_of.
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
This was referenced Jul 4, 2026
Xuanwo
approved these changes
Jul 6, 2026
BubbleCal
pushed a commit
that referenced
this pull request
Jul 6, 2026
…igurable Reconciles #7600's tokens_only DocSet cache with this branch's quantized_scoring flag: the cached num_tokens-only DocSet is built with quantized scoring applied inside the OnceCell init, matching how ensure_loaded stamps the full DocSet.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ensure_num_tokens_loadedrebuilt the num_tokens-onlyDocSetfrom thecached arrow column on every call, copying the whole column (tens of MB per
partition) once per query per partition. Materialize it once in a
OnceCelllike the full DocSet, and account for it in
deep_size_of.Measured vs its merge-base
Per-branch-tip wheels on the 200M-doc legacy index (338 partitions), 1000
queries × 8 concurrent, 400G index cache. The rebuild only happens while a
partition's DocSet is deferred (not yet fully materialized), so the win shows
on the not-yet-prewarmed path, and the fix is exactly neutral once the index
is warm:
In the stacked V3 setup (#7466 and its followers) partitions serve queries
from the tokens-only DocSet indefinitely, where this same rebuild dominated
hot single-term queries (250ms → 3ms when the cache landed there).
Independent of the block-size/impact PR stack; applies to any FTS query.
🤖 Generated with Claude Code