Skip to content

Faster byte range cache - #6683

Open
PSeitz wants to merge 4 commits into
quickwit-oss:mainfrom
PSeitz:faster_byte_range_cache
Open

Faster byte range cache#6683
PSeitz wants to merge 4 commits into
quickwit-oss:mainfrom
PSeitz:faster_byte_range_cache

Conversation

@PSeitz

@PSeitz PSeitz commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

Optimize byte range cache lookups.

In regex queries we have a lot of FileHandle::read_bytes lookups, which are passed to NeedMutByteRangeCache::get_slice. The path comparison was very expensive in these cases, although a lookup will never cross path boundaries.

This PR removes the repeated path lookups by moving it to one layer above to the FileHandle

Screenshot 2026-08-12 at 20 47 40

PSeitz added 3 commits August 12, 2026 17:19
Partition cached ranges by path using an FxHashMap so range lookups only
compare integer offsets within the selected file.
Resolve each path's cache when opening a file so range reads avoid repeated path lookups.
@PSeitz
PSeitz requested review from a team as code owners August 12, 2026 12:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant