[feat][bk] Add per-read no-read-ahead hint#4772
Open
Denovo1998 wants to merge 1 commit intoapache:masterfrom
Open
[feat][bk] Add per-read no-read-ahead hint#4772Denovo1998 wants to merge 1 commit intoapache:masterfrom
Denovo1998 wants to merge 1 commit intoapache:masterfrom
Conversation
Author
|
@zymap @lhotari @merlimat @nodece @hangc0276 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Descriptions of the changes in this PR:
Fix #xyz
Main Issue: #xyz
BP: #xyz
Motivation
Pulsar delayed delivery uses
BucketDelayedDeliveryTrackerwhen there are too many delayed messages to keep all delayed indexes in broker memory. In that mode, delayed indexes are persisted as bucket snapshot segments in BookKeeper, and the broker lazily loads the next snapshot segment entry only after the current segment has been drained by time-driven delivery.This access pattern is a sparse point-read workload, not a normal sequential scan. A single lazy-load read for snapshot entry
Ncan currently trigger bookie-sidedbStorage_readAheadCacheprefill forN+1,N+2, and later entries, but those entries may not be read until much later, or may be evicted before use. This can waste bookie IO and direct memory, and can pollute the read-ahead cache needed by ordinary sequential reads such as backlog consumption, catch-up reads, and ledger scans.This change adds a per-read no-read-ahead hint so callers such as Pulsar can opt out only for these point-read paths while preserving the default read-ahead behavior for normal sequential workloads.
Changes
ReadOptionswithReadOptions.DEFAULTandReadOptions.builder().disableReadAhead(true).build()for request-scoped read options.ReadOptionsoverloads toReadHandleandLedgerHandle, including legacy callback APIs and unconfirmed read APIs that share the same internal read path.PendingReadOpto bookies through a newBookieProtocol.FLAG_NO_READ_AHEADbit.ReadRequest.readFlagsso bitmask-style read flags can coexist with the existing single-valueReadRequest.flagenum.Bookie.readEntry(..., noReadAhead).DbLedgerStorage/SingleDirectoryDbLedgerStoragesonoReadAhead=truestill uses and populates the target entry cache, but skips the extrafillReadAheadCache(...)prefill for following entries.PendingReadOpflag construction.