[RangeIndex] Replace busy-spin snapshot claim with SemaphoreSlim to avoid CPU spikes#1897
Draft
tiagonapoli wants to merge 15 commits into
Draft
[RangeIndex] Replace busy-spin snapshot claim with SemaphoreSlim to avoid CPU spikes#1897tiagonapoli wants to merge 15 commits into
tiagonapoli wants to merge 15 commits into
Conversation
…ck, don't spin) The per-tree snapshot claim in TreeEntry serialized concurrent CPR snapshots (OnFlush / checkpoint / migration) with a CAS + Thread.Yield() busy-spin. Because a CPR snapshot is multi-second (bftree's snapshot() sleeps ~1s per phase), any waiter burned a full CPU core for the entire hold. Under contention (e.g. a migration/checkpoint snapshot of a hot index while flushes land on the same tree) this spikes CPU to N cores for N waiters. Replace the CAS + spin with a per-tree SemaphoreSlim so waiters block instead of spinning. A SemaphoreSlim (vs lock/Monitor) keeps an async snapshot path (WaitAsync) open for the future. It is intentionally not disposed: TreeEntry has no disposal lifecycle and the semaphore holds no unmanaged handle (AvailableWaitHandle is never used), mirroring CollectionItemObserver.ResultFoundSemaphore — this also avoids a dispose-vs-in-flight-snapshot race against the lock-free checkpoint path. Adds a DEBUG-only fault injection (RangeIndex_Snapshot_Inject_Latency + WaitOnClearBlocking) and a standalone test that holds the claim with injected latency while 10 concurrent snapshots contend, asserting they block (don't spin) and all complete after release. Measured (10 waiters, 500ms hold window, net10.0 Debug): busy-spin (old): ~10 cores (cpu ~5.1s / 0.5s window) SemaphoreSlim: ~0.03 cores (cpu ~16ms) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ms OS tick on Windows) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Use Task.Run instead of manual Thread management so exceptions propagate via Task.WaitAll; drop the parallel exception arrays, CountdownEvent, and redundant completion counter. ~120 -> ~55 LOC, same deterministic assertions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Drop the Process/Stopwatch CPU logging (and the now-unused System.Diagnostics using); the deterministic block/release assertions are the test's purpose. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add a 'Snapshot performance' subsection with measured snapshot times for tree sizes 32MB-4GB (~3s fixed floor + ~linear with data size) and a key-count variation showing snapshot time is independent of key count at constant data volume. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The per-tree snapshot claim in
TreeEntryserialized concurrent CPR snapshots —OnFlush(flush),SnapshotAllTreesForCheckpoint(checkpoint), andSnapshotForMigration(cluster migration) — using a CAS + busy-spin:However, in case of pretty big BFTree, and since we may be writing multi-gigabyte files, snapshot may take more time, so any waiter will burn CPU for the entire wait. If migration happens during a checkpoint, or flushes happen with migration/checkpoint, we might burn CPU unnecessarily.
Fix
Replace the CAS + spin with a per-tree
SemaphoreSlimso waiters block instead of spinning: