FileLogger2: drain the transition queue in the destructor after joining the writer thread - #1198
Open
simobenziane wants to merge 1 commit into
Open
Conversation
…oining the writer thread FileLogger2::writerLoop() checks `loop` between batches, and ~FileLogger2() sets `loop = false` and joins without draining transitions_queue. A transition pushed after the writer's last swap -- typically the root's RUNNING->IDLE from a haltTree() issued right before the logger is destroyed -- is lost, so the .btlog ends mid-flight. Reproduced with a serve loop that halts the tree on a broken pipe and then resets its logger; under load the last IDLE is missing in a fraction of runs. This extracts the batch write into writeBatch(), adds drainQueue() (swap under the mutex, then write), and calls it in the destructor after the join, before the file is closed. No behaviour change for callers. A new looping test, FileLogger2_DrainsQueueAfterHaltThenImmediateDestroy, ticks a tree to RUNNING, halts it, and destroys the logger immediately, then reads the .btlog back and asserts the root's last transition is IDLE, 300 times per run to exercise the scheduling gap. On the unpatched destructor it fails intermittently under load (3/10 and 10/10 process invocations failed across two load runs); with the fix, 3000/3000 iterations passed under the same load. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Author
|
CI note: the one red check, |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
FileLogger2::writerLoop()checksloopbetween batches, and~FileLogger2()setsloop = falseand joins without drainingtransitions_queue. A transition pushed after the writer's last swap — typically the root's RUNNING→IDLE from ahaltTree()issued right before the logger is destroyed — is lost, so the.btlogends mid-flight. Reproduced with a serve loop that halts the tree on a broken pipe and then resets its logger; under load the last IDLE is missing in a fraction of runs (12 of 300 with the destructor instrumented, every failure with a non-empty queue at the join).This PR extracts the batch write into
writeBatch(), addsdrainQueue()(swap under the mutex, then write), and calls it in the destructor after the join, before the file is closed. No behaviour change for callers. A looping gtest (LoggerTest.FileLogger2_DrainsQueueAfterHaltThenImmediateDestroy, 300 iterations) ticks a small tree to RUNNING, halts it, destroys the logger at once and asserts the root's last transition in the file is IDLE — it fails on the unpatched code under CPU load and passes with the patch.🤖 Generated with Claude Code