Repository navigation
fix(mcp): idle query workers are recycled, so a burst's memory is given back - #3126
Merged
Merged
Conversation
…en back A session grows its query pool for concurrent reads, and every worker kept its graph caches, SQLite pages and heap until shutdown, so a server that had served one burst held that memory for the rest of its life. After five minutes with no work the pool now retires every worker that has been used and starts one fresh one, which is ready for the next call. A retired worker is asked to close: it closes its projects, runs a full collection and exits by itself, the path worker-teardown.ts measured as safe on Windows. Only one that hasn't exited after 5 seconds is terminated, through the pool's own start-aware path. Calls that arrive while the fresh worker starts are served in process, the same way as before the pool's first worker is up. Changed on top of the PR: - The idle delay is five minutes, not the PR's 60 seconds, so main doesn't get slower than 1.6.2. Agents routinely pause over a minute to edit or build, and the first explore on a fresh worker takes about 200 ms more. The memory still comes back when a session really goes quiet. - The recycle also arms when a burst's last worker reports ready after the last result. Before, that pool stayed at full size until the next call. - A warm worker that has served nothing is never recycled, so that readiness arming can't replace a clean worker over and over. - destroy() waits for retirements underway, so a worker is not left to end with the process. PR #2862 had this; the PR here did not. PR #2862 by @doomsday1906 made the same idle-worker change and is closed in favor of this one. On its own fork it measured idle RSS going from about 673 MiB to 426 MiB. Validation: - Red on main: six pool tests fail, including a real worker that retires with exit code 0 and is never terminated. Each added case was also proven red by undoing its line. - Windows 11 x64, on a busy machine: 480 child processes, each with four real workers on an excalidraw index and retiring right after a burst, all exited 0 (1,920 worker exits). The AGENTS.md crash did not reproduce here even with an owner terminate in place of the close (0 of 480), so this shows the path works, not that the crash can't happen. - excalidraw, four workers after a burst of explores: 611 MB RSS idle on main, 113 MB here. The first explore after a recycle took 299 ms against 93 ms on a warm worker, and the second was back to 92 ms. Original PR #1583 by @danusha2345. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
colbymchenry
force-pushed
the
claude/pr-1583-idle-query-workers
branch
from
October 31, 2026 00:23
93d0492 to
66d0fa9
Compare
This was referenced Oct 31, 2026
colbymchenry
added a commit
that referenced
this pull request
Nov 1, 2026
…ys warm (#3167) PR #3126 ends every query worker after five minutes idle so a burst's memory comes back, which fixed issue #3027. It also made the first call after a pause slower than 1.6.2: 634 to 729 ms on django against 565 to 587, where warm is 165. Root cause. The replacement worker is already up when that call arrives, so the 500 ms is not worker startup. Reopening the project inside a warm isolate costs nearly the same, so it isn't compiled code either, and a compile cache for the worker moved nothing. It is what a connection works out once and remembers: the dominant file's scan of every edge (230 ms on django), the name indexes (85 ms) and a git probe (30 ms), then tree-sitter's first parse. A new connection has to work all of that out again, and a worker's memos can't be handed over, because their change stamp only means something on the connection that took it. A call landing as the recycle ran fared worse. With no warm worker it ran on the main thread, cold: 884 to 1,154 ms, the next call just as slow, and the transport held for up to 417 ms. Fix. The pool keeps the worker that served last and retires the rest. The kept worker closes every project but the last one used, and its connections give up their page cache and file mapping, which the next reads fill again. Everything else stays: compiled code, the memos, tree-sitter. No worker stops, so there is no moment when a call finds none, and a call that lands during the trim waits a few milliseconds. A worker still holding over 256 MB afterwards is replaced the old way, so keeping one can't keep hold of what a large query left behind. A trimmed worker reports about 80 MB, so this has not fired on any repo here. There is no forced collection: V8 already shrinks an idle isolate's heap by itself, well before the five minutes are up. A last-resort collection added 3 MB at that point, so it came out. Options measured on django, one worker (working set / private MB, then the first call): - a clean worker, as on main: 106/79, 732 ms - a clean worker made to compute the dominant file and git probe first: 144/80, 410 ms - the same plus every memo: 213/153, 319 ms - a clean worker replaying the last query: 295/258, 189 ms - a kept worker, trimmed (this): 168/175, 152 ms Validation, on Windows, through the real MCP server under 1.6.2's node and flags: - First call after five real idle minutes, three gaps: django: 1.6.2 886/676/443, main 744/878/660, this 170/171/200 (warm 214) flask: 1.6.2 111/79/82, main 180/172/166, this 43/41/43 (warm 41) elasticsearch: 1.6.2 956/641/674, main 2752/2561/1419, this 764/234/243 (warm 256) The first gap's extra 500 ms shows only on arms that release the page cache, and is gone by the second, which fits its pages being read back once. - A call at the deadline on django: 142 to 226 ms at every offset from -300 ms to +3 s, the next call warm, the transport never held over 19 ms. - Memory after a burst of four workers and five idle minutes (ws/priv): excalidraw: 1.6.2 505/482, main 177/221, this 244/295 django: 1.6.2 695/486, main 179/223, this 247/295 elasticsearch: main 221/265, this 298/348 So about 70 MB more than main, on any index, in exchange for the first call. - 30 busy-then-quiet cycles, 600 mixed calls, no errors and no creep. - 240 processes, each with four real workers, shut down after, during and right behind a trim: all exited 0. 960 worker exits, no crash. - The real shared daemon behaves the same: 155 ms at the deadline. - Red on main: 13 new tests fail, counting worker starts rather than time. Twelve mutations undoing each new line are all caught. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The query pool now recycles its workers after five minutes idle, so a server that served one burst of large queries gives that memory back instead of holding it for the rest of its life. This is the worker part of #1583, split out as its own commit.
Root cause
A session grows its pool for concurrent reads, and each worker kept its graph caches, SQLite pages and heap until shutdown.
Fix (from the PR)
After a stretch with no work the pool retires every worker that has been used and starts one fresh one for the next call. A retired worker is asked to close: it closes its projects, runs a full collection and exits by itself, the path
worker-teardown.tsmeasured as safe on Windows. One that hasn't exited after 5 seconds is terminated through the pool's start-aware path. Calls that arrive while the fresh worker starts are served in process, as before the pool's first worker is up.What we changed
destroy()waits for retirements underway, so a worker is not left to end with the process. perf(query): retire excess workers after idle bursts #2862 had this; the PR here did not.ready, tests for the three changes, and the CHANGELOG bullet.#2862 by @doomsday1906 made the same idle-worker change and is closed in favor of this one. On its own fork it measured idle RSS going from about 673 MiB to 426 MiB. The difference in design: #2862 keeps one used worker warm, and this one replaces it, so a worker's caches go too.
Validation
codegraph_exploreoutput and budgets are unchanged; workers hold no session state.CODEGRAPH_KERNEL_EXPECT=1, 4 workers): 842 files, 10,209 passed, 0 failed, 87 skipped. That suite ran with the 60-second delay. Since then this commit rebased cleanly onto cc86361, sharing no file with main's new commits besidesCHANGELOG.md, and only the default delay, its comments and the CHANGELOG wording changed. On this head,query-poolandmcp-daemonpass: 54 tests, 1 skipped. No test depends on the default.Lands #1583 (worker part) by @danusha2345.
🤖 Generated with Claude Code