WAL grows unbounded (115 GB in 4.5 h, fills system drive): checkpoint starvation from accumulated per-conversation servers + 11 concurrent index workers on one repo

[9981H]

Environment

• codebase-memory-mcp 0.9.0 (Windows 11, installed via install.ps1)
• Multiple concurrent MCP clients against the same repo: Claude Code sessions (spawned by Cursor) + Codex sessions (ChatGPT desktop app). Each conversation spawns its own codebase-memory-mcp.exe; over one work day 30+ server processes accumulated (clients keep running after their conversations are closed, so servers are never reaped).

What happened

The main DB ~/.cache/codebase-memory-mcp/<repo>.db is only 60 MB, but its -wal file grew from 0 to 115 GB in ~4.5 hours (created 10:47, 115 GB by 15:30 local) and completely filled the system drive (free space went 11.3 GB -> 7.8 GB while we watched, ~0.5 GB per 2 minutes).

Process snapshot at 15:32 showed, besides 20+ long-lived server processes, 11 concurrent cli --index-worker index_repository workers all indexing the same repo path at the same time.

Diagnosis

Classic SQLite WAL checkpoint starvation:

• dozens of long-lived connections (one per abandoned conversation) hold read marks on the WAL, so wal_checkpoint can never truncate;
• each full re-index (clients run index_repository at the end of every task) appends the whole dataset to the WAL again;
• log only grows, never shrinks.

Confirmation: at 15:39 a checkpoint finally succeeded and the WAL collapsed from 115 GB to a few MB — so the 115 GB was pure un-checkpointed log, not data volume.

Probably related: #914 (orphan cbm process), #937 (watcher re-index write amplification).

Suggestions

1. Set PRAGMA journal_size_limit and force wal_checkpoint(TRUNCATE) after every index run (and/or periodically), logging a warning when the WAL exceeds a threshold.
2. Serialize index workers per DB with a file lock: if an index_repository is already running for the same repo, queue or refuse instead of running 11 full indexes concurrently.
3. Defense in depth: the server should exit when its client stdio disconnects (EOF), so abandoned conversations do not accumulate connection holders that block checkpoints.
4. Consider one shared server per DB (with a supervisor/broker) instead of one server per conversation.