perf: enrichment still pays a full cold scan on every app launch — persist the mtime-keyed cache

[grimmerk]

Status (2026-08-20): the mtime-keyed cache this issue asked for shipped in PR #132 (v1.0.83). Cold start went from 8–10s to ~4s and stopped repeating. What is left is the last item the original fix direction explicitly deferred — the first launch still pays one full scan — plus two optional follow-ups. Retitled and rewritten so the open part is what the title says; the original text is preserved below under "Original report" for the reasoning and measurements.

What remains

1. One cold scan per app launch (the actual open item).

Nothing survives process exit: enrichedFileState is a module-level Map, so every launch re-greps all ~100 transcripts (~894MB corpus on the reference machine) before custom titles appear. Measured ~4s. Within a run it is already incremental — a stat pass of a few milliseconds, re-reading only transcripts that actually grew.

Two ways to close it, and they are alternatives, not both:

• Persist the cache — write {sessionId → {mtimeMs, size, title, branch, prLink}} next to the other user-level state in ~/.config/codev/, load it at startup, and keep the same stat check as the freshness test. Small, self-contained, no new dependency.
• Let the Batch-2 full-text index subsume it — the SQLite FTS5 index (docs/session-finding-plan.md §5.2) already has to track per-file byte offsets and persist them, so once it exists this cache is duplicated state. If FTS is close, do nothing here.

2. Optional follow-ups, only if the cold scan still hurts after the above.

• Single-pass grep. Each transcript is still read four times per scan: three grep spawns (custom-title, ai-title, pr-link) plus a tail read for gitBranch (claude-session-utility.ts, loadSessionEnrichment). One pass matching all patterns would cut process spawns ~3×.
• Stream results per batch. loadSessionEnrichment awaits the whole scan before returning, so nothing paints until everything is done. Emitting each batch of 25 would light the top rows up first — the same "show what you have" philosophy as perf: terminal/IDE badge appears 0.5-1s late — show last-known badge first (SWR) #133.

What shipped (verified against the code on 2026-08-20 — do not redo)

Original problem	Status	Where
TTL flaw — titlesCacheTimestamp stamped at function entry, so a scan longer than the 5s TTL wrote an already-stale cache and the next caller started another full scan	Fixed. The wall-clock TTL is gone from this path entirely; the stat check is the freshness test	enrichedFileState: Map<string, {mtimeMs, size}>
No in-flight dedup — top-100 pass, VS Code pass and deep-search enrichment each started independent full scans	Fixed. Scans serialize through a promise queue, which stays correct when callers pass different session sets	enrichmentQueue
Per-call maps rebuilt from scratch, so a caller with a different session set could observe holes	Fixed. Module-level accumulators persist across calls	cachedCustomTitles ??= new Map() and siblings
Random title/branch dropout — ~400 concurrent exec greps starved the biggest transcripts past a silent timeout	Fixed. Bounded batches of 25, 5s timeout	runInBatches
Branch read from tail -n 5 — an active session's tail is usually tool output with no gitBranch	Fixed. 256KB tail read, and an explicit detached HEAD clears a stale branch while a tail with no gitBranch line deliberately keeps the old value	readTailUtf8
Shell exposure and lost work on failure	Fixed as well (not in the original report). Reads are shell-free execFile, and a transcript is marked scanned only when every read succeeded, so a timeout retries next pass instead of being cached as empty

CACHE_TTL_MS still exists in that file, but it belongs to readClaudeSessions (the session-list cache) — a different cache from the one this issue is about.

"Would rewriting in Rust help?"

No — the bottleneck is disk IO plus process spawning plus re-scanning unchanged bytes, not per-byte scan speed. A Rust scanner would speed up the scanning fraction; the durable win is reading less, which is what the mtime cache did and what persisting it would finish. Same conclusion as the ecosystem survey in docs/session-finding-plan.md §2.

Cross-refs

• perf: session tab switching performance regression (PR #103) #106 — session tab switching perf (different scope: renderer tab switch)
• perf: terminal/IDE badge appears 0.5-1s late — show last-known badge first (SWR) #133 — terminal badge SWR (same "show cached first" philosophy)
• search: only the ~100 loaded sessions are searchable; main-side searchClaudeSessions (500-pool) is dead code #131 / PR feat(sessions): Batch 1 — full-prompt search, match snippets, junk fold, highlight fix #132 — the search work that surfaced all of this, and that shipped the fix
• docs/session-finding-plan.md §5.2 — the FTS index that may subsume this

---

Original report (2026-07-12) — kept for the measurements and reasoning

Symptoms (measured on the 1.0.83 build, 2026-07-13)

Fresh app launch → Sessions tab: ~2s until the last-AI-message lines (◀) appear, 8–10s until custom titles appear. Values persist afterwards (renderer merges results), but the cost repeats more than it should (see TTL flaw below).

Two related defects existed before this investigation and were already fixed on the PR #132 branch (d07c385):

• Random title/branch dropout (pre-existing, seen in old builds too): ~400 concurrent exec greps (100 sessions × 4) starved the biggest transcripts past the silent 3s timeout (errors resolve to '') → the busiest session's title/branch vanished at random. Fixed by batching (10/batch) + 5s timeout.
• Branch read from tail -n 5: an active session's tail is often tool output with no gitBranch field (measured on a live 26MB session: tail -5 hit 0, tail -20 hit 12). Fixed by widening to 50 lines.

Remaining structural problems

1. Cold cost: every scan runs 3 full-file greps per session (custom-title, ai-title, pr-link) over up to ~100 transcripts (~894MB corpus on the reference machine). That's the 8–10s.
2. TTL flaw (cubic finding on PR feat(sessions): Batch 1 — full-prompt search, match snippets, junk fold, highlight fix #132, discussion r3566660448): titlesCacheTimestamp is set to the value captured at function entry. When the scan itself takes longer than the 5s CACHE_TTL_MS, the just-written cache is already stale → the next caller starts another full scan — perpetual rescans while the popup is in use. There is also no in-flight dedup, so concurrent callers (top-100 pass, VS Code pass, deep-search lazy enrichment) each start independent full scans.
3. Per-call result maps are rebuilt from scratch, so a caller passing a different session set (e.g. deep-search appended matches) can briefly observe holes within a TTL window.

Fix direction (being implemented on the PR #132 branch)

mtime-keyed per-file incremental cache:

• Module-level accumulator maps (titles/branches/prLinks) + per-file {mtimeMs, size} scan state.
• On each call: stat every requested transcript (~1–2ms total for 100 files); re-grep only new or changed files; serialize scans through a promise queue (in-flight dedup that stays correct for different session sets); drop the wall-clock TTL entirely — the stat check is the freshness test.
• First-ever launch still pays one cold scan; afterwards only actively-growing transcripts are re-read.

Possible follow-ups if still needed: single-pass grep (all 4 patterns in one scan), per-batch streaming of results to the renderer (top rows light up first), and ultimately the Batch-2 FTS index (docs/session-finding-plan.md §5.2) subsumes this entirely (it tracks per-file byte offsets anyway).

🤖 On behalf of @grimmerk — generated with Claude Code

[grimmerk]

Status after the mtime-keyed cache landed (PR #132, f8cfa43)

Measured on the packaged build (2026-07-13): cold start to custom titles went 8–10s → ~4s, and that cost is now once per app launch — subsequent refreshes stat ~100 files (~1–2ms) and re-grep only the transcripts that actually changed (typically the 1–5 active sessions), with results accumulating (no blink-out).

Remaining option: persist the cache to disk

The ~4s that's left is the first full scan of every transcript after each app launch. To eliminate it too:

• Persist {sessionId → {mtimeMs, size, title, branch, prNumber, prUrl}} to a small JSON (or the future SQLite DB) under ~/.config/codev/, loaded at startup.
• On launch, the stat pass then skips every unchanged file — first paint gets titles in the time of a stat sweep + a handful of greps for files that changed while the app was closed.
• Invalidation stays trivial: same mtime+size key; invalidateSessionCache() clears both the in-memory maps and the persisted file.

Alternatively, just wait for the Batch-2 FTS index (docs/session-finding-plan.md §5.2) — it persists per-file scan offsets anyway and would subsume this entirely. The JSON persistence is only worth doing if the 4s bothers us before FTS lands.

🤖 On behalf of @grimmerk — generated with Claude Code