Recover tracing through an event WAL and SQLite span ledger

bt-daemon/Cargo.toml

@@ -18,7 +18,7 @@ required-features = ["cli"]
 
 [dependencies]
 # Keep this pinned exactly, matching the dependency policy used by `bt`.
-braintrust-sdk-rust = { git = "https://github.com/braintrustdata/braintrust-sdk-rust", rev = "019e428ab80d0da4c00dab383d4b0d1fdf4486fd" }
+braintrust-sdk-rust = { git = "https://github.com/braintrustdata/braintrust-sdk-rust", rev = "6cb1f157d950862ced0fe0306bd8e2b0ecba8602" }
 base64 = "0.22"
 anyhow = "1"
 async-trait = "0.1"
@@ -28,7 +28,7 @@ fs2 = "0.4"
 regex = "1"
 rquickjs = "0.12.2"
 rquickjs-serde = "0.6.1"
-serde = { version = "1", features = ["derive"] }
+serde = { version = "1", features = ["derive", "rc"] }
 serde_json = "1"
 semver = "1"
 sha2 = "0.10"
@@ -39,6 +39,7 @@ tokio = { version = "1", features = ["rt-multi-thread", "macros", "net", "io-uti
 tracing = "0.1"
 tracing-subscriber = { version = "0.3", features = ["env-filter"] }
 uuid = { version = "1", features = ["v4", "v5"] }
+rusqlite = { version = "0.40.2", features = ["bundled"] }
 
 [target.'cfg(windows)'.dependencies]
 windows-sys = { version = "0.61", features = ["Win32_Foundation", "Win32_Security", "Win32_Security_Authorization", "Win32_System_Threading"] }

bt-daemon/README.md

@@ -153,10 +153,11 @@ bt trace run --plugin ./local.mjs codex -- "summarize this change"
 bt trace import codex SESSION_ID --plugin ./sanitize-history.mjs
 ```
 
-The journal stores raw input events, not transformed spans. After daemon
-recovery, replayed events therefore pass through the resumed session's current
-route: ordinary sessions use the current globally configured plugins, while a
-managed session continues using only that run's isolated plugins.
+Capture writes complete native input to the NDJSON event WAL. Translation commits
+route-neutral span operations and continuation state to a SQLite span ledger,
+then collects the translated event prefix. Plugins and delivery consume only
+the span ledger, using the resumed route's plugin configuration. Historical
+import reads agent transcripts independently of these recovery caches.
 
 `context.operation` is `"insert"` or `"merge"`; `context.source` and
 `context.session_id` identify the translated event stream; and `context.env`
@@ -169,14 +170,17 @@ span processor is constructed. Plugins execute in bounded, thread-local
 QuickJS runtimes with no filesystem or network host APIs. Modules must be
 self-contained and transforms must be stateless: module globals belong to a
 worker thread, not a session. Every configured plugin is mandatory. If one
-fails, the affected source session and route pause delivery; other sessions
-continue. The daemon sends a payload-free failure marker on the affected span
-with `plugin failure: <path>`, then keeps new events in that session's journal.
+fails, delivery pauses only for that route and source session; other routes
+and sessions continue. The daemon sends a failure marker on the affected span
+with `plugin failure: <path>`, then holds that route's undelivered operations in the span ledger.
+Source translation continues while the route is paused.
 It checks the failed plugin's content digest once a second. When that file
-changes, it reprocesses the paused session through the current plugin chain,
-replaces the marker with the recovered span, and resumes delivery after the
-replay reaches a durable checkpoint. A still failing edit leaves that session
-paused. No daemon restart or new agent event is needed.
+changes, it reprocesses the paused route from the failed operation through the
+current plugin chain, replaces the marker with the ordinary translated span,
+and resumes delivery after the replay reaches a durable checkpoint. The
+successful span has no recovery label or marker-only error. A still failing
+edit leaves that route paused. No daemon lifecycle command or new agent event
+is needed.
 
 Plugin failures are deduplicated in the daemon's private local state, including
 the raw QuickJS exception and stack. Inspect them with:
@@ -186,8 +190,9 @@ bt trace doctor codex
 ```
 
 The doctor output reports the plugin path, full local exception, session and
-span IDs, journal position, pending bytes, and recovery state. Raw exception
-details remain local; the Braintrust marker only includes the plugin path.
+span IDs, first unprocessed span ledger position, and active recovery state. Raw
+exception details remain local; the Braintrust marker only includes the
+plugin path.
 Managed-run diagnostics are copied out of their temporary daemon directory
 before it is removed.
 Plugins are trusted local code: although they have no host APIs, they can copy
@@ -296,25 +301,24 @@ different profiles, organizations, projects, experiments, or parent spans.
 
 ## Recovery and storage
 
-Capture requests return after the event is flushed to its journal. Daemon
-workers handle authentication, translation, and delivery. Restart recovery
-replays journaled events with deterministic span IDs, so repeated delivery
-updates the same spans.
-
-Claude, Codex, and Cursor events reference daemon-owned transcript mirrors. Cursor
-preserves separate immutable generations when its native transcript is rewritten.
-Grok uses
-separate updates and events mirrors. Replay does not depend on the original
-transcript staying at its old path.
-
-The daemon streams journals and transcripts, applies queue backpressure, and
-retires idle sessions. Retired sessions are rebuilt from their journals when
-another event arrives. Journals and mirrors preserve conversation content;
-credentials are excluded from journaled routing data.
-
-Recovery files are removed after seven days without modification. Cleanup runs
-at startup and hourly while the daemon is running. This retention period is
-currently fixed in `src/server.rs`.
+Capture requests return after the native event is synced to its NDJSON WAL.
+Translation atomically commits span operations, continuation state, and its
+input watermark to a private SQLite database with WAL mode and full durability.
+The translated input prefix can then be collected without affecting plugins,
+authentication, delivery, or their recovery.
+
+Each delivery route has an independent span ledger cursor. After checked backend
+acceptance, SQLite commits the cursor and collects operations through the slowest
+registered consumer. Paused routes protect undelivered spans; translation failures
+protect untranslated events. Open sessions and turns retain continuation state
+and stable IDs for later merges, rather than retaining earlier operations.
+
+Restart and idle-session resumption restore translator continuation and replay
+pending span operations. Historical import and attach work directly from agent
+transcripts; previously collected hook details may be unavailable in those files.
+Native payloads, span ledgers, continuation, and transcript mirrors can contain
+conversation content and are private local state. Credentials are excluded.
+Resolved incident history is bounded and does not prevent cache collection.
 
 ## Platform and integration coverage