fix(claude): emit the facts the session sheet reads - #116
Conversation
The deterministic session-facts sheet answers a Codex session exactly and answered a Claude Code session at 0.384 of the same scale. The sheet was not the problem: its logic is harness-neutral, but every fact it reads was emitted by the Codex adapter alone. Four things the Claude transcript records and the adapter dropped: - A spawned agent's transcript is folded into the parent's trace with nothing marking it, so the parent's tool count was the whole tree's: 2,675 where the session's own agent made 332. Every span read from a child transcript now carries `traces.span.subagent`, and the sheet keeps the child's calls out of the parent's totals and counts them in a new `subagentToolSpans` field, so the exclusion is checkable rather than invisible. The same rule keeps a child's edits, prompts and end time out of the parent's changed files, human turns and record window. - A Task call selects the agent to run through `subagent_type`, which is the harness's own signal that a call spawned a subagent. Such a call now carries `traces.agent.spawn` and the task name it named, and one synthesized `subagent.lifecycle` span stands for each child transcript, so `subagents` lists what ran with a span to check it against. - The assistant's prose reached the trace only as `content` on `llm.turn`, so nothing answered "what did the agent say last". Each response now also emits `message.assistant`, the shape the sheet already reads, and a child's message is attributed to that child's task. - The harness backs a file up before it changes it and records the path it resolved. Those records become `file.change` spans, which is a better source than a tool argument and survives a truncated input. Who typed a turn also came from the text when the harness had already recorded it. Claude Code stamps `origin.kind` on every record a person produced, so it decides: `human` is a person, any other kind is the harness or another agent. In a transcript that stamps it, a user record without it is not a human turn however human its text reads, and the turns that reclassifies are stamped so the reading can be checked. A typed slash command is stored as `<command-name>` plus `<command-args>`; the human turn is the line that rejoins them, not the storage format. A message sent while a turn is running never becomes a conversation turn at all — it arrives as a `queued_command` attachment — and is now counted as the turn it is. Measured with the deterministic sheet mapped onto an eight-question audit battery, scored against an independently adjudicated truth: | split | before | after | | --- | --- | --- | | Codex dev (n=8) | 1.000 | 1.000 | | Codex holdout (n=5) | 0.996 | 0.996 | | Claude Code (n=2) | 0.384 | 1.000 | Every Codex sheet is byte-identical before and after apart from the new field, so the Codex result is unchanged rather than merely re-scoring the same. n=2 for Claude Code: this is two sessions, not a general claim. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A message a person sends mid-turn arrives as a `queued_command` attachment, and its `prompt` is a message body: usually a string, but the content blocks of a message when the person attached an image alongside the words. The new reader called a string helper on it directly, so a session with one such message failed to parse at all — five of sixty local transcripts. It now goes through the same `textOf` the adapter already uses for every other message body, which takes a string verbatim and joins the text blocks of an array. The image block is dropped, which is right: the human turn is what they typed. Two neighbouring reads of untrusted transcript fields are hardened the same way rather than left to trip later. `origin.kind` is used only when it is a non-empty string, so a record carrying some other value falls back to the text heuristics instead of being trusted as a label, and a subagent metadata file whose `description` is not a string names no task rather than taking the parse down. Found by parsing sixty local Claude Code transcripts with both builds: the base build read all sixty, this one failed five before the fix and reads all sixty after it. The measured splits are unchanged — Codex dev 1.000, Codex holdout 0.996, Claude Code transfer 1.000. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Follow-up: a crash found by probing beyond the two measured sessionsParsing the 60 largest local Claude Code transcripts with both builds caught a regression the two measured sessions did not contain: a It now goes through the same Probe result: base build 60/60 parsed, this branch 55/60 before the fix and 60/60 after. The measured splits are unchanged — Codex dev 1.000, Codex holdout 0.996, Claude Code transfer 1.000. |
What this fixes
The deterministic session-facts sheet (
traces facts) answers a Codex session exactly and answered a Claude Code session at 0.384 of the same scale.The sheet was not the problem. Its logic is harness-neutral; every fact it reads was emitted by the Codex adapter alone. The Claude Code adapter marked no folded subagent span, emitted no assistant-message span, no file-change span and no spawn marker, and decided who typed a turn from the text when the harness had already recorded it.
What changed, and why the transcript supports it
A subagent's work is the subagent's. Claude Code writes one transcript per spawned agent under the session directory, and the adapter folds them all into the parent's trace. Nothing marked them, so the parent's tool count was the whole tree's — 2,675 tool spans on a session whose own agent made 332. Every span read from a child transcript now carries
traces.span.subagent; the sheet keeps those calls out oftoolCallsand counts them in a newsubagentToolSpansfield, so the exclusion is checkable rather than invisible. The same rule keeps a child's edits, prompts and end time out of the parent'schangedFiles,humanTurnsand record window.Spawns are named by the harness's own signal. A Task call selects the agent to run through
subagent_type, so a call carrying it spawned a subagent — no allowlist of tool names decides it. Such a call now carriestraces.agent.spawnplus the task name, and one synthesizedsubagent.lifecyclespan stands for each child transcript sosubagentshas a span to be checked against.The assistant's prose gets a span. It reached the trace only as
contentonllm.turn, so nothing answered "what did the agent say last". Each response now also emitsmessage.assistant— the shape the sheet already reads — and a child's message is attributed to that child's task instead of the session's.File changes come from the harness's record. Claude Code backs a file up before it changes it and records the path it resolved; those records become
file.changespans, which beats a tool argument and survives a truncated input.Who typed a turn comes from
origin.kind. Claude Code stamps it on every record a person produced, so it decides:humanis a person, any other kind is the harness or another agent. In a transcript that stamps it, a user record without it is not a human turn however human its text reads —[Request interrupted by user]is written by the CLI — and each turn that rule reclassifies is stamped so a reader can disagree span by span. A typed slash command is stored as<command-name>plus<command-args>; the human turn is the line that rejoins them, not the storage format. A message sent while a turn is running never becomes a conversation turn at all — Claude Code surfaces it inside the running turn as aqueued_commandattachment — and is now counted as the turn it is.Measured
The sheet mapped mechanically onto an eight-question session-audit battery (tool calls, subagents, pull requests, changed files, human turns, final message, session window, token total), scored against an independently adjudicated truth. Same harness, same build pipeline, before at
b8ae085and after at this branch's head.Claude Code, per question, before → after: tool calls 0.000 → 1.000, subagents 0.000 → 1.000, pull requests 0.891 → 1.000, changed files 0.100 → 1.000, human turns 0.330 → 1.000, final message 0.000 → 1.000, session window 0.750 → 1.000, token total 1.000 → 1.000.
Every Codex sheet is byte-identical before and after apart from the new
subagentToolSpansfield, so the Codex result is genuinely unchanged rather than re-scoring to the same number by chance.n = 2 for Claude Code. Two sessions is a claim about those two sessions, not a general improvement claim. The rules implemented here are general — they read structural signals the harness records — but their breadth is untested beyond n=2.
What the transcript still cannot support
tokenTotalstays null with that reason rather than summing per-turn deltas, which is a different and smaller number. Both measured sessions score this correctly by refusing to answer.subagent.lifecycleisUNSET; the parent's own tool span carries the status.Gates
pnpm check:source,pnpm typecheck,pnpm test(1003 tests),pnpm build,pnpm check:packageall green. The two bounded-heap regressions pass. Fixtures are synthetic inline JSONL in the style of the existing Claude tests; no content from any real session is in this change.🤖 Generated with Claude Code