feat(claude-code): capture background subagent work durably (SubagentStop + task records)

[peyton-alt]

https://entire.io/gh/entireio/cli/trails/1076

Implements #2058's durable pointer model for subagent work, with the subtree payload location (tasks/ inside the parent session's checkpoint).

The incident

A real session dispatched ~29 background Claude Code subagents; entire captured nothing beyond a 2–8s launch stub for each one, even though the agents ran 5–18 minutes. The root cause: Claude Code's PostToolUse for a backgrounded Task tool fires at the launch acknowledgment, before the subagent has done any work — the real completion is delivered later, out of band, as SubagentStop. Entire never registered a hook for that true-completion signal, so every background subagent's edits and transcript were invisible.

Verified hook facts

• SubagentStop fires per-agent at true completion, including after the parent's own turn already ended. Payload carries session_id (parent), agent_id, agent_transcript_path (the subagent's own transcript), tool_use_id, transcript_path, cwd. Notification-only; default timeout 600s.
• Subagents fire no session-lifecycle hooks of their own (no SessionStart/UserPromptSubmit/Stop/SessionEnd) — SubagentStop is the only completion signal, and a finishing subagent does not fire the parent's Stop.
• Subagent transcripts live at <project>/<session-id>/subagents/agent-<agent_id>.jsonl; the launch-time PostToolUse payload's tool_response.agentId matches that filename (verified against 29 real transcripts).

Parsing stays defensive: an absent agent_transcript_path degrades to the agent-layout convention (ResolveAgentTranscriptPath) rather than failing, and the parser logs the raw payload's key names (never values) at Debug.

Design: durable task records + condensation materializer

Task records (session.TaskRecord, json task_records, on session state) are the durable ledger: ToolUseID, AgentID, StartedAt, SubagentType, TaskDescription, DeclaredTranscriptPath, Files, TokenUsage, CompletedAt (zero = live/in-flight). Mid-turn a record is a pointer, not a payload — the subagent transcript stays where the agent wrote it, and the record remembers how to find it. Producers:

• Background launch creates a live record (labels captured at launch because SubagentStop's payload carries none of them).
• Foreground tasks create-on-completion.
• SubagentStop / foreground post-task / SessionEnd sweep complete exactly-once via strategy.CompleteTaskRecord — one MutateSessionState closure: claim CompletedAt, attach files/tokens/declared path, merge into the session's FilesTouched. Completion happens last, after successful extraction, so a failed capture leaves the record live for the SessionEnd sweep to retry; a racing duplicate completes nothing.
• Factory Droid Workers upsert (UpsertCompletedTaskRecord): multi-turn workers merge files into one record instead of claiming exactly-once.

Materializer: condensation resolves each record's transcript (declared path first, agent-layout fallback), runs the same sanitize → externalize → redact pipeline as the session transcript (prepareTaskTranscriptForStorage), and writes tasks/<toolUseID>/{agent-<agentID>.jsonl, task.json} inside the parent session's checkpoint on entire/checkpoints/v1 — both backends, via the shared applySessionWrite. An unresolvable/unreadable/empty transcript still gets a task.json with a stable, path-free unavailable reason. Poisoned records (empty/unsafe IDs) are skipped-with-warn, never allowed to wedge condensation.

Self-contained checkpoints: live records store transcript-so-far on every condensation and survive for retry; completed records are removed only after a successful write (removeCompletedTaskRecords off resetCheckpointWindow). A mid-task commit carries a partial transcript; a later checkpoint carries the full one.

Trigger currency: State.HasTaskContent() (len(TaskRecords) > 0) replaces the deleted TranscriptOnlyTaskSteps counter everywhere. Records-only sessions (no shadow branch, no steps, empty parent transcript) condense; records never live on the shadow tree, so shadow-branch existence no longer implies task content (shadow pinning keys on StepCount only).

Alignment with #2058

This PR implements #2058's core — a durable record that survives until condensation, plus a materializer that makes every checkpoint self-contained — choosing the subtree payload option (tasks/ inside the session checkpoint) over a separate task checkpoint kind. There is now exactly one producer path into stored task data (the materializer); the never-reachable per-write IsTask persistent route and its Entire-Metadata-Task trailer (#2058's "dead writer") are deleted.

What was removed and why

Once every checkpoint is self-contained, the mid-turn backstops are redundant — they existed only because task data used to die if it wasn't snapshotted before the session ended:

• The turn-end incremental snapshot backstop (rotation, growth-dedup, cap of 8), persistCapturedTranscriptSize, IncrementalTypeBackgroundProgress / FormatBackgroundProgressSubject / FormatIncrementalSubject.
• The per-write IsTask persistent route + Entire-Metadata-Task trailer (dead code, see above).
• claimTaskRecord-as-removal: completion now marks, never deletes — the materializer needs the record.
• SaveTaskStep is now formally incremental-only (errors on !IsIncremental); its sole production caller is the Claude Code post-todo hook.

Behavioral invariants preserved

• Background launches stay lazy: no capture at the launch stub, only a record.
• Foreground completion still merges the subagent's files into the session's FilesTouched.
• Exactly-once completion (the old claim-and-remove dedup, reshaped as mark-completed-once).
• SessionEnd ordering: the live-record sweep runs strictly before endSessionNow's eager condense, so that condense materializes it.
• Read-only subagents' transcripts are stored: a completed record with no files still materializes, and "has records" makes the session condensable.
• The late-arrival zombie guard: a SubagentStop for a missing session state skips outright; PhaseEnded state → complete + eager condense.

Deliberate behavior changes

• (a) Turn-end no longer writes incremental background snapshots — in-flight visibility now comes from condensation materializing the transcript-so-far into each checkpoint.
• (b) doctor --force condenses record-bearing dead-owner sessions instead of discarding them.
• (c) Shadow branches are deletable while records exist — records don't live there.
• (d) Record-bearing sessions are PrepareCommitMsg trailer candidates via sessionHasNewContent.
• (e) checkpoint list --pending --json task rows carry empty id/metadata_dir (no shadow commit exists); external consumers keying on id should use tool_use_id. Rows render with accurate Running/Completed verbs.
• (f) SaveTaskStep errors on non-incremental use.

Repairs shipped alongside

• Hard-killed agents' live records get a terminal state: finalizeExitedSessions (the entire status/doctor sweep) completes live records before ending the session, so their transcript-so-far reaches a permanent checkpoint.
• CondenseSessionByID materializes instead of clearing when records exist; orphan cleanup can't Clear record-bearing states.
• ENDED + FullyCondensed is terminal-healthy even with leftover live records — pre-fix states self-resolve, no migration.

On-disk compatibility

Old state files' transcript_only_task_steps, last_captured_transcript_bytes, and last_snapshot_attempt keys are silently dropped on the next save; no migration. Nothing durable shipped under the old in_flight_tasks schema.

OPF / redaction

git-branch (the default primary): task transcripts get the full 9 layers, automatically. The pre-push OPF rewrite walks the checkpoint tree subtree-recursively and treats every blob except content_hash.txt as redactable, so tasks/<id>/agent-<id>.jsonl and task.json are covered with no new wiring. This PR writes no content_hash.txt into the task directory, so the apply pass's sibling-full.jsonl hash recompute is undisturbed.

git-refs: OPF now runs there too — fixed in this PR. It previously ran on nothing on that backend: prePush returns for refs before ever reaching the OPF block, and RewriteUnpushedV1WithOPF's only call site was the git-branch path, so session transcripts and task transcripts alike shipped at 8 layers while a user who enabled OPF believed 9 ran. That gap was pre-existing and backend-wide — not a consequence of the subtree design and not specific to tasks — but this PR enlarges it by volume (subagent transcripts are the highest-volume, least-reviewed content in the system), so it is fixed here rather than deferred.

RewriteQueuedCheckpointRefsWithOPF (strategy/manual_commit_opf_refs.go) mirrors the v1 rewrite and reuses its machinery rather than duplicating it — the same blob walk and open blob policy, the same raw-byte and leaf-byte caps, the same single batched OPF shell-out per push, the same error taxonomy, and the same two up-front fail-closed gates in the same order. Only discovery and update differ, and both are simpler on refs: the push queue already names exactly which refs are unpushed, so there is no merge-base or divergence analysis, and each ref is updated independently instead of CAS-ing one branch tip.

Every unpushed commit on a queued ref is rewritten, not just its tip — pushing a ref sends its whole unpushed ancestry, so a tip-only rewrite would have shipped un-redacted ancestor blobs alongside the redacted tip. The walk stops at the first commit already carrying Entire-OPF-Applied, which makes the trailer its own watermark: in steady state it stops at the tip's parent and costs nothing. A repo that enables OPF late is bounded by the existing resolveBootstrapLimit / BootstrapTooLargeError pair rather than rewriting unbounded history.

Failure semantics differ by backend, deliberately. Both fail closed. git-branch aborts the user's git push. On refs, unflushed refs simply stay queued, so an OPF failure or an explicit abort withholds the flush and returns cleanly: nothing un-redacted ships and the user's push still succeeds. It is not silent — a logging.Warn plus a stderr line say that checkpoint refs stayed queued and why. OPFSkip (the user opting out for one push) is unchanged: flush as-is, 8 layers, no trailer.

One user-visible copy change on the previously-untouched v1 path: BootstrapTooLargeError now says "N checkpoint commits" instead of naming entire/checkpoints/v1, because the error is shared across both backends and a git-refs user has no v1 branch. The remediation is identical for both.

Known gaps / follow-ups

• Shadow-only capture (Subagent data does not survive condensation — durable storage for subagent transcripts #2058) — FIXED on this branch: subagent transcripts no longer die at the user's commit; the materializer writes them into every permanent checkpoint under tasks/, on both backends.
• Sweep-finalize gap — FIXED on this branch: a session whose agent was hard-killed used to lose in-flight subagent work; finalizeExitedSessions now completes live records before ending the session, so the transcript-so-far is condensed.
• Commit trailer linkage for mid-task subagent commits is the stacked follow-up (PR feat: link mid-task background-subagent commits to their session #2034), rebuilt on records: the trailer trigger stays, the commit-snapshot capture is replaced by the materializer.

🤖 Generated with Claude Code

[cursor]

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit ed5142e. Configure here.}

[Copilot]

Pull request overview

This PR addresses a gap in Claude Code background Task capture by introducing a true-completion signal path (SubagentStop → SubagentEnd with Final=true) plus a turn-end “backstop” that snapshots in-flight background subagent progress until the final completion arrives. It extends the session state model to persist in-flight background task markers and updates hook installation/drift detection so repositories automatically register the new hook.

Changes:

• Add background-task lifecycle handling: launch stub records an in-flight marker; turn-end snapshots incremental progress; SubagentStop performs the authoritative final capture (with late-arrival safeguards).
• Extend session state to persist InFlightTask markers and related helper methods (add/find/remove + transcript-size growth dedup support).
• Wire up Claude Code’s SubagentStop hook end-to-end (hook parsing, hook installation, drift detection), plus unit/integration test coverage and architecture docs.

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
docs/architecture/sessions-and-checkpoints.md	Documents task-step (subagent checkpoint) behavior, including background launch stub, turn-end backstop, and final capture semantics.
docs/architecture/agent-integration-checklist.md	Adds guidance for agents with “launch stub vs true completion” subagent models to map a true-completion hook to SubagentEnd with Final=true.
docs/architecture/agent-guide.md	Updates lifecycle hook mapping and required event fields (Final, SubagentTranscript) for subagent completion.
cmd/entire/cli/strategy/messages.go	Introduces IncrementalTypeBackgroundProgress and a dedicated subject formatter for background-progress incrementals.
cmd/entire/cli/strategy/messages_test.go	Adds/extends tests covering background-progress incremental subject rendering.
cmd/entire/cli/strategy/manual_commit_git.go	Ensures transcript-only final task steps are visible to condensation by incrementing StepCount when appropriate.
cmd/entire/cli/session/state.go	Adds persisted InFlightTasks and helper methods to manage background-task markers and transcript-size dedup state.
cmd/entire/cli/session/state_test.go	Adds tests pinning JSON wire format and add/find/remove semantics for InFlightTasks.
cmd/entire/cli/lifecycle.go	Implements background launch marker recording, turn-end backstop snapshots, final capture via SubagentStop, and session-end sweeping of remaining in-flight tasks.
cmd/entire/cli/lifecycle_test.go	Adds extensive unit tests covering background launch deferral, final capture, dedup/claim behavior, turn-end incremental snapshots, and session-end finalization.
cmd/entire/cli/integration_test/subagent_checkpoints_test.go	Adds end-to-end integration coverage for background launch deferral, turn-end backstop, and foreground double-fire dedup.
cmd/entire/cli/integration_test/hooks.go	Adds integration hook simulation for SubagentStop and supports run_in_background in simulated post-task input.
cmd/entire/cli/integration_test/agent_test.go	Updates expected installed hook count to include SubagentStop.
cmd/entire/cli/hooks.go	Adds isBackgroundLaunch helper to detect run_in_background: true in Task tool input.
cmd/entire/cli/hooks_test.go	Adds unit tests for isBackgroundLaunch.
cmd/entire/cli/agent/event.go	Extends event model with SubagentTranscript and Final to distinguish launch stubs from true completion.
cmd/entire/cli/agent/claudecode/types.go	Adds Claude settings hook struct support for SubagentStop payloads.
cmd/entire/cli/agent/claudecode/lifecycle.go	Adds parsing for subagent-stop hook, emitting SubagentEnd with Final=true and an authoritative transcript path when provided.
cmd/entire/cli/agent/claudecode/lifecycle_test.go	Adds tests for SubagentStop parsing and verifies Final semantics.
cmd/entire/cli/agent/claudecode/hooks.go	Installs/uninstalls SubagentStop, refactors install flow, and updates drift detection to require the new hook.
cmd/entire/cli/agent/claudecode/hooks_test.go	Adds tests ensuring SubagentStop is installed/removed and that drift detection flags missing SubagentStop.
.claude/settings.json	Registers the SubagentStop hook for repo dogfooding/verification.

Suppressed comments (1)

cmd/entire/cli/strategy/messages_test.go:368

• If you want these subtests to run in parallel (recommended for unit tests that don't touch process-global state), shadow the loop variable before calling t.Run; otherwise the closure will capture the last iteration's values.

	for _, tt := range tests {
		t.Run(tt.name, func(t *testing.T) {
			got := FormatBackgroundProgressSubject(tt.subagentType, tt.taskDescription, tt.shortToolUseID)
			if got != tt.want {
				t.Errorf("FormatBackgroundProgressSubject(%q, %q, %q) = %q, want %q",
					tt.subagentType, tt.taskDescription, tt.shortToolUseID, got, tt.want)
			}
		})
	}

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.