Skip to content

fix(telemetry): only announce skill events the outermost frame saved - #2100

Open
jdx wants to merge 3 commits into
jdx/missed-opportunity-signalfrom
jdx/skill-telemetry-nested-save
Open

fix(telemetry): only announce skill events the outermost frame saved#2100
jdx wants to merge 3 commits into
jdx/missed-opportunity-signalfrom
jdx/skill-telemetry-nested-save

Conversation

@jdx

@jdx jdx commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

https://entire.io/gh/entireio/cli/trails/1124

Stacked on #2024.

Review feedback on #2023 that I missed before it landed.

MutateSessionStateSaved reported saved=true for a nested frame on the grounds that the outer frame flushes its mutations. That is a prediction, not a fact: if the outer frame then returns ErrMutationSkip or fails, the caller has already emitted telemetry for an append that never reached disk. The landed doc comment waved this off as "the same exposure the plain error return always had, and self-correcting for events that later re-derive" — which is wrong in the second half, and re-derivation is precisely the mechanism that makes it worse:

  • extraction re-derives skill events from transcript offset 0 on every pass;
  • dedupe is against the ledger in session state (state.SkillEvents);
  • so an event whose ledger entry never landed is re-derived next pass, returned as new, and announced a second time.

The result is duplicate cli_skill_invoked events in PostHog — exactly what the ledger exists to prevent. Not emitting is the recoverable direction; emitting early is not.

The fix

Replace the bool with MutateSessionStateOnSaved(ctx, id, fn, onSaved). The caller hands the effect to the helper instead of deciding from a return value it cannot trust, and the helper runs it from the only frame that knows whether a save happened:

  • a nested frame queues its effect on the session gate;
  • the outermost frame drains the queue after its own save succeeds, and discards it when it skips, fails, or panics;
  • effects still run after release(), so the settings load and detached-process spawn never extend the gate hold.

MutateSessionState becomes a one-line wrapper (onSaved = nil), so the eight migrated call sites lose their stateSaved branch rather than gaining a new concept.

Scope

No behavior change today. I traced every Saved call site before writing this: handleLifecycleSessionStart, handleLifecycleTurnStart, handleLifecycleCompaction, transitionSessionTurnEnd, markSessionEnded, PostCommit, CondenseSessionByID, and CondenseAndMarkFullyCondensed are all reached only at hook-handler or command body level, so the nested branch is currently unreachable. This closes it as a landmine, not as a live duplicate — the next refactor that moves one of those handlers under a gate (transitionSessionTurnEnd already runs HandleTurnEnd, with its reentrant mutations, inside its own closure) would have gotten silent PostHog duplicates with no failing test.

One gap deliberately left open and documented on the helper: a nested frame whose fn errors has already mutated the shared state, and an outer frame that swallows that error still saves those mutations, with no effect registered. That direction loses an announcement rather than duplicating one, and is inherent to nested frames sharing a state pointer.

Also updates the three contract doc comments (skill_events.go, skill_telemetry.go, telemetry_signals.go) that told callers to emit "after the surrounding MutateSessionState returns" to name the new helper.

Tests

The nested contract as a table — outer saves / skips / fails — asserting the effect never runs while the outer frame is still open, that a skipped outer frame leaves the nested mutation off disk, and that a queued effect does not leak into the next frame on the same session. Verified each case fails against the old nested behavior (patching the nested branch to call onSaved() inline reproduces all three failures). The existing "runs outside the session gate" test keeps its durability probe, now driven through the new helper.

Full suite: 9,175 unit / 492 integration / canary green.

🤖 Generated with Claude Code

Copilot AI lite review requested due to automatic review settings August 21, 2026 18:38
@jdx
jdx requested a review from a team as a code owner August 21, 2026 18:38

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes telemetry emission so post-save effects run only after a successful outermost session-state save.

Changes:

  • Adds queued post-save effects.
  • Migrates lifecycle, post-commit, and condensation callers.
  • Updates documentation and adds nested-frame tests.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated no comments.

Show a summary per file
File Summary
cmd/entire/cli/strategy/telemetry_signals.go Updates telemetry guidance.
cmd/entire/cli/strategy/skill_telemetry.go Documents the new emission contract.
cmd/entire/cli/strategy/skill_telemetry_test.go Tests save, failure, skip, and nesting behavior.
cmd/entire/cli/strategy/skill_events.go Updates durable deduplication guidance.
cmd/entire/cli/strategy/session_state.go Implements queued post-save effects.
cmd/entire/cli/strategy/manual_commit_hooks.go Migrates post-commit telemetry.
cmd/entire/cli/strategy/manual_commit_condensation.go Migrates condensation telemetry.
cmd/entire/cli/lifecycle.go Migrates lifecycle telemetry calls.
Suppressed comments (2)

cmd/entire/cli/strategy/session_state.go:635

  • Because MutateSessionState now delegates here with onSaved == nil, this block still creates a backing array with capacity at least one for every successful outer mutation. Session-state mutations include the PostToolUse hot path (see the helper's own comment above), so this adds an avoidable allocation on every hook even when no effect is registered. Only allocate/copy effects when len(gate.afterSave) > 0 || onSaved != nil.
	effects = make([]func(), 0, len(gate.afterSave)+1)
	effects = append(effects, gate.afterSave...)
	if onSaved != nil {
		effects = append(effects, onSaved)
	}

cmd/entire/cli/strategy/session_state.go:609

  • The new contract explicitly promises that queued effects are discarded when the outer frame panics, but the added table only covers save, ErrMutationSkip, and an ordinary error. Please add a panic case that registers a nested effect, recovers outside the mutation, and then runs another frame to prove the queue cannot leak or announce a durable event from the panicking frame.
	var effects []func()
	defer func() {
		gate.activeState = nil
		gate.afterSave = nil
		release()
		for _, effect := range effects {
			effect()
		}
	}()

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

evisdren
evisdren previously approved these changes Aug 21, 2026
jdx and others added 3 commits August 21, 2026 20:24
MutateSessionStateSaved reported saved=true for a nested frame on the
grounds that the outer frame flushes its mutations. That is a prediction,
not a fact: if the outer frame then returns ErrMutationSkip or fails, the
caller has already emitted telemetry for an append that never reached
disk.

For skill events that is not self-correcting. Extraction re-derives from
transcript offset 0 on every pass and dedupes against the ledger in
session state, so an event whose ledger entry never landed is re-derived
by the next pass, returned as new, and announced a second time —
duplicating it in PostHog, which is exactly what the ledger exists to
prevent.

Replace the bool with MutateSessionStateOnSaved(ctx, id, fn, onSaved):
the caller hands over the effect and the helper runs it from the only
frame that knows whether a save happened. Nested registrations queue on
the session gate; the outermost frame drains them after its own save
succeeds and discards them when it skips or fails. Effects still run
after release(), so the settings load and detached spawn never extend
the gate hold.

No behavior change today — every current call site is outermost, so the
nested branch was unreachable. It stops being a landmine for the next
refactor that moves one of these handlers under a gate.

Also updates the three contract doc comments (skill_events,
skill_telemetry, telemetry_signals) that told callers to emit "after the
surrounding MutateSessionState returns" to name the new helper.

Tests: the nested contract as a table (outer saves / skips / fails),
asserting the effect never runs while the outer frame is open, that a
skipped outer frame leaves nothing on disk, and that a queued effect
does not leak into the next frame on the same session. Each case fails
against the old nested behavior.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…c case

Two review findings on this PR.

MutateSessionState now delegates here with onSaved == nil, so the copy
built a one-capacity backing array on every successful outer mutation —
including the PostToolUse hot path, where nothing is ever queued. Guard
it: no effects, no allocation.

The helper's doc comment promises queued effects are discarded when the
outer frame panics, but the table only covered save / ErrMutationSkip /
ordinary error. Add the panic case, which also pins down the part that
makes it safe: release() runs before the effects would have, so the gate
is usable afterwards and the panicking frame's queue does not survive
into the next one. Verified it bites — dropping `gate.afterSave = nil`
from the defer fails it on the leak assertion.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…gnal

Two HIGH trail findings read the absent emitCommitCondensedTelemetry call
in CondenseSessionByID and CondenseAndMarkFullyCondensed as data loss.
The omission is deliberate, but nothing at those call sites said so, and
this PR moving the surviving emission onto an onSaved callback puts the
asymmetry right where a reviewer looks.

Name the reason where the question comes up. The invariant itself lives
with the signal (newCommitCondensedSignal, one commit earlier in the
stack); these are the two pointers to it.

Comments only; no behavior change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@jdx

jdx commented Aug 21, 2026

Copy link
Copy Markdown
Contributor Author

@evisdren sorry — I pushed after your approval and dismissed it. Here's the delta so you can re-review just that.

You approved at fe2fde6a9. Two things landed on top, and the branch was force-pushed once (rebase), so it's now c8317b1e5.

1. Rebased onto the renamed base. #2024 gained 26c4cd871, which renames the telemetry event cli_checkpoint_condensedcli_commit_condensed and its Go identifiers. That rippled through this branch mechanically: condensedTelemetrySignalcommitCondensedSignal, emitCheckpointCondensedTelemetryemitCommitCondensedTelemetry. Two conflicts, both resolved to keep this PR's MutateSessionStateOnSaved shape under the new names. Worth noting one thing I caught in the process: the rebase left my doc comments naming the pre-rename identifiers, so I fixed those — a grep for the four old names now comes back empty.

2. Ten lines of comments at the two condensation onSaved callbacks (c8317b1e5), saying why they emit skill telemetry only and pointing at the invariant that now lives on newCommitCondensedSignal in #2024. This is the readable defect behind the two HIGH trail findings: the omission was deliberate but nothing at those call sites said so, and this PR moving the emission onto a callback puts the asymmetry right where a reviewer looks. Both findings are dismissed with the rationale recorded.

What did not change: anything executable. The whole post-approval delta is git diff fe2fde6a9 c8317b1e5 — every non-comment line in it is the rename, and the only line that changes runtime behaviour is the event-name string in #2024. The alloc guard and the panic-case test you already approved in fe2fde6a9 are untouched.

Lint clean, test:ci green (unit + integration + canary), and -race clean over the strategy package.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants