feat(telemetry): emit skill invocations as cli_skill_invoked events

[jdx]

https://entire.io/gh/entireio/cli/trails/1070

Skill events are already extracted per agent and recorded in session state (SkillEventExtractor, appendEventSkillEventsToState), so internal skill usage is measurable from checkpoint refs — but nothing reached PostHog, leaving external skill adoption invisible while agent-help usage is tracked. That asymmetry biases any skills-vs-agent-help comparison toward the instrumented side.

This PR forwards exactly the newly-appended (deduped) skill events to a detached cli_skill_invoked event: skill name, agent, detection signal, event type.

• Content-free by construction — no prompt text, arguments, or transcript content, mirroring the plugin-name-only rule in BuildPluginEventPayload.
• Gated on the same opt-in telemetry setting as command tracking, plus the ENTIRE_TELEMETRY_OPTOUT env opt-out.
• Hook lifecycle paths never produce cli_command_executed (the hooks tree is hidden), so this is the only telemetry signal that a skill fired.

appendEventSkillEventsToState now returns the appended events instead of a bool so both call sites (TurnStart slash-command path and persistEventMetadataToState) forward exactly what was new.

Tests: payload shape + content-free assertions in telemetry, append/dedup semantics in lifecycle_test. Full suite: 9,170 unit / 492 integration / canary green.

🤖 Generated with Claude Code

Which skill names are sent

Skill names are arbitrary user tokens, so the emitted skill property comes
from a closed vocabulary: a recognized Entire skill name, or one of two fixed
categories (custom, entire:unlisted). Recognition matches the leaf name
after stripping Entire's plugin namespace — not the entire: prefix itself,
which comes from whatever plugin the user installed or authored and so vouches
for nothing.

Two scoping choices, both taking the direction whose error is visible:

• Unlisted Entire skills still count. The leaf list lives in this repo
  while the skills ship from entireio/skills, so a skill released upstream
  before the list is updated reports as entire:unlisted — a lagging list
  undercounts the per-skill breakdown but never adoption volume.
• Unnamespaced installs are not matched. Entire's skills are advertised as
  cross-agent and can arrive without a namespace (Pi normalizes
  /skill:<name> to a bare name; a hand-copied SKILL.md has none), and
  those report as custom. Matching bare leaves would fold Claude Code's
  built-in /review and any local /search into Entire's numbers — an
  overcount invisible in the data, where this undercount has a known cause.
  Worth revisiting if cross-agent installs become a meaningful share.

entireio/pr-review-toolkit skills are deliberately out of scope: the CLI's
install hints for it are still placeholders (skilldiscovery/registry.go), so
nothing ships under that namespace yet.

Counting caveats

Which paths actually emit. The condense path (commit and session-end) is
where transcript-extracted events reach telemetry in practice. The turn-end
snapshot around HandleTurnEnd only yields events when the turn had mid-turn
commits — finalizeAllTurnCheckpoints early-returns on an empty
TurnCheckpointIDs — so in an ordinary turn it is a no-op and the condense
path provides the coverage. Two gaps stay open by design: sessions the
exited-owner sweep marks ENDED without condensing (when sweepCondenseBudget
is exhausted) never emit their tool-based events, and claudeSkillEventID
falls back to a transcript-relative id when tool_use_id is absent, which
shadow-vs-live extraction could double-count — pre-existing, and now costing a
telemetry event rather than a duplicate metadata row.

One action could emit twice. A typed /entire:review can produce a
prompt_slash_command event from the hook path and a skill_tool_use event
from the transcript extractor; the ids differ, so both are legitimately new and
both emit. event_type disambiguates, so count by event_type, or count
distinct actions rather than raw cli_skill_invoked volume — a raw count
would inflate. Deliberately not deduped across signals: the two events are
distinct records the session UI relies on for its collapse behavior, so
collapsing them in telemetry would mean either dropping a recorded event or
adding a correlation identifier to the payload.

Measured rather than assumed: across 72 real Claude Code transcripts on this
machine, 8 contain Skill tool invocations and none pairs one with a literal
/<name> user message, so in practice the tool signal is what appears. The
inflation is a real possibility to guard the insight against, not something
observed.

[Copilot]

Pull request overview

This PR adds detached telemetry for “skill invocation” events surfaced by agent hooks, and wires lifecycle session handling to forward newly recorded SkillEvents as privacy-preserving analytics events (skill/agent/signal/event_type only).

Changes:

• Introduces SkillInvocation, BuildSkillEventPayload, and TrackSkillInvocationsDetached to emit cli_skill_invoked events from a detached subprocess.
• Updates lifecycle session mutation flow to dedupe/append skill events and forward newly appended events to telemetry (gated by the user’s telemetry opt-in setting).
• Adds unit tests validating skill payload shape and that appendEventSkillEventsToState returns only newly appended events.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File	Description
cmd/entire/cli/telemetry/detached.go	Adds skill-invocation telemetry payload + detached tracking loop.
cmd/entire/cli/telemetry/detached_test.go	Adds tests for building skill telemetry payloads and privacy constraints.
cmd/entire/cli/lifecycle.go	Threads newly appended skill events through session state + forwards them to telemetry behind opt-in gating.
cmd/entire/cli/lifecycle_test.go	Adds coverage for “return only newly appended skill events” behavior.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[jdx]

Addressed all three review findings in 9c05a09:

• Lock scope (the substantive one): persistEventMetadataToState now returns the newly-appended skill events, and every MutateSessionState call site forwards them to telemetry only after the mutation returns — settings load and detached spawn no longer run while the session gate is held, and never fire for events that failed to persist.
• Empty agent defaults to "auto", matching BuildEventPayload.
• Process burst capped at 10 detached senders per call (maxSkillInvocationsPerTrack); real turns carry at most a few events, the cap guards the pathological case.

Full suite re-run: 9,170 unit / 492 integration / canary green.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Suppressed comments (2)

cmd/entire/cli/lifecycle.go:1595

• This only forwards hook-provided events that passed through appendEventSkillEventsToState. Claude Code's SkillEventExtractor runs later during condensation (strategy/manual_commit_condensation.go:1116,1162) and its results are merged directly into checkpoint metadata at line 396, so skill_tool_use / tool_invocation events never reach this loop or PostHog. Wire newly merged extractor events into the telemetry path (while deduping against state), otherwise the new event undercounts the main tool-based detection path.

	for _, ev := range appended {
		invocations = append(invocations, telemetry.SkillInvocation{
			Skill:     ev.Skill.Name,
			Agent:     ev.Source.Agent,
			Signal:    ev.Source.Signal,

cmd/entire/cli/telemetry/detached.go:205

• Raw skill names are not content-free. The prompt extractor accepts any leading token matching /<token> (agent/skill_events_prompt.go:12,48-61), and Claude's extractor copies the arbitrary Skill input verbatim (agent/claudecode/transcript.go:328-341). A custom command such as /customer-acme-incident-123 would therefore be sent to PostHog. This differs from plugin telemetry, which explicitly allowlists official names before sending (plugin.go:71-85). Restrict raw names to a known allowlist or map unknown/custom names to a fixed category before adding this property.

		"skill":           inv.Skill,

[jdx]

Addressed the two remaining review findings in 4015a35, and rebased the stack on main:

• Raw skill names are not content-free (the substantive one): names now pass through to PostHog only when on an official allowlist (telemetry/skill_official.go, seeded with entire, the agent-help skill — the skills equivalent of IsOfficialPlugin). Everything else is reported as the fixed custom category, so a /customer-acme-incident-123 never leaves the machine but adoption volume stays measurable. Enforced inside BuildSkillEventPayload so every caller is covered, with a leak-regression test.
• Condensation-time extractor events never reached telemetry: Claude Code's skill_tool_use events only surface via ExtractSkillEvents on the condensation/finalize paths, so the main tool-based detection path was invisible. Those paths now persist newly extracted events into SessionState.SkillEvents — the durable dedup that keeps reporting exactly-once across repeated full-transcript (offset-0) extraction — and forward exactly the new ones: CondenseSession returns them via CondenseResult.NewSkillEvents, its three callers emit after their MutateSessionState saves, and the turn-end finalize is picked up by a state-growth snapshot around HandleTurnEnd. The emitter moved to strategy.EmitSkillInvocationTelemetry (single implementation; lifecycle.go delegates), preserving the outside-the-gate and only-after-successful-save contracts from the previous round.

Full suite re-run: 9,316 unit / 514 integration / canary green.

[jdx]

Removed a stray scripts/entire-dev symlink (an absolute path to my machine's local-dev hook launcher) that had been accidentally swept into an earlier commit — history rewritten so it never appears, no code changes. #2024 rebased accordingly.