fix(telemetry): opt-in default + picker funnel events#3308
Merged
louisgv merged 1 commit intoOpenRouterTeam:mainfrom Apr 15, 2026
Merged
fix(telemetry): opt-in default + picker funnel events#3308louisgv merged 1 commit intoOpenRouterTeam:mainfrom
louisgv merged 1 commit intoOpenRouterTeam:mainfrom
Conversation
Two bugs from the OpenRouterTeam#3305 rollout: 1. Test pollution: orchestrate.test.ts imports runOrchestration directly and never calls initTelemetry, but _enabled defaulted to true in the module so captureEvent happily fired real events at PostHog tagged agent=testagent. The onboarding funnel filled up with CI fixture data. 2. Funnel started too late: funnel_* events fired inside runOrchestration, which is only called AFTER the interactive picker completes. Users who bail at the agent/cloud/setup-options/name prompts were invisible — yet that's exactly where real drop-off happens. Fix 1 — telemetry.ts: - Default _enabled = false. Nothing fires until initTelemetry is explicitly called. Production (index.ts) calls it; tests that need telemetry (telemetry.test.ts) call it with BUN_ENV/NODE_ENV cleared. - Belt-and-suspenders: initTelemetry now short-circuits when BUN_ENV === "test" || NODE_ENV === "test", so even if future code calls it from a test context, events stay local. Fix 2 — picker instrumentation: New events fired before runOrchestration in every entry path: spawn_launched { mode: interactive | agent_interactive | direct | headless } menu_shown / menu_selected / menu_cancelled (only when user has prior spawns) agent_picker_shown agent_selected { agent } — also sets telemetry context cloud_picker_shown cloud_selected { cloud } — also sets telemetry context preflight_passed setup_options_shown setup_options_selected { step_count } name_prompt_shown name_entered picker_completed Wired into: commands/interactive.ts cmdInteractive + cmdAgentInteractive commands/run.ts cmdRun (direct `spawn <agent> <cloud>`) cmdRunHeadless (only spawn_launched) runOrchestration's existing funnel_* events continue to fire unchanged. The final funnel in PostHog: spawn_launched → agent_selected → cloud_selected → preflight_passed → setup_options_selected → name_entered → picker_completed → funnel_started → funnel_cloud_authed → funnel_credentials_ready → funnel_vm_ready → funnel_install_completed → funnel_configure_completed → funnel_prelaunch_completed → funnel_handoff Tests: - telemetry.test.ts: 2 new env-guard tests (BUN_ENV, NODE_ENV), plus updated beforeEach to clear both env vars so existing tests still exercise initTelemetry. - Full suite: 2131/2131 pass, biome 0 errors. Bumps 1.0.12 -> 1.0.13 (patch — auto-propagates under OpenRouterTeam#3296 policy).
louisgv
approved these changes
Apr 15, 2026
Member
louisgv
left a comment
There was a problem hiding this comment.
Security Review
Verdict: APPROVED
Commit: 72c0cf0
Findings
No security issues found. This PR improves telemetry safety:
- Opt-in default: Telemetry starts disabled (
_enabled = false) untilinitTelemetry()is explicitly called - Test environment guard: Blocks telemetry in CI (
BUN_ENV=test,NODE_ENV=test) to prevent test fixtures polluting production analytics - PII protection maintained: All new
captureEvent()calls only send non-sensitive metadata (agent/cloud names, mode strings, counts) - Existing scrubbing active: String values still pass through the scrubber that redacts API keys, tokens, emails, IPs, paths
Tests
- bun test: PASS (2131 tests, all green)
- bash -n: N/A (no shell scripts changed)
- Test coverage: New tests verify the
BUN_ENV=testandNODE_ENV=testguards work correctly
-- security/pr-reviewer
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes two bugs discovered after #3305 merged and data started flowing:
Fix 1 — opt-in default
`telemetry.ts`:
Fix 2 — picker funnel events
New events fired in every entry path before `runOrchestration` is called:
Wired into:
`runOrchestration`'s existing `funnel_*` events continue to fire unchanged, appended to the end of the funnel. The full sequence in PostHog is now:
```
spawn_launched
→ agent_selected
→ cloud_selected
→ preflight_passed
→ setup_options_selected
→ name_entered
→ picker_completed
→ funnel_started (existing)
→ funnel_cloud_authed
→ funnel_credentials_ready
→ funnel_vm_ready
→ funnel_install_completed
→ funnel_configure_completed
→ funnel_prelaunch_completed
→ funnel_handoff
```
Dashboard follow-up (manual, not in this PR)
After merge, the PostHog funnel insight should be rebuilt to start from `spawn_launched` (not `funnel_started`). The existing `testagent` events from before this fix can't be deleted, but they're trivially filterable — add `agent != testagent` as a property filter on the insight.
Tests
Version
Bumps 1.0.12 → 1.0.13. Patch bump — auto-propagates under the #3296 policy so the fix reaches users on their next spawn run.
Closes the regression introduced by #3305.