feat(core): percentage-based canary rollouts + calibration experiment

[vanceingalls]

What

A shared, percentage-based rollout mechanism: enable a change for a stable slice of installs instead of all-or-nothing, decided locally and offline (the render path forbids network fetches), reported to PostHog as native flag properties ($feature/canary-<name>) so breakdowns/funnels/experiments work with nothing configured server-side.

• packages/core/src/canary.ts — pure evaluator (FNV-1a over feature:installId, bucket < pct). No fs/network/process; browser-safe.
• packages/core/src/canaryRegistry.ts — every rollout in one table: name, percentage, description, owner, sunsetAfter (past-due = failing test).
• packages/cli/src/telemetry/canary.ts — CLI binding: anonymousId, HF_CANARY_<FEATURE> override, CI exclusion.
• packages/studio/src/telemetry/canary.ts — browser binding: Studio distinct id (adopts the CLI id when CLI-launched, so both surfaces share a cohort), ?hf_canary_<name>=on override (session-scoped), navigator.webdriver exclusion.
• Both telemetry clients attach assignments to every event; both arms emitted ("false" = control, absent = build predates the canary).
• Leaf subpaths @hyperframes/core/canary{,-registry} via package-subpaths.json — the core barrel breaks the studio browser bundle.

Properties (all tested): inclusive ramping (widening never drops an enrolled install), independent slices per feature, fails closed (no id / unknown name / CI → off), memoized per process/page load.

Calibration experiment

Two inert canaries (calibration-10 @10%, calibration-50 @50%) gate nothing and validate the mechanism on real traffic before anything depends on it. Four pre-registered checks (accuracy, cumulative-exposure drift, cohort stability, cross-surface agreement) + an independence check — documented in docs/contributing/canary-rollouts.mdx, with matching dashboard tiles already built.

de-parallel-router is registered at 0% but deliberately unwired — wiring it through the CLI trial is follow-up work after calibration reads clean.

Install-state (breaker + cohorts)

config.json is hot and wide — ~20 fields rewritten on every command and every render — and readConfig recovers from any parse/permission/IO failure by minting a fresh identity. Two facts must not ride on that, so they live in ~/.hyperframes/install-state.json: a separate file, deliberately the same directory, so rm -rf ~/.hyperframes is a genuine full reset. (An earlier revision put it under ~/.local/state/ to survive that delete; review rejected persisting state to defeat the user's own reset, and the data agreed — install_predecessor_found reads false on 155k/155k events, so the wipe carryover has never once fired.)

• deParallelRouterTrialFired — a tripped breaker stays tripped across a config.json re-mint. Merged back in on every effective read, which makes install-state authoritative for the latch rather than a mirror: a failed write, or a stale concurrent writer rewriting config from a pre-trip snapshot, can no longer re-enrol a machine whose router already failed.
• bucketSeed — the canary bucketing unit, a dedicated UUID distinct from anonymousId, write-once. Canaries hash feature:seed, so cohorts hold across a re-mint. Never emitted.
• markerAt — written unconditionally, so the fraction of fresh mints finding it measures re-mint churn directly. Emitted as install_predecessor_found, with install_state_file_corrupt splitting "we lost this machine's record" from "genuinely new machine".

All three are independently salvageable from a partially corrupt record — most importantly the tripped bit, since dropping it is the one outcome the file exists to prevent. The file holds no telemetry identity. Migration from the pre-move location adopts and deletes it.

CLI → Studio

The CLI publishes window.__HF_CLI_CANARY_DECISIONS — { name: { enabled, forced } } — and a launched Studio adopts it rather than re-deriving, which could not agree with the CLI when telemetry is off, when an HF_CANARY_* override is set, or when no seed was injected. forced carries provenance so a deliberate override outranks Studio's own opt-out while an ordinary cohort roll does not.

Identity is treated separately from decisions: __HF_CLI_DISTINCT_ID / __HF_CLI_BUCKET_SEED go out only for a loopback Host (DNS-rebinding mitigation, applied to the SPA route and the /api/telemetry-identity endpoint alike — which no longer serves the seed at all). The decisions map is non-identifying and always published, so a LAN preview (HYPERFRAMES_PREVIEW_HOST=0.0.0.0) stays in agreement with the CLI.

Telemetry opt-out is canary opt-out

Enrolment is part of measurement: an install that reports nothing cannot be compared, so bucketing it changes that user's code path for no signal. One shared browser policy (packages/studio/src/telemetry/policy.ts) now governs both Studio transports and enrolment — documented and legacy localStorage keys, navigator.doNotTrack, VITE_HYPERFRAMES_NO_TELEMETRY, Vite dev mode, API-key eligibility — with the CLI mirroring shouldTrack()'s inputs. Resolves to telemetry_opt_out before bucketing, so no cohort is assigned at all.

The documented exception: an explicit HF_CANARY_* (or ?hf_canary_*= in Studio) override still wins. It is a deliberate operator choice — the escalation channel for dogfooding, bisects and panic-off — not silent enrolment.

Validation

Canary + carryover + policy tests across core, CLI and Studio: canonical FNV-1a vectors plus a cross-check that the shipped bucket function uses that hash, ±0.16pp accuracy at n=60k, chi-square uniformity 89.0 vs 148.2 critical, binomial independence across 8 concurrent canaries, real fleet ids landing 9.9/24.8/49.5 against 10/25/50 targets. Behavioural coverage: latch rehydration and the stale-writer race, partial-corruption salvage per field, backfill write failure, the full opt-out precedence matrix on both surfaces, each privacy control individually, and the Host split for identity vs decisions. Every fix in the review rounds has a regression that fails when that fix alone is reverted.

🤖 Generated with Claude Code

[mintlify]

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated (UTC)
hyperframes	🟢 Ready	View Preview	Jul 28, 2026, 8:11 PM

💡 Tip: Enable Workflows to automatically generate PRs for you.

[mintlify]

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated (UTC)
hyperframes	🟡 Building	–	Jul 28, 2026, 8:10 PM

💡 Tip: Enable Workflows to automatically generate PRs for you.

[miga-heygen]

SSOT -- single source of truth audit

Clean across the board.

One canonical bucketing function. evaluateCanary in packages/core/src/canary.ts is the single decision maker. Both CLI and Studio bindings import it from @hyperframes/core/canary via the leaf subpath -- neither duplicates the hash, the comparison, or the clamping logic.

One registry. CANARIES in packages/core/src/canaryRegistry.ts is the sole rollout table. Both bindings import findCanary and CANARIES from @hyperframes/core/canary-registry. Percentage lives in the registry, never at the call site.

Symmetric bindings. CLI and Studio expose the same public API (isCanaryEnabled, resolveCanary, canaryEventProperties) with the same signatures and semantics. The three surface-specific inputs (unit id, override mechanism, exclusion signal) differ only in how they're sourced:

Input	CLI	Studio
Unit id	config.bucketSeed ?? config.anonymousId	window.__HF_CLI_BUCKET_SEED ?? resolveStudioDistinctId()
Override	HF_CANARY_<FEATURE> env var	?hf_canary_<name>=on query param + sessionStorage
Exclusion	getSystemMeta().is_ci	navigator.webdriver

Both memoize decisions for the lifetime of the process/page load. Both use parseCanaryOverride for the same on/off/true/false/1/0/yes/no vocabulary. Override always wins over percentage in both directions on both surfaces. Both emit both arms in telemetry ("false" = control, absent = build predates canary).

State file is the single owner of machine-lineage. ~/.local/state/hyperframes/install-state.json carries exactly three fields: markerAt, deParallelRouterTrialFired, bucketSeed. Written only through syncInstallState, called exclusively from writeConfig. No identity leaks into this file (asserted by test: file keys are exactly those three, and bucketSeed !== anonymousId).

No SSOT findings.

---

Correctness

FNV-1a implementation verified. The multiply-by-shift trick decomposes hash * 16777619 (the FNV-1a 32-bit prime) as hash + (hash<<1) + (hash<<4) + (hash<<7) + (hash<<8) + (hash<<24), which equals hash * (1 + 2 + 16 + 128 + 256 + 16777216) = hash * 16777619. I independently confirmed all five test vectors ("" -> 0x811c9dc5, "a" -> 0xe40c292c, "b" -> 0xe70c2de5, "foobar" -> 0xbf9cf968, "hello" -> 0x4f9f2cab) match canonical FNV-1a 32-bit output. The test suite also cross-checks the shipped canaryBucket against a local FNV-1a copy on 200 UUIDs x 3 features -- that cross-check is what prevents the vector test from being tautological.

Bucket comparison is correct. bucket < pct with bucket in [0, 99]:

• 0% -> pct = 0 -> early return pct <= 0 -> off for everyone.
• 100% -> pct = 100 -> early return pct >= 100 -> on for everyone.
• 10% -> buckets 0-9 enrolled (exactly 10 of 100). Using <= would enroll 11.
• Fractional percentages truncated (10.9 -> 10), tested explicitly.
• Out-of-range clamped (negative -> 0, >100 -> 100), tested.

Inclusive ramping. bucket < 10 is a proper subset of bucket < 25 -- widening never drops an enrolled install. Tested on 500 UUIDs.

Independence per feature. Hashing feature:unitId instead of unitId alone. Tested: 8 concurrent 10% canaries on 20k UUIDs show binomial(8, 0.1) distribution (43% in none, 0 in all 8 -- correlated slices would put ~10% in all). Plus overlap between feat-a and feat-b at 10% is <70/2000, not ~200 (which would indicate identical slices).

Uniformity. Chi-square on 30k UUIDs across 100 buckets: asserted < 148.2 (critical at p=0.001, df=99). Plus accuracy within 1pp at n=20k for percentages 1/5/10/25/50.

Memoization. Both bindings memoize in a Map<string, CanaryDecision>. A late env change / URL param change after initial resolution does NOT flip the decision -- tested explicitly. Registry changes require a process/page restart anyway (new package version). This is correct and intentional.

No correctness findings.

---

State management

Write-once bucketSeed. In nextInstallState: bucketSeed: state?.bucketSeed ?? config.bucketSeed -- an existing lineage seed always wins. Tested: "a later install never overwrites the lineage seed" writes config multiple times and verifies the state file seed stays anchored to the first install.

Atomic writes. Both writeConfig and writeInstallState use the tmp+rename pattern (${path}.${process.pid}.tmp -> rename). Correct for crash safety.

Breaker carryover. latchedFired returns true if either the state file or the config has it tripped (latching OR). mintConfig reads state and carries deParallelRouterTrialFired forward. Tested: survives config wipe, survives config corruption, does NOT invent an untripped breaker.

No identity in state file. Test asserts Object.keys(stateFile()).sort() === ["bucketSeed", "deParallelRouterTrialFired", "markerAt"]. Seed asserted distinct from anonymousId. JSON.stringify(stateFile()) asserted to NOT contain the anonymousId.

Legacy backfill. Configs predating bucketSeed get a one-time backfill (preferring the lineage seed from the state file if one exists, else minting a fresh UUID). Persisted immediately so the seed doesn't re-roll per readConfigFresh. Tested: backfill is stable across fresh reads, and a legacy config adopts the lineage seed.

State-write failure isolation. syncInstallState swallows errors and leaves sync memos unset so a later write retries. The writeConfig return value reflects only the config write, not the state sync. Tested: a state-file write failure (EACCES) does not break the config write.

No state management findings.

---

Security / Privacy

bucketSeed never emitted in telemetry. The CLI telemetry client spreads only canaryEventProperties() (which returns $feature/canary-<name>: "true"/"false" for each registered canary) and install_predecessor_found (boolean). The seed is passed locally from CLI to Studio via window.__HF_CLI_BUCKET_SEED and /api/telemetry-identity, but these are localhost-only paths. Verified: no code path attaches bucketSeed to a PostHog event.

Flag vector as cross-wipe linker. The PR acknowledges this explicitly in the docs: with N canaries, the k-bit assignment vector is a low-entropy linker. At 3 canaries (current), that's 8 possible vectors -- not individually identifying. The calibration docs note this as the residual privacy cost, traded against cumulative-exposure-drift reduction. Acceptable.

XSS hardening on injected values. encodeInlineScriptValue escapes < and / in the JSON-stringified seed before injecting into the inline <script> tag, preventing </script> injection. Belt-and-suspenders since the values are randomUUID().

No security findings.

---

Standards

No bare as T in production code. The only production-side casts are (1) JSON.parse() as Partial<InstallState> followed by shape validation in readInstallState, and (2) as RecentRenderRecord inside a type-guard filter in parseRecentRenders. Both are the standard defensive patterns. All other as casts are in test files.

No non-null assertions. None found in the diff.

Package subpaths properly configured. ./canary and ./canary-registry added to package.json (development exports with bun/node/import/types conditions, publishConfig exports with import/types), and to package-subpaths.json (source/runtime/types/environments). Pattern matches the existing ./beats entry. Both subpaths list all three target environments (browser, bun, node).

Barrel re-exports. packages/core/src/index.ts re-exports both modules for consumers that don't use subpaths. The CLI and Studio bindings import from the subpaths (not the barrel), which is correct -- the barrel pulls the whole core surface and would bloat the Studio browser bundle.

No standards findings.

---

Side-effect invariant

Pure evaluator. evaluateCanary takes all inputs as arguments, does no I/O, reads no global state, writes nothing. Browser-safe by construction.

State writes only in the writeConfig path. syncInstallState is called exclusively from writeConfig. The readConfig path triggers writeConfig only during mint (first run or corrupted config) -- which is correct, since a fresh install needs to create the state file.

Both arms emitted in telemetry. canaryFeatureProperties iterates over ALL registered canaries, mapping enrolled to "true" and control to "false". Absent means "this build predates the canary" -- a different fact from "not enrolled". Tested on both CLI and Studio.

No side-effect findings.

---

CI

Both required checks pass:

• WIP: completed, success
• Mintlify Deployment: completed, success

---

Nits

1. Orphaned JSDoc in telemetryIdentity.ts. The insertion of resolveCliBucketSeed and encodeInlineScriptValue between the old JSDoc for buildCliIdentityScript and the function itself means buildCliIdentityScript no longer has its own JSDoc directly above it. The old comment ("Empty string when there's nothing to seed") now floats above resolveCliBucketSeed. Worth moving it back down or replacing it.

2. CanaryReason JSDoc says "Attach to telemetry" but the reason is deliberately not attached (only the boolean assignment is). The calibration docs say "Emitting a reason property is deliberately skipped -- add it only if this check comes back dirty." The type-level comment could mislead a future reader into thinking the reason IS on every event. Consider softening to "Available for telemetry if segmentation by reason is ever needed."

3. as const on the CANARIES array is redundant with the readonly CanaryDefinition[] type annotation (the annotation already prevents mutations and widens the element types back to CanaryDefinition). Harmless, but if the intent is deep immutability, satisfies would be clearer.

---

Verdict

Approve. This is well-designed infrastructure with an unusually strong test suite (chi-square uniformity, binomial independence cross-check, FNV vector verification, breaker corruption survival, write-once lineage seed). The SSOT is clean, the fail-closed invariant is consistent across both surfaces, the state-file separation is the right call for breaker carryover, and the calibration experiment is a thoughtful "measure before you depend on it" approach.

The three nits above are readability touches, none affect correctness or safety.

-- Miga

[miguel-heygen]

Reviewed exact head a2f2827f421b529bf33c52317d39c3305da6f1fd. The canary evaluator/registry/bindings and machine-lineage state separation are coherent, and Miga’s invariant review matches the implementation I sampled. I am withholding the approval stamp at this head for one procedural blocker: GitHub reports mergeStateStatus=DIRTY, and the only checks attached are WIP + Mintlify—not the repository CI lanes. This is a large rollout/telemetry/state change, so it needs a clean rebase onto current main and fresh build, typecheck, lint, tests, Fallow, runtime/CLI/Studio contract checks before approval. Ping me on the rebased exact head and I will delta-review/stamp it.

[james-russo-rames-d-jusso]

Reviewed at a2f2827.

Cross-cutting take: the design is a big lift and it reads as thoughtful — the pure evaluator, the registry with sunset guard, the state-file lineage, the "both arms emitted" flag-property semantics, the cumulative-exposure analysis in the docs, and the pre-registered calibration checks all read as a well-considered rollout mechanism rather than a percentage-flag with docs stapled on. docs/contributing/canary-rollouts.mdx is the clearest primitive-doc I've seen land in HF and I'd trust it as spec.

Concerns below are about specific places where the code diverges from the docs' own invariants, or where the state carryover has recovery paths that quietly break the "cohorts survive" claim under adverse conditions. Inline for each concrete site.

Highest-priority

• studio:* events do not carry canary properties. Only studio_* events (via packages/studio/src/telemetry/client.ts — which IS in this diff) got the canary wiring; the sibling packages/studio/src/utils/studioTelemetry.ts (session_start, render_start, crash, commit_transaction, panel_toggle, tab_switch, ~30 call sites) does NOT — it never imports canaryEventProperties. docs/contributing/canary-rollouts.mdx:60 claims "Every telemetry event carries the assignment." Calibration check 4 (cross-surface agreement) will be biased on the studio:* stream, and any Studio-consumer canary added later will silently miss half its Studio events. Not attaching an inline because the file isn't in the diff — this is a missing wiring rather than a bad line — but the concrete fix mirrors client.ts's import + merge into the batch payload.

• CLI HF_CANARY_<FEATURE> env override does not propagate to a CLI-launched Studio. See inline on telemetryIdentity.ts:70. The docs sell the env override as the primary escalation channel; in practice CLI-only override + Studio-percentage means the two surfaces silently diverge for the exact use case (support / dogfooding / panic-off) the mechanism was built to serve.

• Cohort divergence when CLI telemetry is off. See inline on telemetryIdentity.ts:54. A canary is a behavioral switch, not just a reporting field — CLI still buckets on config.bucketSeed when telemetry is disabled, but Studio never receives the seed and falls back to its own distinctId. Two surfaces, same install, different code paths.

Concerns worth resolving before ramp

• /api/telemetry-identity returns the raw bucket seed to any localhost caller with no origin check (inline).
• readInstallState returns null on any partial corruption, discarding a salvageable bucketSeed (inline).
• predecessorFound: state !== null conflates "no predecessor" with "corrupt state file", polluting the recoverable-churn metric the feature exists to measure (inline).
• Legacy backfill ignores writeConfig's return value; on unwritable HOME the machine mints a fresh seed per process invocation with no diagnostic (inline).
• autoUpdate.ts (not in this diff) launches a detached child that read-modify-writes ~/.hyperframes/config.json directly, bypassing writeConfig. If the main process trips deParallelRouterTrialFired: true while the child holds a stale snapshot, the child's rename silently drops the tripped fact — and because the config parse path never re-hydrates from the state file (only the mint path does), the tripped fact is gone until the trial fails again. Directly contradicts the PR body's "sync lives inside writeConfig so no breaker write site can forget it."

Nits / follow-ons

• FNV-1a hashes UTF-16 code units, not UTF-8 bytes (inline).
• No test pins de-parallel-router at 0% (inline on the registry).
• Test flake risk at pct=50: 1pp band ≈ 2.83σ at n=20000 = ~0.4% baseline flake; the chi-square is asserted against p=0.001 = 0.1% baseline flake. UUIDs are unseeded (randomUUID per run), so re-runs draw independently. Not blocking, but consider seeding the UUID population for these two tests to remove the noise.
• Barrel at packages/core/src/index.ts:333-340 re-exports canaryBucket/evaluateCanary/parseCanaryOverride + types but silently drops canaryFeatureKey/canaryFeatureProperties/CANARY_FEATURE_PREFIX. Consumers use leaf imports so nothing breaks today — cleanup nit.
• Studio URL-param lookup is lowercase-only (canaryParamName force-lowercases in packages/studio/src/telemetry/canary.ts:47-48). ?HF_CANARY_DE_PARALLEL_ROUTER=on (upper-snake, matching CLI env-var muscle memory) silently no-ops. Docs are lowercase; a case-insensitive read or an assertion at usage would surface the mistake.
• ?hf_canary_x=reset only takes effect on cold cache (resolveCanary returns memoized decision). A page reload works around; docs could clarify.

CI + merge state

• mergeStateStatus: DIRTY — merge conflict against origin/main (37 ahead / 13 behind).
• CI is largely un-run on this branch — only Mintlify Deployment + WIP fired. The GitHub Actions checks (Typecheck / Lint / Format / regression-shards / Test / Preflight / etc.) that gate every other PR in this repo have not reported. Worth confirming what triggered the CI silence — the PR won't be mergeable until it's rebased + CI reads green.

What I didn't verify

• I read the pure evaluator, both bindings, config.ts + telemetryIdentity.ts + studioServer.ts + client.ts + relevant tests; I did not exercise the CLI end-to-end.
• I dispatched 4 sub-agents in parallel for the CLI binding, Studio binding, core evaluator, and config carryover — each was asked to quote file:line for every finding. Findings above have been spot-verified against the actual code before landing here.

— Review by Rames D Jusso

[james-russo-rames-d-jusso]

🟠 Seed injection gated on telemetry-tracking, but CLI canary evaluation is not. telemetryShouldTrack() is a reporting gate — the CLI still evaluates canaries (which control code paths, e.g. process.env.HF_DE_PARALLEL_ROUTER = "true" in render.ts) regardless of telemetry state, bucketing on config.bucketSeed.

When telemetry is off (HYPERFRAMES_NO_TELEMETRY=1 / DO_NOT_TRACK / CI), resolveCliBucketSeed() returns null → buildCliIdentityScript() skips the __HF_CLI_BUCKET_SEED line entirely → Studio has no CLI seed to adopt → Studio falls back to resolveStudioDistinctId(). Same install, two different unit ids, potentially two different cohorts, two different code paths on the same render. Directly breaks canary-rollouts.mdx:203-205 ("the bucketing unit … shared with a CLI-launched Studio so both surfaces agree") and calibration check 4.

The gate should probably be "is the seed safe to expose" (which it is by design — a random UUID with no PII, no identity linkage per docs), not "is telemetry on". — Rames D Jusso

[vanceingalls]

Fixed in 4f464dc4. You and @jrusso1020 landed on this independently, which is what moved it to the top of the list.

You had the asymmetry exactly right: telemetryShouldTrack() gated seed injection while evaluation ran regardless, so an opted-out install was still bucketed and still had HF_DE_PARALLEL_ROUTER flipped underneath it.

The fix closes it at evaluation rather than at injection — resolveCanary returns a new telemetry_opt_out reason before evaluateCanary is ever called, so there's no bucket to inject a seed for. Same rule mirrored in Studio against hyperframes-studio:telemetryDisabled.

One implementation note, since it looks like an obvious refactor otherwise: the CLI check mirrors shouldTrack() rather than importing it. client.ts already imports canary.ts for canaryEventProperties, so depending on it here would be an import cycle. Both read the same two inputs — policy.ts's runtime override, then the persisted preference — so they can't drift apart; that's noted at the call site.

[james-russo-rames-d-jusso]

🟠 HF_CANARY_<FEATURE> env override does not propagate to CLI-launched Studio. This script injects __HF_CLI_DISTINCT_ID + __HF_CLI_BUCKET_SEED but NEVER process.env.HF_CANARY_*. Studio reads overrides only from window.location.search / sessionStorage (packages/studio/src/telemetry/canary.ts:98-119).

A support session running HF_CANARY_DE_PARALLEL_ROUTER=on hyperframes preview gets: CLI → forced_on (correct); Studio → whatever the percentage says for that seed (silently wrong). The docs (canary-rollouts.mdx:54-56) sell overrides as the escalation channel — dogfooding, bisects, panic-off — and this is exactly where the two surfaces should agree.

Cheapest fix: forward the resolved override decision (not the raw env-var) via a per-feature injected map, e.g. window.__HF_CLI_CANARY_OVERRIDES = { "de-parallel-router": true } — resolved once by the CLI, adopted by the Studio binding, wins over URL param + registry percentage on both sides. — Rames D Jusso

[vanceingalls]

Fixed in 98b23a88, using your suggested shape — generalised slightly to cover the sibling finding at the same time.

The CLI now publishes window.__HF_CLI_CANARY_DECISIONS ({ name: boolean }) and Studio takes it as authoritative over its own seed, URL override and registry percentage. Injecting the resolved decision rather than the raw env var is what you proposed; extending it from overrides to all decisions also closes 3670330763, because the same re-derivation was the root of both.

Worth flagging that 3670330763 was not fully closed by the telemetry-opt-out commit before it, as I'd initially thought. That gated each surface independently — which stopped silent enrolment per surface but left exactly the divergence you described: DO_NOT_TRACK=1 hyperframes preview gave CLI → telemetry_opt_out, Studio → separate localStorage flag unset → evaluates on its fallback id → may enrol. Publishing decisions is what actually closes it, since the map is emitted even when telemetry is off — that's the case it exists for.

On your framing ("the gate should be is the seed safe to expose, not is telemetry on"): agreed in spirit, and the decisions map sidesteps it — booleans about features are safe to publish where the seed isn't, and Studio no longer needs the seed to agree with the CLI. That also shrinks the surface behind 3670330769; the endpoint may not need to serve bucketSeed at all.

Studio still evaluates locally when standalone or for a canary the CLI didn't publish, and ignores a non-boolean value rather than trusting it. Tests: 6 Studio, 4 CLI (including script-tag escaping on a hostile canary name and a throwing resolver degrading to identity-only). Fault injection: dropping the adoption fails 4.

[james-russo-rames-d-jusso]

🟡 Unauthenticated endpoint exposes the raw bucketSeed to any localhost caller. No Origin/Referer/Host check, no CORS gate.

Standard SOP blocks a cross-origin fetch() from reading the JSON body, but:

• Any other local process on the machine can GET this endpoint and read the anonymous distinct id + raw seed at will — a strictly worse privacy surface than the inline <script> injection at telemetryIdentity.ts:70-77, which is scoped to Studio's own document.
• DNS-rebinding: a remote page can rebind a hostname to 127.0.0.1 and read the response as same-origin. The seed then leaks and can be joined to any prior PostHog events sharing that distinct id.

Docs comment above the handler calls this a "fallback for clients that can't rely on the injected global" but doesn't name the trade-off. Cheapest close: require a Host: localhost (or the exact bound host) check via c.req.header("host"), and/or serve the endpoint only when req.headers.get("origin") matches the studio's own origin. — Rames D Jusso

[vanceingalls]

Fixed in 31361b8e5 — closed by removal rather than a guard.

The endpoint no longer serves bucketSeed at all. Now that Studio adopts window.__HF_CLI_CANARY_DECISIONS, nothing needed the seed over HTTP, so the field just went away — and you were right that an unauthenticated local endpoint is a strictly worse surface than the inline script scoped to Studio's own document.

The endpoint itself predates this PR and still serves distinctId, so it also gained the Host guard you suggested: isLoopbackHost() accepts localhost, the whole 127.0.0.0/8 block and [::1] (with or without port), and refuses everything else. That's what makes DNS rebinding catchable — the rebound request still arrives carrying the attacker's hostname. 17 cases pinned, including 127.0.0.1.evil.com and localhost.evil.com.

[james-russo-rames-d-jusso]

🟡 Any partial corruption of the state file drops bucketSeed even when it was salvageable. The nuclear-drop on missing markerAt at line 57 discards the whole record — so if a hand-edit or partial disk write mangles markerAt while leaving bucketSeed intact, the next mintConfig mints a fresh seed and writeConfig → syncInstallState overwrites the recoverable seed with the new one. Cohort lineage lost, no diagnostic.

The docstring at line 84-85 acknowledges the trade-off ("a corrupted state file silently reads as absent"), but the choice sacrifices the invariant the PR body foregrounds ("cohorts survive the wipe"). Suggested: if markerAt is malformed, still attempt bucketSeed recovery — a valid UUID-shaped string is the higher-value fact to preserve, and markerAt can be regenerated. The corruption case is rare, but the recovery path only has to be right for a small minority to matter for fleet-level cohort math. — Rames D Jusso

[vanceingalls]

Fixed in 31361b8e5. Took your suggestion directly: a malformed markerAt no longer discards the record.

markerAt is only a timestamp and gets restamped; bucketSeed is unrecoverable, and losing it silently re-rolls the install's cohort — so the seed is the fact worth preserving, exactly as you argued. Only when neither field survives is the record treated as corrupt.

Also updated the docstring you quoted: it claimed a corrupt file 'silently reads as absent', which is no longer true and was the thing making the trade-off invisible.

[james-russo-rames-d-jusso]

🟡 predecessorFound conflates "no predecessor" with "corrupt state file". readInstallState returns null for BOTH "file absent" AND "file present but unparseable" (line 66 catch, plus the markerAt drop at line 57). So the mint reports predecessorFound: false in the corruption case, even though the machine did have a previous install.

Per the PR body, install_predecessor_found is the metric that "splits check 2's drift into the recoverable part (config wiped, machine persisted) and the part no local mechanism can link (fresh machine, container, new user)." Silently emitting false for corruption events biases that split — a machine with a partial disk write looks like a genuinely fresh mint, understating recoverable churn and overstating fresh mints.

Cheapest fix: distinguish "absent" (fs stat says no file) from "corrupt" (file exists but parse failed) in readInstallState, and emit predecessorFound: "absent" | "corrupt" | true — or a boolean plus a separate stateFileCorrupt counter for the residual. — Rames D Jusso

[vanceingalls]

Fixed in 31361b8e5 — and this one mattered more than its 🟡, because it biases the exact number the rollout gets judged on.

readInstallState now distinguishes absent from corrupt, predecessorFound reports true for corruption (the machine did have an install), and install_state_file_corrupt ships alongside so the true share can be split rather than just inflated.

Went with your boolean-plus-counter option over the tri-state string: it keeps the existing property's type stable for anything already querying it, and the residual is what you actually want to filter on.

Worth noting the field currently reads false on 155k/155k events, so it has no measured signal yet either way — which is precisely why I didn't want it silently under-reporting once it does.

[james-russo-rames-d-jusso]

🟡 writeConfig's return value is ignored — a failed backfill leaves the process running on a seed that will re-roll next invocation. If ~/.hyperframes is unwritable (root-owned after a sudo mishap, read-only bind mount, disk full), writeConfig(config) returns false, syncInstallState never runs (both writes are in the same try), and the state file has no seed to inherit from. The in-process fallback (line 384: cachedConfig = config) papers over the failure for the life of this process — but the next process invocation re-enters this branch, readInstallState() returns null again, and randomUUID() mints a different seed. No log, no diagnostic — the install silently churns cohorts across invocations.

The field's own docstring at line 255-258 claims "backfilled once for configs predating the field" — true only when the write actually lands. The failure mode is silent by design, but the docs claim doesn't match. Suggested: at minimum, log to the transport (like the swallowed telemetry-write failures), and consider surfacing via hyperframes telemetry doctor. — Rames D Jusso

[vanceingalls]

Fixed in 31361b8e5. The backfill now uses writeConfigWithResult and warns once per process with the underlying error, rather than dropping the boolean.

You were right that the docstring was the tell — 'backfilled once' was only true when the write landed, and the failure mode was silent by construction. The warning names the consequence ('canary cohort assignment will not be stable across runs') rather than just the write failure, since that's the part a user can act on.

Once per process, not per call: the backfill re-runs on every readConfig until it succeeds, so an unwritable home directory would otherwise print on every command.

[james-russo-rames-d-jusso]

🟢 FNV-1a hashes UTF-16 code units, not UTF-8 bytes. charCodeAt(i) yields a 16-bit code unit (up to 0xFFFF), two surrogate halves for astral code points. Reference FNV-1a is byte-oriented; the two agree only for ASCII.

All current inputs (kebab-case canary names, v4 UUIDs) are ASCII, so this is not a live bug — the canonical vectors at canary.test.ts:41-50 pin the ASCII case correctly. But: any future canary name that grows a non-ASCII character (an accented owner tag, an emoji, a full-width dash typed by autocorrect) will silently diverge from every other FNV-1a implementation, and any external tool that hashes canary cohorts (fleet analytics, doctor, debug tooling) won't agree.

Two ways to close: (a) require ASCII in the registry (already true by convention — a name-shape assertion on the registry test would lock it), or (b) hash the UTF-8 encoding (TextEncoder in browser, Buffer.from(s, 'utf8') in Node — but that pulls in Buffer, breaking the browser-bundle constraint). (a) is cheaper. — Rames D Jusso

[vanceingalls]

Went with your option (a), in 31361b8e5 — with the note that the guard already existed and was simply undocumented.

The registry test's kebab-case assertion (/^[a-z0-9]+(-[a-z0-9]+)*$/) already made non-ASCII names unreachable; nothing linked it to the hash, so anyone loosening it would have had no idea it was load-bearing. Now both ends say so: the test is renamed 'has unique, ASCII kebab-case names — the hash depends on this' with an explicit range check, and fnv1a32 documents that it is ASCII-only by contract, that both inputs are constrained (names by the registry, units by being UUIDs), and that widening either means switching to UTF-8 encoding first.

Agreed on (b) being the wrong trade — TextEncoder would be fine but Buffer breaks the browser-bundle constraint that motivated the six-line hash in the first place.

[james-russo-rames-d-jusso]

🟢 No test pins de-parallel-router at percentage: 0. The registry-shape tests at canary.test.ts:231-239 check bounds (percentage in [0..100]) and name uniqueness, but no per-name assertion. The description here explicitly says "Ramp only alongside the per-install circuit breaker" — a patch that bumps this to 5 without also landing the breaker wiring lands green.

If the intent is "stays at 0 until further notice", a targeted assertion —

expect(findCanary("de-parallel-router")?.percentage).toBe(0);

with a comment linking to the breaker-wiring PR — would guard the invariant against a hurried ramp. — Rames D Jusso

[vanceingalls]

Added in 31361b8e5, essentially your snippet:

it("keeps de-parallel-router at 0% until the circuit breaker is wired", () => {
  expect(findCanary("de-parallel-router")?.percentage).toBe(0);
});

With a comment making the reasoning explicit — the registry is data, so a ramp is a one-line edit with no code-review surface, and the canary's own description says to ramp only alongside the breaker. The test name states the unblocking condition so whoever hits it knows what to land first rather than just deleting the assertion.

[jrusso1020]

if someone is eleting their hyperframes config I think it should wipe all hyperframes state

i'm not sure we should try and persist state elsewhere to get around this

[vanceingalls]

Agreed, and done — this is now the design in both this PR and #2904 (merged).

install-state.json moved into ~/.hyperframes alongside config.json, so rm -rf ~/.hyperframes really is a full reset. Nothing hyperframes-related persists outside that directory, and a pre-move file is migrated in and deleted so nobody is left with a stale copy at the old path.

Your instinct was also right on the data: install_predecessor_found is false on all ~155k events since the marker shipped, so the config-wipe carryover has never once fired. (Two days is too short to prove wipes are rare, but it's certainly not evidence they're common.)

The file still earns its place, just against a different threat — config.json is rewritten on every command and every render, and readConfig re-mints a fresh identity on any parse/permission/IO failure. That re-mint is the churn that actually happens, and it's what would reshuffle canary cohorts mid-rollout. So: config re-mint → cohorts hold; directory deleted → cohorts go too.

Docs now carry an explicit removal-path section, and hyperframes telemetry status prints both paths.

[jrusso1020]

should this be in install state?

maybe the name is confusing but i feel like we would want their to be "install state" then per feature flag values separate from each other?

[vanceingalls]

Fair — the naming was doing two jobs. install-state.json now holds three facts and the header enumerates them explicitly rather than calling itself generic "install state":

1. markerAt — a previous install existed here (measures re-mint churn)
2. deParallelRouterTrialFired — circuit-breaker tripped bit
3. bucketSeed — canary bucketing unit

On splitting per-feature values into their own records: I'd rather not yet. There's exactly one canary in the registry (de-parallel-router, at 0%), so a per-feature store would be one file with one field and no second consumer to shape it. The seed is also deliberately not per-feature — bucketing hashes feature:seed, so one seed yields independent slices per feature and a per-feature seed would actually break that property.

If a second canary needs its own persisted state beyond the shared seed, that's the point to split — happy to revisit then.

[jrusso1020]

are these FFs are configuration values for optional long lived opt ins?

i think it may be conflating the two?

[vanceingalls]

Wired up in 4f464dc4 — telemetry opt-out is now canary opt-out, on both the CLI and Studio.

Every opt-out route counts: the persisted preference (hyperframes telemetry disable), the runtime env vars (HYPERFRAMES_NO_TELEMETRY, DO_NOT_TRACK), dev/telemetry-disabled builds, and Studio's hyperframes-studio:telemetryDisabled. It resolves to a new telemetry_opt_out reason before bucketing, so an opted-out install isn't assigned a cohort at all (there's a test asserting bucket stays undefined — a bucket number would mean we hashed them into a slice anyway).

An explicit HF_CANARY_<FEATURE> / ?hf_canary_<feature>= override still wins. That's a deliberate local choice rather than silent enrolment, and it stays the documented way to exercise a canary with telemetry off.

On your other question in this thread — whether these are feature flags or long-lived opt-in config — I think the gate makes the distinction concrete rather than papering over it. A canary is a measured rollout: we enrol a slice specifically so it can be compared to everyone else, which is why an install that reports nothing has no business being in one. A long-lived opt-in would be the opposite: it should work regardless of telemetry, because its value is to the user, not to us. The registry today holds only the former (de-parallel-router, at 0%). If we ever want the latter, I'd argue it belongs in config, not here — happy to make that split explicit in the docs if you'd like it stated outright.

Fault injection: removing the CLI gate fails 6 tests, removing the Studio gate fails 3.

[jrusso1020]

should we add this into/connected to the existing telemetry doc?

we shoudl also make sure if a user opts out of telemetry they opt out of this

[vanceingalls]

Both halves now done — I'd only addressed the second one earlier, thanks for re-surfacing it.

Opt-out (4f464dc4): telemetry opt-out is canary opt-out, on CLI and Studio. Resolves to a telemetry_opt_out reason before bucketing, so an opted-out install isn't assigned a cohort at all — there's a test asserting bucket stays undefined. Covers the persisted preference, HYPERFRAMES_NO_TELEMETRY, DO_NOT_TRACK, dev builds, and Studio's hyperframes-studio:telemetryDisabled. An explicit HF_CANARY_* override still wins, since that's a deliberate local choice rather than silent enrolment.

Docs connection (54534d53c): there's no standalone telemetry page — the canonical disclosure is the telemetry command section in packages/cli — so I linked all three surfaces both ways rather than adding a fourth:

• packages/cli #telemetry → telemetry state also controls canary enrolment, every opt-out route named, links here.
• this page → a <Note> at the very top, above any of the how-to, stating canaries sit behind the telemetry switch and why: a canary is a measured rollout, and an install that reports nothing can't be compared against anyone, so enrolling it buys no signal and only changes that user's code path.
• guides/feedback "Opting Out" → disabling telemetry also ends canary enrolment, listed with the feedback prompt and usage tracking.

The framing throughout is your point back to me: this isn't a separate system with its own opt-out, it's part of telemetry.

[james-russo-rames-d-jusso]

R2 delta-verify at 54534d5

Diffed from R1 SHA a2f2827f through the 11 canary-scoped commits — all R1 findings addressed, plus James's out-of-directory concern and the telemetry-opt-out-as-canary-opt-out policy landed cleanly. CI now green across all lanes (Player perf, regression shards, Windows renders, CodeQL, CLI smokes, Producer/SDK tests).

Finding	Resolution
Seed injection gated on telemetry, canary eval was not	feat(cli,studio): telemetry opt-out is canary opt-out — CLI resolves every canary to telemetry_opt_out when telemetry is off and publishes decisions to Studio via window.__HF_CLI_CANARY_DECISIONS. Divergence closed at the source.
HF_CANARY_<FEATURE> env override didn't propagate to Studio	Same commit — CLI applies env overrides during decision resolution, then injects decisions. Studio adopts them at canary.ts:80 (window.__HF_CLI_CANARY_DECISIONS?.[name]) with test coverage in canary.test.ts:233-265.
studio:* events lacked canary properties	feat(core): emit canary assignments as PostHog flag properties — moved from per-event opt-in to trackEvent's property-mixin at client.ts:77 (...canaryEventProperties()). Auto-tags every event with $feature/canary-<name>; native to PostHog flag surfaces.
Unauthenticated /api/telemetry-identity exposed seed to any local caller	isLoopbackHost() DNS-rebinding guard at telemetryIdentity.ts:69-78, applied at studioServer.ts:671. Handles [::1], dotted-quad, and the full 127.0.0.0/8 block.
Partial corruption dropped bucketSeed even when salvageable	config.ts:95-97 — corruption is now defined as BOTH markerAt AND bucketSeed absent; malformed markerAt alone synthesizes a fresh timestamp and keeps the seed.
predecessorFound conflated "absent" and "corrupt"	config.ts:280 — predecessorFound: state !== null || read === "corrupt"; corruption is now treated as "predecessor existed", with a new stateFileCorrupt field to distinguish. The docs commit 10536e847 notes override contamination as an innocent explanation for calibration check 3.
writeConfig return ignored on backfill (unwritable dir loses seed)	New writeConfigWithResult (config.ts:582) exposes ok/error; syncInstallState moved INSIDE writeConfigWithResult at line 591, so the state-file mirror is guaranteed to run only on successful atomic config write. Coupled at the source.
autoUpdate.ts detached child bypassed writeConfig (breaker drop risk)	The detached child now writes only config-only fields (completedUpdate, clears pendingUpdate) at autoUpdate.ts:105-118. Install-state (markerAt / bucketSeed / deParallelRouterTrialFired) lives in a separate file the child never touches — the race I was worried about no longer exists because the surfaces don't overlap.
FNV-1a UTF-16 vs UTF-8 discrepancy	canary.ts:85-92 — documented as ASCII-only-by-contract with an explicit widening path (TextEncoder in browser). Pragmatic given the browser bundle constraints.
No test pinning de-parallel-router at 0%	canary.test.ts:247-248 — it("keeps de-parallel-router at 0% until the circuit breaker is wired") with expect(findCanary("de-parallel-router")?.percentage).toBe(0).

Also worth calling out:

• Out-of-directory state file (James's in-thread concern): dae1b63d4 + 9f2b892a7 move install-state inside CONFIG_DIR, so rm -rf ~/.hyperframes genuinely resets everything.
• Machine-lineage seed (bucketSeed) is now decoupled from anonymousId — commit 8dd20ac2e. Correct primitive for cohort stability across a telemetry-id re-mint.
• Calibration docs got extensive clarification (checks 2-4, override contamination, per-surface comparison, dashboard link, loose-toggle count in the registry header).
• The canary-rollouts.mdx guide is still one of the clearer primitive docs in HF — the added trust-boundary discussion Vance surfaced in Slack should land in the runbook too.

LGTM from my side — clean, thoughtful rework of a subtle set of cross-surface concerns.

— Review by Rames D Jusso

[miga-heygen]

Re-review: PR #2854 — canary rollouts, post-feedback

Overview

+2518/−70 across core, CLI, Studio, and docs. Reviewed the full 3007-line diff with focus on the three concerns from James's feedback: state storage location, telemetry opt-out gate, and CLI→Studio canary agreement. All three are properly addressed.

The design is well-reasoned. The decision to publish decisions not inputs to Studio, the bucketing on feature:seed for independent slices, the telemetry-opt-out gate before bucketing, and the DNS-rebinding guard on the identity endpoint are all thoughtful choices.

---

1. State storage — ADDRESSED

The bucketSeed lives in install-state.json inside ~/.hyperframes/ alongside config.json. Deleting the config dir resets canary enrollment — test explicitly verifies: "the seed does NOT survive deleting the config dir — that reset is real."

The seed survives config.json re-mints (which happen on every command) — test verifies: "the seed survives a config.json re-mint — cohorts hold, only the id re-rolls."

The only ~/.local/state/ reference is a legacy migration path that adopts a pre-move seed and then deletes the legacy file. Clean.

2. Telemetry opt-out = canary opt-out — ADDRESSED

telemetryActive() checks BOTH the runtime override (HYPERFRAMES_NO_TELEMETRY, DO_NOT_TRACK, dev mode) AND the persisted preference (hyperframes telemetry disable). The check runs before bucketing — no cohort is ever assigned for opted-out installs. The CanaryDecision gets reason: "telemetry_opt_out" without a bucket field.

Studio mirrors this with its own isOptedOut() (localStorage flag).

An explicit HF_CANARY_* override still wins — checked first, before the telemetry gate. This is the documented path to exercise a canary with telemetry off.

3. CLI→Studio cross-surface — ADDRESSED

The CLI publishes window.__HF_CLI_CANARY_DECISIONS — a plain { name: boolean } map — and Studio takes it as authoritative. Three divergence cases eliminated:

1. Telemetry off: CLI resolves telemetry_opt_out, but Studio's opt-out is a separate localStorage flag. Without the injected decisions, Studio would evaluate independently.
2. HF_CANARY_* override: Env vars never cross into the browser. Studio would not see them.
3. No seed injected: Studio falls back to a different unit id. Without the decisions, it would land in a different bucket.

Decisions are published even when telemetry is off — precisely the case they exist for. The identity block (distinct id, seed) is suppressed; the decisions block (booleans) is emitted. Test verifies this explicitly.

4. Rollout mechanism — CORRECT

FNV-1a bucketing: Canonical test vectors verified. Distribution validated via chi-square (30k UUIDs), share accuracy within 1pp, and binomial independence across 8 concurrent features. The feature:unitId hash input gives each canary an independent slice — tested that overlap at 10% × 10% ≈ 1%, not 10%.

Registry: Single file, one entry per rollout. Each carries name, percentage, description, owner, and sunsetAfter — the sunset date is enforced by a test (overdueCanaries()). Current entries: two calibration canaries (10%, 50%) and one real canary (de-parallel-router at 0%, guarded to stay at 0% until the circuit breaker is wired).

evaluateCanary() precedence:

1. Override → forced_on/forced_off (no bucketing)
2. Telemetry opt-out → telemetry_opt_out (no bucketing, CLI/Studio only)
3. pct ≤ 0 → out_of_cohort
4. exclude (CI/webdriver) → excluded
5. No unit id → no_unit_id (fail-closed)
6. pct ≥ 100 → in_cohort
7. bucket < pct → in_cohort / else → out_of_cohort

Override-before-exclusion is correct: an explicit HF_CANARY_*=on reaches CI installs, which is how you test a canary in CI.

Ramping is inclusive: bucket < pct means widening 10→25 keeps everyone already at 10. Tested with set containment across ramp steps.

5. Telemetry integration — CORRECT

Every event carries /canary-<name>: "true" | "false" for ALL registered canaries. Both arms emitted ("false" vs absent distinguishes control from old-build). The canary- infix prevents namespace collision with real PostHog flags.

6. Security considerations — WELL-HANDLED

• DNS rebinding guard: /api/telemetry-identity now checks isLoopbackHost(c.req.header("host")). Comprehensive test coverage (localhost, 127.x.x.x, [::1], plus rejections for evil.example.com, localhost.evil.com, 192.168.x.x, 0.0.0.0).
• Script injection: JSON.stringify for the decisions map + test that a canary name containing </script> is escaped.
• Seed not exposed over HTTP: The /api/telemetry-identity endpoint deliberately does NOT serve bucketSeed. Studio gets booleans, not the value cohorts derive from.
• Resolver resilience: A throwing canary resolver does not break preview — buildCliIdentityScript catches and continues without canary data.

7. Documentation — THOROUGH

The docs/contributing/canary-rollouts.mdx page (287 lines) is excellent — it covers adding a canary, overriding, measuring, cumulative exposure drift (with the math), the calibration experiment, removal, and the four pre-registered checks. The calibration checks are stated before the data arrives, which makes the read non-post-hoc.

Minor note

One test gap: Studio client.ts adds canary properties via ...canaryEventProperties() but has no dedicated client.test.ts test verifying they appear on emitted events (the CLI does). Low risk since the pattern is identical and the Studio canary tests validate canaryEventProperties() output directly.

---

Verdict

Approve. All three original review concerns are properly addressed — state inside the config dir, telemetry opt-out as canary opt-out (before bucketing), and CLI→Studio agreement via published decisions (not re-derivation). The rollout mechanism is correct (FNV-1a verified, independent slices, inclusive ramping, fail-closed). Security is well-handled (DNS rebinding, injection escaping, seed isolation). Test coverage is comprehensive across all layers. All required CI checks pass.

— Miga

[miguel-heygen]

Reviewed exact head 54534d53c257fda8fa0f14d4d91fd58724f6b6a6. The R2 direction is substantially better, but I found four blocking gaps:

1. P1 — DNS-rebinding guard is bypassed by the SPA response. /api/telemetry-identity rejects a non-loopback Host, but the catch-all SPA route still calls buildStudioHeadScripts() and returns __HF_CLI_DISTINCT_ID plus the new __HF_CLI_BUCKET_SEED in readable HTML for the same hostile Host. A rebound origin can fetch / instead of the guarded endpoint. Guard the SPA response or omit identity/seed injection for untrusted Hosts; add a route-level hostile-Host regression.

2. P1 — a CLI decision can override Studio's own telemetry opt-out. decideStudioCanary() accepts __HF_CLI_CANARY_DECISIONS[name] before checking isOptedOut(). If CLI telemetry is enabled but Studio's documented localStorage opt-out is set, an injected percentage decision of true still enrolls Studio. A boolean map also loses the provenance needed to distinguish a deliberate HF_CANARY_* override from ordinary cohort enrollment. Preserve provenance or otherwise make Studio opt-out win for percentage enrollment while retaining explicit-override behavior, with the interaction matrix tested.

3. P1 — the legacy studio:* telemetry path is outside both new contracts. packages/studio/src/utils/studioTelemetry.ts sends a large set of studio:* events directly. It neither adds canaryEventProperties() nor honors the documented hyperframes-studio:telemetryDisabled key (it checks only legacy hf-studio-telemetry-opt-out). This contradicts “every telemetry event carries the assignment” and means the documented opt-out still emits those events. Route this path through the shared client or share the same opt-out and canary-property SSOT.

4. P2 — partial state salvage can drop a tripped breaker. salvageInstallState() treats a record as corrupt when both markerAt and bucketSeed are unusable, even if deParallelRouterTrialFired === true. mintConfig() then cannot restore the latch and can re-enroll a machine whose experimental router already failed. Treat the tripped bit as independently salvageable and add the damaged-marker/no-seed regression.

The exact-head CI is terminal green; these are behavioral/security gaps not covered by it. I also noticed the PR description and canary-rollouts.mdx:82-83 still describe superseded state/opt-out semantics, so please refresh those alongside the fixes. No merge or deployment performed.

[vanceingalls]

R3 blockers addressed — f81ab0162

All four, plus the doc refresh. Each fix has a regression that fails when the fix alone is reverted.

P1-1 — SPA route bypassed the rebinding guard. Correct and the more important half: guarding /api/telemetry-identity while the catch-all still injected identity into HTML meant the guard bought nothing — a rebound origin just fetches /. The SPA response now applies the same isLoopbackHost() check. An untrusted Host still gets a working Studio, only without identity, seed or decisions. Route-level regression asserts a hostile Host response contains none of the three globals.

P1-2 — a CLI cohort roll could override Studio's own opt-out. You identified the root cause exactly: a boolean map loses the provenance needed to tell a deliberate override from an ordinary roll. The injected map now carries { enabled, forced }:

CLI decision	Studio opted out	Result
forced on/off	yes	honoured — documented escalation channel, same as a local URL override
cohort roll	yes	refused, telemetry_opt_out
cohort roll	no	adopted, in_cohort/out_of_cohort
none	yes	refused

Full matrix tested. The precedence chain is documented on decideStudioCanary.

P1-3 — legacy studio:* path outside both contracts. Fixed at the source rather than by routing: utils/studioTelemetry.ts now calls isOptedOut() and keeps its legacy hf-studio-telemetry-opt-out check, so nobody already opted out gets quietly re-enabled, and it mixes in canaryEventProperties(). "Every telemetry event carries the assignment" is now actually true. Added studioTelemetry.test.ts (the module had no tests).

P2-4 — salvage could drop a tripped breaker. Right, and the worst of the four in consequence — losing the latch re-enrols a machine whose router already failed, which is the single thing install-state exists to prevent. All three fields are now independently salvageable; only a record where none survive is corrupt.

Docs. canary-rollouts.mdx:82-83 said "disabling telemetry disables the reporting, not the enrolment" — exactly backwards since the opt-out gate landed. Corrected, and I grepped for other copies of that claim (none). PR description refreshed below.

Verification: 257 CLI + 3,140 Studio + 24 core tests pass; typecheck, oxlint, oxfmt clean; fallow audit exit 0 (dead code 0, complexity 0 — decideStudioCanary hit 10 cyclomatic on the first pass and was restructured). Fault injection: reverting each fix independently fails 2 CLI / 1 Studio / 2 Studio / 2 CLI respectively.

[miguel-heygen]

Reviewed exact head f81ab0162e2add6c278518de8dc237450b5f8d93.

The R3 delta correctly fixes three prior blockers: hostile-Host SPA responses no longer contain the CLI identity/seed, {enabled, forced} preserves override provenance so Studio opt-out can beat ordinary enrollment, and the tripped breaker is independently salvageable. I cannot stamp this head yet:

1. P1 — the required Test check is red. packages/cli/src/server/studioServer.test.ts:90 expects the hostile-Host SPA request to return 200, but the fixture never provides a Studio bundle, so the route returns the existing “Studio bundle missing” 500 page. CI reports 1 failed / 2320 passed / 2 skipped. Give this route test a real index.html fixture (or mock the resolved bundle) so it actually exercises the injection branch, then rerun the required Test lane.

2. P1 — studio:* still bypasses several privacy opt-outs. packages/studio/src/utils/studioTelemetry.ts:40-47 now honors both localStorage keys, but unlike telemetry/client.ts:60-68 it still ignores navigator.doNotTrack, VITE_HYPERFRAMES_NO_TELEMETRY, Vite dev mode, and API-key eligibility. Those events can therefore send—and canaryEventProperties() can evaluate—under controls the public docs say disable telemetry and enrollment. Extract a cycle-free browser telemetry policy shared by both transports, and add no-fetch/no-beacon/no-enrollment coverage for every opt-out.

3. P1 — supported LAN/remote preview loses the authoritative CLI decisions. HYPERFRAMES_PREVIEW_HOST=0.0.0.0 is an explicit supported mode, but studioServer.ts:824-827 withholds the entire head script for every non-loopback Host. That drops the safe {enabled, forced} map along with identity/seed, so Studio re-derives locally and can disagree with CLI opt-out, cohort, or env overrides again. Split non-identifying decision injection from identity/seed injection (or reject non-loopback Studio access entirely), and cover the supported remote-host matrix.

4. P1 — the install-state breaker is not authoritative or truthfully persisted. writeConfigWithResult() returns success even when syncInstallState() fails, and the safety retry in render.ts:1254-1262 verifies only config.json. Conversely, normal reads take deParallelRouterTrialFired only from config.json; they do not rehydrate a latched true from install-state. A failed mirror—or a stale concurrent config writer after the breaker fires—can therefore leave state latched but config false, and a later process re-enrolls. Make the safety write verify both stores and/or merge the latched state into every effective read; add deterministic state-write-failure and stale-writer regressions.

5. P2 — the public contracts still describe superseded behavior. canary-rollouts.mdx:119-121 says a config wipe loses the breaker, while :184-197 says the opposite; :225-233 documents a {name:boolean} map and unconditional CLI precedence instead of {enabled,forced} plus the current precedence. The PR body still names ~/.local/state/..., says state survives deleting ~/.hyperframes, describes the old identity/seed bridge, and has stale validation counts. Please update both before landing; also state explicitly that deliberate HF_CANARY_* / URL overrides are the exception to telemetry opt-out.

Security note, not charged to this PR: explicitly LAN-bound Studio APIs remain broadly unauthenticated. The new Host check is a DNS-rebinding mitigation for identity-bearing responses, not general access control.

No merge or deployment performed.

[vanceingalls]

R4 blockers addressed — 5e2a9432f

P1-1 — red Test lane, and it was my test. You diagnosed it exactly: studioServer.test.ts:90 asserted a 200, which only holds when packages/studio/dist is built — true on a dev box, false in CI's Test lane. I reported that round green off a local run, which was the actual error. Rather than add a bundle fixture, the Host split moved into a pure buildStudioHeadScriptsForHost() and is asserted there; the route test no longer touches the injection branch. Verified by running the full CLI suite with the bundle moved aside: 2330 pass (was 1 failed / 2320).

P1-2 — studio:* bypassed most privacy controls. Correct, and worse than the previous round's fix implied: honouring both localStorage keys still left navigator.doNotTrack, VITE_HYPERFRAMES_NO_TELEMETRY, Vite dev mode and API-key eligibility unenforced — and canary enrolment was gated on a different single control. New packages/studio/src/telemetry/policy.ts is the one answer to "may this profile be measured", consumed by both transports and by enrolment. It imports only ./config, so there is no cycle with the modules that consume it. policy.test.ts asserts each control individually, including that it is deliberately un-memoized so a mid-session opt-out takes effect.

P1-3 — LAN/remote preview lost the authoritative decisions. Right, and this was a regression I introduced fixing the rebinding gap: withholding the whole head script also dropped the non-identifying map, sending a supported HYPERFRAMES_PREVIEW_HOST=0.0.0.0 Studio back to re-deriving. Identity and decisions are now gated separately — identity loopback-only, decisions always published. Covered across a remote-host matrix (evil.example.com, 192.168.1.10, *.local, absent Host).

P1-4 — the latch was neither authoritative nor truthfully persisted. Both halves confirmed. Rather than only verifying the write, the latch is now merged into every effective read, which makes install-state authoritative and closes the failed-mirror and stale-concurrent-writer paths in one place — verifying a write cannot help when a later writer clears the flag. writeConfigWithResult additionally reports mirrored: false instead of swallowing. Regressions for the stale-writer race and the state-write failure.

P2-5 — public contracts. canary-rollouts.mdx:119-121 claimed a config wipe loses the breaker (it does not, and :184-197 said so) and :225-233 documented the superseded {name: boolean} map with unconditional CLI precedence. Both corrected, precedence ladder written out, and the HF_CANARY_* / URL-override exception to telemetry opt-out stated explicitly as you asked. PR body rewritten — it still named ~/.local/state, claimed state survives deleting ~/.hyperframes, described the old identity/seed bridge, and carried stale counts.

Noted, not actioned: agreed that the Host check is a rebinding mitigation for identity-bearing responses, not access control for LAN-bound Studio APIs generally. Out of scope here.

Verification: 2330 CLI (bundle absent, matching the CI lane) + 3151 Studio + 24 core; typecheck, oxlint, oxfmt clean; fallow audit exit 0. Fault injection: reverting each fix alone fails 5 CLI / 5 Studio.

[miguel-heygen]

Re-reviewed exact head 5e2a9432f142416de7d3d72f0d6b08ad999a23e4.

The R4 delta fixes the bundle-dependent Test failure, routes both Studio transports and canary enrolment through the shared browser policy, preserves non-identifying CLI decisions for LAN/remote Hosts, and refreshes the public contracts. One safety blocker remains in two connected pieces:

1. P1 — a negative install-state latch is cached forever in a long-lived process. config.ts:150-170 memoizes both true and false, but only true is monotonic across processes. Sequence: process A reads unlatched state and caches false; process B trips and mirrors the breaker; a stale writer clears config.json; A calls readConfigFresh(), which clears only cachedConfig, reuses the cached false, and can re-enrol in the experiment that already failed. Cache only true, or invalidate/re-read the negative latch on a fresh effective read. Add the same-module false → external state true → stale config false → fresh read true regression.

2. P1 — the real breaker writer still discards mirrored:false. writeConfigWithResult() exposes a failed install-state mirror, but render.ts:1254-1262,1313-1319 calls writeConfig(), which collapses {ok:true, mirrored:false} to success. The follow-up fresh read sees config.json:true and returns before retrying or warning. A later stale write/corruption can then erase the only durable copy. Consume writeConfigWithResult() in this safety path, require/retry the mirror, and cover mirror-failure followed by stale-clobber through the real caller.

The added tests separately preload an already-true state and assert that mirrored:false is returned, so neither transition is currently exercised. Targeted config/policy tests pass and required Build/Test/Lint/Typecheck/CodeQL are green; several Windows/regression jobs are still running. No merge or deployment performed.

[vanceingalls]

R5 blockers + independent review findings — 3f69a2c63

Your two blockers

P1-1 — negative latch cached forever. Right, and the asymmetry is the crux: only true is monotonic across processes. Only the positive is cached now; false re-reads. Regression added for the exact sequence you named — false → external state true → stale config false → fresh read true.

P1-2 — the real writer discarded mirrored:false. persistDeParallelRouterTrialFired now consumes writeConfigWithResult and requires BOTH stores before returning success, retrying otherwise. Also correct that the existing tests preloaded an already-true state and so exercised neither transition; the new test drives a mirror failure through the real caller and asserts the retry.

Plus 15 from an independent max-effort review

I ran one against this branch. It surfaced 15 confirmed defects, nine of which I introduced across the previous fix rounds — worth stating plainly, since the pattern across R2–R5 has been that each fix creates the next finding.

Most consequential:

• **Storage-restricted Studio profiles all bucketed on the literal "anonymous"".** Computed against the shipped hash: calibration-50:anonymous` → bucket 44, so 100% of that population was enrolled at a nominal 50%, and they merged into one PostHog person. The calibration numbers this PR exists to produce were being read against that.
• The override path minted and persisted a tracking id for opted-out profiles — resolveBucketUnit() evaluated eagerly as an argument that evaluateCanary discards unread. One click on a support link created a durable identifier for someone who opted out. Mine, from the precedence restructure.
• CLI decisions memoized per process, so telemetry disable during a running preview server was ignored for hours — the docs say that cannot happen.
• bucketSeed had read/write authority backwards. I gave the latch read-side authority and not the seed, so the two stores could disagree until a re-mint flipped every cohort.
• isLoopbackHost rejected 0.0.0.0, so my own Host guard broke identity stitching in the documented LAN mode. Identity is now allowed when the operator explicitly opted into LAN binding — and I corrected the comment claiming the guard refuses spoofed Hosts, which is false for any non-browser client. It is a browser rebinding mitigation, as you said, and now says so.
• A corrupt pre-move state file was never deleted, so a machine reset with rm -rf ~/.hyperframes reported predecessorFound forever — poisoning the one metric this work produces.
• overdueCanaries() was asserted against wall-clock time, so the entire core suite would have gone red on 2026-09-15 for every unrelated PR. Also > against midnight made a canary overdue ON its sunset date.
• Statistical assertions ran unseeded and failed roughly 1 run in 200.

Also broke a config → policy → transport → config import cycle I had introduced, by moving POSTHOG_API_KEY to a leaf.

Two pre-existing tests asserted behaviour these findings identify as wrong — shouldTrack memoization and 100% still excludes CI — and were rewritten with the reasoning stated rather than worked around.

Verification: 2347 CLI (bundle absent, matching the CI lane) + 3153 Studio + 1450 core; typecheck, oxlint, oxfmt clean; fallow audit exit 0. Fault injection covers the latch, seed authority, LAN identity, webdriver exclusion and the anonymous-bucket fix.