perf(web): stop blocking every API request on a Clerk token fetch

[Makisuo]

Why

The session replays list felt slow. It turned out the replay queries weren't the problem — the page fires exactly two requests, and the ClickHouse work behind them is ~400 ms and ~140 ms.

Measuring the per-trace gap between the browser's client span and the API Worker's server span, across all maple-web → maple-api traces on 2026-08-04 (n=1485):

pre-handler:  p50 580ms   p75 1056ms   p90 3224ms   p99 6806ms   (16% over 2s)
handler:      p50 204ms                p90 2832ms

Not clock skew (per trace, pre_handler + handler ≈ client span duration) and not retries (POST is excluded from transport retry, and the traces carry exactly one server span). It's a fixed per-request cost, uniform across every endpoint in the app — /query-engine/execute p50 1109 ms, /billing/customer p50 828 ms, /auth/session p50 807 ms, all against ~200 ms handlers.

The one thing on that path for every request: mapleFetch awaited getMapleAuthHeaders(), which resolved to Clerk's getToken() with no caching at any layer. Clerk session JWTs live 60 s, so a page loaded more than a minute after the last request paid a full cross-origin round-trip to Clerk before the first byte went to the API.

What changed

Token cache (auth-headers.ts) — the main fix, and it affects every request in the app, not just replays.

• Serves the cached bearer while >10 s of life remains; refreshes in the background between 10–30 s so no request ever blocks on Clerk.
• Refresh is single-flighted, so a burst (this page fires two requests) triggers one Clerk call.
• A generation counter discards a refresh that resolves after the identity changed.
• Invalidated on org switch — the JWT encodes the active org, so a token outliving a switch would query the previous org's data.
• Only JWT-shaped bearers are cached; an opaque token (self-hosted, read synchronously from sessionStorage) has no expiry we can trust, so that path is unchanged.
• A 401 invalidates the cache and retries once (atom-client.ts), so a token stale ahead of its own exp self-heals. Bounded per-request via a WeakSet so concurrent 401s don't consume each other's allowance, and a second 401 still fails fast.

Route loader on /replays — the router runs defaultPreload: "intent", so both queries now start on hover rather than after hydration + route-chunk evaluate. The detail route already did this; the list route didn't.

Narrowed the org ClickHouse settings hot-path read — SELECT * (14 columns, including encrypted password blobs) → the 8 that are actually cached. This query runs on every warehouse execution and measures p50 2098 ms / p99 6002 ms in the API Worker (vs 313 ms for identical code in alerting). It is a child of executeSql, which is why nearly every query.context in the catalogue shares a ~3 s p95 regardless of what it scans.

Also removed a comment asserting EdgeCacheService single-flights in-flight computes — it deliberately doesn't, because Cloudflare ties I/O objects to the request that created them.

Reviewer notes

The key-drift risk in the loader is tested, not assumed. A prefetch only pays off if the loader and the component key to the same atom-family entry; a mismatch would silently double the requests and nothing else would notice. The filter-input builder is extracted to replays-filter-inputs.ts and replays-filter-inputs.test.ts pins the invariant — including a fake-timer test proving the keys still match 14 s apart, since encodeKey snaps timestamps to a 15 s grid.

isWarehouseWriteReady deliberately keeps its uncached read. Routing it through the shared 5-minute in-isolate memo failed CloudflareAnalyticsService.test.ts, correctly: that gate decides which warehouse a read is answered from, and a stale false right after onboarding sends reads to Tinybird while the gateway is already writing to the org's ClickHouse — data silently missing until the memo expires. It reads the narrow projection now, but it reads it fresh.

The auth.token span was dropped. A proper child span needs header injection moved out of the fetch shim and into the Effect client pipeline, which risks a request path going out unauthenticated for a measurement nicety. The pre-handler-gap query below measures the same thing from telemetry that already exists.

Not in this PR

Two items with trade-offs that need a call rather than a default:

• Hoist resolveRoute to once per request. Still the right fix for the shared ~3 s p95, but OrgClickHouseSettingsService is built per isolate, not per request, so a naive in-flight map hits the exact "Cannot perform I/O on behalf of a different request" failure the edge cache documents. Needs request-scoped state threaded through the execution core every query runs through.
• Cache the two replay handlers. Biggest remaining win for this page (removes both ClickHouse queries and the Postgres read on repeat loads), but this is a page about live sessions and refreshVersion never reaches the list query — so a server TTL could return a byte-identical response to an explicit refresh.

Verification

bun typecheck clean for both apps. 216 web tests and 87 API tests pass (scoped to touched areas).

After deploy, the decisive before/after — this needs no new instrumentation:

SELECT quantile(0.5)(pre_ms), quantile(0.9)(pre_ms) FROM (
  SELECT TraceId,
    dateDiff('millisecond',
      minIf(Timestamp, ServiceName='maple-web' AND SpanKind='Client'),
      minIf(Timestamp, ServiceName='maple-api' AND SpanKind='Server')) AS pre_ms
  FROM traces WHERE $__orgFilter AND $__timeFilter(Timestamp)
    AND ServiceName IN ('maple-web','maple-api') GROUP BY TraceId)

Watch pre_p50 fall from 580 ms toward the network floor, and SessionReplays.listReplays / .facets client-observed p50 toward handler-time-plus-network.

🤖 Generated with Claude Code

---

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

[pullfrog]

Your Pullfrog Router balance is exhausted.

You have a payment method on file but auto-reload is disabled, so runs paused once your balance went past the overdraft buffer.

Top up balance → · Enable auto-reload →

  ^{｜ Rerun failed job ➔ ｜ View workflow run ｜ via Pullfrog ｜ Using Claude Opus ｜ 𝕏}

[github-actions]

🍁 Maple PR preview

Note

Preview resources were removed when this pull request closed.

Final commit 5103bac · View workflow run