Serve document bytes, not the app shell, to <object>/<embed> loads

[lukemelia]

What this does

Opening a PDF FileDef in the embedded or isolated view rendered the entire Boxel app recursively nested inside the preview. The chain: the PDF viewer mounts a native <object data="…/file.pdf" type="application/pdf">; the browser issues <object>/<embed> loads as frame-style navigations whose Accept header includes text/html; and the realm serves the host app for any request that accepts HTML (the same negotiation that lets a pasted file URL open in operator mode) — so the <object> receives the app shell, boots it, deep-links to the PDF card, mounts another <object>, and recurses.

The negotiation is corrected on the server, because no client-side layer can do it: service workers are spec-required to pass <object>/<embed> loads straight to the network without dispatching a fetch event (ServiceWorker spec §6.7; confirmed empirically in Chrome — a controlling SW sees <img> loads but never object/embed loads). These loads do self-identify on the wire, though: browsers stamp Sec-Fetch-Dest: embed/object on them (on trustworthy origins — HTTPS and localhost), while an address-bar navigation carries Sec-Fetch-Dest: document.

serveIndex/serveHostApp in the realm server now decline to serve the app shell when the request's Sec-Fetch-Dest is embed or object, falling through to the realm, which serves the file's own bytes with the file's own content type. Consequences of fixing at this layer:

• Pasting a file URL in the browser still opens the app (Sec-Fetch-Dest: document), and requests with no Sec-Fetch-Dest keep their existing behavior.
• Prerendered HTML carrying real file URLs in <object data="…"> stays valid in any browsing context, where a component-fetched blob: URL would be dead outside the context that created it.
• Cacheable (ETagged) shell responses add Sec-Fetch-Dest to Vary, so a cached shell can never satisfy an embed load.

Known limitation, unchanged by this PR: <object>/<embed> loads can't carry Authorization (the same service-worker bypass blocks token injection), so documents render in the native viewer only from publicly readable realms. Private-realm document auth through native embeds needs its own design — follow-up.

Test plan

• Wire-level tests in packages/realm-server/tests/server-endpoints/index-responses-test.ts (document embed negotiation module): a .pdf GET with a frame-style Accept plus Sec-Fetch-Dest: embed or object returns application/pdf bytes, not the shell; the same URL with Sec-Fetch-Dest: document or no Sec-Fetch-Dest still returns the app shell; the published-realm ETag response's Vary includes Sec-Fetch-Dest. All touched modules green locally (the full file exhausts the local test-pg tmpfs in unrelated later modules, so they were run per-module).
• Realm-server typecheck and eslint clean; base package template-lint clean.

Fixes CS-12557.

🤖 Generated with Claude Code

[lukemelia]

[Claude Code 🤖] This review focused on verifying the fix's transport-layer premise end-to-end — does the pinned Accept header actually reach the wire on an <object>/<embed> load? — because the whole change hangs on the service worker seeing those requests.

Bottom line: blocking. Service workers never receive fetch events for <object>/<embed> loads — the ServiceWorker specification makes them fall back to the network without dispatching an event, and I reproduced exactly that in Chrome 151 — so acceptOverrideFor never fires, the pinned Accept never leaves the browser, and the recursive app boot is still reachable. Full evidence (spec text, live-browser repro, captured request headers) is in the inline thread on acceptOverrideFor in packages/host/public/auth-service-worker.js.

What lands right. The diagnosis of the recursion is verified correct on both halves. The captured <embed> request really does go out frame-style with accept: text/html,application/xhtml+xml,... and sec-fetch-mode: navigate. And on the realm side, serveIndex in packages/realm-server/handlers/serve-index.ts serves the app shell for any text/html-accepting request without a vnd. type regardless of file extension, while with Accept: */* an extension-carrying path takes the hasExtension branch to next(), reaching the realm's fallbackHandle → serveLocalFile, which returns the document bytes. If the pinned header reached the wire, the realm would behave exactly as this description says. The interception point is the only wrong link in the chain.

Recommendations

1. Refix at the realm's content negotiation: serveIndex should decline to serve the app shell when the request carries Sec-Fetch-Dest: embed or object — these loads self-identify on the wire, address-bar navigations carry Sec-Fetch-Dest: document and keep working, and no client change is needed. Detail in the acceptOverrideFor thread.
2. Pin whichever fix lands with a wire-level realm-server integration test (frame-style request for a file URL asserting bytes, not shell) — detail in the thread on the test scaffold in packages/host/tests/unit/auth-service-worker-test.ts.
3. Follow-up worth filing: private-realm documents cannot authenticate through native embeds at all — the same bypass blocks Authorization injection, so the native-viewer approach needs an auth story (authed fetch + blob:, delegated/signed URLs, or cookie-scoped sessions).

Adjacent, out of scope. The comment in packages/base/file-formats/pdf-viewer.gts stating the auth service worker injects the realm token on the native <object> request describes a path that does not exist (same bypass). Pre-existing and not this change's doing, but worth correcting when the real fix lands so the next reader doesn't build on it again.

[github-actions]

Host Test Results

    1 files      1 suites   2h 12m 7s ⏱️
3 795 tests 3 781 ✅ 14 💤 0 ❌
3 812 runs  3 798 ✅ 14 💤 0 ❌

Results for commit 4b46c8d.

Realm Server Test Results

    1 files      1 suites   16m 3s ⏱️
2 178 tests 2 178 ✅ 0 💤 0 ❌
2 258 runs  2 258 ✅ 0 💤 0 ❌

Results for commit 4b46c8d.