diff --git a/.github/workflows/ci-lint.yaml b/.github/workflows/ci-lint.yaml index 72941a54fbb..1719f93fc1f 100644 --- a/.github/workflows/ci-lint.yaml +++ b/.github/workflows/ci-lint.yaml @@ -35,6 +35,12 @@ jobs: - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - uses: ./.github/actions/init + - name: Check rendering-protocol statement/test bijection + # RP-0.3: a conformance test citing a statement the spec no longer + # makes fails the build. Coverage of the reverse direction is + # report-only until the spec reaches NORMATIVE status (--strict). + if: ${{ !cancelled() }} + run: pnpm run lint:rp-bijection - name: Lint Boxel Icons if: ${{ !cancelled() }} run: pnpm run lint diff --git a/.github/workflows/preview-host.yml b/.github/workflows/preview-host.yml index 591826f3dad..b692ef9618d 100644 --- a/.github/workflows/preview-host.yml +++ b/.github/workflows/preview-host.yml @@ -97,7 +97,11 @@ jobs: aws-region: us-east-1 - name: Load app environment variables shell: bash - run: cat packages/host/config/production.env >> $GITHUB_ENV + run: | + cat packages/host/config/production.env >> $GITHUB_ENV + # Production preview nonce hosts use the DNS-ready preview edge until + # the boxelusercontent.com wildcard DNS record is provisioned. + echo "BOXEL_SANDBOX_RUNTIME_URL=https://boxelusercontent.dev" >> $GITHUB_ENV - name: Deploy boxel-host preview uses: ./.github/actions/deploy-ember-preview env: diff --git a/.github/workflows/test-web-assets.yaml b/.github/workflows/test-web-assets.yaml index f518f6e4e1c..de7fad11b67 100644 --- a/.github/workflows/test-web-assets.yaml +++ b/.github/workflows/test-web-assets.yaml @@ -100,6 +100,12 @@ jobs: # vite chunks. Empty preserves the standard-mode defaults. BOXEL_ENVIRONMENT: ${{ inputs.boxel_environment }} + - name: Verify Host test environment config + run: pnpm test:environment-config + working-directory: packages/host + env: + BOXEL_ENVIRONMENT: ${{ inputs.boxel_environment }} + - name: Write assets manifest shell: bash run: | diff --git a/docs/boxel-capsule-glimmer-dom-review.md b/docs/boxel-capsule-glimmer-dom-review.md new file mode 100644 index 00000000000..eeab27b1844 --- /dev/null +++ b/docs/boxel-capsule-glimmer-dom-review.md @@ -0,0 +1,1335 @@ +# Capsule execution through Glimmer and the Host DOM + +**Status:** branch-specific implementation review for +`codex/boxel-execution-runtime-architecture`. + +**Audience:** Ember and GlimmerVM maintainers evaluating whether Boxel's +Capsule renderer uses supported framework contracts, where it depends on +compiler/VM internals, and whether the shared-document containment argument is +sound enough to admit more unknown authored cards. + +This is a companion to the +[execution runtime reviewer's guide](boxel-execution-runtime-reviewers-guide.md). +The normative Boxel boundary contract remains the +[Boxel Rendering Protocol](boxel-rendering-protocol.md). + +## Executive assessment + +A Capsule is **not a DOM sandbox** in the iframe sense. It is a JavaScript +authority boundary plus a constrained rendering bridge: + +1. authored component classes, state, getters, and actions run in an SES + `Compartment` with no ambient `window`, `document`, storage, or `fetch`; +2. Boxel captures the component's compiled Glimmer template descriptor inside + that compartment; +3. it validates the serialized Glimmer block and rewrites every scope entry to + one of three boundary references: another authored component, an approved + trusted export, or JSON data; +4. the Host reconstructs a component definition with Ember's component-manager + and template APIs; and +5. the Host's own Glimmer VM executes that template and mutates the shared Host + document. + +No live `Element`, `Event`, owner, service, Store, Loader, or authored function +is handed to authored Capsule JavaScript. Nevertheless, the compiled template +is a consequential channel: it is an instruction program interpreted by a +trusted Glimmer VM with access to the Host DOM. Containment therefore depends +on the completeness of Boxel's template, scope, trusted-export, event, and CSS +policies and on the stability of the Glimmer wire format they inspect. + +The current design has valuable properties: + +- one Host Glimmer runtime renders Direct and Capsule content; +- trusted Base/Cardstack components are reused by reference rather than + serialized or reimplemented; +- component and DOM identity survive ordinary argument updates; +- authored actions receive reduced event data and can emit only named Host + effects; +- nested authored fields and cards re-enter the same execution router; and +- unsupported browser behavior is promoted to an origin-isolated Sandbox + iframe instead of being emulated by a generic DOM escape hatch. + +The main concern is not an identified direct `document` leak. It is that Boxel +currently treats a partly validated, unversioned Glimmer compiler artifact as +an executable boundary format. Before expanding Capsule admission materially, +we should either obtain an Ember-supported portable-template contract or make +the present coupling explicit, versioned, exhaustively validated, fuzzed, and +fail-closed across Ember upgrades. + +## The exact boundary + +The phrase "Capsule renders into the Host DOM" can be misleading. The +authored component does not call DOM APIs remotely. Instead, two programs +cooperate across a synchronous membrane: + +```mermaid +flowchart LR + Source["Authored GTS source"] + SES["SES Compartment\ncomponent JS and state"] + Bundle["Validated template bundle\nwire block + typed scope refs"] + Manager["Host component manager\nstable proxy context"] + VM["Host Glimmer VM"] + DOM["Shared Host DOM\nCapsule slot subtree"] + + Source --> SES + SES --> Bundle + Bundle --> Manager + Manager --> VM + VM --> DOM + VM -->|"safe action arguments"| SES + SES -->|"state + named effects"| Manager +``` + +What crosses from the Capsule toward the Host: + +- a protocol-versioned graph of template descriptors; +- serialized Glimmer `block` strings; +- typed scope references, never arbitrary scope functions; +- cloneable component state and getter return values; +- reduced action return values; and +- named effects such as `view-card` and `set`. + +What crosses from the Host toward the Capsule: + +- a membrane whose reads clone the current named arguments; +- cloneable render/model data; +- reduced action arguments; and +- stable closures for specifically granted effects. + +What does **not** cross: + +- live DOM nodes or native browser events; +- an Ember owner or dependency-injection container; +- the Host Card API instance, Store, Loader, network, or service objects; +- arbitrary callbacks from a template scope; or +- the Host's component class for a trusted portal. + +Trusted components are resolved on the Host side. The Capsule bundle carries +only a `(module, export)` identity for them. + +## End-to-end implementation + +### 1. Routing chooses Capsule before any template runs + +[`boxel-source-classifier.ts`](../packages/host/app/lib/boxel-source-classifier.ts) +classifies an authored module and its static import graph. Browser globals, +DOM-dependent libraries, arbitrary modifiers, dynamic inline styles, global +CSS, top-layer markup, and similar signals promote full formats to Sandbox. +Otherwise authored modules default to Capsule. + +Classification is a compatibility decision and an early security layer, but +it is not the final verifier. The Capsule evaluator independently rejects +inadmissible template and scope content. This second check is important: a +classifier miss must become a visible refusal, not Host DOM authority. + +The common runtime interface is +[`BoxelRuntime`](../packages/host/app/lib/boxel-runtime.ts): + +```ts +export interface BoxelRuntime { + readonly mode: 'direct' | 'capsule' | 'sandbox'; + + loadBoxel(ref: CodeRef): Promise; + createFromSerialized( + resource: LooseCardResource, + document: LooseSingleCardDocument, + relativeTo: RealmResourceIdentifier | undefined, + purpose: MaterializationPurpose, + ): Promise; + buildRenderRecord(card: BoxelInstanceHandle): Promise; + dispose(handle: RuntimeHandle): Promise; +} +``` + +[`CapsuleBoxelRuntime`](../packages/host/app/lib/capsule-boxel-runtime.ts) +implements this contract with opaque handles and cloneable records. It caches +render slots per type, format, and explicit component provider. A missing +authored format returns a trusted Base render slot rather than copying Base's +default template into SES. + +### 2. Authored modules execute in SES + +[`CapsuleModuleEvaluator`](../packages/host/app/lib/capsule-module-evaluator.ts) +calls `lockdown()` once and creates a principal-named `Compartment`. The Host +currently chooses compatibility-oriented taming for the trusted start +compartment: + +```ts +lockdown({ + evalTaming: 'unsafe-eval', + consoleTaming: 'unsafe', + errorTaming: 'unsafe', + localeTaming: 'unsafe', + overrideTaming: 'severe', +}); +``` + +These settings preserve the existing Host Loader, diagnostics, locale +formatting, Monaco, and generated prototype code. They do not endow those +globals into the Capsule. The evaluator's ambient probe asserts that the +ordinary Capsule receives no `window`, `document`, `localStorage`, `fetch`, or +`XMLHttpRequest`. It receives hardened/pure facilities such as `Intl`, bounded +`structuredClone`, and URL parsing. + +Dynamic import is rewritten by the Boxel Loader, so the evaluator supplies a +denial-shaped `import.meta.loader` rather than the real Loader: + +```ts +loader: harden({ + import(): never { + throw new Error( + 'CAPSULE_DYNAMIC_IMPORT_DENIED: dynamic import requires Sandbox execution', + ); + }, + fetch(): never { + throw new Error( + 'CAPSULE_DYNAMIC_FETCH_DENIED: resource fetch requires an explicit capability', + ); + }, +}), +``` + +Framework imports inside the compartment are explicit facades, not the Host +namespaces. Examples include a minimal Glimmer component base, tracking/action +decorators, `on`, template-only components, selected helpers, Card API data +semantics, and an explicit subset of runtime-common. Adding an export to the +real package does not automatically grant it to authored code. + +### 3. Template registration is captured, not installed + +The evaluator shims `@ember/component` and `@ember/template-factory`. Compiled +GTS still performs its normal registration calls, but the Capsule versions +capture the descriptor: + +```ts +private emberComponentFacade() { + return harden({ + setComponentTemplate: (factory, component) => { + let descriptor = factory()?.parsedLayout; + if (!descriptor) { + throw new Error('Card template factory returned no layout'); + } + this.templateByComponent.set( + component, + this.captureDescriptor(descriptor), + ); + return component; + }, + }); +} + +private templateFactoryFacade() { + return harden({ + createTemplateFactory: (descriptor) => + harden(() => ({ parsedLayout: descriptor })), + }); +} +``` + +`captureDescriptor()` retains `id`, serialized `block`, `moduleName`, strict +mode, scoped stylesheet requests, and a lazy `scope()` closure. Scope must stay +lazy because compiled templates may refer to bindings declared later in the +module; eagerly calling it during `setComponentTemplate()` creates temporal +dead-zone failures that ordinary Ember does not. + +The captured closure never leaves SES. `bundleFor()` invokes it only after +module evaluation completes and translates each returned value into a typed +boundary reference: + +```ts +type CapsuleScopeReference = + | { kind: 'authored-component'; component: string } + | { kind: 'trusted-export'; module: string; name: string } + | { kind: 'literal-value'; value: JSONValue }; +``` + +An authored component is recursively included in the template graph. An +approved Cardstack/Base export becomes a Host-resolved identity. A literal is +JSON-cloned. Any other function or symbol is rejected: + +```ts +if (typeof value === 'function' || typeof value === 'symbol') { + throw new Error( + 'template scope contains an executable value without a trusted module identity', + ); +} +``` + +This translation is the most important protection against a template scope +becoming an arbitrary-code tunnel into the Host. + +### 4. Boxel validates selected DOM effects in Glimmer wire format + +Before a non-trusted template is bundled, +`validateTemplateDOMPolicy()` in +[`capsule-module-evaluator.ts`](../packages/host/app/lib/capsule-module-evaluator.ts) +walks the JSON-decoded block. Today it recognizes numeric Glimmer opcodes for +static and dynamic attributes and refuses: + +- popover/command attributes that enter the browser top layer; +- dynamic inline style except the Host-owned `cssVar` helper; +- unvalidated literal inline style; and +- literal unscoped ` + +
+``` + +The RP permits a validated, versioned `TemplateBundle` with typed dependencies +and requires an unknown dependency feature to fail closed (RP-14.1 through +RP-14.3). + +**Harder Capsule option.** This is the largest and most important middle-zone +project: bind the bundle to compiler, VM, and DOM-policy versions; validate an +exhaustive opcode vocabulary; allowlist element namespaces and +attribute/property effects; reject unknown instructions; and add fixtures for +every admitted declarative DOM operation. An Ember/Glimmer-supported DOM +construction delegate would be preferable to permanently interpreting private +wire opcodes if one can enforce these rules without changing template +semantics. + +**Sandbox line.** Any compiler instruction or declarative browser feature that +has not been assigned shared-document semantics must route to Sandbox. Unknown +must never mean “let Host Glimmer try it.” + +### Boundary 4: executable template scope + +**Current enforcement.** Scope entries are converted to a typed dependency +union. Authored components are recursively captured, trusted components and +helpers are resolved by Host export identity, and literal data is cloned. +Unknown executable functions, symbols, helpers, modifiers, and components are +rejected instead of crossing as callable values. + +Allowed today: + +```gts + + +``` + +Here `CardsGrid` may be a trusted Base export installed by reference, while +`MyAuthoredRow` remains authored logic captured in the same Capsule bundle. + +Refused today: + +```gts + +
+``` + +**Harder Capsule option.** Replace package-level trust with an export-level +registry. Each trusted export should declare its accepted argument schema, +effects, whether it renders outside its invocation bounds, and which named +Host capabilities it may use. Forbid trusted portals from returning Elements, +owners, services, native events, or arbitrary callbacks to authored code. + +**Sandbox line.** An arbitrary third-party Glimmer component, modifier, helper, +or custom element that has not received an export-level capability review +belongs in the iframe. “Distributed in an npm package” is not a trust grant. +That iframe uses `credentialless` isolation where supported and an +`allow-scripts`-only opaque origin in Safari and Firefox; this browser +negotiation changes +transport mechanics, not the card's RP entitlement. + +### Boundary 5: component lifecycle, reactivity, and Ember ownership + +**Current enforcement.** A custom Host component manager owns the Glimmer +definition and maintains one SES component instance across compatible argument +updates. The Host proxy resolves property reads and action calls into the +Capsule. Capsule-authored instances have no Ember owner, so ordinary tracked +state and getters work while service injection and owner lookup do not. + +Allowed today: + +```ts +@tracked expanded = false; + +@action toggle() { + this.expanded = !this.expanded; +} +``` + +Unavailable today: + +```ts +@service router; +get arbitraryService() { + return getOwner(this)?.lookup('service:anything'); +} +``` + +**Harder Capsule option.** Define a supported scheduler/tag bridge, explicit +destruction semantics, re-entrancy rules, and per-generation error handling. +Where a common service use case is legitimate, expose its minimum operation as +a versioned RP capability rather than attaching an Ember owner to authored +instances. + +**Sandbox line.** Components that require arbitrary Ember DI, an application +owner, browser-service singletons, or unbounded synchronous work should not be +admitted to Capsule. Note that an iframe provides origin and DOM isolation but +does not guarantee CPU isolation; hard CPU budgets need a Worker or process +boundary. + +### Boundary 6: DOM construction and mutation + +**Current enforcement.** Host Glimmer creates ordinary DOM descendants inside +`.boxel-execution-capsule-slot`. The Capsule has no Element reference and no +generic DOM-request operation. Known direct-DOM imports, globals, methods, +modifiers, top-layer constructs, and unsafe styles classify to Sandbox or are +rejected by the verifier. + +Allowed today: + +```gts +
+ +
+``` + +Not admitted as Capsule authority: + +```ts +element.closest('.operator-mode').remove(); +new MutationObserver(...).observe(document.body, ...); +canvas.getContext('webgl2'); +``` + +There is no ShadowRoot, separate custom-element registry, or Glimmer +principal check. A literal custom element, an `{{in-element}}` destination, +or an insufficiently reviewed trusted modifier is therefore a policy gap, not +something Glimmer independently confines. + +**Harder Capsule option.** Add an explicit declarative DOM vocabulary: +allowlisted HTML/SVG elements, URL-bearing attributes, reflected properties, +form behavior, custom-element rejection, `in-element` rejection, and strict +namespace rules. If the framework exposes a supported DOM operations delegate, +perform the same checks at element/attribute creation as a defense in depth. + +**Sandbox line.** Direct Element access, custom elements, arbitrary modifiers, +portals outside the card slot, canvas/WebGL, observers, pointer lock, dialogs, +and browser-owned widgets remain iframe work. + +### Boundary 7: browser events and authored actions + +**Current enforcement.** Only the trusted Ember `on` modifier is reified. +Native events are reduced to the versioned `SafeEvent` record: event scalar +fields plus an allowlisted target projection containing values such as +`tagName`, `value`, `checked`, and `dataset`. Action arguments are cloned or +rejected. The live Event and Element never enter SES. + +Allowed today: + +```ts +@action choose(event: SafeEvent) { + this.selection = event.currentTarget?.dataset?.id; +} +``` + +Unavailable today: + +```ts +event.target.closest('[data-card]'); +event.composedPath(); +event.target.setPointerCapture(event.pointerId); +``` + +Calling `preventDefault()` is also not equivalent to receiving a native Event; +the current record reports `defaultPrevented` but carries no methods. + +**Harder Capsule option.** Add narrowly named event effects—such as +`prevent-default`, `stop-propagation`, or pointer capture—only when their timing +and target identity can be validated synchronously by the Host. Define typed +drag, keyboard, input, and composition projections separately rather than +continually widening one event-shaped bag. + +**Sandbox line.** Native Event identity, live targets, composed-path access, +unbounded `DataTransfer`, and arbitrary browser event methods stay in the +iframe unless a use case is captured by a small named effect. + +### Boundary 8: CSS and presentation + +**Current enforcement.** Capsule accepts compiler-produced Glimmer scoped CSS, +adds a Capsule ancestor, parses the result in the browser, and refuses escaped +selectors, network-bearing declarations, document-global rules, named global +layers, and top-layer/view-transition declarations. Literal inline styles are +parsed; dynamic inline styles are limited to the trusted `cssVar` helper. + +Allowed today: + +```gts + +``` + +Classified to Sandbox or refused in Capsule: + +```css +:global(body) { + overflow: hidden; +} +@font-face { + src: url('https://example.test/font.woff2'); +} +.card { + view-transition-name: whole-page; +} +``` + +**Harder Capsule option.** Version the CSS grammar and policy, namespace +keyframes and other registries, define an exhaustive at-rule/property policy, +and test normalized browser CSS rather than relying only on source spelling. +Projected resource references could eventually support specific image/font +cases without granting general CSS network egress. + +**Sandbox line.** Global CSS, page registries, top-layer effects, unrestricted +external resources, view-transition ownership, and styling that intentionally +escapes the card subtree belong in the iframe. + +### Boundary 9: resource URLs and browser-initiated egress + +**Current enforcement.** JavaScript network egress is absent from Capsule and +CSS network-bearing forms are refused. An authored `` under realm auth is +currently allowed to load natively in the shared Host document, as recorded in +the RP-21 capability matrix. This preserves existing cards, but it means +URL-bearing HTML attributes are a distinct authority channel from `fetch`. + +Allowed today: + +```gts +{{@model.title}} +``` + +A potentially unsafe pattern that needs stronger policy is authored data being +assembled into an arbitrary request URL: + +```gts +Open +``` + +Potentially admitted today, but not yet covered by a complete declarative +navigation/form policy: + +```gts +Leave Boxel +
...
+``` + +**Harder Capsule option.** Rewrite navigation and submission into typed intent +records. The Host validates the target, mounted surface, user gesture, +authorization, and destination before acting. Add focus, clipboard write, and +top-layer presentation only as separate named capabilities with revocable +surface grants. + +**Sandbox line.** Native navigation control, arbitrary form submission, +unmediated clipboard or focus ownership, fullscreen, dialogs, and top-layer UI +remain iframe behavior until a specific operation has a protocol contract. + +### Boundary 11: trusted portals and Boxel context + +**Current enforcement.** Trusted Base and `@cardstack/*` components can be +installed into a Capsule template by Host reference. They may use the real +Glimmer and Boxel implementation, while authored code sees only their declared +arguments and output. The Capsule context facade exposes a small projection, +not the full Ember service container. This is how trusted NumberField, +CardContainer, RichMarkdown, and other Boxel building blocks can compose with +authored templates without cloning their implementations. + +Allowed today: + +```gts +<@fields.quantity /> +<@fields.body @format='embedded' /> +``` + +The dangerous shape is a trusted portal that accidentally returns a live +Element, owner, service, Loader, native Event, or outside-slot destination to +authored code. Glimmer will not reject that authority after the portal grants +it. + +**Harder Capsule option.** Move from broad package provenance to an +export-level portal manifest, one-way argument schemas, explicit effects, and +adversarial tests for every portal capable of DOM work. A trusted export should +be reviewed as a capability adapter, not merely as code published by Boxel. + +**Sandbox line.** Unknown community components, external UI packages, and any +portal that cannot prove one-way data flow and subtree-bounded effects run in +the iframe. Trust does not flow from a trusted child back into its authored +parent. + +### Boundary 12: named Surface capabilities + +**Current enforcement.** RP-16 defines exactly five Host-owned surface +operations: `presentation`, `layout`, `observe`, `view-card`, and `patch`. +Grants are attached to one mounted surface, use cloneable records and named +effects, and fail closed after release. They do not hand an Element or service +to the Capsule. + +Allowed today: + +```text +Capsule → presentation({ headerColor }) → Host validates mounted surface +Capsule → view-card({ cardId }) → Host validates and navigates +``` + +Not allowed today: + +```text +Capsule → dom({ selector: 'body', method: 'append', value: ... }) +Capsule → service({ name: 'store', method: 'search', args: ... }) +``` + +**Harder Capsule option.** The deferred `surface*` family—pointer, focus, +style, transition, schedule, clipboard, haptics, slot, playback, viewport, and +canvas—can keep some future cards in Capsule when each operation has a small +typed request, a bounded response, surface-local authority, revocation, and +cross-tier conformance tests. Add an operation only after multiple real use +cases demonstrate the same semantic need. + +**Sandbox line.** A generic selector/method channel, generic service invocation, +or a surface API whose useful payload is a live browser object is simply +`document` under another name and must not ship. Cards requiring that shape use +the iframe. + +### Boundary 13: recursively nested composition + +**Current enforcement.** Every authored node is routed independently. A +Capsule component may render a trusted Base field, which may render another +authored Capsule or an origin-isolated Sandbox child. The Host retains the +canonical graph and passes opaque handles and bounded projections at every +edge. A trusted descendant does not upgrade its authored ancestor, and an +iframe descendant does not expose its document to its parent. + +Allowed today: + +```text +Capsule card + → trusted Base field portal + → Capsule linked card + → Sandbox media card +``` + +Forbidden authority flow: + +```text +Sandbox child Element → trusted Base field → Capsule parent +Capsule parent closure → Sandbox child as an unrestricted callback +``` + +**Harder Capsule option.** Generate graph-shaped conformance tests—not only +pairwise tests—for tier alternation, formats, field delegation, lifecycle, +mutation, errors, focus, scroll, and teardown. Include principal and surface +identity in every handle so a capability from one node cannot be replayed by a +sibling. + +**Sandbox line.** Routing remains per authored module and format. A parent +cannot force a browser-dependent child into Capsule for layout convenience, +and a child cannot cause the whole graph to inherit broader data entitlement. + +## How large is the useful middle zone? + +There is a real and worthwhile area between today's Capsule and “put it all in +an iframe,” but it is not an invitation to proxy the browser. It divides into +three groups. + +### Group A: harden what Capsule already claims — do this + +These changes reduce risk without materially increasing Capsule authority: + +1. Version and exhaustively validate the Glimmer template/DOM vocabulary. +2. Validate or rewrite all URL-bearing attributes using opaque projected + resources. +3. Replace package-wide trusted portals with export-level manifests and typed + one-way argument/effect contracts. +4. Version the SES endowment, trusted-module, SafeEvent, and CSS policies. +5. Add graph-shaped and browser-version conformance tests that fail closed on + an unknown compiler instruction or platform feature. + +This is a medium-sized platform project, not an unbounded compatibility layer. +The difficult part is obtaining a stable Glimmer interception point; the +policy vocabulary itself is finite and Boxel's Card/Field APIs are already +tight. These improvements are worth doing before classifying substantially +more unknown code as Capsule. + +### Group B: add narrow semantic capabilities when real cards converge + +Several browser-shaped needs can be expressed without browser authority: + +- focus a known surface-local target; +- observe a frozen viewport or size record; +- request pointer capture for the element associated with the current event; +- write explicitly supplied text to the clipboard after a gesture; +- coordinate playback with timestamps and commands; +- request a Host-owned transition, navigation, or presentation change; and +- resolve an authorized image, media, or download resource. + +Each can be a revocable, typed `surface*` operation. They are good Capsule +features only when the Host can validate the request without accepting a CSS +selector, Element, callback, service name, arbitrary URL, or arbitrary method. +The protocol should gain these one at a time, backed by multiple use cases and +Direct/Capsule/Sandbox conformance—not as one generic bridge. + +### Group C: browser/application emulation — do not add this to Capsule + +The following should remain strong Sandbox signals: + +- `window`, `document`, Element, ShadowRoot, native Event, and custom elements; +- arbitrary Ember owner/service injection or application container access; +- third-party modifiers and browser UI packages without a trusted adapter; +- Canvas/WebGL/WebGPU, map/3D engines, observers, pointer lock, fullscreen, and + top-layer ownership; +- arbitrary fetch, storage, sockets, workers, dynamic imports, or external + resource egress; +- global CSS, page registries, and cross-slot portals; and +- generic DOM, service, module, or callback RPC. + +Trying to support these in Capsule would make the shared Host document the +security perimeter while rebuilding a partial browser membrane in application +code. That would be more complex and easier to get wrong than the +origin-isolated iframe. If hard CPU isolation is required as well, an iframe is +not sufficient by itself; that use case needs a Worker or separate process. + +### Recommendation + +Keep Capsule's identity as **data + declarative Glimmer + audited trusted +portals + named surface effects**. Complete Group A, add Group B only where the +real corpus supplies repeated semantics, and route Group C to Sandbox without +apology. This should admit a broad class of ordinary Boxel cards—forms, lists, +computed displays, nested fields, markdown, themes, navigation, and bounded +media—while retaining a simple explanation of why 3D engines, browser-native +widgets, and application-like cards get their own document. + +## Framework API stability review + +This table deliberately distinguishes an exported API from a stable boundary +artifact. An import can be public while the data shape supplied to it is still +compiler-private. + +| Usage | Current assessment | Why it matters | +| ----------------------------------------------------------------------- | ----------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- | +| `capabilities('3.13')`, `setComponentManager()` | Public component-manager API | Appropriate basis for a Host-side adapter, subject to normal manager-version migration. | +| `setComponentTemplate()` | Public association API | The usage is conventional once a valid template factory exists. | +| Glimmer component base and `@tracked` | Public | Ordinary framework usage. | +| `createTemplateFactory({ id, block, moduleName, scope, isStrictMode })` | Exported, but Boxel relies on compiler-output details | We need confirmation that external reconstruction from a captured descriptor is supported. | +| Serialized `block` JSON | Glimmer compiler/VM wire format | Boxel treats it as a boundary program and inspects nested arrays/opcode numbers. No explicit compiler/VM version is carried today. | +| Lazy positional `scope()` | Compiler-generated convention | Correctly preserves TDZ ordering, but scope position and expression encoding are VM/compiler coupled. | +| Numeric opcode checks in `validateTemplateDOMPolicy()` | Private/internal coupling | An opcode addition or semantic change can bypass a check or cause a false refusal unless upgrades fail closed. | +| Reading `args.named` in the custom manager | Low-level manager contract | Works in current Ember; maintainer confirmation on the supported typed access pattern would reduce risk. | +| Microtask-incremented tracked revision | Boxel scheduling convention | Avoids current backtracking assertions but needs guidance for re-entrancy and future autotracking semantics. | +| Trusted component identity resolved to a Host export | Boxel policy layered on public Glimmer composition | Safe only when the trusted-export registry is capability-aware. | + +## What the current implementation does well + +### It keeps object authority narrower than the visual composition graph + +The Host can render a deeply interleaved tree—authored Capsule component, +trusted Base FieldDef, another authored card, Sandbox child—without passing +live instances or DOM references between those owners. Opaque handles and +tracked paths preserve identity while the execution router is re-entered at +each authored node. + +### It delegates rendering by reference, not by serializing component code + +Trusted framework components stay native. Authored component constructors stay +in SES. The only code-like artifact crossing from authored code is the compiled +template block, whose scope is reconstructed from typed references. This is +smaller and more maintainable than attempting to proxy all of Ember or clone +arbitrary Glimmer components. + +### It preserves the interaction properties users notice + +The custom manager retains the SES instance and Host context. Data updates +change argument paths instead of remounting the component, which preserves +focus, selection, scroll position, local component state, and DOM identity in +the normal case. + +### Its failures are explicit + +Unknown executable scope values, unscoped styles, unsupported inline styles, +and unavailable browser APIs are refused rather than silently dropped. The +classifier sends known browser-dependent modules to Sandbox. There is no +generic `getElement()` or `runDOMCommand()` escape hatch. + +### It has direct adversarial and composition coverage + +Useful starting points for review are: + +- [`capsule-boundary-probe-test.ts`](../packages/host/tests/unit/lib/capsule-boundary-probe-test.ts): ambient authority, object leakage, event projection, and the documented CPU-termination gap; +- [`capsule-module-registration-test.ts`](../packages/host/tests/unit/lib/capsule-module-registration-test.ts): template capture, trusted references, state/actions, getters, nested FieldDefs, and rejected scope values; +- [`capsule-css-policy-test.ts`](../packages/host/tests/unit/lib/capsule-css-policy-test.ts): selector, network, global CSS, and confinement cases; +- [`rp-realm-mirror-compatibility-test.gts`](../packages/host/tests/integration/components/rp-realm-mirror-compatibility-test.gts): realistic nested cards, relationships, Rich Markdown, formats, and Capsule→Sandbox composition; and +- [`rp-continuity-test.gts`](../packages/host/tests/integration/components/rp-continuity-test.gts): stable slot and DOM identity across updates. + +## Shared-document channels and residual concerns + +The following are concerns to evaluate, not claims that every row is currently +exploitable. + +| Channel | Current control | Residual concern | +| ------------------------------------------------------ | -------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Template instructions | Selected opcode/attribute validation; strict typed scope references | The validator does not define an exhaustive, versioned allowlist of every Glimmer instruction that can create a browser side effect. Compiler/VM drift is the highest-risk maintenance issue. | +| Elements and attributes | Classifier signals plus top-layer/style checks at bundle time | Resource-loading attributes, URL protocols, forms/navigation, focus/autofocus, SVG/MathML, `srcdoc`, and future declarative browser features need a systematic policy rather than an accumulating denylist. | +| Custom elements | Browser globals classify obvious uses as Sandbox | A literal custom-element tag can invoke a constructor already registered in the Host document. The reifier currently has no explicit authored custom-element allowlist. | +| Trusted exports | Only recognized trusted module identities cross; authored functions refuse | Package-level trust is broad. A trusted modifier/component that passes a live element, owner, service, or capability-bearing object to an authored action would amplify authority. Trust should ultimately be export-level and capability-described. | +| Events | Native events become bounded `SafeEvent` records | New event types and values require explicit projection. Trusted components must not bypass this path by invoking authored callbacks with arbitrary Host objects. | +| CSS | Scoped anchor validation, Capsule ancestor, network/global CSS rejection, ref-counted styles | CSS shares parsing, cascade, fonts, custom properties, layout, animation names, and performance with the Host. Browser parser differences, expensive selectors, and compiler keyframe namespacing need continuous testing. | +| Focus, selection, accessibility, and event propagation | Content is rooted under one slot | The slot is not Shadow DOM. Authored markup participates in the document focus order, accessibility tree, bubbling, IDs/names, form association, drag/drop, and selection. These need explicit accepted semantics and tests. | +| Layout and timing | No direct DOM read in SES | Shared main thread and rendering can expose coarse timing/layout side channels through interaction behavior and can degrade Host responsiveness. Capsule is not a confidentiality boundary against all same-process side channels. | +| CPU and memory | SES removes ambient authority; handles are released on destroy | SES supplies no preemption. A non-terminating getter/action or pathological template/selector can freeze the Host. The adversarial test records this as a known gap. | +| Reactivity | Live cloned reads and a tracked revision proxy | A synchronous SES getter runs while Host Glimmer is rendering. Re-entrancy, tag consumption, exceptions, and action-driven updates rely on custom scheduling rather than a framework cross-realm primitive. | +| Global SES lockdown | One guarded call, narrow compartment endowments | `lockdown()` mutates shared intrinsics and uses compatibility-oriented unsafe tamings for the trusted start realm. Ember/Monaco/library upgrades need start-realm compatibility tests. | +| Cached definitions and styles | Render-slot caches and ref-counted stylesheet lifetime | Long-running multi-realm sessions need bounds and lifecycle tests so component definitions, captured bundles, modules, or failed promises do not become app-lifetime leaks. | + +### A useful threat-model distinction + +Capsule is intended to stop authored presentation code from obtaining Host +object authority. It is not intended to provide: + +- process isolation; +- CPU or memory quotas; +- a separate browser security origin; +- complete same-process side-channel resistance; or +- support for arbitrary DOM libraries. + +Cards requiring those properties belong in Sandbox. Expanding Capsule should +mean broadening a reviewed declarative rendering vocabulary, not weakening +that distinction. + +## Improvements required before materially broader Capsule admission + +### Priority 0 — make the Glimmer artifact a real, versioned boundary + +1. **Add a compiler/VM compatibility identity to every template bundle.** It + should cover Ember/Glimmer version, template compiler version, Boxel DOM + policy version, scoped-CSS compiler version, and required feature bits. The + Host must reject an unknown combination before calling + `createTemplateFactory()`. +2. **Obtain or define a supported portable-template API.** The ideal upstream + primitive would carry a strict-mode block plus symbolic scope references + and expose a supported validation/instantiation contract. If that is not an + Ember goal, Boxel should isolate the current wire-format adapter in one + versioned module instead of spreading opcode knowledge through classifier + and evaluator code. +3. **Fail closed on unknown instructions.** Replace the current selected + denylist with a complete instruction visitor for the pinned wire format. + Every opcode must be categorized as pure render, data read, trusted + invocation, DOM effect, or forbidden/unknown. +4. **Prefer source/AST capability analysis before compilation, then verify the + compiled result.** Source analysis provides readable author diagnostics; + wire validation catches compiler transformations and classifier mistakes. + Neither should be the sole gate. + +### Priority 0 — formalize the Host DOM capability vocabulary + +5. Define allowlists for HTML/SVG/MathML elements, namespaces, attributes, + property writes, URL-bearing attributes, and protocol schemes. Route + unsupported constructs to Sandbox. +6. Refuse or explicitly allow custom elements. A Host-registered custom + element is executable Host code even if the template contains no modifier. +7. Treat resource loads as capabilities. Prefer Host-projected, exact resource + references or mediated URLs rather than letting arbitrary authored + `src`/`href` values initiate browser requests from the Host origin. +8. Specify focus, form, navigation, selection, drag/drop, accessibility, and + event propagation semantics at the Capsule slot. Add conformance tests for + accepted behavior and visible refusal for the rest. + +### Priority 0 — narrow trusted portals + +9. Replace broad package trust at the rendering bridge with an export-level + registry. Each admitted helper/component/modifier should declare the data it + accepts and the named effects it may invoke. +10. Enforce one-way invocation mechanically: no authored callbacks as generic + trusted-component arguments, no live owner/service/element results, and all + trusted-to-authored action delivery through the SafeEvent/data projector. +11. Test recursive graphs, not only pairs: Capsule → trusted Base → Capsule → + Sandbox → Host effect, with independently routed siblings and repeated + formats. + +### Priority 1 — align reactivity and lifecycle with supported Glimmer idioms + +12. Ask Glimmer maintainers for the supported external-runtime reactivity + primitive. Replace the ad hoc microtask invalidation if a stable tag or + cache API exists. +13. Define re-entrancy semantics for SES getters invoked during render and + actions that settle during a render turn. Errors must stay inside the + component boundary and last-known-good output should remain visible. +14. Add bounds for template count, block bytes, scope entries, component + instances, DOM nodes, action duration, getter duration, and selector + complexity. Strong CPU availability cannot be achieved in same-process + SES; workloads requiring it should route to Sandbox or a worker-backed + design. +15. Key reconstructed-template caches by source hash plus every compiler and + policy version, and test teardown/eviction over long cross-realm sessions. + +### Priority 1 — make browser and upgrade drift observable + +16. Differentially render a curated corpus through Direct and Capsule and + compare semantics, interaction, DOM shape, accessibility, styles, and + lifecycle behavior. +17. Fuzz template blocks and scope graphs at the evaluator/reifier boundary. + Unknown or malformed artifacts must refuse without invoking Host Glimmer. +18. Run the containment suite on Chromium, Safari/WebKit, and Firefox. CSSOM + normalization and declarative DOM behavior are browser-dependent. +19. Add an Ember/Glimmer upgrade gate that inventories changed opcodes, + template descriptor fields, manager capabilities, and compiler output + before dependency updates can merge. + +## Questions for Ember and Glimmer maintainers + +These answers determine whether the current bridge is a reasonable supported +extension point or a prototype that needs an upstream primitive. + +1. Is constructing a template with + `createTemplateFactory({ id, block, moduleName, scope, isStrictMode })` + outside generated compiler output supported? +2. Is the serialized `block` shape versioned or documented anywhere, and can a + consumer reliably determine the compiler/VM compatibility range? +3. Is there an official visitor or decoder for Glimmer wire format that lets a + host enumerate every DOM-affecting instruction without copying opcode + numbers? +4. Is rebuilding the strict-mode scope array with Host component/helper + references a supported pattern? What invariants beyond positional order + must Boxel preserve? +5. Is reading `args.named` in a custom component manager supported, or should + Boxel use another public argument-access contract? +6. Is a stable Host proxy context whose getters synchronously call another JS + realm compatible with Glimmer's render and autotracking assumptions? +7. What is the supported way to invalidate such a context after an external + runtime changes state without triggering backtracking? Is a microtask-updated + tracked revision acceptable? +8. Are nested dynamically created definitions—some reconstructed Capsule + components and some ordinary trusted components—within the expected + lifecycle/destructor semantics of the component-manager API? +9. Can Ember expose a portable strict-mode template artifact whose executable + scope is symbolic rather than a live closure? +10. Would an upstream DOM-effect/capability description at the compiler level + be useful, or should Boxel perform this analysis on Glimmer AST before + compilation and treat postcompile inspection as a pinned private adapter? +11. Which parts of the current approach would prevent future SSR or hydration + support for a Host-reconstructed Capsule component? + +## Suggested maintainer review order + +1. Start with + [`capsule-module-evaluator.ts`](../packages/host/app/lib/capsule-module-evaluator.ts): + `ensureCapsuleLockdown`, `emberComponentFacade`, `templateFactoryFacade`, + `captureDescriptor`, `bundleFor`, `validateTemplateDOMPolicy`, + `scopeReference`, `liveComponentArgs`, and action projection. +2. Read + [`capsule-component.ts`](../packages/host/app/lib/capsule-component.ts): the + custom manager, trusted scope resolution, `createTemplateFactory`, and + stylesheet registry. +3. Read + [`capsule-component-runtime.ts`](../packages/host/app/lib/capsule-component-runtime.ts): + stable context, live argument path, getter/action calls, revision scheduling, + and teardown. +4. Read + [`capsule-css-policy.ts`](../packages/host/app/lib/capsule-css-policy.ts) + and + [`boxel-source-classifier.ts`](../packages/host/app/lib/boxel-source-classifier.ts) + together. They are defense-in-depth layers that must not drift. +5. Finish at + [`boxel-execution-renderer.gts`](../packages/host/app/components/boxel-execution-renderer.gts) + and + [`boxel-field-portal.gts`](../packages/host/app/components/boxel-field-portal.gts) + to see where the Host DOM slot is mounted and how recursive composition + re-enters policy. + +## Bottom line + +Capsule is a compelling middle tier when its contract is stated precisely: +authored JavaScript remains in SES, while a bounded declarative template +program is executed by trusted Host Glimmer inside a designated subtree. It +preserves native Ember composition and substantially better interaction and +startup characteristics than one iframe per compact card. + +It should not be described as equivalent to origin isolation. The Host is +accepting an untrusted rendering program into a shared VM/document, and the +current program validator knows selected Glimmer internals rather than an +exhaustive, upstream-supported capability model. The right path to wider +Capsule use is to make that program format and its DOM authority explicit, +versioned, allowlisted, and tested—not to grow a collection of exceptions as +new cards are encountered. diff --git a/docs/boxel-execution-graph-testing.md b/docs/boxel-execution-graph-testing.md new file mode 100644 index 00000000000..39fbecf80d5 --- /dev/null +++ b/docs/boxel-execution-graph-testing.md @@ -0,0 +1,220 @@ +# Boxel execution graph testing + +This is the cheap correctness gate between implementation work and the real +workspace smoke corpus. It treats Boxel rendering as an alternating-owner +graph, not as three unrelated renderers. + +## One fast command + +From `packages/host`, with the ordinary Host build and test services available: + +```sh +pnpm exec ember test --path dist --filter "Boxel execution graph" \ + 2>&1 | tee /tmp/host-test-execution-graph.log +``` + +The filter runs both the small deterministic graph checks and the authoritative +realm-mirror integration module. It checks the routing truth table, all +required boundary-edge types, the structural authority/lifecycle axioms, and +exact Capsule composition for nested FieldDefs, linked cards, independent +formats, and Rich Markdown card embeds. +Every minimum-gate path now names its executable evidence. The historical +labels remain available for new scenarios while they are being developed: + +- `exact`: an integration or acceptance test already exercises the behavior; +- `protocol-only`: the contracts and route are tested, but not the whole UI; +- `browser-gated`: a real child document, interaction, or prerender handoff is + still required before we can claim end-to-end proof. + +That label is part of the fixture. A protocol test cannot silently turn a +missing browser proof green. + +## The axioms + +1. Every independently loaded nested Boxel re-enters Host routing policy. +2. A runtime-local FieldDef/component stays in its parent Capsule or Sandbox. +3. A trusted Base portal is Direct but receives projected data and bounded + callbacks; it cannot transfer its owner or Store to authored code. +4. Every Surface or mutation capability terminates in the Host, where grants + and writes are revalidated. +5. A relationship is a graph edge, not authority. The child keeps the viewer + principal unless the Host issues a separate explicit grant. +6. Sandbox identity is the stable mounted surface, not the card URL or Realm. +7. Compact formats never allocate inline iframes. Browser-heavy renderers use + Sandbox for isolated/embedded/authored-edit and must provide a safe compact + module or fail closed in Capsule. +8. Prerendered HTML is inert and can only hand off through Host policy to an + interactive runtime. +9. Teardown is local: releasing a child cannot invalidate a surviving parent, + sibling, shared Capsule, or trusted Direct runtime. +10. Unknown protocol features retain last-known-good output instead of + partially rendering an unrecognized record. + +## The thirteen-path gauntlet + +The executable declarations live in +`tests/helpers/boxel-execution-graph.ts`. They cover ordinary Capsule, trusted +Base FieldDef portals, recursive fields, linked cards, Rich Markdown embeds, +Capsule-to-Sandbox delegation, Sandbox-local composition, writes and +reconciliation, prerender handoff, warm formats, split compact/browser modules, +Surface capabilities through both boundary tiers, and the full alternating +Capsule → Direct → Capsule → Host → Sandbox → Host → Capsule reconciliation +path. + +The minimum gate admits no `protocol-only` or `browser-gated` rows. Each row's +`evidence` list points at its browser-QUnit or signed-in product smoke proof; +the graph suite fails if a row loses that evidence or is demoted. New rows may +start at a weaker label, but cannot enter the minimum gate until their exact +proof lands in the same change. + +## Performance baselines + +Performance has two layers and should not be reduced to one flaky CI timeout: + +1. The deterministic suite records the median warm routing/retention cost for + Direct, Capsule, and Sandbox over five 10,000-route samples. Expand the + passing performance assertion in QUnit or search the captured log for + `BOXEL_EXECUTION_ROUTING_BASELINE` to see the values. This deliberately + excludes runtime construction, module evaluation, DOM, and iframe startup. +2. The persistent in-app-browser smoke runner records cold total time, warm + total time, and Sandbox prerender-to-interactive handoff for the six-card + cohort in `boxel-realm-mirror-compatibility-strategy.md`. + +Pass `performanceRepeats: 1` (or more) to the browser runner when collecting a +baseline. Its `performanceBaseline` result separates three non-equivalent but +user-meaningful timings: + +- Direct: the trusted Base click-to-edit-ready transition; +- Capsule: semantic page readiness on cold and warm navigation; and +- Sandbox: semantic page readiness plus the distinct iframe-interactive + handoff, on cold and warm navigation. + +The labels are intentional. A Direct in-document transition must not be +presented as though it were a cold document navigation, and prerender text must +not be presented as Sandbox interactivity. + +When localhost authentication is scoped to an existing in-app-browser tab, +pass that handle as `candidateTab`. The runner reuses it instead of opening an +unauthenticated scratch tab; this keeps authentication setup outside the +timing window. + +**Staging-auth invariant:** no **Continue with Google** option means the Host is +not staging-backed. Stop; do not enter staging credentials or record results. + +The local Host must still be built against the same services as the reference. +For the staging differential, use `packages/host/config/staging.env` (staging +Matrix plus staging realm, Base, Catalog, Skills, and OpenRouter URLs). The +sign-in screen is the human-visible preflight: it must offer the same staging +providers, including **Continue with Google**. If that option is absent, the +Host is pointed at local Matrix and the browser gate must stop before accepting +credentials or measuring a card. An HTTP 200 from localhost is not proof that +the candidate is staging-backed. + +Run focused QUnit builds **before** starting the staging-backed browser Host. +`vite build --mode development` regenerates Embroider's environment entry with +local Matrix/Base URLs; a dev build performed while a staging Vite server is +open can therefore reload that server into local mode. The reliable order is: + +1. finish the focused build and QUnit runs; +2. start (or restart) the Host with `scripts/start-host.sh staging`; +3. inspect the environment meta or, more visibly, require **Continue with + Google** before signing in; +4. run the in-app-browser smoke without another Host build in parallel. + +Use environment mode for collision-free tests that need a complete local +service stack, isolated Postgres database, and isolated realm root. It is not +a substitute for the staging differential: this gate intentionally uses the +staging launcher and the staging identity provider. + +For review, compare medians and retained-runtime counts to the previous local +record. Correctness still blocks immediately; a performance regression is +reported separately so it cannot be “fixed” by weakening a semantic assertion. + +### Initial record — 2026-08-09 + +Chrome 151, local development build, focused graph suite: + +| Warm routing decision | Median ms/op | +| --------------------- | -----------: | +| Direct | 0.00002 | +| Capsule | 0.00048 | +| Sandbox | 0.00055 | + +These figures show that policy routing and retained-runtime lookup are not a +material source of UI latency. They do not measure evaluation or rendering. + +The same browser runner produced this reference-only readiness record against +the current staging Host (one warm repeat): + +| Cohort/transition | Cold median | Warm median | +| -------------------------------- | ----------: | ----------: | +| Trusted default-edit transition | 3211 ms | — | +| Capsule-designated cards | 2418 ms | 2249 ms | +| Sandbox-designated cards on main | 2486 ms | 2217 ms | + +The final row is a semantic control cohort: main does not run those cards in +the branch's iframe runtime, so it has no iframe handoff measurement. The +branch-side browser record is intentionally still blank because the local +preview session returned to the sign-in screen on hard navigation. It must be +collected from a signed-in, reload-persistent localhost session; an +authentication screen is not a renderer performance result. + +The broad reference lane is also established: main rendered +`FormatPreviewBatchOne` (35 delegated format boundaries) in 4131 ms with 89 +headings, 41 controls, and 15 loaded image elements. The staging-backed +candidate initially remained human-auth gated; the runner correctly stopped at +the Google-enabled sign-in screen instead of recording that screen as a +renderer failure or timing sample. + +### Signed-in staging differential — 2026-08-09 + +After the human completed Google sign-in, the same persistent tabs passed the +full six-card differential. This run includes three trusted Direct edit +transitions, four Capsule cards, two real Sandbox children, media playback, +default nested editing, Rich Markdown delegation, computed fields, image +delivery, and Sandbox teardown. + +| Cohort/transition | Staging/main | Branch candidate | +| -------------------------------------------- | -----------: | ---------------: | +| Trusted default-edit transition | 3111 ms | 3081 ms | +| Capsule cold median | 2524 ms | 2711 ms | +| Capsule warm median | 2474 ms | 2935 ms | +| Sandbox-designated cold/main vs real Sandbox | 2160 ms | 4485 ms | +| Sandbox-designated warm/main vs real Sandbox | 2176 ms | 4349 ms | +| Real Sandbox cold interactive handoff | n/a | 1476 ms | +| Real Sandbox warm interactive handoff | n/a | 1208 ms | + +All semantic, interaction, execution-tier, lifecycle, and teardown assertions +passed. The Sandbox timing is expected to be slower than main's in-document +semantic control, but the current roughly two-second additional cost remains a +concrete optimization target. + +The 35-boundary broad card also passed in both tabs: 2567 ms on main and 5090 +ms on the candidate. The candidate exposes additional trusted default Head +preview structure, so raw heading totals are diagnostic rather than an exact +visual-parity assertion. The required authored content, controls, images, and +delegated formats were present. + +### Same-document lifecycle soak — 2026-08-09 + +`runExecutionRuntimeNavigationSoak()` opens and closes six representative +cards through the compatibility workspace's real buttons. It intentionally +does not use `page.goto()` between samples, because a hard document navigation +would hide app-lifetime retention. The cohort covers nested Capsule fields, +Rich Markdown, computed values, and two Sandbox children. + +After three complete cycles on both main and the candidate: + +- all 18 opens settled and all 18 closes returned to the workspace; +- the candidate used the expected Capsule/Sandbox tier on every open; +- no Sandbox iframe or loading indicator survived any close; +- candidate style count was unchanged across the last two cycles; +- candidate DOM and style counts were identical across the last two cycles; +- main style count was unchanged across the last two cycles and its residual + DOM varied by one node. + +The cold pass legitimately materializes templates and styles. Report it +separately from the last-cycle delta; treating cold cache population as a leak +would create a misleading failure. These DOM/style/iframe counts are a cheap +lifecycle gate. Exact retained-heap and Core Web Vitals analysis still requires +the Chrome DevTools MCP server. diff --git a/docs/boxel-execution-performance-plan.md b/docs/boxel-execution-performance-plan.md new file mode 100644 index 00000000000..a05e67eb6ac --- /dev/null +++ b/docs/boxel-execution-performance-plan.md @@ -0,0 +1,497 @@ +# Boxel Execution Runtime — Performance Audit and Prioritized Plan + +**Status:** Phase 0 instrumentation implemented; focused live baseline captured, +full corpus pending. Revised +2026-08-11 on +`codex/boxel-execution-runtime-architecture` after checkpoint `652dabe7a8`. +Line references were measured against the audit working tree and will drift. +No optimization in this document is authorized until Phase 0 records a green +correctness-qualified baseline. + +**Scope:** the three execution tiers (Direct, Capsule/SES, Sandbox/iframe), the +orchestration layer that feeds them, and the store/projection/search layer beneath. +Direct, Capsule, and Sandbox stay consistent and share the `BoxelRuntime` API +throughout — nothing here changes tier semantics, isolation, or the protocol's +observable behavior. Per R5, no item de-escalates isolation: caches hold bytes and +derived pure artifacts; authority checks stay per-process. + +## The lens + +The first version of this runtime is a **reference implementation**: the code should +read like protocol documentation. Some performance fixes hide intent behind machinery; +others make the code state its intent _better_ than it does today. Every item below is +graded on three axes: + +- **Legibility** — **A**: the fix improves intent legibility. **B**: about the same. + **C**: the fix trades clarity for speed. +- **LOC** — net signed estimate of lines changed (negative is a reduction). +- **Perf** — quantitative estimate plus what the cost scales with. All numbers are + pre-measurement estimates; measure before and after. + +The scoring produced a finding worth stating up front: **the biggest wins are mostly +A-class, because the slowest paths are slow precisely where the code contradicts its +own stated intent.** The Capsule computes a full projection and then discards it +against the adopted Host projection; the Sandbox child keeps measuring renders the +parent has already said it ignores; `reloadSandbox()` documents a clean authority +slate but does not replace `resourceAuthority`; the GC sweep dedupes with a `WeakSet` +in one phase and forgets to in the other. Fixing those _can_ be spec work, but only +after the relevant ownership and identity rule is made explicit. The genuinely +C-class speedups (delta sync, warm pools, speculative boot, handshake-loading) +cluster at the bottom on all three axes for a v1 and are deferred deliberately. + +## Correctness is the first performance gate + +A fast result is not a sample when it rendered stale data, retained a prerender +placeholder, selected the wrong execution tier, dropped an interaction, crossed an +authority boundary, or leaked a child process. Every baseline and every before/after +comparison must pass these gates before its timing is admitted: + +1. **Semantic parity:** required text, computed values, relationships, cardInfo, + presentation, and delegated formats match the staging reference. +2. **Visual and interaction parity:** required DOM primitives exist; images decode; + scrolling, text entry, media controls, drag/drop, and navigation work where the + case declares them. +3. **Execution truth:** Direct, Capsule, and Sandbox labels match policy; a prerender + placeholder is not counted as an interactive Sandbox. +4. **Authority truth:** module and resource grants are exact, principal-scoped, and + generation-scoped. Cache hits never bypass a fresh authorization decision. +5. **Identity truth:** two simultaneous occurrences of the same card remain distinct; + compatible rerenders retain identity; format switches and back-navigation do not + accidentally share one DOM/process occurrence. +6. **Lifecycle truth:** errors retain last-known-good output where promised; explicit + reload replaces the intended generation; close/teardown leaves no iframe, pending + RPC, observer, timer, style, or retained authority beyond its documented TTL. + +The existing browser smoke runner already enforces most of gates 1–3 and part of 6. +The graph tests and focused transport tests enforce 4–5. A performance change must +strengthen missing assertions before relying on the path they cover; it must never +weaken an assertion to improve a number. + +## The cost model + +Three multipliers dominate, and they compound: + +1. **Per-property-read (Capsule).** Every authored `this.args.x` read crosses the + membrane via `cloneIntoCompartment(jsonClone(value))`, and `cloneIntoCompartment` + is `compartment.evaluate('JSON.parse("…escaped…")')` — a full SES compile of a + fresh source string per read + (`packages/host/app/lib/capsule-module-evaluator.ts:2564`). The host side of the + same read calls the uncached `getFields` and deep-expands linked subtrees per read + (`packages/host/app/lib/boxel-projection.ts:170`). +2. **Per-card-render (all tiers).** `requestFor` performs an uncached network fetch of + the module source, a full `serializeCard({withIncluded, includeUnrenderedFields})`, + and a whole-graph `settleHostProjection` fixpoint per renderer instance + (`packages/host/app/services/boxel-execution.ts:319`) — and there is one renderer + per card _and per field portal_. A 30-tile grid of one card type fetches the same + source 30+ times and serializes 30 full documents. The Direct tier — the majority + of renders — pays all of this and then uses `canonicalCard` natively. +3. **Per-boot (Sandbox).** The iframe does not start loading until two serial network + phases finish (request build, then a strictly sequential depth-first classification + walk, one fetch per module, up to 256). The child then boots the full host Ember + app, then ~6 sequential round-trips + (`listening → connect → ready → createFromSerialized → buildRenderRecord → render → diagnostic`), + every RPC costing 3 messages instead of 2, the child re-fetching the whole module + graph one round-trip at a time through the parent — each response blocked on a + parent-side main-thread `TextDecoder` + `es-module-lexer` parse the child then + repeats for itself. + +--- + +## Phase 0 — correctness-qualified baseline + +This phase changes instrumentation and documentation only. It does not add a cache, +pool, shortcut, parallel graph walk, or altered execution semantic. + +### Fixed environments + +Measure two environments separately: + +- the uniquely named, staging-backed development Host at + `https://host.codex-execution-runtime.localhost`, started through + `scripts/start-host.sh staging`; and +- a production preview build once one exists. + +Use `https://realms-staging.stack.cards` as the behavioral reference. The reference +does not run the branch's Sandbox and therefore cannot supply an iframe-handoff +number; it supplies semantic/visual/interaction parity and an unsandboxed latency +control. Never label an authentication screen as a renderer sample. The local Host +must show the staging identity providers (including **Continue with Google**) before +staging credentials or timings are accepted. + +Do not build Host/QUnit assets while collecting a staging-backed browser run: a build +can regenerate environment modules and silently switch a running Vite Host back to +local services. Finish focused builds first, restart the staging launcher, verify the +identity provider, then measure. + +### Fixed corpus + +Run one tab sequentially; never start multiple media cards concurrently. + +| Cohort | What it proves | +| ------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Six-card commit gate in `execution-runtime-browser-smoke.mjs` | Direct edit transition, representative Capsule composition, two real Sandboxes, interactions, teardown | +| `FormatPreviewBatchOne/sample` | 35 delegated format boundaries and repeated trusted portals | +| `RecipeGallery/home` | query readiness, relationship results, navigation, and scrolling | +| `TierList/national-fast-food-ranking` | image/resource projection, retained Sandbox lifecycle, and mutation UI | +| `Release/opening-night` | deeply nested computed values, trusted Base portals, edit scrolling, and theme | +| FileDef live subset: PDF, GLB, MIDI, DOCX, XLSX | scalar and relationship resource authority, canvas/media/Office/data adapters; XLSX remains a known authored `DataFilePreview` correctness blocker until fixed | +| Same-document navigation soak | retention, teardown, DOM/style growth, and back-navigation | +| Two simultaneous occurrences of one card | occurrence identity and non-aliasing | +| Same card under two distinct grants/principals | cache isolation and fresh authority checks | + +Do not expand the timed cohort merely for breadth. The broader 50-card and 44-case +FileDef matrices remain correctness smoke; promote a case into the timed cohort only +when it represents a new multiplier or boundary. + +### Samples and metrics + +For each admitted case collect five cold and five warm samples. Preserve raw samples; +report median and p95 rather than only a single aggregate. A cold Sandbox sample uses +a newly minted child while keeping the signed-in Host document alive. A warm sample +reuses only the lifecycle the protocol promises to retain—never a hidden speculative +pool. + +The existing browser runner records total semantic readiness, Direct edit readiness, +and Sandbox prerender-to-interactive handoff. Add named marks around the existing +runtime stages so the same run also records: + +- request construction, source fetch count/bytes/cache status, classification, and + graph size/depth; +- serialization, projection-settle duration/pass count, and projection bytes; +- Capsule evaluation, membrane read/clone counts, render-record construction, and + first visible output; +- Sandbox child navigation, listening, connect, ready, create, build-render-record, + render acknowledgement, and first interactive output; +- MessagePort request/response counts, pending-map high-water marks, and timeouts; +- DOM nodes, style elements, live iframes, retained runtimes, and JS heap before load, + after cold load, after warm load, after close, and after idle eviction. + +Instrumentation must be passive: stable names, monotonic timestamps, counters, and +sizes. It must not add awaits, graph traversal, serialization, or production-visible +authority. Keep detailed events behind the existing debug/test diagnostics gate. + +### Baseline runbook + +Use the persistent in-app Browser and the existing runner: + +```js +let smoke = + await import('/Users/chris/Projects/boxel/packages/host/scripts/execution-runtime-browser-smoke.mjs'); +let result = await smoke.runExecutionRuntimeBrowserSmoke({ + browser, + candidateOrigin: 'https://host.codex-execution-runtime.localhost', + candidateTab, + referenceOrigin: 'https://realms-staging.stack.cards', + referenceTab, + performanceRepeats: 5, + timeoutMs: 30_000, +}); +let summary = smoke.summarizeExecutionRuntimeSmokeRun(result); +``` + +Run correctness first. If either origin drifts, authentication is required, a declared +interaction fails, execution differs, or Sandbox teardown fails, stop and diagnose; +do not publish timing rows for that case. Save the raw result as JSON and append the +qualified summary to `boxel-execution-runtime-cold-start-baseline.md`, recording the +commit, browser/build mode, environment origin, date, cache state, and repeat count. + +### Initial budgets and stop conditions + +The first run establishes distributions, not aspirational SLAs. Until two comparable +runs exist, use only regression guardrails: + +- no Direct or Capsule median regression greater than 5% without an explained common + Host variance; +- no Sandbox median or interactive-handoff regression greater than 10%; +- no new request, clone, projection pass, live iframe, pending RPC, DOM/style, or heap + growth after the final warm/close cycle; +- any correctness, authority, identity, or teardown regression is an immediate stop, + regardless of latency improvement. + +After the baseline, choose one optimization whose measured segment dominates and +whose correctness proof is already present. Do not implement a whole phase at once. + +### Baseline implementation patch + +Prepare the baseline as one reviewable, instrumentation-only patch: + +1. Add a small typed stage recorder at the execution-engine boundary. It accepts a + stable operation id, occurrence id, execution tier, stage name, monotonic start/end + time, and inert numeric counters. It must not receive card instances, grants, + loaders, services, DOM nodes, or source text. +2. Place marks at the existing orchestration seams rather than wrapping internals + with new control flow: request/classification/materialization in + `boxel-execution-engine.ts`; projection settlement in `services/boxel-execution.ts`; + Capsule evaluation and cloning in `capsule-boxel-runtime.ts` and + `capsule-module-evaluator.ts`; Sandbox lifecycle and RPC in + `sandbox-runtime-process.ts` and the existing clients. +3. Expose a bounded snapshot through the existing debug/test diagnostics surface. + Production behavior remains unchanged when diagnostics are disabled. Reading the + snapshot must not reset runtime state or trigger work. +4. Extend `execution-runtime-browser-smoke.mjs` to capture the snapshot immediately + after declared semantic readiness and after close/idle. Keep its existing + correctness assertions, execution labels, teardown checks, and sequential tab use. +5. Add focused tests for recorder ordering, exactly-once completion, bounded storage, + disabled-mode no-op behavior, and operation/occurrence separation. Do not add + timing assertions to QUnit; the Browser run owns distributions and budgets. +6. Run the fixed corpus, save raw JSON outside the product bundle, append the + qualified summary to `boxel-execution-runtime-cold-start-baseline.md`, and only + then select an optimization. + +This patch is deliberately disposable infrastructure: stable enough to compare +commits, small enough to delete or replace, and incapable of becoming a second +execution protocol. + +### Phase 0 implementation status — 2026-08-11 + +The first instrumentation slice is implemented in the working tree: + +- a bounded, data-only recorder that is inert until explicitly enabled; +- operation and occurrence correlation without card, source, authority, service, or + DOM references; +- request, source, serialization, Card API, projection settlement, classification, + materialization, runtime creation, render-record, and generation spans; +- projection-pass, included-resource, source-size, module-graph, field, and format + counters; +- per-tier/stage median and p95 aggregation in the existing sequential browser smoke + runner, with recorder reset between occurrences; and +- focused recorder and aggregation tests. + +The development build, focused recorder QUnit tests, smoke-runner Node tests, ESLint, +and template lint pass. Full Host type lint is still blocked by the working tree's +pre-existing `.at()` target-library errors and the existing missing Sandbox `context` +argument; none originates in the instrumentation files. + +Chrome DevTools MCP now controls an authenticated staging-backed tab at the custom +`.localhost` origin. A correctness-qualified, five-sample focused baseline for +`Release/opening-night` was admitted on 2026-08-11 and used to evaluate optimization +#1. The full fixed corpus and the resource/lifecycle snapshots after close and idle +eviction remain pending, so Phase 0 is not complete. Add deeper Capsule clone and +Sandbox RPC substage marks only where the coarse spans cannot identify the dominant +segment; do not widen the runtime protocol merely for diagnostics. + +--- + +## Tier 1 — A-class legibility, small or negative LOC, large perf + +| # | Fix | Legibility | LOC | Perf estimate | +| --- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | +| 1 | Capture `JSON.parse` from the compartment once (`this.compartmentParse = compartment.evaluate('JSON.parse')`) instead of a per-read `compartment.evaluate` of an escaped source string; the 4-pass escape gauntlet becomes unnecessary. `capsule-module-evaluator.ts:2564` | **A** | **−20** | ~100–1000× per membrane read (SES compile → function call); 10–100 ms per interactive Capsule render | +| 2 | Split adopted projection work by semantic owner. Seed declared field values from the Host projection, but continue to execute every authored semantic the protocol assigns to Capsule—including `computeVia`, getters, and action-derived values—and merge only those results. Do **not** simply skip `projectionFor()`: the current contract explicitly says Capsule owns authored execution. `capsule-boxel-runtime.ts:167,189` | **A after the ownership split is specified and tested** | +20–40 | Avoids rebuilding Host-owned declared values while preserving authored computation; estimate only after projection-stage instrumentation | +| 3 | Clone-once discipline: delete the duplicate `cloneJSONRecord`s on the render-record path, the `structuredClone` of an already-hardened bundle, and `validateTemplateDescriptor`'s throwaway `JSON.parse`; memoize `fieldsFor`'s clone per instance. The same field array is deep-cloned 3× and the projection 3× per Capsule materialize. `capsule-boxel-runtime.ts:133,182,197,364`, `boxel-render-record.ts:37,52`, `capsule-component.ts:265` | **A** — "cloned once at the trust boundary, immutable thereafter" is a protocol statement; triple clones leave the reader unsure which copy is authoritative | **−25** | ms–tens of ms per materialize (MB-scale clones on large cards); scales with card size | +| 4 | Validate, then ungate the `getFields` memo for the live app. The cache exists, is instance-keyed and validity-tokened, but is bypassed unless the prerender context global is set. Before changing it, prove invalidation for inheritance, polymorphic overrides, relationship changes, field configuration, compute passes, and live Monaco/AI edits. `packages/base/field-support.ts:416` | **A only after conformance proof** | **−3** | Potentially large projection win. **Own PR — `packages/base`, changes main's behavior too** | +| 5 | Principal- and generation-scoped source/classification memo. Dependencies are re-fetched and re-classified per distinct entry card. Cache inert bytes/pure classification by principal or equivalent authorization partition + canonical URL + source generation/ETag + compiler/classifier version + draft generation. Every hit still performs the current authority decision; `invalidate()` removes dependent entries. Never cache evaluated exports, grants, instances, DOM, or services. `boxel-source-classifier.ts:660`, `services/boxel-execution.ts:1148`, `card-service.ts:247` | **A** — removes entry/dependency asymmetry without weakening authority | +25–40 | Grid of N same-type tiles: N× fetch/classify → 1× per authorized generation; measure request and Babel counters | +| 6 | Split lifecycle truth into independently testable changes: (a) replace resource authority on explicit generation/reload/destroy, not blindly on retained unmount; (b) add deadlines and cancellation to every pending transport; (c) stop child diagnostics after the parent has acknowledged first interactive output unless debug diagnostics are enabled; (d) fan out Surface observations only with observers; (e) stabilize `capsuleContextProjection` identity for an unchanged Host context. `sandbox-runtime-process.ts`, `sandbox-*-transport.ts`, `boxel-sandbox-runtime.gts`, `boxel-execution-renderer.gts` | **A** | +30–50 | Reliability/security first; then removes steady-state messages, forced measurements, and spurious Capsule rerenders | +| 7 | One Babel pass instead of two per classified module; hoist the 19 per-call `RegExp`s to named module constants; make ContentTag's `Preprocessor` a singleton. `boxel-source-classifier.ts:388,436,341,410,184` | **A** — the named constants document what classification looks for, in one place | **−10** | Halves per-module classification (~10–50 ms → 5–25 ms) | +| 8 | Evaluate `content-visibility: auto` + `contain-intrinsic-size` on fitted/gallery tiles only, with explicit exclusions for intrinsic-height Sandbox surfaces, animated/media cards, accessibility discovery, ElementTracker, and screenshot/prerender paths. Add height and offscreen-interaction tests first. | **B until those semantics are proven** | +15 CSS/tests | Potentially large layout win on offscreen galleries; not safe as a universal execution-slot rule | +| 9 | Group-commit equivalent sync fan-out: settle/serialize once per instance generation **and authorization-equivalent projection key** (principal, grants/policy, protocol version, format-relevant projection), then deliver that immutable document to matching views. Different grants never share a document. `services/boxel-execution.ts:557` | **A after the equivalence key is explicit** | +10–25 | k equivalent views: k× settle+serialize → 1×; measure projection passes and bytes | +| 10 | Keep transport lanes independently owned unless profiling shows dispatch overhead. A central dispatcher may improve teardown legibility, but it is not currently a measured performance lever and can turn typed, bounded capabilities into one broad switch. | **B/neutral** | ~0 | Expected latency win is negligible; defer from the performance path | + +## Tier 2 — B-class or modest scope; after Tier 1 + +| # | Fix | Legibility | LOC | Perf estimate | +| --- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------- | +| 11 | Use a bounded classifier work queue with an in-flight promise memo per canonical module. Do not use raw `Promise.all` over the existing `visited` set: converging branches can observe “visited” before classification resolves and incorrectly treat the shared dependency as Capsule-safe. Preserve graph limits, deterministic diagnostics, cancellation, and propagation semantics. `boxel-source-classifier.ts:697` | B+ | +30–50 | Wall time approaches graph depth rather than node count without a policy race; concurrency limit is selected from baseline data | +| 12 | Open a compute pass around each projection (`beginComputePass`/`endComputePass`), so repeated `computeVia` executions within one settle pass collapse to one each. Only the prerender route opens one today. `field-support.ts:164`, `routes/render/meta.ts:261` | B+ — states the snapshot semantics explicitly | +6 | Large across the up-to-32-pass `settleHostProjection` loop | +| 13 | Remove duplicated module parsing only after proving the authority ordering. The child must not receive executable bytes early enough to request or evaluate a dependency before the parent has admitted that exact edge. Prefer returning bytes with an already-observed dependency manifest, or let the child parse while the parent keeps subsequent fetches closed until observation commits. `sandbox-fetch-transport.ts:329`, `sandbox-module-authority.ts:46`, `boxel-sandbox-runtime.gts:361` | B− until the ordering proof exists | +10–30 | Can remove 1–10 ms × modules from the boot path; any authority race is a stop | +| 14 | CSS memoization: `confineCapsuleStylesheet` reuses the sheet `validateCapsuleStylesheet` already parsed (one CSSOM parse instead of two); content-keyed bounded LRU for `validateCapsuleInlineStyle` (the `cssVar` helper re-validates per render per invocation). `capsule-css-policy.ts:74`, `capsule-component.ts:68` | A for the sheet reuse, B for the LRU | −5 / +10 | O(rows×updates) CSSOM parses → O(distinct declaration strings); 10–100 ms on list re-renders | +| 15 | Per-key invalidation in the Capsule component runtime: consume the `changed` keys the action-result update already computes and currently ignores, instead of one shared `@tracked revision`. `capsule-component-runtime.ts:78,96,34` | B+ — computed-and-ignored data is an intent smell | +25 | One action stops re-running every getter through SES + `jsonClone`; 1–10 ms/action | +| 16 | Membrane read memo keyed by the instance version cell: `{version, value}` per property, re-project only when the cell bumped. `capsule-module-evaluator.ts:2314`, `services/boxel-execution.ts:746` | B — expresses RP-20.2's "stable until the instance changes" | +15 | N reads of one property per render → 1 per version bump | +| 17 | Split the store's global mutation counter per instance so an edit invalidates only the searches whose membership it can affect (already ticketed: CS-11419). `services/store.ts:251`, `resources/search.ts:535` | A — declares the real dependency | +40 | Any-edit-invalidates-every-search → affected rows only; 10–100 ms/keystroke with large result sets | +| 18 | GC sweep: add the visited-set dedupe to the graph builders (the sweep's own mark phase already uses one — uniformity), and skip the sweep when nothing changed since the last one. Instances are keyed under both local and remote ids, so the builders currently do everything twice. `gc-card-store.ts:1232,1254` | A / B | +10 | Halves+ the 2-minute background jank; scales with store size | +| 19 | `preconnect`/`dns-prefetch` for the sandbox origin in `index.html` (currently absent); `modulepreload` for the child's own boot bundle in the child document head | B — declarative hints | +3 | ~1 RTT (20–100 ms) off cold boot | +| 20 | Add deadlines, abort/close rejection, and teardown assertions to every pending client map (`SandboxFetchClient`, `SandboxSurfaceClient`, view-card, write, render, and Boxel-runtime requests). A lost response must settle exactly once and remove its entry. | **A; move to the correctness-first slice** | +25–40 | Prevents permanent promises/map leaks and converts hangs into typed lifecycle failures | +| 21 | Stable **occurrence** retention keys: principal + authorization partition + stable stack/surface occurrence, with card/module generation as compatibility metadata. Do not key only by card/module family: two simultaneous occurrences must receive distinct DOM/process identity. Revisit can reclaim the same inactive occurrence within TTL; concurrent mounts cannot alias it. `services/boxel-execution.ts:169`, `boxel-runtime-router.ts:70`, `retained-runtime-registry.ts:22` | B+ after occurrence semantics are tested | +25–40 | Eliminates a full Sandbox boot on compatible revisit without process aliasing | + +## Tier 3 — big-LOC or C-class; hold until the reference implementation is stable, then measure first + +| # | Fix | Legibility | LOC | Perf estimate | +| --- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- | --------------------- | ---------------------------------------------------------------------------------------------------------------------- | +| 22 | Slim child entry bundle: Glimmer + Loader + the Sandbox runtime shell, one route—no Matrix/realm/telemetry/operator-mode; delete stub/gate initializers that exist only because the child boots everything. Instrument this in Phase 0: prior local evidence attributes about 4.5 s of a 5.3 s Sandbox startup to the full child app boot. If the new baseline confirms that dominance, promote this into its own immediate workstream rather than waiting behind fine-grained reactivity. `routes/boxel-sandbox-runtime.ts`, `instance-initializers/stub-matrix-service-for-sandbox.ts` | **A** — states what the child _is_; "boot everything, stub the rest" is the anti-spec | +150 net (build work) | Potentially the dominant Sandbox win; target must come from production-preview and development measurements separately | +| 23 | Admitted module-graph snapshot: after classification and authorization, transfer an immutable graph of bytes + canonical dependency metadata to the child instead of N discovery-driven round-trips. Every edge is admitted before delivery and the snapshot is scoped to principal, generation, and compiler identity. | B — unifies grant-of-authority with delivery-of-content, but adds a lane | +100 | Re-evaluate after scoped memoization, bounded classification, Deck delivery, and the slim child; may be unnecessary | +| 24 | `materializeAndRender` combined wire op (`createFromSerialized` → `buildRenderRecord` → `render` are always issued back-to-back with no parent-side decision between them). Keep the engine's three steps; collapse only the Sandbox wire | B — names the real operation | +60 | ~10–50 ms (port RTTs are cheap; the win is deadline simplification) | +| 25 | Handshake-loaded initial render (0-RTT analog: the `connect` message carries the initial document/format/permissions) | **C** — complicates the bootstrap protocol's clean phases | +80 | Modest once #24 exists | +| 26 | JSON-Patch delta sync on the instance lanes | **C** — full-state push is _the_ self-healing story ("every push carries full current state; a missed one self-heals"); deltas break it | +100+ | 1–10 ms/push; bandwidth only | +| 27 | Warm pool / speculative boot / hidden cache-warm boot | **C** — all three contradict "born in place; classification decides" | +40–150 each | Seconds on first visit; the honest subset (stable retention keys) is already Tier 2 #21 | +| 28 | Worklist-based `settleHostProjection` (reproject only dirty nodes per fixpoint pass) | **C** — the 8-line "repeat until stable" loop is clearer | +40 | Mostly obsoleted by #4 + #12 | +| 29 | Classifier in a Worker (it is pure: string in, plain object out) | B− — mechanical, but adds indirection | +60 | Main-thread responsiveness, not latency; partly obsoleted by #7 + #11 | + +## Recommended order + +### Phase 0 — measure a green system + +Execute the baseline runbook above and land only the passive timing/counter marks +needed to decompose the observed critical path. The output is one raw JSON record and +one appended Markdown summary. Do not combine this commit with an optimization. + +### Correctness-first slice — smallest risk, required regardless of latency + +1. Add deadlines, close/abort rejection, and exactly-once cleanup to all pending + transport requests (#20/#6b). +2. Make resource/module authority replacement explicitly generation-scoped and test + reload, retained unmount/remount, card replacement, and denial after revocation + (#6a). Do not clear retained authority accidentally and do not let it accumulate + across incompatible generations. +3. Gate idle Surface/diagnostic observation and stabilize unchanged Capsule context + identity (#6c–e), with message/reflow counters proving the removed churn. + +These changes are reliability/spec work. They may improve steady state, but they do +not need a latency claim to justify themselves. + +### First measured optimization — one change at a time + +Prefer the first item whose segment is dominant in Phase 0 and whose correctness proof +already exists. The lowest-risk/high-payoff candidates are: + +1. **#1 captured compartment `JSON.parse`**—local implementation, no cache, no + identity change, easy before/after membrane microbenchmark plus full Capsule smoke. +2. **#3 clone-once render records**—make boundary immutability explicit; assert input + objects remain unchanged and all Direct/Capsule/Sandbox records stay equivalent. +3. **#7 one classifier parse/Babel pass**—pure string-in/data-out work; snapshot every + existing classification signal before removing the duplicate pass. +4. **#5 scoped source/classification memo**—only after the baseline proves repeated + fetch/classify work is material and the principal/generation/draft key has denial, + invalidation, and concurrent-request tests. + +Land and remeasure each separately. Keep it only when the intended counter falls, the +qualified cohort remains green, and Direct/Capsule/Sandbox guardrails hold. This is +the initial implementation queue; #2, #4, #9, #11, #13, and #21 are not part of the +first slice because each changes semantic ownership, authority equivalence, +concurrency, or identity. + +**Provisional first choice: #1, captured compartment `JSON.parse`.** It changes only +how an already-authorized JSON value enters the same compartment; it adds no cache, +does not alter projection ownership, does not change execution-tier selection, and +does not affect occurrence or lifecycle identity. Prepare its focused benchmark and +Capsule conformance test now, but do not land the implementation unless Phase 0 shows +membrane evaluation is material. If it is not, follow the measured critical path +rather than defending this choice. + +**Implemented and retained on 2026-08-11.** Three independent benchmark runs on a +2,412-byte representative value measured a 3.88–4.22× speedup and a 74.2–76.3% +reduction in the clone operation. Five full pre/post Release loads remained +correctness-green and kept navigation within the 5% guardrail (+3.2% median), but +their coarse render-record spans were inconclusive because the post-change Host +request median was 51% slower. The claim is therefore limited to the measured +boundary operation; no page-level latency win is claimed. + +**Second measured batch implemented and retained on 2026-08-11.** Three +semantics-preserving slices from the queue were small enough to assess together: + +- #3 now passes the Capsule projection directly to the shared render-record + assembler, which remains the sole output clone boundary. A focused mutation test + proves the returned record cannot mutate runtime-owned projection data. The more + invasive bundle/field/template-parser parts of #3 remain unimplemented. +- #7 combines executable global and DOM-method collection into one Babel traversal, + hoists the signal patterns, and reuses ContentTag's preprocessor. The existing + classification matrix plus a combined `document`/`getContext` case remains green. +- the A-class half of #14 makes validation return its parsed CSSOM sheet to + confinement. The bounded inline-style LRU remains unimplemented. + +The higher-iteration focused benchmark measured the classifier at 3,389.96 → +1,556.67 µs/operation (**2.18×, −54.1%**) and the representative render-record +assembly at 216.86 → 151.42 µs/operation (**1.43×, −30.2%**). ContentTag constructor +reuse was median-neutral (55.59 → 56.63 µs) under the same run, so no latency claim +is attached to that subchange. A Chrome-native 80-rule stylesheet benchmark measured +validation-plus-confinement at 1,320 → 902 µs/operation (**1.46×, −31.7%**) with +identical serialized CSS. + +Five authenticated Release loads remained semantic-parity green, Capsule-only, with +zero iframes and zero dropped records. Root render-record median improved from the +previous 655.9 ms to 586.2 ms (**−10.6%**), while root request median worsened from +356.0 ms to 413.8 ms (**+16.2%**) and one render outlier lifted p95 to 1,113.7 ms. +The 20.9 s page-navigation median was dominated by current Host/staging delay and is +not comparable to the earlier run. The retained claim is therefore the focused work +reduction plus the improved render-record median, not a whole-page latency win. + +**Safe lifecycle batch implemented and retained on 2026-08-11.** The low-risk parts +of #6 are now explicit protocol behavior: + +- fetch and Surface requests have a 10 s deadline, closed-state rejection, + exactly-once settlement, timer cleanup, and harmless late responses; +- Sandbox Surface observation is subscribed across the port only while the child + has a listener, and the Host installs DOM observers only while the service has a + subscriber; +- after the first visible render the parent acknowledges the diagnostic and removes + that listener, while the child stops DOM measurement and posts; explicit + performance diagnostics keep the lane enabled, and runtime-error reporting remains + independent and live; and +- Capsule context projection preserves object identity across fresh Host context + wrappers until one of the two projected presentation capabilities changes. + +A Chrome-native alternating benchmark measured unchanged Capsule context projection +at 3.1 → 0.2 ms per 100,000 accesses (**−93.5%**) and 100,000 → 1 facade allocations. +The lifecycle gates have stronger exact work counters than timing claims: after +first-paint acceptance, 1,000 simulated post-paint diagnostics perform 0 DOM +measurements/posts instead of 1,000; an attached surface with no subscribers creates +0 observers and performs 0 initial layout reads instead of two observers and one +read. A silent fetch or Surface request now lives for at most 10 s instead of +unbounded time. + +Three warmed authenticated `Release/opening-night` loads remained parity-green with +all five semantic signatures, Capsule-only execution, nine headings, zero iframes, +zero dropped records, and 946–957 DOM nodes. Medians were 26.18 s readiness, 241.2 ms +root request, and 612.9 ms root render-record. Relative to the prior retained run, +request was 41.7% lower, render-record was 4.6% higher, and readiness was 25.3% +higher; this contradictory movement is treated as Host/staging variance, so no +page-level latency claim is attached to the batch. + +**Compatible Sandbox format-switch batch implemented and retained on +2026-08-11.** A same-card `isolated` ↔ authored-`edit` toggle now keeps one +component-owned execution session and transfers the already-mounted Sandbox slot +between resource generations. `switchSandboxFormat()` repeats policy admission +against the retained source classification; if the destination is not Sandbox, or +the card or base-template identity changed, it returns to the ordinary full-update +path. + +For an admitted switch, the retained path asks the existing child card handle for +the new format's render slot. It does not repeat Host request construction, source +preparation, classification, serialization, projection settlement, child +materialization, or iframe boot. Instance-sync and lifecycle listeners are +reconnected for the new resource generation, and the `format-switch` performance +stage records the remaining child-render work. + +Focused engine coverage asserts one child materialization, no child-card disposal, +and one retained semantic generation across the round trip. Integration coverage +toggles `isolated → edit → isolated → edit → isolated` and asserts exact iframe DOM +identity after every switch. These exact work and identity assertions justify the +retained implementation; comparable before/after browser samples were not captured, +so no page-level or format-switch latency improvement is claimed here. + +### Sandbox boot decision + +The existing 2026-08-09 development record attributes about 4.46 s of a 5.31 s +Sandbox startup to child Vite/Ember boot and only about 440 ms to first module +materialization. Repeat that decomposition on the current checkpoint and on a +production preview. If child boot still dominates, promote #22 (the slim child) into +its own architecture workstream before fine-grained reactivity. Do not spend months +making a 440 ms segment perfect while retaining a multi-second child boot. + +Only after the slim-child decision should we consider bounded classifier concurrency +(#11), authority-ordered parse deduplication (#13), or an admitted graph snapshot +(#23). Deck/Realm Server immutable delivery may remove enough network cost that those +features are unnecessary. + +### Later measured reactivity + +Items #12, #14–18, and occurrence-safe #21 follow only when profiles show steady-state +projection, CSS parsing, broad invalidation, GC, or revisit boot as the limiting path. +Item #4 remains a separate Base change with its full invalidation conformance suite. +Item #8 remains a fitted/gallery experiment, not a universal execution-slot rule. + +Everything C-class stays out of v1 deliberately. The full-state protocol, explicit +bootstrap phases, and born-in-place execution identity are more valuable than small +bandwidth or first-visit wins until measured evidence says otherwise. + +## Borrowed patterns referenced above + +For the record, the cross-domain patterns each item draws on: compartment-function +capture (#1) is standard membrane practice; per-module memoization (#5) is query-based +incremental compilation (salsa / rust-analyzer); group commit (#9) is the database +write-coalescing pattern; bounded graph walk (#11) is worklist fan-out with in-flight +deduplication from dataflow analysis; admitted graph snapshot (#23) is an immutable +content-addressed deployment artifact, not a broad authority cache; handshake loading +(#25) is TLS 1.3 0-RTT; delta sync (#26) is video-codec I-frame/P-frame; pooling (#27) +is game-engine object pooling; occurrence-safe retention (#21) is cache keying by +stable UI occurrence plus compatibility metadata rather than card family alone—the +same derivation-over-authority stance as the Deck name ruling. diff --git a/docs/boxel-execution-runtime-architecture.md b/docs/boxel-execution-runtime-architecture.md new file mode 100644 index 00000000000..88618b7f7f3 --- /dev/null +++ b/docs/boxel-execution-runtime-architecture.md @@ -0,0 +1,1855 @@ +# Boxel execution runtime architecture + +## Status and purpose + +This document describes a target architecture for executing and rendering +Boxels through three execution tiers while preserving one authored API and one +user experience: + +- **Direct** — trusted modules execute in the Ember Host. +- **Capsule** — user-authored modules execute in an SES Compartment and render + through Host-owned Glimmer. +- **Sandbox** — user-authored modules execute and render in an origin-isolated + iframe, communicating with the Host through `MessageChannel`. + +It is a near-future implementation design, not a claim that the branch already +has every interface described below. It refines the boundary-record work in +[realm-sandbox-boundary-v2-plan.md](realm-sandbox-boundary-v2-plan.md) by +putting the fixed Boxel semantic API and the Glimmer rendering boundary into +one layered runtime model. + +This document is intentionally self-contained for architectural review. It +includes the authored and implicit API inventory, the complete `surface*` +capability plane, the validation conclusions drawn from real applications, and +the one twelve-case cumulative acceptance suite. The working coverage, +compatibility, and real-example ledgers remain useful implementation records, +but a reviewer does not need them to understand or evaluate this design. + +### Delivery approach: freeze the POC and rebuild from `main` + +The production implementation starts on a new branch from `origin/main`. It +does not continue restructuring `codex/code-preview-instant-reload` in place. +That branch is the **frozen reference implementation**: it proves that Capsule +and Sandbox execution, staging-backed cards, editing, HMR, iframe sizing, +media, prerender placeholders, and compatibility shims can produce a useful +product experience. + +At the decision point, the reference branch was 50 commits ahead of +`origin/main`, changed 242 files, and added approximately 42,500 lines. About +20,300 of those additions were Host production code, and the ten central +sandbox files alone contained approximately 13,350 lines. Its working behavior +is valuable; its aggregate ownership and review surface are not the target +architecture. + +Freeze means: + +- no new product features or architectural layers are added to the reference + branch; +- only critical fixes needed to keep its preview and comparison corpus usable + are accepted; +- its preview build, compatibility corpus, screenshots, import classifications, + protocol/security tests, and observation ledgers remain available as the + behavioral oracle; +- every delivery slice compares Direct and sandboxed output against both + `main` and the frozen preview; and +- implementation code is ported only when its ownership already matches this + architecture. Tests, fixtures, policies, and hard-won edge cases are ported + more aggressively than orchestration code. + +The canonical exploratory oracle is the staging-backed +[Sandbox Compatibility Corpus](https://realms-staging.stack.cards/ctse/sandbox-compatibility-corpus-20260803/index). +Runtime work must exercise that same Realm through both the deployed Host and +the branch's local Host; synthetic fixtures remain the deterministic CI layer, +not a replacement for this real-card comparison. + +The new main-based branch is delivered as a sequence of independently +reviewable vertical slices. The smallest architectural slice—versioned Boxel +records, canonical projection, and a Direct adapter—should be roughly +2,000–3,000 production lines across 10–20 files, plus focused tests. It is +valuable before isolation ships because it removes implicit reflection and +makes Direct the conformance oracle. It is infrastructure, however, and is not +called a useful execution-runtime milestone by itself. + +The second phase must deliver both untrusted adapters: Capsule execution +through Host Glimmer and Sandbox execution through an origin-isolated iframe. +The same bounded, interactive fixture must run through Direct, Capsule, and +Sandbox before the new runtime is considered useful. Capsule-only or +Sandbox-only delivery would leave the classifier and authored API unproven. +The cumulative Phase 1–2 target is approximately 6,000–9,000 production lines +plus focused tests. + +The smallest honest implementation with the same important behavior as the +latest preview is expected to be roughly 9,000–14,000 production lines and +8,000–12,000 focused test lines across approximately 40–70 files. This is a +planning guardrail, not a line-count target: exceeding it requires explaining +which previously implicit Boxel semantic or lifecycle requirement was missing, +while beating it must not come from dropping compatibility, visual behavior, +editing, security, or cleanup. + +The sequencing principle is **semantic spine first, then complete vertical +capabilities**. Do not define every future API before exercising it, and do not +build sandboxes around ad hoc snapshots that must later be replaced. Establish +one canonical Boxel interface and Direct behavior, then add the minimum +capability transport plus Capsule and Sandbox adapters as one useful vertical +slice. Mutation, HMR, additional Surfaces, and BXL authorization then extend +that same spine. + +### Vocabulary: Boxel means Box Element + +In this architecture, **Boxel** is the technical noun **Box Element**. It does +not mean the Boxel product as a whole. A Boxel is any visually present, +interactive building block derived from `BaseDef`, including `CardDef`, +`FieldDef`, `FileDef`, and future compatible kinds. A Boxel may be a persisted +card, a field renderer, a file-backed visual element, or a nested part of +another Boxel. + +The narrower nouns retain narrower meanings: + +- **Card** means a `CardDef` or its persisted Store document/instance. +- **Realm** means the server-side location and authorization boundary for data + and modules. It is not the name of a Host rendering or execution service. +- **Module** means executable source and its dependency graph. +- **Surface** means a mounted presentation environment and its bounded visual + or interactive capabilities. +- **Runtime** names who executes a Boxel: Direct, Capsule, or Sandbox. + +Consequently, cross-kind runtime types use the `Boxel*` prefix. `Card*` remains +appropriate for Card JSON:API documents, Store operations, serialization, and +card-specific mutations. `Realm*` remains appropriate for Realm URLs, grants, +fetching, indexing, and cross-Realm authorization. + +The names and pseudo-code in this document follow the conventions already used +by Boxel `main` and Ember/Glimmer: + +- preserve the existing Card API verbs when the operation has the same + semantics: `loadCardDef`, `createFromSerialized`, `getFields`, `getField`, + `getComponent`, and `serializeCard`; +- use `Boxel*` for runtime contracts shared by CardDef, FieldDef, FileDef, and + future BaseDef-derived visual kinds; +- use `Card*` only where the contract truly requires a CardDef or persisted + card document; +- suffix opaque cross-boundary identities with `Handle` and cloneable + descriptions with `Description`; +- suffix Ember services with `Service`, transport endpoints with `Client` or + `Server`, and implementation adapters with their execution tier; +- reserve `ComponentManager` and its lifecycle hook names for their actual + Glimmer meanings; and +- document exported authority-bearing interfaces with TSDoc, while keeping + implementation commentary close to the mechanism it explains. + +The snippets are target TypeScript, not a new author-facing Card API. Where a +boundary operation mirrors an existing Card API operation, its return value is +a handle or cloneable description of that same result rather than a differently +named semantic. + +The naming and lifecycle choices were cross-checked against these existing +framework seams: + +- Boxel's Card API in `packages/base/card-api.gts` and its field support in + `packages/base/field-support.ts`; +- Boxel's current custom-manager pairs in + `packages/host/app/lib/html-component.ts` and + `packages/host/app/lib/hydratable-entry-component.ts`; +- Ember's public `setComponentManager` and `capabilities` exports in + `packages/@ember/component/index.ts`; and +- Glimmer's public `ComponentManager` contract in + `packages/@glimmer/interfaces/lib/managers/component.d.ts` and its adapter in + `packages/@glimmer/manager/lib/public/component.ts`, plus the corresponding + helper and modifier contracts in that same package family. + +Those sources are the compatibility boundary. Glimmer VM implementation types +may explain the machinery, but they are not protocol vocabulary and must not +appear in Boxel's execution-runtime interfaces. + +The same architecture is presented at four zoom levels. A reader can stop +after the level that answers their question: + +1. **System overview** — the thirty-second model and non-negotiable rules. +2. **Runtime topology** — ownership and the complete Direct/Capsule/Sandbox + flows. +3. **Protocols and Glimmer mechanics** — the interfaces, records, handles, + reactivity, blocks, effects, mutations, and failure behavior. +4. **Implementation plan** — current-code mapping, migration sequence, + testing, performance instrumentation, and deletion criteria. + +--- + +## Zoom level 1: system overview + +### The architecture in one paragraph + +The Store owns the canonical card document. Each execution tier owns exactly +one executable copy of a module in its permitted environment. Every tier +exposes the same fixed Boxel semantic interface for loading card definitions, +creating instances from serialized documents, resolving fields, and selecting +formats. Direct and +Capsule rendering use one Host-owned Glimmer runtime; Direct component logic +runs in the Host while Capsule component logic remains behind stable handles +in SES. Sandbox rendering uses a Glimmer runtime inside an isolated iframe and +speaks the same semantic protocol over `MessageChannel`. Trusted Base and +Catalog components remain ordinary Ember/Glimmer programs loaded once in the +Host. Only inert values and named capabilities cross a trust boundary. + +```mermaid +flowchart LR + Store["Canonical Store\ndocuments and relationships"] + Semantic["Fixed Boxel semantic API\ntypes, fields, instances, formats"] + HostGlimmer["Host Glimmer\nDOM and trusted components"] + Direct["Direct runtime\ntrusted Host module"] + Capsule["Capsule runtime\nmodule in SES"] + Sandbox["Sandbox runtime\nmodule + Glimmer in iframe"] + + Store <--> Semantic + Direct --> Semantic + Capsule --> Semantic + Sandbox <-->|"typed MessageChannel"| Semantic + Semantic --> HostGlimmer + Direct --> HostGlimmer + Capsule -->|"template and component handles"| HostGlimmer +``` + +### Runtime matrix + +| Tier | Executable owners | Semantic owner | Glimmer/DOM owner | Trusted Base presentation | +| ------- | ------------------------------------------------- | ------------------------------------------ | ----------------- | ---------------------------------------------------------- | +| Direct | Host Loader | Host module | Host | Shared Host module graph | +| Capsule | Host canonical module + Compartment render module | Host canonical class + SES presentation | Host | Shared Host module graph through trusted component portals | +| Sandbox | Host canonical module + iframe render module | Host canonical class + iframe presentation | Iframe | Loaded in the isolated child as allowed by child policy | + +The shipped prototype deliberately keeps canonical Card API evaluation in the +Host while separately evaluating presentation code in its selected cage. A +consumer must not mix identities between those owners: + +- trusted source has one executable Host module; +- authored Capsule/Sandbox source has a Host canonical constructor for + deserialization, getters/computeds, relationships, and serialization; +- its render owner uses a separate Compartment/iframe module and receives only + projected records/capabilities for presentation. + +This is a presentation-containment boundary, not a claim that all authored +JavaScript is absent from the Host. Removing canonical authored evaluation +requires the larger store/Card API split described as future architecture. + +### Two layers, not one giant sandbox service + +```text +Layer 1 — Boxel semantics + CardDef, FieldDef, fields, documents, getters, computeVia, + configuration, relationships, format selection, mutations + +Layer 2 — rendering + captured Glimmer templates, component lifecycle, reactive cells, DOM operations, + trusted component portals, events, blocks, modifiers, styles +``` + +The first layer is intentionally bounded because the Boxel Card API is +bounded. The second layer does not attempt to serialize or emulate arbitrary +Ember components. It lets trusted Base components execute natively and gives +untrusted component logic a Capsule component manager implemented with Ember's +public custom component manager API. + +### The `surface*` capability plane + +`surface*` is an explicit coordination plane attached to rendering. It is not +part of card schema, the Store, or type introspection, and it is not ambient +browser authority. + +```text +Authored API + @cardstack/boxel-ui/surface + surfacePresentation, surfaceObserve, surfaceFocus, surfacePointer, ... + | + v +Render-tier adapter + Direct/Capsule: trusted Glimmer token or modifier manager + Sandbox: child SurfaceCapabilityClient over MessageChannel + | + v +Host SurfaceService + registration, grants, validation, coordination, lifetime, cleanup + | + v +Host DOM/browser operation or coordinated notification +``` + +The Host service is the semantic owner. A mounted render generation gets +one Host-only `SurfaceRegistration` containing its root element, execution +identity, card, format, principal, grants, and cleanup scope. The card receives +only an inert surface id and the operations granted to that surface. Unmounting +or replacing the generation revokes the registration and all observers, +listeners, timers, captures, media coordination, and pending requests. + +The public API belongs in Boxel UI because it is authored presentation code. +The transport-neutral request/response schema belongs in a small +runtime-common protocol module. The implementation and all browser authority +belong in a Host `SurfaceService`. Direct and Capsule calls dispatch to it +locally; Sandbox calls reach the same service through the iframe protocol. +Network, Store, Realm search, persistence, AI, and command authority remain +separate capability families because they are scoped to data principals rather +than a mounted surface. + +The portable authored Surface vocabulary is finite and capability-specific. +There is no generic DOM request escape hatch: + +| API | Portable semantic | Important boundary rule | +| --------------------- | ------------------------------------------------------- | --------------------------------------------------------------------------------------------------- | +| `surfaceRoot` | stable mounted Surface identity and nested path | Host registers the honest root element; authored code receives only an opaque id | +| `surfaceLifecycle` | mount, readiness, suspend, resume, teardown | callbacks are bounded component-handle messages, never Host functions | +| `surfaceObserve` | size, visibility, and intersection records | returns frozen numeric/boolean records and owns observer cleanup | +| `surfaceFocus` | focus, focus return, and bounded scroll intent | requires a registered descendant target and mode-appropriate authority | +| `surfacePointer` | pointer stream, capture, drag activation | returns reduced pointer records; the browser event and DOM nodes remain Host-owned | +| `surfaceStyle` | validated dynamic geometry/style values | allowlists properties and typed values; it cannot inject selectors or URLs | +| `surfacePresentation` | header and container/background presentation | publishes validated presentation tokens; it does not clone arbitrary iframe body CSS | +| `surfaceTransition` | scoped view-transition intent and lifecycle | transition names are namespaced by Surface/render-slot identity | +| `surfaceSchedule` | bounded timers, animation ticks, pause/resume | Host owns clocks, quotas, cancellation, and background throttling | +| `surfaceClipboard` | user-activated copy/paste of typed Boxel placement data | no unrestricted clipboard read; payload re-enters the placement validator | +| `surfaceHaptics` | bounded success/error feedback | optional result only; denial cannot change command semantics | +| `surfaceSlot` | placement into an approved Host presentation slot | named slots only; never arbitrary Host selectors or component access | +| `surfacePlayback` | media intent, leader/lease, position, rate, sequence | group membership uses Host-issued ids and monotonic state, not guessed Surface ids | +| `surfaceViewport` | pan/zoom intent and effective viewport state | coordinate conversions are typed and scoped to one mounted viewport | +| `surfaceLayout` | intrinsic measurement and allocated rectangles | parent allocation wins for fitted content; intrinsic formats report bounded size | +| `surfaceCanvas` | safe canvas/graph drawing and input coordination | Capsule receives a reviewed drawing/graph contract; arbitrary browser libraries remain Sandbox-only | + +Foundation components consume this capability plane, but the first runtime +gate does not require every foundation Surface to ship at once. The cumulative +suite uses representative composites: RichMarkdown proves `Layout` and `Run`, +a data table proves `Table` and `Cell`, and a campaign board proves +`PosterBoard` and `Frame`. Further foundation components use the same contract +and add focused conformance coverage when they ship. Product content that is +or may become data-bound remains a Surface, including direct product children +of composite Surfaces. Raw HTML is limited to native local structure, +decoration, accessory chrome, or text inside an existing leaf Surface. + +### Data, commands, Realm Script, and asynchronous AI + +Long-running work does not belong in `surface*` and does not receive ambient +Host authority. A mounted Capsule or Sandbox may invoke a separately granted, +typed command. The Host owns command authorization, provider proxying, +credentials, file persistence, and Store writes. + +Realm Script is another confined execution principal, not an extension of the +rendering Capsule. It receives bounded JSON input and named Realm operations, +produces schema-validated JSON, and is subject to input/result byte limits, +operation budgets, cancellation, and wall-clock timeout. Preview mode cannot +write. Commit mode still routes each mutation through an authorized Host +operation. The invoking harness selects the model; Realm Script cannot read or +override provider credentials. + +The canonical asynchronous image path is: + +```text +Authored action + -> granted typed command + -> one canonical Run/Job card with contained stages and logs + -> capability-scoped Realm Script produces a validated generation plan + -> Host provider command dispatches bounded requests + -> Host binary-file command persists each successful result + -> Store links durable ImageDefs and advances progress + -> SSE/index messages acknowledge, but never overwrite, the active generation +``` + +Partial success is first-class. Results may finish out of order while their +requested slots remain stable. Cancellation, retry, timeout, duplicate +acknowledgement, and stale-generation rejection cannot erase already durable +outputs. UI surfaces show stable placeholders and each image as soon as it is +persisted; linting, indexing, and acknowledgement are subsequent state, not a +reason to hide or remount the local result. + +### Non-negotiable rules + +1. The Store remains canonical for card data and relationship identity. +2. User-authored TypeScript never executes in the Host. +3. Trusted Base component internals are not mediated; the whole component + invocation is a trusted portal. +4. A live Store, Loader, service, constructor, component instance, callback, + DOM node, Glimmer tag, or browser event never crosses an untrusted boundary. +5. Data crossing a boundary is bounded, validated, cloneable, and versioned. +6. Authority crosses only as a named, revocable capability whose Host handler + rechecks authorization. +7. DOM-operation interception is necessary but not sufficient: component, + helper, modifier, block, and dynamic-resolution paths are controlled too. +8. Unsupported protocol semantics fail atomically and retain last-known-good + UI instead of partially rendering an unknown record. +9. Execution-tier selection is Host policy. A card may request stronger + isolation but cannot request Direct execution. +10. Receiving a document, Surface registration, or render handle never grants + Realm search, mutation, network, command, AI, or credential authority. +11. Long-running work is represented by canonical Run/Job state; progress, + partial success, retry, and acknowledgement do not depend on keeping the + initiating component mounted. +12. Client-side authorization may only reduce the server-authorized Boxel + graph. It never grants access, and every read, search, relationship + traversal, Command, and mutation is independently enforced by the server. + +--- + +## Zoom level 2: runtime topology and ownership + +### Canonical ownership + +| Concern | Owner | What other layers receive | +| -------------------------------------------- | ----------------------------- | --------------------------------------------------------------- | +| Card documents and relationship identifiers | Store | Versioned document projection | +| Trusted Base/Catalog executable code | Host trusted Loader | Trusted component/type identity | +| User code selected for Capsule | Per-principal Capsule runtime | Semantic records, captured template bundles, component handles | +| User code selected for Sandbox | Isolated child runtime | Typed protocol responses and effects | +| DOM for Direct and Capsule | Host Glimmer | Nothing; DOM remains Host-owned | +| DOM for Sandbox | Iframe Glimmer | Size, presentation, readiness, and named effects | +| `surface*` coordination | Host `SurfaceService` | Inert ids, bounded requests, responses, and notifications | +| Guide cards and cascade order | Store + trusted Guide runtime | Resolved labels, constraints, defaults, visibility, field order | +| Annotation, Actor, and Workflow documents | Store | Authorized target/anchor/body/state projections | +| Run/Job progress and durable outputs | Store | Versioned progress, logs, links, terminal state | +| Realm Script planning | Confined script runtime | Schema-validated JSON result and bounded activity records | +| Provider IO, binary writes, and credentials | Authorized Host commands | Durable FileDef/ImageDef identifiers, never credentials | +| Authentication and server authorization | Realm Server | Already-authorized resources and an upper-bound decision record | +| BXL UI authorization projection | Host authorization service | Frozen capability and redacted Boxel projection | +| Source generations and last-known-good state | Host orchestration | Monotonic revision and acknowledgements | + +### The common semantic contract + +Host orchestration depends on the `BoxelRuntime` interface defined +below. Rendering is deliberately tier-specific because only Capsule needs to +bridge authored component logic into a different Glimmer owner. + +The ownership distinction is deliberate: + +```text +DirectBoxelRuntime + Implements BoxelRuntime over the trusted Host module. Rendering uses + the existing Card API getComponent() result directly. + +CapsuleBoxelRuntime + Implements BoxelRuntime in the principal's Compartment and exposes a + CapsuleComponentRuntime adapter consumed by Host Glimmer. + +SandboxBoxelRuntimeClient + Implements BoxelRuntime and Sandbox surface lifecycle over the + private port. It does not expose child Glimmer component state to the Host. + +SandboxBoxelRuntimeServer + Implements BoxelRuntime beside child Glimmer. The child uses its + local Card API getComponent() result directly. +``` + +The Sandbox names must contain `Client` and `Server`. Using the same ambiguous +class name on both sides makes it too easy to invoke child authority in the +parent or treat a transport object as a local Glimmer runtime. In particular, +`getContext()` is never an iframe RPC: it is called synchronously by the +component manager in the same JavaScript realm as that Glimmer instance. + +### Direct flow + +```mermaid +sequenceDiagram + participant Store + participant Host as Direct runtime + participant Glimmer as Host Glimmer + + Host->>Store: read canonical document + Host->>Host: import trusted CardDef/FieldDef + Host->>Host: loadCardDef + createFromSerialized + Host->>Glimmer: real component + stable model projection + Glimmer->>Store: named write capability + Store-->>Glimmer: persisted revision +``` + +The adapter is still important in Direct mode. It prevents Host features from +depending on constructor reflection that Capsule and Sandbox cannot reproduce. +Direct is the reference implementation of the semantic contract, not a bypass +around it. + +### Capsule flow + +```mermaid +sequenceDiagram + participant Store + participant Host + participant Capsule as Capsule semantic runtime + participant Glimmer as Host Glimmer + + Host->>Store: read canonical document + Host->>Capsule: createFromSerialized(resource, document, relativeTo, purpose) + Capsule-->>Host: instance handle + tracked Host context + Host->>Capsule: getComponent(instance, format) + Capsule-->>Host: component definition + captured template bundle + Host->>Glimmer: mount through Capsule component manager + Glimmer->>Capsule: invokeAction(component, action, safeEvent) + Capsule-->>Glimmer: CapsuleComponentUpdate + named effects + Glimmer->>Host: apply effects + Host->>Store: authorized canonical mutation +``` + +The Capsule has no DOM. It owns authored classes, getters, actions, component +state, and pure computation. Host Glimmer owns rendering and creates only the +DOM permitted beneath the assigned render root. + +Capsule runtimes are shared per realm principal rather than per card. Module +evaluation and pure runtime support are amortized across the realm, while +module identity and invalidation remain isolated from Base and other realms. + +### Sandbox flow + +```mermaid +sequenceDiagram + participant Store + participant Parent as Host Sandbox client + participant Child as Sandbox server + participant Glimmer as Child Glimmer + + Parent->>Store: read bounded document projection + Parent->>Child: init(protocol, grant, source, document, format) + Child->>Child: loadCardDef + createFromSerialized + Child->>Glimmer: render inside isolated document + Child-->>Parent: ready(type presentation, height, revision) + Glimmer->>Child: local action + Child-->>Parent: named effect or mutation request + Parent->>Store: reauthorize and persist + Store-->>Parent: persisted revision + Parent-->>Child: acknowledgement or rejection +``` + +The child can use a real browser document and packages such as Three.js. It +does not inherit parent credentials, Store access, or parent-origin storage. +The Host client implements the common runtime contract as asynchronous +transport operations. + +### Trusted Cardstack components are one-way portals + +Components exported from `@cardstack/*`, Base, and Catalog are arbitrary, +evolving Ember/Glimmer/TypeScript programs. They cannot be reproduced by +copying static properties or proxying every method. The boundary therefore +treats an approved trusted component as an atomic Host-owned portal: + +```ts +interface TrustedComponentReference { + kind: 'trusted-component'; + module: string; + export: string; +} +``` + +When a Capsule template invokes that reference, the Capsule holds only an +inert module/export token. Host Glimmer resolves that token and runs the real +component from the shared trusted module graph. A Sandbox uses the equivalent +protocol reference and mounts the trusted child through a Host-owned slot. +Trust is deliberately one-way: trusted components may be composed into any +Capsule or Sandbox presentation, but authored constructors, closures, +services, Store objects, DOM nodes, and browser events cannot flow back into +the trusted module graph. Only projected inputs and named effects cross: + +```ts +interface TrustedFieldInvocation { + fieldType: CodeRef; + fieldName: string; + value: JSONValue; + configuration: JSONValue; + writable: boolean; + setCapability?: CapabilityID; +} +``` + +The trusted component may use any trusted Host service internally. It must not +receive an authored function, live card instance, Store, or DOM value from the +Capsule or Sandbox. + +### Server indexing and prerender + +Realm Server execution is a consumer of the same semantic contract, not a +fourth browser execution tier. It materializes authorized computed values and +produces versioned prerender output. Browser Host code may use that output as: + +- initial Host-mode HTML; +- inert last-known-good display; +- an immediate placeholder while a Sandbox becomes interactive; +- the materialized source for iframe-only computed values that must not open a + browser merely to compute an index field. + +### Guides and durable collaboration + +A Guide is a canonical data card, not an authored callback surface. The trusted +Guide runtime resolves base, domain, realm, and inline layers and evaluates +JQXL against the authorized card projection. Direct, Capsule, and Sandbox +renderers receive the same bounded result: labels, helper text, constraints, +defaults, visibility, field order, and named Command affordances. They do not +receive the Guide engine, Store, or executable functions. + +An Annotation is also canonical Store data. Its target and typed anchor +(field, text range, Cell, image region, timeline cue, or another supported +anchor kind) cross a boundary only after the target is authorized. Body, +author Actor, assignee, state, replies, and workflow step remain stable Boxel +identities. Reply, resolve, assign, and advance are typed Commands; mounting an +Annotation never grants mutation or neighboring-target access. This makes +human-to-agent review durable across Direct, Capsule, and Sandbox rather than +recreating it as component-local comment state. + +Coordination ownership is also a Boxel semantic, not a property of the mounted +renderer. A single card may have an ordinary revisioned title, a Yjs-concurrent +rich-text field, Command-owned approval fields, temporarily frozen release +terms, and computed readiness. The Store and server project one bounded, +field-scoped coordination description for every execution tier. They never +send a Policy engine, Yjs document, Command executor, or live collaboration +service across the boundary. + +The invariant is one write owner per field path at a time. Ordinary revision +writes, collaborative materialization, ordered Commands, and admitted atomic +snapshots all commit field-scoped patches through the same canonical document +revision. Policy custody is episodic and minimal: paths not named by the +active, versioned Policy retain their declared ordinary behavior, and custody +ends when the bounded term lapses. Cursor, selection, focus, and presence are +ephemeral awareness and never become Store data. An Annotation or message may +propose or explain a transition, but only a separately admitted typed Command +may enact consequential state. + +AI source/data rewrites use snapshot compare-and-swap rather than pretending +to be CRDT operations. A current snapshot briefly fences affected bindings, +settles accepted collaborative updates, closes their epochs, installs the +candidate atomically, and starts new epochs for remaining collaborators. A +stale snapshot is rejected without overwriting official state. Published views +similarly observe two clocks: compatible instance data may follow the source +change feed, while code, schema, templates, theme, and projection policy stay +pinned until republish. Incompatible changes retain last-known-good output and +surface `republish-required`. + +### BXL authorization projects the usable Boxel graph + +The detailed companion contract is +[boxel-execution-runtime-authorization-projection.md](boxel-execution-runtime-authorization-projection.md). + +BXL authorization is a Boxel semantic and data-capability concern, not a +`surface*` capability. `surface*` coordinates a mounted visual surface; BXL +decides which data and operations may enter that surface at all. + +The clinical-access example demonstrates the intended shape. A patient record +links a policy, facility, people, and recursively nested teams. The policy +evaluates capabilities such as `ViewClinicalSummary`, `ViewInternalNotes`, and +`EditCarePlan` from a finite snapshot. A positive nested-team seat can grant +eligibility, while a separate `Seat.Suspended` refusal wins. Access to one +patient resource does not imply access to another. The current demo prepares +BXL and constructs the dashboard projection inside the card; the runtime +architecture moves that repeated work into a Host-owned +`BoxelAuthorizationService`. + +The resource or owning application may link a BXL policy card, but that policy +reference is an input to the Host service, not authority handed to authored +code. The service resolves the authorized policy version, principal, resource, +bounded relationship graph, and request inputs; evaluates BXL synchronously +when a safe client snapshot is available; and returns a frozen projection. It +does not send the policy evaluator, live Store, hidden membership graph, or +unprojected resource values into a Capsule or Sandbox. + +```text +resource + linked policy + principal + request input + | + v + BoxelAuthorizationService + server upper bound ∩ client BXL decisions + | + v + authorization projection + capabilities + visible Boxel graph + | + v + buildBoxelRenderRecord() + | + Direct / Capsule / Sandbox +``` + +"Client-side filtering" is therefore implemented as **client-side +projection**. Filtering after materialization is too late: a denied field must +not briefly appear, become a loading placeholder, affect a count, leak a title +or URL, or cross an execution boundary. Projection can remove: + +- field values and FieldDef render slots; +- relationships, children, and query results; +- formats and presentation sections; +- Guide affordances and menu items; and +- Commands and mutation paths. + +The server remains the security boundary. It must not send secrets merely so +the client can hide them, and it rechecks every fetch, search, relationship +traversal, Command, and mutation. A locally evaluated BXL result may only +intersect with and reduce the server-authorized upper bound; it cannot widen +it. Missing, stale, incompatible, or failed authorization projection fails +closed while preserving a non-sensitive last-known-good shell. Explicit BXL +refusal wins over positive eligibility in both client projection and server +enforcement. + +Authorization changes are targeted semantic invalidations. Changing the +viewer, policy version, membership, resource state, or request input rebuilds +the affected projection and render slots without remounting unrelated Boxels. +The same projection record is consumed by Direct, Capsule, Sandbox, Code +preview, delegated rendering, Rich Markdown, fitted galleries, inspector +schema, and AI schema sharing, so a different execution tier cannot restore a +field or action that projection removed. + +--- + +## Zoom level 3: protocols and Glimmer mechanics + +### Semantic interface + +```ts +type RuntimeHandle = string & { readonly __runtimeHandle: unique symbol }; +type BoxelTypeHandle = RuntimeHandle & { readonly __boxelType: unique symbol }; +type CardInstanceHandle = RuntimeHandle & { + readonly __cardInstance: unique symbol; +}; + +/** + * The execution-tier-neutral subset of the existing Card API. + * + * Implementations execute these operations in Direct, Capsule, or Sandbox. + * Values returned to the caller are handles or cloneable records; live CardDef + * classes and instances never cross an untrusted boundary. + */ +export interface BoxelRuntime { + loadBoxel(ref: CodeRef): Promise; + + createFromSerialized( + resource: LooseCardResource, + document: LooseSingleCardDocument | CardDocument, + relativeTo: RealmResourceIdentifier | undefined, + purpose: MaterializationPurpose, + ): Promise; + + describeBoxel(boxel: BoxelTypeHandle): Promise; + + getFields( + boxel: BoxelTypeHandle | CardInstanceHandle, + ): Promise; + + getField( + boxel: BoxelTypeHandle | CardInstanceHandle, + fieldName: string, + ): Promise; + + serializeCard( + card: CardInstanceHandle, + options: SerializeOpts, + ): Promise; + + serializeCardPatch( + card: CardInstanceHandle, + changes: Record, + ): Promise; + + dispose(handle: RuntimeHandle): Promise; +} +``` + +`MaterializationPurpose` is explicit. Indexing, Host display, command +validation, Code preview, and interactive editing do not receive authority +merely because they all call `createFromSerialized()`. + +`serializeCardPatch()` is intentionally new rather than overloading +`serializeCard()`. The existing function serializes a complete card document; +the new operation accepts named edits and returns only canonical `PatchData`. +It never treats the richer render projection as a writable document. + +### Type and field descriptions + +The template inventory is open-ended. Formats shown here are examples, not an +exhaustive union embedded in the boundary record. + +```ts +export interface BoxelDescription { + protocolVersion: number; + requiredFeatures: string[]; + ref: CodeRef; + boxelKind: string; + ancestors: CodeRef[]; + fields: FieldDescription[]; + formats: FormatDescription[]; + presentation: TypePresentation; + executionHints: { + prefersFullSandbox: boolean; + }; +} + +export interface FormatDescription { + format: string; + provider: + | { kind: 'authored'; ref: CodeRef } + | { kind: 'trusted-base'; ref: CodeRef }; +} + +export interface ResolvedField { + fieldName: string; + fieldType: CodeRef; + kind: 'contains' | 'containsMany' | 'linksTo' | 'linksToMany'; + value: JSONValue; + resolvedConfiguration: JSONValue; + presentation: Record; + writable: boolean; +} +``` + +Executable configuration providers run with their semantic owner. A Capsule +configuration function runs in the Capsule and returns validated data. The +Host never attempts to copy the function or rediscover field-specific statics +such as currency symbols. Trusted Base presentation consumes the resolved +record. + +### Canonical Boxel render projection + +Authorization has two records. The first is Host-internal input to the policy +service; the second is the cloneable result that render consumers may receive. +Capability names remain domain policy vocabulary rather than a hard-coded +clinical or music union. + +```ts +export interface BoxelAuthorizationRequest { + policy: CardIdentifier; + principal: string; + resource: CardIdentifier; + resourceRevision: string; + requestedCapabilities: string[]; + input: Record; +} + +export interface BoxelAuthorizationProjection { + protocolVersion: number; + policy: CardIdentifier; + policyRevision: string; + principal: string; + resource: CardIdentifier; + resourceRevision: string; + inputHash: string; + capabilities: Record< + string, + { effect: 'allow' | 'refuse' | 'not-applicable'; reasonCode?: string } + >; + visibleFields: string[]; + visibleRelationships: string[]; + visibleFormats: string[]; + visibleSections: string[]; + availableCommands: string[]; +} +``` + +The Host may expose the frozen result through render context for custom +section composition, but this is descriptive state, not an authorization +closure. Generic Base templates consume the same field, relationship, format, +and Command lists without requiring card-specific conditionals. Explanations +use bounded reason codes; sensitive policy traces and membership paths remain +Host/server diagnostic data. + +All tiers consume one record assembled by a pure Host pipeline. Boxel normally +uses `build*` for a pure record assembler, so the target name is +`buildBoxelRenderRecord()`: + +```ts +buildBoxelRenderRecord({ + canonicalDocument, + boxelType, + projectedValue, + relationshipProjection, + authorizationProjection, + cardInfoProjection, + presentationProjection, + policy, +}): BoxelRenderRecord; +``` + +The record contains no capability closures. Host-only capabilities are stored +separately and keyed by execution identity, Boxel identity, format, grant, and +lifetime. `canonicalDocument` is present for persisted cards; FieldDef, +FileDef, and future Boxel kinds can project a value without pretending to be a +card document. + +### Mutation is a reverse projection + +The detailed companion contract is +[boxel-execution-runtime-mutation-protocol.md](boxel-execution-runtime-mutation-protocol.md). + +The rendered projection is deliberately richer than the writable document. +It may contain computed values, expanded relationships, presentation values, +and side-loaded data. It must never be sent back as a card PATCH. + +```text +rendered projection + | user edit + v +named field changes + | semantic runtime serializeCardPatch() + v +canonical attributes + relationship identifiers + | Host authorization and JSON-API validation + v +Store PATCH +``` + +`serializeCardPatch()` rejects computed fields, presentation-only values, +foreign side-loaded resources, unknown relationships, and fields outside the +current write grant. + +### Capsule component runtime + +```ts +type CapsuleComponentHandle = RuntimeHandle & { + readonly __capsuleComponent: unique symbol; +}; + +type CapsuleComponentInstanceHandle = RuntimeHandle & { + readonly __capsuleComponentInstance: unique symbol; +}; + +export interface CapsuleComponentDefinition { + component: CapsuleComponentHandle; + template: CapturedTemplateBundle; +} + +/** + * Resolves Capsule components and implements their authored lifecycle. + * + * The Host's Glimmer ComponentManager delegates to this interface. Direct and + * Sandbox rendering use their local Card API components and do not implement + * this cross-owner bridge. + */ +export interface CapsuleComponentRuntime { + getComponent( + card: CardInstanceHandle, + format: string, + ): Promise; + + createComponent( + definition: CapsuleComponentDefinition, + args: RenderArguments, + ): CapsuleComponentInstanceHandle; + + getContext(component: CapsuleComponentInstanceHandle): ComponentContext; + + updateComponent( + component: CapsuleComponentInstanceHandle, + args: RenderArguments, + ): CapsuleComponentUpdate; + + invokeAction( + component: CapsuleComponentInstanceHandle, + action: string, + event: SafeEvent, + ): Promise; + + destroyComponent(component: CapsuleComponentInstanceHandle): void; + dispose(handle: RuntimeHandle): Promise; +} +``` + +`createComponent`, `getContext`, `updateComponent`, and `destroyComponent` +deliberately match Ember's public custom component manager hooks. Boxel's +manager delegate has `capabilities('3.13', { updateHook: true, destructor: +true })`; it must not implement or expose Glimmer's internal `getSelf`, +`getDestroyable`, VM `Program`, or state-bucket APIs. + +The lifecycle hooks are synchronous because Ember's public manager contract is +synchronous. Any module loading, template capture, or policy negotiation must +finish in `getComponent()` before Glimmer receives the definition state. The +manager's `getContext()` returns a stable Host object backed by tracked cells; +it never waits on the Capsule or returns a Promise to the VM. + +Boxel supplies `_CapsuleComponent` as the definition object passed to +`createComponent()`. `CapsuleComponentState` is the manager's private state +bucket. Neither object is serialized, stored in the Card boundary record, or +exposed to card authors. + +### Captured template bundles + +A captured template bundle contains validated Glimmer wire data and explicit +references. It never contains an executable authored closure in Host memory. + +```ts +export interface CapturedTemplateBundle { + protocolVersion: number; + root: TemplateHandle; + templates: Record; + stylesheets: StylesheetReference[]; + dependencies: RenderDependency[]; +} + +type RenderDependency = + | TrustedComponentReference + | SandboxComponentReference + | TrustedHelperReference + | SafeModifierReference + | SandboxBlockReference; +``` + +The Host validates the bundle before Glimmer sees it. Unknown required +dependency kinds reject the complete generation. + +Glimmer needs both a component manager and a template factory. For every +validated entry in the bundle, the Host creates one private component +definition object, associates the common Capsule manager through its +prototype, and calls `setComponentTemplate()` exactly once on that definition +object: + +```ts +import { setComponentTemplate } from '@ember/component'; +import type { ComponentLike } from '@glint/template'; + +type TemplateFactory = Parameters[0]; + +type CapsuleComponent = ComponentLike<{ + Args: Record; + Element: Element; +}>; + +function createCapsuleComponentDefinition( + runtime: CapsuleComponentRuntime, + definition: CapsuleComponentDefinition, + template: TemplateFactory, +): CapsuleComponent { + let component = new _CapsuleComponent(runtime, definition); + setComponentTemplate(template, component); + return component as unknown as CapsuleComponent; +} +``` + +This is supported by Glimmer's public API: manager and template lookup both +walk the definition object's prototype chain, while a template may be +associated directly with an object. The Host caches the resulting definition +by runtime identity, component handle, source hash, and captured-template +signature. It never calls `setComponentTemplate()` twice on the same object, +which Glimmer explicitly rejects in debug builds. + +A template factory is immutable for the lifetime of that definition. An HMR +generation that changes only component state or arguments reuses the +definition. A generation that changes the captured template creates a new +definition; the stable outer template island then adopts compatible serialized +DOM or replaces only that island. The public custom component manager API does +not provide a mutable-layout hook, so the architecture must not claim that the +manager alone can replace a template while preserving DOM. + +### Host-owned Glimmer bridge + +Capsule component logic stays in SES while Host Glimmer owns DOM and +reactivity. A custom component manager performs the bridge: + +1. create a Capsule component instance and receive a stable handle; +2. create Host-owned cells for cloneable component state and getters; +3. render its validated captured template bundle; +4. dispatch actions to the Capsule with a reduced event record; +5. apply returned cell changes and dirty only the corresponding Glimmer tags; +6. execute named Host effects after policy checks; +7. destroy the component state on teardown. + +```ts +export interface CapsuleComponentUpdate { + generation: number; + componentRevision: number; + changed: Record; + effects: SurfaceEffect[]; +} +``` + +Host-owned cells—not SES objects or tags—connect the component to Glimmer. +Synchronous action updates return in `CapsuleComponentUpdate`. Asynchronous +authored changes use a bounded invalidation channel naming a component handle +and changed paths; they never receive a Host Glimmer tag. + +The manager itself should look ordinary to an Ember reviewer. The transport +and policy machinery stays behind `CapsuleComponentRuntime`: + +```ts +import { capabilities, setComponentManager } from '@ember/component'; + +type ComponentManager = ReturnType[0]>; +type ComponentArguments = Parameters[1]; + +class _CapsuleComponent { + constructor( + readonly runtime: CapsuleComponentRuntime, + readonly definition: CapsuleComponentDefinition, + ) {} +} + +class CapsuleComponentState { + constructor( + readonly runtime: CapsuleComponentRuntime, + readonly handle: CapsuleComponentInstanceHandle, + ) {} +} + +class CapsuleComponentManager implements ComponentManager { + capabilities = capabilities('3.13', { + destructor: true, + updateHook: true, + }); + + static create(_owner: unknown) { + return new CapsuleComponentManager(); + } + + createComponent(definition: _CapsuleComponent, args: ComponentArguments) { + let handle = definition.runtime.createComponent( + definition.definition, + renderArguments(args), + ); + return new CapsuleComponentState(definition.runtime, handle); + } + + getContext(component: CapsuleComponentState) { + return component.runtime.getContext(component.handle); + } + + updateComponent(component: CapsuleComponentState, args: ComponentArguments) { + applyComponentUpdate( + component.runtime.updateComponent( + component.handle, + renderArguments(args), + ), + ); + } + + destroyComponent(component: CapsuleComponentState) { + component.runtime.destroyComponent(component.handle); + } +} + +setComponentManager( + (owner) => CapsuleComponentManager.create(owner), + _CapsuleComponent.prototype, +); +``` + +`renderArguments()` and `applyComponentUpdate()` in this sketch are private +adapter helpers. They reduce Glimmer's stable argument proxy to the bounded +runtime argument shape and apply returned values to Host-owned tracked cells. +They are not Card API methods, Glimmer extensions, or author-facing +capabilities. + +This follows Boxel's existing `HTMLComponentManager` and +`HydratableEntryComponentManager` pattern: an internal component definition +value, a small manager delegate, `capabilities('3.13', ...)`, and +`setComponentManager()` on the definition prototype. The detailed runtime +does not pretend to be a Glimmer VM `InternalComponentManager`. + +### Components, helpers, and modifiers + +DOM interception alone is not a sufficient boundary. Resolution is also +explicit: + +- trusted component references resolve through the trusted Host Loader; +- authored component references resolve through the Capsule component manager; +- trusted helpers receive cloneable arguments and return cloneable values; +- authored helpers execute in the Capsule through handles; +- safe modifiers are named Host capabilities with reviewed argument/result + schemas and teardown; +- raw Ember modifiers from user code require Sandbox routing; +- dynamic component resolution accepts only a validated reference produced by + the semantic runtime. + +Boxel should use each public Glimmer manager for its actual purpose rather than +inventing one universal "sandbox manager": + +| Glimmer seam | Public hooks Boxel implements | Boundary rule | +| ----------------------------------- | ------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- | +| Component manager (`3.13`) | `createComponent`, `getContext`, optional `updateComponent`, `destroyComponent` | Host state bucket contains only runtime/handle/cell references; authored instance stays in Capsule | +| Helper manager (`3.23`, `hasValue`) | `createHelper`, `getValue`, optional destroyable | Capsule helper returns a cloneable value through a helper handle; Host-owned tracking invalidates it | +| Modifier manager (`3.22`) | `createModifier`, `installModifier`, `updateModifier`, `destroyModifier` | The trusted Host manager owns the `Element`; Capsule receives only validated arguments and named `surface*` operations | +| Component template | `setComponentTemplate` once per definition object | Host attaches a validated `TemplateFactory`; authored code never receives the factory or wire data back | + +All of these hooks are synchronous. Module loads, dependency resolution, and +policy checks happen before Glimmer resolves the definition. A helper or +modifier that cannot satisfy the bounded synchronous contract routes the +format to Sandbox rather than blocking the Host render transaction. The +manager state buckets are destroyable lifetime records, not serialized Boxel +state. + +### DOM and CSS boundary + +Host Glimmer uses a sandbox-aware owner/resolver and DOM policy beneath a +stable render root. The policy covers: + +- permitted elements and attributes; +- URL-bearing attributes and protocols; +- event listener ownership; +- literal and dynamic style values; +- stylesheet scoping, network-bearing declarations, global rules, and + keyframes; +- lifecycle cleanup and reference-counted stylesheet removal. + +The DOM policy complements source classification and captured-template +validation. No one layer is treated as the complete security boundary. + +### Blocks and yields + +No live block closure crosses between trusted and untrusted execution. + +```ts +interface SandboxBlockReference { + kind: 'sandbox-block'; + handle: TemplateHandle; + parameters: JSONValue[]; +} +``` + +When a trusted Base component yields, the Host projects and validates the +yielded parameters, then asks the Capsule component manager to render the +captured authored block. A block cannot recover the trusted component +instance, owner, Store, or original event through its parameters. + +### Events and effects + +Browser events are reduced to an allowlisted record: + +```ts +interface SafeEvent { + type: string; + key?: string; + code?: string; + value?: string; + checked?: boolean; + clientX?: number; + clientY?: number; + dataset?: Record; +} +``` + +Imperative behavior becomes a named effect rather than a method on a live +event or element. Examples include field mutation, navigation, focus, scroll, +selection, measurement, presentation, media, and playback. Each effect defines +its argument schema, result schema, lifetime, cleanup, and authorization rule. + +### `surface*` registration, dispatch, and coordination + +`surface*` capabilities are mounted-render capabilities. They do not belong on +`BoxelRuntime`, because loading a CardDef or creating a card instance +must not allocate DOM authority. `CapsuleComponentRuntime.createComponent()` +receives a Host-created surface context, and the Glimmer bridge makes its +granted author API available to the rendered component. + +The Host-only registration is approximately: + +```ts +interface SurfaceRegistration { + surfaceId: string; + executionId: string; + renderSlotId: string; + principal: string; + cardId: string; + format: string; + generation: number; + root: Element; + grants: ReadonlySet; + abort: AbortController; +} +``` + +`root`, `grants`, and `abort` never cross the boundary. The transport record is +smaller: + +```ts +interface SurfaceCapabilityRequest { + protocol: 'boxel-surface-capabilities/1'; + requestId: string; + surfaceId: string; + generation: number; + capability: string; + operation: string; + args: JSONValue; + activation?: string; +} +``` + +The injected Host service owns the operation boundary. Boxel services are +default-exported concrete classes, so its public declaration follows that +convention rather than introducing a parallel `SurfaceCoordinator`: + +```ts +/** Host authority for capabilities attached to a mounted card surface. */ +declare class SurfaceService extends Service { + registerSurface(registration: SurfaceRegistration): void; + unregisterSurface(surfaceId: string, generation: number): void; + + request( + request: SurfaceCapabilityRequest, + ): Promise; + + notify(notification: SurfaceCapabilityNotification): void; +} + +export default SurfaceService; +``` + +The implementation is an ordinary concrete Ember service in +`services/surface-service.ts`; the declaration shows only its relevant public +contract. + +`registerSurface()` and `unregisterSurface()` are Host-only. The author-facing +module exports capability-specific names such as `surfacePresentation`; it +does not expose this service or a generic `request()` escape hatch. The +Sandbox client serializes the same typed request that the Direct and Capsule +adapters pass locally. + +The proposed code ownership is: + +| Piece | Location and responsibility | +| ------------------- | ----------------------------------------------------------------------------------------------------------------------------- | +| Author imports | `@cardstack/boxel-ui/surface`; typed modifiers/helpers/services with no Host objects | +| Protocol records | A runtime-common `surface-capability-protocol` module shared by Host and iframe child | +| Host semantic owner | `SurfaceService`; registrations, grants, validation, rate limits, coordination state, cleanup | +| Direct adapter | Trusted Boxel UI implementation dispatches to the service for consistent semantics | +| Capsule adapter | Trusted Glimmer token/modifier manager dispatches directly to the service; callbacks return through Capsule component handles | +| Sandbox adapter | Child `SurfaceCapabilityClient` sends requests over the private port; parent validates and dispatches to the service | + +Cross-card features such as synchronized playback are `SurfaceService` +concerns. They use a Host-issued coordination-group id and monotonic sequence/clock +state; cards cannot join a group by guessing another surface id. Pan/zoom, +selection, presentation, and playback can share this envelope while retaining +separate operation schemas and grants. A service notification contains only +bounded state such as `{ playing, position, rate, sequence }`, never a +media element or controller object. + +Direct rendering should use the same public `surface*` semantics even though +it could touch the DOM directly. This makes behavior portable across tiers and +keeps a later Capsule/Sandbox classification change from changing the authored +contract. Trusted Host-internal components remain free to use private Host APIs +when they are not presenting an author-facing portable surface contract. + +### Generation, HMR, and stable identity + +Source generation, semantic instance identity, component identity, and DOM +identity are separate: + +- a source hash identifies compilation and classification cache entries; +- a monotonic generation identifies a proposed module update; +- stable instance/component handles preserve state across compatible updates; +- stable render-slot keys preserve Glimmer and DOM identity; +- server/index echoes acknowledge an already-rendered generation; +- incompatible template bundles or explicit Reload Card create a new component + generation; +- failed generations keep the last-known-good template bundle mounted. + +Compatibility is decided by an explicit captured-template signature, not by +whether two generated JavaScript classes happen to be referentially equal. + +### Protocol negotiation and failure behavior + +Semantic records and iframe transport have independent protocol versions. +Every message or record includes: + +- protocol version; +- required features; +- execution identity; +- card/instance identity; +- source generation; +- request correlation id where applicable. + +An unsupported required feature rejects the complete operation. The Host keeps +last-known-good content visible, surfaces one actionable diagnostic, and does +not silently substitute `undefined` for a missing semantic. + +### Resource lifetime + +Every handle has an owner and release point. Runtimes track: + +- module generations; +- instances; +- component instances; +- captured templates; +- blocks; +- style references; +- effect subscriptions; +- pending requests. + +Per-principal Capsule runtimes are retained while they have active consumers +and evicted after an idle TTL. Settled loads and released handles are removed +immediately; runtime eviction is not the only cleanup mechanism. + +--- + +## Zoom level 4: implementation and migration plan + +### Frozen reference branch mapping + +The reference branch is evidence and a source of focused tests, policies, and +proven algorithms. It is not the base of the production implementation. The +table maps its working POC responsibilities so that each can be deliberately +covered, ported, or replaced on the new main-based branch. + +| Target responsibility | Current starting point | +| ---------------------------------------------- | --------------------------------------------------------------------------------- | +| Capsule module evaluation and Card API facade | `packages/host/app/lib/realm-compartment-module-runtime.ts` | +| Per-principal runtime lifetime | `packages/host/app/lib/realm-sandbox-runtime-registry.ts` | +| Opaque records and symbols | `packages/host/app/lib/realm-sandbox-boundary.ts` | +| Orchestration, classification, projection, HMR | `packages/host/app/services/realm-sandbox.ts` | +| Common render entry | `packages/host/app/components/card-renderer.gts` | +| Capsule rendering | `packages/host/app/components/realm-sandbox-render.gts` | +| Stable render island | `packages/host/app/components/realm-sandbox-template-island.gts` | +| Delegated trusted rendering | `packages/host/app/components/realm-sandbox-delegated-render.gts` | +| Contextual field rendering | `packages/host/app/lib/realm-sandbox-field-component.gts` | +| Sandbox parent component | `packages/host/app/components/realm-sandbox-iframe.gts` | +| Sandbox transport | `packages/host/app/lib/realm-iframe-sandbox-protocol.ts` | +| Sandbox child application | `packages/host/app/templates/realm-sandbox-frame.gts` | +| Height and media capabilities | `realm-iframe-height-service.ts`, `realm-iframe-media-bridge.ts` | +| First author-facing surface capability | `packages/boxel-ui/addon/src/surface.gts` and `modifiers/surface-presentation.ts` | +| Current Capsule/Sandbox presentation adapters | `realm-sandbox-render.gts` and `realm-sandbox-frame.gts` | +| Stylesheet policy/lifetime | `realm-sandbox-styles.ts` service and modifier | + +The legacy `RealmSandboxService` combines several of these concerns. Its name +describes the POC rather than the target architecture: Realm is a server-side +data/module location, not a Host execution service. Do not copy this service +and split it afterward. Port each proven behavior directly into its target +owner only when the corresponding vertical slice reaches it. + +### Target module and symbol names + +These are the intended names once behavior has moved behind the interfaces. +They follow Boxel's lower-kebab-case module names, named library exports, and +default-exported Ember service classes. + +| Module | Principal exports | +| ------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | +| `packages/runtime-common/boxel-execution-protocol.ts` | Cloneable `BoxelDescription`, `FieldDescription`, `ResolvedField`, handle ids, protocol requests, responses, and version checks; no Ember imports | +| `packages/host/app/lib/boxel-runtime.ts` | `BoxelRuntime` and shared semantic handles | +| `packages/host/app/lib/capsule-component-runtime.ts` | `CapsuleComponentRuntime`, `CapsuleComponentDefinition`, and Capsule component handles; no iframe transport | +| `packages/host/app/lib/direct-boxel-runtime.ts` | `DirectBoxelRuntime` | +| `packages/host/app/lib/capsule-boxel-runtime.ts` | `CapsuleBoxelRuntime`; selectively ports proven evaluator behavior without importing the legacy orchestration service | +| `packages/host/app/lib/sandbox-boxel-runtime-client.ts` | `SandboxBoxelRuntimeClient`; parent-side `BoxelRuntime`, surface lifecycle, correlation, and transport only—never Host Glimmer state | +| `packages/host/app/lib/capsule-component.ts` | `_CapsuleComponent`, `CapsuleComponentManager`, and its Glimmer definition/state wrappers | +| `packages/host/app/services/surface-service.ts` | default `SurfaceService extends Service`; mounted-surface registrations and browser capabilities | +| iframe child module beside `realm-sandbox-frame.gts` | `SandboxBoxelRuntimeServer`; child dispatch only | + +Do not introduce a `SandboxBoxelEnvironment`, `RemoteGlimmerProgram`, or other +parallel vocabulary. `Boxel*` identifies the cross-kind semantic boundary; +`Card*` is reserved for persisted Card documents and card-specific operations; +`Capsule*` and `Sandbox*` identify execution policy; and Glimmer lifecycle names +appear only on the component manager that actually implements them. + +### Phase 0: freeze the reference and establish evidence + +Create the delivery branch from `origin/main` and stop adding features to the +reference branch. Before implementation: + +1. preserve a reachable reference preview and record its commit; +2. port the deterministic compatibility fixtures and boundary assertions before + porting runtime code; +3. record Direct, Capsule, and Sandbox cold/steady render timings; +4. count module evaluations, cache hits, component remounts, DOM replacements, + stylesheet installations, and active handles; +5. capture current green/red corpus results and representative screenshots; +6. freeze `Direct`, `Capsule`, and `Sandbox` vocabulary in diagnostics and + tests; and +7. prohibit new constructor introspection in Host consumers. + +The reference branch is an oracle, not a dependency. A delivery test passes +only when it proves the intended semantic and visible behavior; matching an +accidental POC implementation detail is not a goal. + +### Phase 1: ship the semantic spine through Direct + +Add versioned `BoxelRuntime` records, `ResolvedField`, stable Boxel/render-slot +identity, and `buildBoxelRenderRecord()` without changing visible rendering. +Implement `DirectBoxelRuntime` over the existing trusted Loader and Card API. +Direct rendering must continue through main's canonical `transpileJS()` and +Loader pipeline, including the existing `glimmer-scoped-css` transform. The +runtime adapter owns execution and semantic projection; it does not introduce +another stylesheet compiler, registry, or selector-rewriting path. + +The canonical pipeline includes: + +- inherited field metadata and configuration; +- per-usage configuration functions; +- getters, `computeVia`, and their dependency/error state; +- relationship identity and bounded hydration; +- `cardInfo`, theme, title, description, thumbnail, and presentation; +- an open-ended authored format inventory plus trusted Base fallback slots; +- writability and mutation schema; and +- reserved authorization and capability-projection fields, even though their + services are implemented later. + +Add a development-only missing-path diagnostic proxy. Move one Host consumer +at a time from constructor reflection to the adapter. Direct is the executable +conformance oracle: if a semantic cannot be expressed here, the interface is +incomplete before Capsule or Sandbox work begins. + +This is the first independently mergeable slice. Its target is approximately +2,000–3,000 production lines across 10–20 files plus focused protocol, Direct, +and visual conformance tests. + +### Phase 2: deliver Capsule and Sandbox as one useful vertical slice + +Phase 2 is one milestone with three inseparable parts: the minimum Host +capability broker, the Capsule adapter, and the Sandbox adapter. Implementing +only the broker, only Capsule, or only Sandbox does not count as a useful +execution runtime. The same bounded authored fixture must execute through +Direct, Capsule, and Sandbox before this phase is complete. + +First create the Host `SurfaceService`, execution identity, registration +lifetime, and shared request/response/notification envelopes. Do not implement +the full `surface*` catalog. The initial capabilities are only those required +for honest ordinary rendering: + +- `surfacePresentation` for header/container intent; +- `surfaceLayout` for intrinsic and parent-allocated sizing; +- `surfaceObserve` for bounded size/visibility records; and +- the trusted modifier/manager path needed to bind those capabilities to the + real Host element. + +Direct uses a local dispatcher, Capsule uses trusted Glimmer managers, and +Sandbox uses a private `MessageChannel`; all three target the same Host-owned +service methods. Data access, authorization, and mutation remain outside +`SurfaceService`. + +Then implement both untrusted adapters: + +- `CapsuleBoxelRuntime` owns per-principal SES module lifetime, + source/import classification, stable component handles, deterministic + teardown, and the Host Glimmer component bridge; +- `SandboxBoxelRuntimeClient` owns parent-side correlation, capability + admission, lifecycle, and last-known-good placeholder state; and +- `SandboxBoxelRuntimeServer` owns the origin-isolated iframe's child Loader, + authored module, Glimmer runtime, and DOM. + +Selectively port proven evaluator, classifier, and transport behavior from the +reference branch; do not port `RealmSandboxService`. The vertical fixture must +prove, in all three tiers: + +1. text, tracked state, one action, scoped CSS, and asset loading; +2. a trusted Base field/component rendered as a native Host portal; +3. a yielded block, helper, and safe modifier through explicit managers; +4. FieldDef/CardDef/FileDef invocation records and field configuration; +5. contained and linked relationships plus delegated nested rendering; +6. `surfacePresentation`, intrinsic and allocated `surfaceLayout`, readiness, + and observation; and +7. isolated, embedded, fitted, atom, head, edit, and Markdown fallback behavior + sufficient to prove format routing and composition. + +The Sandbox path must additionally prove origin isolation, a transferred +private port, a persistent compatible child, an immediate non-interactive +prerender placeholder, and zero fallback to Direct after child failure. +Semantic and transport protocol versions remain separate. + +Support explicit `prefersFullSandbox` and classifier-required browser globals. +Classification remains module-based: if one executable module imports a +browser-global dependency, every format defined by that module is +Sandbox-classified. Authors split safe and browser-dependent formats into +separate modules to recover Capsule execution. Safe atom, head, fitted, or +Markdown formats must not pay for an iframe merely because another module in +the same card family requires one. + +Special-case field/component shims are accepted only as temporary compatibility +adapters around canonical records. Phase 2 exits only when tier diagnostics +prove the intended route, visible output and the authored action agree across +Direct/Capsule/Sandbox, nested delegated rendering crosses mixed boundaries, +and neither untrusted adapter receives a live Store, Loader, service, CardDef +instance, or ambient Host object. + +The cumulative Phase 1–2 target is approximately 6,000–9,000 production lines +plus focused protocol, security, visual, and interaction tests. This is the +first milestone called a useful new execution runtime; it is not yet full +editing or HMR parity. + +#### Phase 2 implementation ledger + +Phase 2 is implemented on `codex/boxel-execution-runtime-architecture` as one +vertical system, not as three independent prototypes: + +1. **Rendering is wired through the Host boundary.** Ordinary card render + entry points use `BoxelExecutionRenderer`, which asks `BoxelExecutionService` + for one execution session and mounts the selected Direct, Capsule, or + Sandbox-owned render slot. Indexed HTML is an inert, non-interactive loading + placeholder; it is never treated as the live renderer. +2. **Capsule is end to end.** Authored modules evaluate in retained SES + compartments without browser globals, produce cloneable type and instance + records, retain authored component state, and emit only explicitly granted + effects. Trusted Glimmer managers render the resulting component and scoped + styles in the Host document. +3. **Sandbox is end to end.** Browser-dependent formats run in an + origin-isolated iframe on the configured Sandbox origin: credentialless in + supporting browsers, and an `allow-scripts`-only opaque origin in Safari and + Firefox. A + transferred private `MessageChannel` carries versioned render and Surface + messages. Authored module fetches are Host-brokered, GET-only, recursively + admitted from literal imports, bounded in size, and cancelled when the + process is destroyed. +4. **Composition crosses mixed tiers.** `BoxelFieldPortal` is the Host-owned + invocation capability for nested authored FieldDefs, CardDefs, and FileDefs. + It recursively routes contained and linked Boxels through the same engine; + trusted Base fields stay native. Child process teardown does not invalidate + its Capsule parent or the Direct Base runtime. +5. **Format routing is explicit.** Isolated, embedded, and edit may use the + iframe Sandbox when the module requires browser authority. Fitted, atom, + head, and Markdown stay Capsule-rendered so compact composition never + creates inline iframes. Missing authored formats use a Host-owned Base + fallback over the same record instead of executing authored code Direct. +6. **One contract is exercised in all three tiers.** Acceptance coverage sends + the same resource and render request through Direct, Capsule, and Sandbox and + verifies the selected owner and cloneable render-record shape. Layered + integration coverage adds getters and `computeVia`, field configuration, + linked snapshots, scoped CSS, retained actions, nested mixed boundaries, + Surface presentation/layout/observation, and the seven format decisions. +7. **Security and lifecycle are evidence, not assumptions.** Tests cover ambient + browser denial in Capsule, stylesheet confinement, exact recursive Sandbox + module authority, header stripping, origin and protocol validation, + browser-negotiated iframe isolation, last-known-good retention, + deterministic Surface release, iframe removal, and runtime eviction. No + runtime request or record contains a live Store, Loader, service, CardDef + instance, or Host DOM object. + +`data-boxel-execution` on the mounted slot is a temporary development and test +diagnostic (`direct`, `capsule`, `sandbox`, or `prerender`), not a card-author +API. The runtime, not URL state or authored input, remains authoritative. Full +mutation parity, source volatility, and HMR intentionally begin in Phase 3 and +Phase 4. + +### Phase 3: make editing canonical across every adapter + +Implement the companion mutation protocol: write grants, +`serializeCardPatch()`, edit-session/generation identity, optimistic overlays, +structured rejection, and matching server/index acknowledgements. + +Test primitive, compound, contained, contained-many, linked, and linked-many +edits. Test read-only denial, side-loaded-data pruning, relationship +replacement, stale revision rejection, save/reload identity, and no transient +read-only flash. Direct, Capsule, and Sandbox use the same request/result +semantics; no tier receives a Store-write shortcut. + +### Phase 4: restore preview-speed behavior on stable identities + +Add module volatility, source generations, source-hash classification and +transpilation caches, last-known-good output, persistent render islands, and +server acknowledgement handling. + +Monaco, AI patches, Boxel CLI writes, and other out-of-band module updates enter +the same generation coordinator. Compatible CSS/template updates preserve the +render slot and authored DOM; an incompatible generation or explicit Reload +Card remounts deliberately. Matching SSE/index events acknowledge current state +and never restore an older generation. + +Cover rapid file navigation, format switching, new/broken GTS recovery, +prerender handoff, and Code/Interact parity without making file trees or Monaco +wait for preview execution. + +### Phase 5: add further `surface*` capabilities one at a time + +Only after Capsule and Sandbox both exercise the shared capability broker, add +viewport, playback, canvas, transitions, focus/pointer coordination, clipboard, +slots, scheduling, and other capabilities required by the cumulative suite. + +Each capability requires: + +1. public Boxel UI types and author semantics; +2. request, response, and notification schemas; +3. Direct, Capsule, and Sandbox adapters where meaningful; +4. grant, rate-limit, user-activation, lifetime, and cleanup rules; and +5. positive cross-tier behavior tests and negative authority tests. + +This avoids designing a large Surface API without real boundary pressure while +also avoiding one-off iframe bridges. + +### Phase 6: add Host-owned BXL authorization projection + +Implement `BoxelAuthorizationService` using the projection slot reserved in +Phase 1. Initially it may consume an explicitly linked policy card and bounded +finite snapshot matching the clinical-access example: + +1. the server supplies the authoritative upper bound and independently + enforces every operation; +2. the Host resolves the policy card and evaluates client BXL only over data + the principal is already allowed to receive; +3. local decisions can only reduce the server upper bound; +4. `buildBoxelRenderRecord()` omits denied values and slots before rendering; +5. Direct, Capsule, Sandbox, inspector, query, and AI consumers receive the + same frozen projection; and +6. policy/membership/input changes invalidate only affected projection keys. + +Test nested usersets, `via(...)`, capability composition, request inputs, +resource-scoped grants, explicit refusal, stale policy revisions, and failure. +Authorization is intentionally not a prerequisite for proving basic sandbox +parity, but sensitive BXL-governed applications cannot ship until this phase is +complete. + +### Phase 7: migrate all consumers and delete duplicates + +Migrate: + +- Interact card rendering; +- Code preview and Monaco HMR; +- inspector/schema/type presentation; +- trusted Base default templates; +- Rich Markdown and delegated rendering; +- format preview and fitted galleries; +- AI/tool schema sharing; +- indexing/prerender and Host-mode rehydration. + +Delete a legacy path only when all consumers use the canonical interface, the +conformance matrix is green, and the frozen preview comparison shows no +unexplained visual or interactive regression. Likely deletion targets include +duplicated snapshot builders, stored custom-format booleans, direct constructor +introspection, generated component-class identity as an invalidation signal, +and per-feature static-property tunnels superseded by resolved semantics. + +The full parity target is approximately 9,000–14,000 production lines and +8,000–12,000 focused test lines across 40–70 files. If the implementation grows +toward the POC's blast radius, stop and identify which owner has started +absorbing unrelated policy, rendering, mutation, HMR, or capability state. + +### Validation basis + +A separate working audit pressure-tested the architecture against real Boxel +Labs programs and realm applications. It initially found strong ownership for +eleven of twenty-three mechanism families, partial ownership for six, and no +deterministic owner for six. The gaps were concentrated at the execution +boundary: Surface modes/focus, inline-versus-lifted editing, typed placement, +structured Table/Cell mechanics, scheduling and collaboration, and +clipboard/haptics/view-transition behavior. Canvas/Scene, BXL mutation, +Host-tool authority, and asynchronous AI were only partial. + +The design below assigns every one of those mechanisms to the single +cumulative suite. That closes a planning gap, not an evidence gap: a mechanism +is implemented only when its deterministic assertions pass in every applicable +tier and in the final nested graph. + +### One cumulative twelve-case acceptance suite + +This is one suite, not a primary suite plus add-ons. It grows one independent +music-release graph. Each case reuses actual Boxels from earlier cases, and the +final multimedia timeline composes them across Direct, Capsule, Sandbox, Store, +Surface, query, mutation, media, and asynchronous command boundaries. + +| Case | Fixture growth | Mechanisms that must be proved | +| ---: | --------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 1 | Release identity | primitive/Base fields, dates, percent, getters/computeVia, cardInfo, title/header/wide hint, isolated/embedded/fitted/atom/head/markdown/edit | +| 2 | CatalogMetadata, Guide, Credit, Price, VenueAddress | FieldDef identity, contains/containsMany, nested paths, configuration, Guide/JQXL cascade, currency symbol, country options, writable compound values | +| 3 | DeluxeRelease, Theme, BrandGuide, cover assets | inheritance, enum variants, tags, theme/CSS variables/scoped CSS, brand tokens and examples, ImageDef/URL fallback, polymorphic images | +| 4 | MusicPlayer and Track | audio FileDef, actions/tracked state, `surfacePlayback`, loading/error/cleanup, actual reusable player identity | +| 5 | Playlist and linked/query tracks | linksTo/linksToMany, relationship states, search projection, explicit Partner grant, nested Direct↔Capsule routing | +| 6 | Rich LinerNotes | RichMarkdown, Mermaid, CodeMirror, recursive atom/embedded/fitted rendering, sanitization, Layout/Run composition, modes, keyboard/focus, accessories | +| 7 | ReleaseEditor | custom/default edit, Guide-driven form behavior, primitive/compound/contained/linked writes, validation, optimistic state, canonical PATCH, stale/rejected saves, no read-only flash | +| 8 | Campaign PosterBoard and Frames | PosterBoard/Frame identity, paths and context; modes/focus/selection; inline/lift; x/y/w/h; pointer/keyboard/paste placement; denial, ghost, FLIP, transitions | +| 9 | MerchArtifact, Canvas graph, Sandbox Scene | split safe/browser modules, Three.js/3MF, iframe origin/protocol/height/prerender, Canvas nodes/edges/reconnect/minimap, Scene camera/effects, teardown | +| 10 | ReleasePlanningSheet and BXL access policy | concrete music-release rows, complete query filters/sorts/pagination, nested membership and resource-scoped BXL projection, explicit refusal, narrow Store grants/revocation, Table/Cell identity, pinned/resized columns, typed cell editors, selection, lifted editor | +| 11 | Live production, approval Policy, async AI, volatile code | field-scoped CRUD/Yjs/Command/frozen ownership, Actor-attributed Annotation evidence, awareness, AI snapshot CAS, publication clocks, BXL paths, Host-tool grants, HMR/LKG, image generation, retry/timeout/acknowledgement | +| 12 | Multimedia production timeline | actual player, notes, playlist, PosterBoard, Table, governed Release Approval Room, Annotation, generated assets, Canvas and 3D Scene; nested coordination, playback/viewport, lifted edit, revocation, failure isolation, cleanup | + +Every case requires five kinds of proof: + +1. **Semantic** — correct values, identities, query membership, formats, and + mutation payloads. +2. **Visual** — visible text/images/icons, CSS variables, background handoff, + responsive geometry, loading/error/empty states, and no raw JSON or blank + placeholder accepted as success. +3. **Interactive** — focus, typing, selection, pointer/keyboard placement, + playback, pan/zoom, commands, progress, save, retry, and cancellation. +4. **Boundary** — expected Direct/Capsule/Sandbox routing, bounded grants, and + no leaked constructors, functions, services, Store, credentials, events, or + DOM nodes. +5. **Lifecycle** — stable render slots, generation acknowledgement, + last-known-good behavior, no unnecessary remount, and complete release of + styles, listeners, observers, ports, handles, media, timers, and WebGL. + +CI uses deterministic providers, clocks, completion ordering, ids, and test +realms. External network availability is not part of acceptance. The async AI +test still runs the real Realm Script schema/limits, command authorization, +binary persistence, ImageDef linking, Store updates, acknowledgement state +machine, and visual progressive-results path. + +### Acceptance-test matrix + +Tests should generate a cross-product rather than rely only on hand-picked +examples. + +| Axis | Required cases | +| ------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Tier | Direct, Capsule, Sandbox where compatible | +| Definition | CardDef, FieldDef, FileDef | +| Format | isolated, embedded, fitted, edit, atom, head, markdown, unknown custom format | +| Fields | primitive, contains, containsMany, linksTo, linksToMany, polymorphic | +| Behavior | getter, computeVia, resolved configuration, action, async update | +| Presentation | title, icon, header color, wide hint, theme, thumbnail, scoped CSS | +| Composition | trusted component, authored component, yielded block, Rich Markdown embed | +| Mutation | writable, read-only, stale, side-loaded input, relationship update | +| Guide | base/domain/realm/inline cascade, JQXL, visibility, constraints, defaults, field order | +| Authorization | server upper bound, linked BXL policy, nested membership, `via(...)`, resource scope, request input, explicit refusal, redacted UI graph, operation reauthorization | +| Collaboration | Actor, Annotation target/anchor/body/state, reply/resolve, workflow step; field-scoped ordinary/Yjs/Command/frozen ownership, ephemeral awareness, Policy term and Command receipts | +| Surface | RichMarkdown Layout/Run, Table/Cell, PosterBoard/Frame, focus, lift, Canvas, Scene | +| Async work | BXL, Realm Script, command, progress, partial result, cancel, retry, acknowledgement | +| Lifecycle | cold load, format switch, HMR, server acknowledgement, failure, reload, teardown | + +Each semantic fixture declares: + +1. its semantic owner; +2. its boundary representation; +3. its Direct/Capsule/Sandbox consumers; +4. its expected DOM/state/effect result; +5. its security-negative assertions. + +The tests compare observable Boxel behavior, not private implementation +objects or byte-identical DOM where Glimmer legitimately differs. + +### Performance plan + +Measure before assigning absolute budgets. Required metrics include: + +- cold and warm module evaluation time by tier; +- time to first prerendered content and time to interactive content; +- semantic projection and field-configuration time; +- Glimmer initial render and update duration; +- action round-trip duration; +- component and DOM remount counts; +- format-switch latency; +- active runtime, instance, component, template bundle, style, and subscription + counts; +- memory after long cross-realm navigation and after idle eviction. + +Expected shape: + +- Direct remains the lowest-overhead baseline. +- Capsule pays one per-principal startup cost, then uses module and captured + template-bundle caches plus synchronous same-thread handle calls. +- Trusted Base remains one shared Host graph rather than one Base/Ember graph + per principal. +- State updates batch changed cells rather than invoking one boundary call per + template property. +- Sandbox pays child startup and transport costs but presents prerendered HTML + immediately and retains the child across compatible format/source updates. + +### Security review checklist + +- Does any authored constructor or function execute in the Host? +- Can any captured reference resolve outside the audited component/helper/ + modifier vocabulary? +- Can a trusted component receive an authored callback, owner, service, Store, + Loader, DOM node, event, or Glimmer tag? +- Can a record or guessed identifier expand Store authority? +- Can DOM/CSS values create network requests outside policy? +- Are yielded values, event records, action results, and configuration bounded + and cloned? +- Does every capability recheck execution identity, grant, card, format, + generation, and lifetime? +- Can Realm Script select a provider/model, read credentials, write during + preview, exceed its operation/byte/time budgets, or return executable data? +- Can cancellation, retry, duplicate completion, or a stale index + acknowledgement overwrite a newer or already durable async result? +- Does teardown release handles, observers, ports, object URLs, styles, and + timers? +- Does a protocol mismatch retain last-known-good output and fail closed? + +### Completion criteria + +The architecture is complete when: + +- all Host consumers use `BoxelRuntime` and the selected tier's render + adapter rather than authored constructor introspection; +- Direct and Capsule share Host Glimmer without sharing authored executable + objects; +- trusted Base components execute normally through generic portals; +- Sandbox uses the same semantic contract through an explicit client/server + transport; +- field configuration, computed values, relationships, cardInfo, media, and + mutations match Direct behavior in every compatible tier; +- compatible HMR updates preserve component and DOM identity; +- incompatible or failed updates preserve last-known-good UI; +- asynchronous commands preserve durable partial results and canonical Run/Job + state across cancellation, retry, unmount, and delayed acknowledgement; +- all twelve cumulative cases and the generated semantic cross-product are + green, with the fifty-example inventory mapped to passing owners or an + explicit unsupported/denied decision; +- performance and long-navigation memory results are recorded and acceptable; +- duplicate projection, introspection, and render-selection paths are removed. + +At that point, adding a Boxel semantic is mechanically incomplete until it +declares an owner, boundary representation, runtime consumers, Glimmer +behavior, and Direct/Capsule/Sandbox conformance proof. diff --git a/docs/boxel-execution-runtime-authorization-projection.md b/docs/boxel-execution-runtime-authorization-projection.md new file mode 100644 index 00000000000..d5e700008d3 --- /dev/null +++ b/docs/boxel-execution-runtime-authorization-projection.md @@ -0,0 +1,353 @@ +# Boxel execution runtime: BXL authorization projection + +## Status and scope + +This is a proposed companion contract for the Boxel execution runtime. Boxel +does not yet have this complete authorization layer. The purpose of the design +is to ensure that Direct, Capsule, and Sandbox rendering can consume the same +authorization result without confusing client-side UI reduction with the +server security boundary. + +In this document, **Boxel means Box Element**: a CardDef, FieldDef, FileDef, or +future compatible visual building block. Realm remains the server-side data +and module location. + +The proposal covers: + +- linking or otherwise selecting a BXL authorization policy for a resource; +- evaluating a finite, bounded authorization snapshot; +- projecting the Boxel graph to the fields, relationships, query results, + formats, sections, and Commands a principal may use; +- transporting the result across Direct, Capsule, and Sandbox boundaries; and +- preserving independent server enforcement for all reads and effects. + +It does not define authentication, credential storage, organization policy +administration, or the complete mutation protocol. + +## Evidence from BXL Clinical Access + +The local clinical example currently performs four operations inside +`patient-dashboard.gts`: + +1. resolve a linked `ClinicalAccessPolicy`, facility, patient resource, + principals, and nested team membership; +2. build a finite authorization snapshot; +3. run `prepareBxlAuthorizationSafe()` and `listCapabilities()` for the viewer, + resource, and request input; and +4. hand-build a dashboard projection whose denied values are `undefined` and + whose actions are capability booleans. + +The example demonstrates semantics the shared service must preserve: + +- nested usersets are relationship data and are expanded cycle-safely within + limits; +- authorization is resource-scoped: a care-team seat on one patient does not + grant another patient; +- `via(Resource.Facility; Capability.ViewOperationalContext)` delegates a + specific parent-resource capability rather than exposing the parent graph; +- capabilities can depend on other capabilities; +- request input can refine a result, such as break-glass plus incident ticket; + and +- explicit refusal is evaluated separately and wins after positive + eligibility, such as `Seat.Suspended` removing notes and mutations. + +Those are general BXL mechanics. The runtime must not hard-code clinical +capability names. + +## Security and UX invariants + +1. The Realm Server is the authoritative security boundary. +2. Client-side BXL evaluation can only reduce a server-authorized upper bound. +3. The server does not send secrets merely so the client can hide them. +4. Every fetch, search, relationship traversal, Command, and mutation is + independently authorized by the server. +5. Visibility is not mutation authority. An available Command is descriptive; + invocation still requires a Host capability and server authorization. +6. A policy card reference is configuration, not a grant. +7. No live Store, policy evaluator, membership service, component, callback, + credential, or unprojected resource crosses a Capsule or Sandbox boundary. +8. Explicit refusal wins over positive eligibility. +9. Missing, stale, failed, or protocol-incompatible authorization fails closed. +10. Denied data produces no value, child render slot, loading placeholder, + count, title, URL, menu item, diagnostic payload, or timing-distinct shell. +11. Direct, Capsule, Sandbox, inspector, query, delegated render, Rich + Markdown, fitted gallery, Code preview, and AI schema consumers use the + same projection. +12. A different execution tier cannot restore a semantic removed by + authorization projection. + +## Ownership + +| Concern | Owner | Boundary result | +| ------------------------------------- | ------------------------------------- | --------------------------------------------------------- | +| Authentication and principal identity | Host + Realm Server | stable principal id, never credentials | +| Authoritative data authorization | Realm Server | already-authorized resources and upper-bound decisions | +| BXL policy card and relationship data | Store + Realm Server | authorized policy reference/revision and bounded snapshot | +| Client BXL evaluation | Host `BoxelAuthorizationService` | frozen capability decisions | +| Boxel graph projection | Host render projection pipeline | redacted `BoxelRenderRecord` | +| Visual rendering | Direct/Capsule/Sandbox adapter | only projected data and descriptive decisions | +| Commands and mutation | Host capability broker + Realm Server | admitted result/receipt or bounded refusal | + +The service is deliberately not part of `SurfaceService`. Surface authority is +scoped to a mounted visual root. Authorization is scoped to principal, +resource, policy, data revision, and operation. + +## Policy binding + +A resource or owning application may link a policy card. The initial design +can support an explicit relationship such as `authorizationPolicy`; later it +may also support policy selection from workspace/application configuration or +server policy. The binding rules must be deterministic and inspectable. + +The rendered card does not choose a weaker policy. The Host resolves the +effective binding and the server-authorized policy revision. A card that is +itself an authorized policy editor may receive the policy card as ordinary +projected data, but merely rendering a governed resource does not expose the +policy document or membership graph. + +## Host service contract + +The Host service accepts an internal request and returns a cloneable +projection. Names are aligned with the execution runtime's `Boxel*` semantic +vocabulary. + +```ts +export interface BoxelAuthorizationRequest { + policy: CardIdentifier; + principal: string; + resource: CardIdentifier; + resourceRevision: string; + requestedCapabilities: string[]; + input: Record; +} + +export interface BoxelCapabilityDecision { + effect: 'allow' | 'refuse' | 'not-applicable'; + reasonCode?: string; +} + +export interface BoxelAuthorizationProjection { + protocolVersion: number; + policy: CardIdentifier; + policyRevision: string; + principal: string; + resource: CardIdentifier; + resourceRevision: string; + inputHash: string; + capabilities: Record; + visibleFields: string[]; + visibleRelationships: string[]; + visibleFormats: string[]; + visibleSections: string[]; + availableCommands: string[]; +} + +export interface BoxelAuthorizationService { + project( + request: BoxelAuthorizationRequest, + ): Promise; +} +``` + +`requestedCapabilities` keeps evaluation bounded and makes dependencies +observable. `inputHash` identifies the exact request-input state without +echoing potentially sensitive inputs across all consumers. Bounded reason +codes support safe UI explanations; full policy traces and membership paths +remain server/Host diagnostics unless separately authorized. + +The projection is descriptive. It contains no `can()` closure and no method +that can grant authority. Custom authored formats may read the frozen decision +record from render context, while generic Base templates consume the concrete +visible field, relationship, format, section, and Command lists. + +## Evaluation and intersection + +```text +Realm Server + authenticates principal + authorizes resource/policy/snapshot reads + returns an upper-bound authorization envelope + | + v +BoxelAuthorizationService + resolves effective policy revision + evaluates BXL over an already-authorized finite snapshot + applies nested membership, via, composition, input, refusal + intersects local decisions with the server upper bound + | + v +buildBoxelRenderRecord() + removes denied values and render slots + records descriptive capability decisions + | + v +Direct / Capsule / Sandbox / non-render consumers +``` + +Two production implementations are valid: + +- the server computes the complete projection and the client validates and + consumes it; or +- the server supplies an upper bound and an authorized finite snapshot, and + the Host evaluates BXL synchronously for responsive UI changes. + +In both cases the client result is an intersection. A forged or buggy local +`allow` cannot widen the server envelope. The second model enables interactions +like changing a viewer or local request input without a network round trip, +but is a UX optimization rather than enforcement. + +## Projecting the Boxel graph + +Authorization must run before the canonical render record is assembled: + +```ts +buildBoxelRenderRecord({ + canonicalDocument, + boxelType, + projectedValue, + relationshipProjection, + authorizationProjection, + cardInfoProjection, + presentationProjection, + policy, +}); +``` + +The projection controls these semantic domains: + +| Domain | Denied behavior | +| ------------ | ------------------------------------------------------------------- | +| field | value and FieldDef slot absent; no placeholder or validation detail | +| relationship | identifier, resolved child, and traversal affordance absent | +| query | unauthorized rows excluded before count/sort/page presentation | +| format | format unavailable; trusted fallback only if separately authorized | +| section | custom presentation section absent, not CSS-hidden | +| Guide | label/help/control/constraint visible only for remaining semantics | +| menu | item absent; no inert control that reveals a capability name | +| Command | affordance absent, while direct invocation remains server-denied | +| schema/AI | unauthorized field and relationship descriptions omitted | + +Projection needs a deterministic mapping from capabilities to these domains. +That mapping may be declared by trusted Base/schema metadata, the linked +policy/application contract, or a future Guide integration. It cannot be an +unvalidated callback copied from authored code into the Host. + +## Execution-boundary behavior + +### Direct + +Trusted code receives the same projected model and frozen authorization +record. Direct execution does not bypass the projection pipeline. + +### Capsule + +The Capsule receives the redacted `BoxelRenderRecord` and descriptive decision +record. It cannot load omitted data through a trusted Base portal: every nested +render and data request returns to the Host router with the same execution and +authorization identity. + +### Sandbox + +The iframe receives the same redacted records over the versioned protocol. +The policy evaluator, source snapshot, Store, credentials, and denied data +remain outside. A child request for a nested Boxel re-enters the Host router; +guessing an id or URL does not expand the projection. + +## Reactivity, caching, and last-known-good behavior + +Projection cache identity includes: + +- principal; +- resource id and revision; +- policy id and revision; +- server upper-bound revision; +- finite-snapshot revision; +- request-input hash; and +- requested capability set. + +Policy, membership, resource-state, principal, or request-input changes +invalidate only affected authorization keys. The Host diffs old and new +projections and preserves unrelated Table/Cell, field, relationship, and child +render-slot identity. + +An authorization failure must never fall back to an earlier broader +projection. The only reusable last-known-good UI is a non-sensitive shell whose +contents are independently safe under the new upper bound. When that cannot be +proved, the Host shows a generic denied/unavailable state. + +## Mutation interaction + +Authorization projection and mutation admission are related but distinct: + +```text +availableCommands / writable field paths + | + v + visible affordance + | + user requests operation + v +Host capability broker checks execution identity + current projection + | + v +Realm Server reauthorizes against current data/policy revisions + | + v +admitted PATCH/Command receipt or bounded refusal +``` + +A projected writable field does not make the rendered projection itself +writable. The mutation pipeline still serializes only named canonical changes, +rejects computed/presentation/side-loaded data, and detects stale revisions. +Revocation between display and invocation must be safe and ordinary. + +## Acceptance coverage + +The cumulative music-release suite applies the clinical mechanics to a +non-clinical domain through `ReleaseAccessPolicy` and +`ReleasePlanningSheet`. It must prove: + +- artist, nested release team, rights team, finance, guest, and suspended + collaborator views; +- one resource grant does not authorize a sibling release; +- nested membership and `via(Resource.Label; ...)` are bounded; +- explicit refusal removes internal notes and mutations after positive team + eligibility; +- request-input changes update only dependent semantics; +- denied rows do not influence counts, totals, sort, pagination, or timing; +- Direct, Capsule, Sandbox, inspector, query, and AI schema projections agree; +- forged allows and direct Command attempts fail; +- server revocation wins over cached client state; and +- unaffected DOM/render-slot identity survives a projection update. + +Unit tests cover record codecs, hashing, version rejection, intersection, +refusal precedence, and redaction. Adapter conformance tests run the same +projection through every execution tier. Browser tests assert visible output, +absence of leaks, interactions, targeted updates, and revocation behavior. + +## Implementation sequence + +1. Define the versioned request, upper-bound, decision, and projection records. +2. Add a test-only linked BXL policy binding to the music fixture graph. +3. Implement `BoxelAuthorizationService` over a bounded authorized snapshot. +4. Intersect local decisions with a server upper-bound fixture. +5. Insert authorization before `buildBoxelRenderRecord()` materialization. +6. Feed the same frozen result to Direct, Capsule, Sandbox, inspector, query, + Code preview, delegated rendering, and AI schema consumers. +7. Route every operation through the existing/future Host capability broker + and authoritative server check. +8. Add targeted invalidation and safe non-sensitive failure shells. +9. Add the cross-tier and browser assertions above. +10. Replace application-local authorization projection code only after parity + is proven. + +## Open design decisions + +- exact policy-binding precedence across resource relationship, application, + workspace, and server policy; +- whether the server returns a fully computed projection, an upper bound plus + finite snapshot, or both; +- the declarative capability-to-field/relationship/format/section mapping; +- safe, localized explanation records versus privileged audit traces; and +- whether authorization projection gets its own revision stream or is folded + into the canonical Store document/change acknowledgement protocol. diff --git a/docs/boxel-execution-runtime-cold-start-baseline.md b/docs/boxel-execution-runtime-cold-start-baseline.md new file mode 100644 index 00000000000..0198d4bdc2b --- /dev/null +++ b/docs/boxel-execution-runtime-cold-start-baseline.md @@ -0,0 +1,258 @@ +# Boxel execution runtime cold-start baseline + +Status: measurement baseline, 2026-08-09. No runtime caching changes were made +as part of this measurement. + +## Purpose + +This baseline separates three costs that are easy to conflate when evaluating +Direct, Capsule, and Sandbox execution: + +1. common Host, authentication, Store, and application startup; +2. module retrieval, transformation, and evaluation; +3. Sandbox child-document startup and interactive handoff. + +The distinction matters because the Deck backport into Realm Server may change +module addressing, cache headers, and transpilation behavior. We should measure +that backport before adding another Host-owned artifact cache. + +These numbers are diagnostic, not service-level objectives. They were captured +against the local Vite development Host at `https://localhost:4219`, using the +staging compatibility corpus and an already-warm browser asset cache. Local +Vite serves a large unbundled module graph, so raw Sandbox child startup is not +representative of a deployed production bundle. + +## Page-level observations + +Three fresh Host-document samples were collected for each execution tier. + +| Tier | Representative card | Observed result | +| ------- | ------------------------------ | ------------------------------------------------------------------------------------------- | +| Direct | Base Skill card | 5,730 ms, 4,753 ms, 5,041 ms; median **5,041 ms** | +| Capsule | compatibility-corpus workspace | 5,453 ms, 4,767 ms, 5,011 ms; median **5,011 ms** | +| Sandbox | Browser Canvas | 6,252 ms and 7,159 ms successful; one run had not reached interactive readiness at 8,789 ms | + +Direct and Capsule cold-document time is effectively identical at this level. +The common Host startup dominates, so these samples do not support adding a +Capsule-specific cache merely to improve initial page navigation. + +## Sandbox critical path + +One Browser Canvas startup was decomposed using the runtime's lifecycle logs: + +| Segment | Approximate duration | +| ------------------------------------------ | -------------------: | +| Sandbox process creation to rendered child | 5.31 s | +| Child Vite/Ember application boot | 4.46 s | +| First module materialization request | 440 ms | +| First Glimmer render | 28 ms | + +The dominant local cost is starting the child application, not rendering the +card. Immutable module delivery can improve the materialization segment and +remove repeated network validation, but it cannot eliminate the approximately +4.5-second development iframe boot. If that cost remains material in a +production build, the appropriate levers are prebooted or retained Sandbox +processes and a smaller child runtime. + +## Current Base module delivery + +`https://realms-staging.stack.cards/base/card-api` currently returns a 146,859 +byte compiled module with 54 unique direct import specifiers. + +Five requests of each kind produced: + +| Request | Median TTFB | Median total | Response | +| ------------------ | ----------: | -----------: | ---------------------- | +| Full module | 71 ms | 113 ms | `200`, 146,859 bytes | +| Conditional module | 61 ms | 61 ms | `304`, zero body bytes | + +Relevant response headers were: + +```text +cache-control: public, max-age=0 +etag: 1786136959:module +x-boxel-cache: hit +x-boxel-canonical-path: https://cardstack.com/base/card-api.gts +``` + +Realm Server is already serving a transpilation-cache hit, but the browser must +revalidate the module. A roughly 60 ms conditional round trip can compound over +module-graph depth and across independently owned Capsule and Sandbox loaders. + +## Deck interaction and cache correctness + +Deck's immutable-address work is likely to improve repeated Base and published +module retrieval, but its cache classes must remain precise: + +- raw bytes at an exact published Version may use a one-year immutable cache; +- live or movable addresses remain non-immutable; +- compiled GTS/TS output depends on the compiler and must carry a build key or + equivalent compiler identity, plus a short TTL/conditional response rather + than inheriting raw-source immutability. + +Only inert bytes and pure compilation artifacts may be shared across execution +principals. Evaluated exports, Store/CardDef instances, SES values, grants, +services, DOM nodes, and MessagePorts remain runtime-local. + +## Decision before implementation + +Do not add a Host AMD/module-artifact cache until the Realm Server Deck +backport lands and this baseline is repeated. After the backport, measure: + +1. full and conditional Base module fetch latency and cache headers; +2. Sandbox process creation to child-ready; +3. child-ready to module materialized; +4. module materialized to first render; +5. a second Sandbox surface using the same immutable Base graph; +6. a second principal using the same bytes, verifying that only inert artifacts + are shared; +7. local development and production-preview builds separately. + +If module materialization falls from roughly 440 ms toward 100 ms and repeat +requests disappear, Realm Server delivery is doing the useful caching and a +second Host cache would add complexity without enough benefit. If the module +segment remains large, a Host cache should be content/build-key addressed and +store only inert transformed artifacts. If child boot remains dominant, work +on Sandbox process lifecycle instead of module caching. + +## Compatibility context for the measurement + +Performance numbers are useful only when the same runtime is still producing +correct cards. Three real-browser checks were run against staging/main and the +local branch after recording the cold-start samples: + +- the 35-boundary format gauntlet passed (`FormatPreviewBatchOne/sample`), at + 3,693 ms on staging and 7,604 ms locally; +- a two-cycle, same-document navigation soak passed across Primitive Profile, + Nested Field Host, Rich Markdown, Browser Canvas, Computed Flight Plan, and + Poster Board; every close left zero Sandbox iframes and zero loading + affordances, while the second cycle had zero net DOM or style growth; +- an additional ten-card mechanism cohort passed on both origins, including + linked and recursive graphs, cardInfo projection, editable/tracked UI, + native video, Leaflet, Three.js/3MF, and native popovers. + +For the extended cohort, the local Capsule median was 2,842 ms across six +cards. The local Sandbox median was 4,608 ms across four cards, with a 1,613 ms +median from prerender readiness to interactive child handoff. The matching +staging buckets were 2,162 ms and 2,129 ms. These are navigation observations, +not isolated compiler benchmarks, but they preserve an important constraint: +future cache work must improve these timings without weakening the same +semantic, DOM-primitive, execution-tier, lifecycle, and teardown assertions. + +## Focused Capsule parser optimization — 2026-08-11 + +This is a focused, correctness-qualified comparison for performance-plan item #1, +not completion of the full Phase 0 corpus. Chrome DevTools MCP drove five pre-change +and five post-change document loads of +`Release/opening-night` against the authenticated staging-backed development Host at +`https://host.codex-execution-runtime.localhost`. Every admitted sample rendered all +five declared semantic signatures, selected only Capsule, retained zero iframes, and +reported zero dropped instrumentation records. + +| Metric | Before median / p95 | After median / p95 | Interpretation | +| ------------------------------ | ------------------: | -----------------: | --------------------------------------------------------- | +| Full document navigation | 6,119 / 7,849 ms | 6,314 / 8,306 ms | +3.2% median; inside the 5% guardrail | +| Host request construction | 235 / 379 ms | 356 / 520 ms | +51%; direct evidence of non-comparable Host/staging load | +| Root Capsule render record | 587 / 687 ms | 656 / 725 ms | +11.7%; inconclusive under the common variance above | +| DOM nodes at readiness | 893–902 | 893–902 | no growth | +| Live iframes / dropped records | 0 / 0 | 0 / 0 | unchanged | + +The matching DevTools traces reinforce the variance diagnosis rather than a +page-level regression claim: the pre-change trace observed 204 ms TTFB and 11.35 s +LCP, while the post-change trace hit 6.42 s TTFB and 35.30 s LCP. Render-blocking +insights estimated 0 ms savings in the pre-change trace. These development-load LCP +values are not used to assess the local parser change. + +The focused benchmark compares the removed implementation with the captured parser +in one process, alternating order across nine rounds of 250 clones. The payload is +2,412 bytes and includes HTML-comment tokens, Mermaid arrows, and JavaScript line +separators. + +| Boundary clone | Median per clone | p95 batch (250) | +| --------------------------------- | ---------------: | --------------: | +| Per-read `Compartment.evaluate` | 55.81 µs | 19.99 ms | +| Captured compartment `JSON.parse` | 14.38 µs | 3.85 ms | + +The captured parser is **3.88× faster**, a **74.2% reduction**. Two further +independent runs measured 3.99–4.22× and 74.9–76.3%, supporting the same result. +Reproduce with +`pnpm --dir packages/host bench:execution-runtime-clone`. + +Raw root-operation samples, in milliseconds: + +```text +before render-record: 541.2, 678.4, 587.3, 511.0, 686.8 +before navigation: 5076.3, 7528.3, 7848.5, 6119.0, 6021.3 +after render-record: 534.0, 689.2, 572.3, 655.9, 724.5 +after navigation: 6045.0, 5515.7, 7935.2, 8305.9, 6313.5 +``` + +The new JSON-text boundary test passes in the clean prebuilt Host runner. The broader +render-record parity filter could not run because its required local Base realm at +`https://localhost:4201` was not started; the authenticated real Release card +nevertheless preserved the parity signatures exercised by this focused comparison. + +## Runtime simplification batch — 2026-08-11 + +This follow-up retains three small, ownership-preserving changes: the Capsule render +projection is cloned only by the shared render-record assembler; browser globals and +DOM-method signals are collected in one Babel pass; and Capsule CSS confinement +reuses the stylesheet parsed by validation. No tier, authority, occurrence, protocol, +or lifecycle rule changed. + +Focused alternating-order benchmarks compared the removed implementations with the +new paths and asserted output equality before timing: + +| Operation | Legacy | Retained | Result | +| ----------------------------------------------- | ----------: | ----------: | ----------------: | +| Classifier Babel traversal | 3,389.96 µs | 1,556.67 µs | **2.18×; −54.1%** | +| Render-record projection assembly | 216.86 µs | 151.42 µs | **1.43×; −30.2%** | +| ContentTag preprocessor construction | 55.59 µs | 56.63 µs | median-neutral | +| CSS validation + confinement (Chrome, 80 rules) | 1,320 µs | 902 µs | **1.46×; −31.7%** | + +Reproduce the Node cases with +`pnpm --dir packages/host bench:execution-runtime-simplifications`. The stylesheet +case used Chrome's native `CSSStyleSheet`, 50 operations per round, and nine +alternating rounds. + +Five warmed authenticated loads of `Release/opening-night` produced: + +```text +render-record: 623.4, 570.3, 581.0, 586.2, 1113.7 ms +root request: 413.8, 528.0, 408.8, 376.7, 580.0 ms +navigation: 22081.8, 20261.5, 19803.3, 21477.5, 20898.0 ms +``` + +| Metric | Previous retained run | Simplification batch | Interpretation | +| --------------------------------------- | --------------------: | -------------------: | --------------------------------------------- | +| Root Capsule render-record median / p95 | 655.9 / 724.5 ms | 586.2 / 1,113.7 ms | **−10.6% median**; p95 has one outlier | +| Root request median / p95 | 356 / 520 ms | 413.8 / 580 ms | +16.2% median; Host/staging variance worsened | +| Full navigation median / p95 | 6,313.5 / 8,305.9 ms | 20,898 / 22,081.8 ms | non-comparable current environment delay | + +All five samples rendered the five declared semantic signatures, selected Capsule +only, retained zero Sandbox iframes, and dropped zero instrumentation records. DOM +size was stable at 957 nodes throughout the batch. A navigation trace was attempted, +but the DevTools navigation timeout expired under the same Host/staging delay, so no +trace-level LCP claim is admitted. + +## Safe lifecycle batch — 2026-08-11 + +The next retained batch bounded the remaining fetch and Surface request tables, +made Surface observation subscriber-driven, acknowledged and stopped ordinary +post-paint diagnostics, and stabilized unchanged Capsule context projection. + +Chrome-native focused measurements found: + +| Work item | Before | After | +| ------------------------------------- | -----------------------: | --------------------: | +| 100,000 unchanged context projections | 3.1 ms / 100,000 facades | 0.2 ms / 1 facade | +| 1,000 accepted post-paint diagnostics | 1,000 measurements/posts | 0 | +| idle attached Surface | 2 observers + 1 box read | 0 observers / reads | +| silent fetch or Surface request | unbounded | bounded to 10 seconds | + +Three warmed authenticated `Release/opening-night` samples were semantic-parity +green, Capsule-only, with nine headings, zero iframes, zero dropped records, and +946–957 DOM nodes. Median readiness/root request/root render-record were +26.18 s / 241.2 ms / 612.9 ms. Their mixed movement versus the preceding run is +classified as Host/staging variance; the retained performance claim is the direct +work elimination above, not a page-level speedup. diff --git a/docs/boxel-execution-runtime-composition-suite.md b/docs/boxel-execution-runtime-composition-suite.md new file mode 100644 index 00000000000..b380cafe5cc --- /dev/null +++ b/docs/boxel-execution-runtime-composition-suite.md @@ -0,0 +1,1143 @@ +# Boxel execution runtime composition suite + +## Purpose + +This is the deterministic acceptance suite for the Boxel execution runtime. +It replaces a collection of isolated sandbox demos with one deliberately +interlinked graph that grows through twelve use cases. Each use case reuses +Boxels introduced earlier, adds one new semantic pressure, and proves both the +visual result and the behavior across Direct, Capsule, and Sandbox execution. + +In this document, **Boxel means Box Element**: a visually present, +interactive `BaseDef`-derived building block. It does not mean the Boxel +product as a whole. CardDef, FieldDef, FileDef, and future compatible visual +kinds are Boxels. A persisted Card document is only one kind of Boxel state. + +This suite is intentionally tighter than the exploratory compatibility corpus: + +- the corpus remains useful for discovery, random sampling, and soak testing; +- this suite is small enough to run in CI on every change; +- every fixture is crafted locally and deterministic; +- every later use case composes actual earlier Boxels rather than copying + their markup or behavior; +- visual and interactive expectations are part of correctness; +- each boundary crossing has an expected runtime route and authority; +- a passing placeholder, raw JSON dump, blank panel, or inert control is a + failure even when no exception was thrown. + +The architecture that this suite verifies is described in +[boxel-execution-runtime-architecture.md](boxel-execution-runtime-architecture.md). +The BXL projection and server-enforcement contract is specified in +[boxel-execution-runtime-authorization-projection.md](boxel-execution-runtime-authorization-projection.md). +Canonical edits and Commands are specified in +[boxel-execution-runtime-mutation-protocol.md](boxel-execution-runtime-mutation-protocol.md). +The broader inventory of current POC behavior is in +[boxel-execution-runtime-coverage-audit.md](boxel-execution-runtime-coverage-audit.md). +The fifty-example pressure test that selected the mechanisms in these twelve +cases is in +[boxel-execution-runtime-real-example-audit.md](boxel-execution-runtime-real-example-audit.md). + +## What this suite must prove + +The suite has seven non-negotiable outcomes: + +1. **One authored API.** A Boxel author uses normal Boxel Card API, Field API, + Glimmer, and approved `surface*` capabilities. The author does not write + MessageChannel, SES, iframe, or boundary-record code. +2. **Composition survives a graph, not merely a pair.** A Capsule parent may + delegate to a trusted Base field, which may render a linked Capsule card, + whose isolated view may contain a Sandbox child. Each nested render + re-enters the Host router without flattening identity or trust. +3. **Provenance selects execution.** A parent cannot weaken a child's + isolation. Trusted Base can remain Direct; ordinary user source runs in a + Capsule; browser-dependent source runs in a Sandbox. +4. **The Store remains canonical.** Execution runtimes receive bounded + representations. Writes are explicit Host capabilities, re-authorized and + committed through the Store. +5. **Visual parity is semantic parity.** Themes, images, CSS variables, + layout, height, accessible roles, focus, loading states, and formatted + values must match the Direct reference within declared tolerances. +6. **No partial unknown records.** Missing protocol features retain + last-known-good output and show a diagnostic. They never degrade silently + to `undefined`, blank UI, raw JSON, or the wrong format. +7. **Authorization reduces the graph before rendering.** A linked BXL policy + projects only usable fields, relationships, query results, formats, + sections, and Commands. The client can never widen the server-authorized + graph, and denied values never cross Direct/Capsule/Sandbox boundaries. + +## Fixture topology and provenance + +The suite uses three deterministic test realms and trusted Base fixtures. The +word Realm is reserved for server-side data/module location and authorization; +it does not name a Host execution service. + +| Source lane | Example URL root | Default execution | Purpose | +| ----------- | ----------------------------------------------------------- | ------------------------------------------- | ------------------------------------------------------------------------ | +| Official | `https://cardstack.com/base/` and test-only trusted modules | Direct | Base fields, trusted components, Host commands, standard format fallback | +| Studio | `https://studio.test/album/` | Capsule | Primary user-authored release workspace and most composition | +| Partner | `https://partner.test/licensed/` | Capsule after an explicit grant | Cross-Realm authorization, inherited user code, linked media | +| Lab | `https://lab.test/artifacts/` | Sandbox when browser capability is required | Three.js/WebGL and other document/window-dependent programs | + +The test Store gives Studio no implicit access to Partner. A test chooser adds +one explicit Partner card link and a narrow read grant. A later test revokes +the grant. Lab data is similarly projected; loading a Sandbox does not grant +search access to the whole Store. + +```mermaid +flowchart LR + B["Official Base\nDirect"] + S["Studio modules\nCapsule"] + P["Partner modules\nCapsule after grant"] + L["Lab browser modules\nSandbox"] + H["Host router + Store"] + + S --> H + P --> H + L <-->|"typed protocol"| H + H --> B + H --> S + H --> P + H --> L +``` + +Every rendered node records a test-only execution trace containing: + +- Boxel URL and type reference; +- source module generation and source hash; +- selected format; +- execution tier (`direct`, `capsule`, or `sandbox`); +- parent render-slot id and child render-slot ids; +- Store document revision; +- granted capabilities; +- component mount/unmount counts; and +- stylesheet acquisition/release counts. + +The trace is diagnostic evidence, not an author-facing API and not permission +to introspect live constructors. + +## The cumulative graph + +The domain is an independent music release. It is visually rich enough to +exercise Boxel as an interactive system while remaining deterministic and +small. The final use case is a multimedia production timeline that reuses the +actual player, notes, playlist, poster, image, query, and 3D artifact Boxels +introduced earlier. + +```mermaid +flowchart TD + U1["1 Release identity"] + U2["2 Catalog metadata + Guide"] + U3["3 Theme + brand guide"] + U4["4 Music player"] + U5["5 Playlist and relationships"] + U6["6 Rich liner notes"] + U7["7 Release editor and writes"] + U8["8 Campaign poster board"] + U9["9 3D merch artifact"] + U10["10 Release-planning spreadsheet"] + U11["11 Live production + Annotation"] + U12["12 Multimedia timeline"] + + U1 --> U2 --> U3 + U1 --> U4 + U3 --> U4 + U4 --> U5 + U4 --> U6 + U3 --> U6 + U2 --> U7 + U3 --> U7 + U4 --> U8 + U6 --> U8 + U4 --> U9 + U5 --> U10 + U9 --> U10 + U7 --> U11 + U10 --> U11 + U4 --> U12 + U5 --> U12 + U6 --> U12 + U8 --> U12 + U9 --> U12 + U10 --> U12 + U11 --> U12 +``` + +## Shared assertion vocabulary + +Each case declares five forms of evidence. A test is incomplete if it checks +only the first. + +| Evidence | What it proves | +| ----------- | ------------------------------------------------------------------------------------------------------------------------------------ | +| Semantic | Correct field values, computed values, relationship identity, query membership, format selection, and mutation payload | +| Visual | Required visible text, images, icons, formatted values, computed CSS variables, solid background handoff, and bounded geometry | +| Interactive | Focus, typing, selection, pointer input, playback, drag/resize, commands, or save behavior changes visible state | +| Boundary | Expected Direct/Capsule/Sandbox route, no leaked constructors/services/Store, and correct capability set | +| Lifecycle | Stable render-slot identity, loading/error/last-known-good behavior, HMR acknowledgement, cleanup, and no duplicate listeners/styles | + +High-value layouts use small screenshot regions in addition to DOM assertions. +Screenshot comparison is not used for rapidly changing browser chrome or font +antialiasing. The invariant manifest carries the durable visual contract: + +```ts +interface BoxelVisualExpectation { + format: string; + visibleText?: string[]; + roles?: Array<{ role: string; name?: string }>; + images?: Array<{ alt: string; loaded: boolean }>; + cssVariables?: Record; + geometry?: Array<{ + selector: string; + minWidth?: number; + minHeight?: number; + contains?: string; + }>; + interactive?: Array<'focusable' | 'editable' | 'draggable' | 'playable'>; +} +``` + +## Use case 1 — Release identity and primitive fields + +### Boxels introduced + +- `Release` CardDef in Studio. +- Official Base fields rendered through trusted Direct component portals. +- One `Release/opening-night.json` instance. + +### Semantics exercised + +- `@field` with `contains` for `StringField`, `NumberField`, `BooleanField`, + `BigIntegerField`, `TextAreaField`, `EmailField`, `UrlField`, + `PhoneNumberField`, `EthereumAddressField`, and `ColorField`. +- `DateField`, `DateTimeField`, `TimeField`, `DateRangeField`, and + `DateTimeStampField`, with exact date-versus-datetime JSON shapes. +- `PercentageField` formatting. +- `cardTitle`, `cardDescription`, `cardInfo`, `headerColor`, and + `prefersWideFormat` projection. +- `computeVia` for a short release status string and a chained ordinary getter. +- `isolated`, `embedded`, `fitted`, `atom`, `head`, `markdown`, and default + `edit` format selection. The format set is data, not a closed enum. + +### Visual and interactive contract + +The isolated card shows coverless release identity, release date, availability +status, completion percentage, contact link, and an explicit color swatch. +Embedded is a compact two-row identity block. Fitted responds at badge, strip, +tile, and card container sizes without overflow. Atom is one readable pill. +Head supplies only head metadata. Markdown is stable copy, not `[object Object]`. +Default edit renders appropriate trusted Base controls and remains writable. + +### Required assertions + +- Capsule author code never runs in Host. +- Every Base field component is Direct and sees the same field value/config as + the authored template would see without a boundary. +- `0`, `false`, empty string, null optional fields, and the BigInt transport + codec do not collapse into one another. +- Date formatting does not throw and a wrong Date/DateTime fixture is rejected + as a fixture error rather than becoming a blank preview. +- Header title is not `Untitled`; header color and wide-format preference match + the Direct reference. + +## Use case 2 — Catalog metadata FieldDef, configuration, and Guide + +### Boxels introduced + +- `CatalogMetadata` FieldDef contained by `Release`. +- `Credit` FieldDef used by `containsMany` with exactly two small entries. +- `Price` using `AmountWithCurrency` and linked `CurrencyField`. +- `VenueAddress` using `AddressField`, `CountryField`, and `CoordinateField`. +- `ReleaseGuide`, a data-authored Guide linked from `cardInfo.guide`. + +### Semantics exercised + +- FieldDef versus CardDef identity: contained metadata has no independent card + URL and does not cross the Store as a fake card document. +- `contains(FieldDef)`, `containsMany(FieldDef)`, nested field paths, and + delegated `<@fields.* />` rendering. +- Static field configuration, configuration callback using owner context, and + per-use configuration override. +- `ConfigurationInput`, resolved field-configuration merging, invalidation of + the per-instance field-configuration cache, and `enumConfig` callbacks. +- `CurrencyField.symbol` and nested getters across trusted Base FieldDefs. +- `@field`, `@model`, `@format`, `@context`, `@configuration`, and writable + `@set` component arguments. +- Compound validation and fallback presentation. +- Guide cascade across base, domain, realm, and inline layers; JQXL-backed + visibility, constraints, computed defaults, helper text, and field order. + +### Visual and interactive contract + +The metadata section is a two-column grid with producer, region, origin +country, farm/venue coordinate, process, price with currency symbol, and two +credits. Edit mode exposes a country selector, numeric price control, currency +selector, coordinate inputs, and configured labels/placeholders. It must never +remain on `Loading countries...` after the Base options resolve. +Changing the Guide changes help text, field visibility, ordering, constraints, +and suggested defaults without loading new authored TypeScript. + +### Required assertions + +- A nested edit updates the canonical Release document through one authorized + mutation and retains unrelated fields. +- No side-loaded contained FieldDef is serialized as a card resource. +- Configuration callbacks run in the owning runtime and cross the rendering + boundary as resolved configuration, not executable functions. +- Guide data and evaluated JQXL results cross the boundary as bounded records; + no Guide introduces a callback, Store handle, or ambient authority. +- Scalar/list/field-order cascade rules produce the same form in Direct, + Capsule, and Sandbox. +- `CurrencyField.symbol`, nested `computeVia`, and an undefined optional nested + path produce their declared fallback, not `undefined 24`. + +## Use case 3 — Enumerations, inheritance, theme, brand guide, and images + +### Boxels introduced + +- `DeluxeRelease extends Release` in Studio. +- `ReleaseTheme` CardDef using `CssValueField` and `TypographyField`. +- `ReleaseBrandGuide`, a local instance adopting the trusted `BrandGuide` + definition and linked through `cardInfo.theme`; it is not a subclass. +- Cover image pair: `linksTo(ImageDef)` plus `contains(UrlField)` fallback. +- One linked `PngDef` and one polymorphic `WebpDef` fixture. + +### Semantics exercised + +- inherited fields, inherited formats, an overridden format, and a new field; +- `enumField` static scalar options; +- rich enum labels/icons; +- dynamic enum options resolved from owner state; +- per-use enum configuration override; +- null enum value and invalid-value validation; +- `Tag` cards through `linksToMany`; +- `cardInfo.theme`, theme inheritance, CSS-variable injection, `headerColor`, + and authored scoped styles; +- the trusted Theme → StructuredTheme → StyleReference → BrandGuide behavior, + with local values on one linked BrandGuide instance rather than copied token + fields or a new subclass; +- BrandGuide palette-to-functional CSS-variable computation and `cssImports` + font loading; +- the URL/ImageDef pair, polymorphic image links, and broken-image fallback. + +### Visual and interactive contract + +The deluxe release has a deliberate poster-like theme, cover art with alt +text, typography tokens, a stage pill from a rich enum, and visible tags. The +Brand Guide visibly demonstrates the same palette, type scale, spacing, +imagery treatment, and representative controls. Direct reference and Capsule +must have matching background, foreground, typeface family category, spacing +tokens, and loaded image. Styles may not leak into Host chrome or sibling +cards. + +### Required assertions + +- Inherited type metadata preserves the child's source provenance and does not + promote it to Direct because its parent is trusted. +- Every declared/overridden format appears in `BoxelDescription.formats`. +- Theme and Brand Guide retain one canonical token source and render the same + tokens in Direct and Capsule. +- Relinking `cardInfo.theme` to a rotated-palette BrandGuide changes every + visible themed element without remounting the release or retaining a + hard-coded color/font leak. +- Image identity, content URL, alt text, intrinsic dimensions, and load/error + state survive the boundary. +- Enum option labels/icons and dynamic configuration are identical in Direct + reference and Capsule rendering. + +## Use case 4 — Music player + +This Boxel is deliberately introduced early because use case 12 must reuse the +actual player, not a visually similar copy. + +### Boxels introduced + +- `Track` CardDef in Studio. +- linked audio `FileDef` fixture and linked cover `ImageDef` from use case 3; +- `MusicPlayer` component used by Track's isolated and embedded formats. + +### Semantics exercised + +- FileDef link identity, MIME type, filename, content URL, and metadata; +- `@tracked` play state, current time, duration, volume, and scrub position; +- Glimmer actions, modifiers, event cleanup, and derived display getters; +- `surfacePlayback` registration and commands: play, pause, seek, rate, + current-time reporting, ended, and error; +- safe media element access through the Surface capability plane rather than + ambient navigator/window authority; +- isolated, embedded, fitted, atom, and edit presentation. + +### Visual and interactive contract + +The player shows cover, track title, artist, play/pause control, elapsed and +duration text, seek bar, and volume control. Atom is a non-playing identity +pill; fitted is a bounded mini-player only when interaction is allowed by the +container. Edit changes Track metadata but does not duplicate audio bytes into +card JSON. + +### Required assertions + +- Clicking play advances time and changes accessible control name to Pause. +- Seek and volume changes round-trip once; no duplicate event listeners appear + after format switches or HMR. +- Two player surfaces can be assigned to the same playback group and converge + on play state/time within tolerance without recursively echoing commands. +- The player is Capsule-capable because it uses approved capabilities and no + browser-dependent external package. + +## Use case 5 — Playlist, relationships, and query fields + +### Boxels introduced + +- `Playlist` CardDef in Studio. +- Three Track instances, including one explicitly granted Partner Track. +- One deliberately broken relationship slot. +- A query-backed `recentTracks` field. + +### Semantics exercised + +- `linksTo`, `linksToMany`, indexed relationship JSON keys, null, loading, + present, and broken states; +- `getRelationshipMembershipState(...).isLoading`, including the requirement + that the template also reads the field; +- `undefined` holes without changing relationship-array length; +- defensive link traversal and `.filter(Boolean)` only where intended; +- query-backed `linksTo` and `linksToMany`, `$this`, `$REALM`, sort, and page; +- aggregate `computeVia` across loaded related values; +- delegated Track rendering in atom, fitted, and embedded formats. + +### Visual and interactive contract + +The playlist has ordered rows, cover thumbnails, duration, and play controls. +While loading, it shows the existing Boxel spinner and stable row geometry. +A broken slot shows a bounded unavailable-row state rather than disappearing, +throwing, or shifting later row identity. Query refresh visibly returns to a +loading state and then preserves sort order. + +### Required assertions + +- The Partner Track is unreadable before the explicit grant and readable after + it without granting search over the Partner realm. +- Query-backed fields are materialized for rendering but omitted from PATCH. +- The child Track runtime is selected from the Track module's provenance, not + inherited from Playlist. +- Nested path is at least Capsule Playlist → Host router → Direct Base field → + Host router → Capsule Track. + +## Use case 6 — Rich liner notes and recursive delegated rendering + +### Boxels introduced + +- `LinerNotes` CardDef in Studio using `RichMarkdownField`. +- Markdown fixture containing headings, lists, a table, a Mermaid diagram, + cover image, Track atom, Track embedded player, Playlist fitted card, and a + link to the Release. + +### Semantics exercised + +- MarkdownField and RichMarkdownField rendering and editing; +- trusted CodeMirror and Mermaid shims distributed by Boxel; +- Boxel-flavored Markdown embeds through `viewCard`/delegated rendering; +- nested format propagation and stable child identities; +- image/file embeds and sanitization; +- editable body mutation through the trusted Base RichMarkdown editor; +- a RichMarkdown-owned Layout → Run projection of the same canonical document; +- Surface use/change/inspect mode propagation, keyboard traversal, and focus + return through the outline and editor; and +- cue label, description, status, and accessory semantics as non-product + chrome. + +### Visual and interactive contract + +Read mode shows formatted prose, a rendered Mermaid SVG, a loaded cover image, +and visibly distinct atom/embedded/fitted child formats. Edit mode shows a +working rich editor with the full body loaded, toolbar, slash menu, and live +preview behavior. Raw Markdown, raw Mermaid source, and raw JSON are failures. + +### Required assertions + +- CodeMirror and Mermaid execute as explicitly trusted shim modules, not as + ambient Host imports granted to all authored code. +- Markdown → Base RichMarkdown Direct portal → Host router → Capsule Track → + Base media component proves the recursive graph. +- Focus, selection, and edits remain stable when a nested child finishes + loading. +- RichMarkdown Layout and Run nodes project into stable Surface paths without + copying their content or creating a second document state. +- A nested iframe is never created for atom/head/markdown merely because the + same card type has a Sandbox-only isolated renderer in another module. + +## Use case 7 — Release editor, writable fields, and canonical PATCH + +### Boxels introduced + +- `ReleaseEditor` custom edit component in Studio. +- `BoxelSelect` for rich enum selection. +- Save and validation status components. + +### Semantics exercised + +- custom versus default edit templates; +- writable `@set` for primitive, compound, contained, and linked fields; +- BoxelSelect delegated rendering; +- validation errors, dirty state, optimistic local state, server + acknowledgement, and last-known-good rollback; +- cardInfo aliases without duplicate synthetic fields; +- canonical Boxel JSON:API PATCH shape and removal of query-backed fields; +- read/write capability independent of execution tier; and +- `ReleaseGuide` remains the source of labels, constraints, ordering, and + conditional visibility in both default and custom edit formats. + +### Visual and interactive contract + +Edit is visibly an editor: labels, controls, validation, save status, and no +read-only flash while authority settles. A successful local edit appears +immediately. Server acknowledgement does not remount or revert it. A rejected +save keeps the draft, restores the last valid preview, and shows an actionable +error overlay in the standard bottom position. + +### Required assertions + +- A field writable Direct remains writable in Capsule or Sandbox when the + capability is granted; sandboxing alone cannot turn it read-only. +- The Host rejects side-loaded non-card resources and unauthorized links. +- Mutation intent contains changed field path/value and expected document + revision, not a live Store object or authored instance. +- Reauthorization happens on every mutation, including after a grant is + revoked. + +## Use case 8 — Release campaign PosterBoard and Frame coordination + +### Boxels introduced + +- `CampaignBoard` CardDef in Studio. +- `PositionedCardField` entries referencing Release, LinerNotes, Track, and + ImageDef, each with x/y/width/height. +- a trusted `PosterBoard` Surface containing one `Frame` per campaign asset; +- two ordered placement lanes and one positioned target. + +### Semantics exercised + +- `PositionedCardField` identity and geometry; +- `surfacePresentation` background/header presentation; +- `surfaceViewport` pan, zoom, focus, and coordinate conversion; +- `surfaceLayout` allocated rectangles and intrinsic-size reporting; +- `surfaceObserve`, `surfaceStyle`, `surfaceFocus`, `surfacePointer`, and + `surfaceSlot` where already shipped or explicitly stubbed by the fixture; +- drag/resize mutation through a Host-authorized capability; +- theme variables and scoped authored CSS in nested render slots; +- stable PosterBoard/Frame identity/path, parent context, coordinate source, and + use/change/inspect posture propagation; +- keyboard focus ladder, one selected Frame, and inspect hover; +- inline-versus-lifted editing with anchor geometry, commit/cancel, and focus + return; +- CSS-like Surface rule matching by specificity/order and component choice; +- self/children/descendants/subtree directive scope and posture inheritance; +- typed cross-container placement shared by pointer, keyboard, and paste; and +- placement ghost, insertion wedge, structured denial, autoscroll, and scoped + FLIP/view-transition lifecycle. + +### Visual and interactive contract + +The board is a designed poster canvas, not a list. Cards have deterministic +positions and sizes, cover images load, pan/zoom retains crisp geometry, and +selection/focus is visible. Embedded children fit their allocated rectangles. +Background presentation avoids the double-frame effect without copying +untrusted arbitrary CSS into Host chrome. + +### Required assertions + +- x/y/width/height cross the boundary as typed values and persist after drag. +- Surface capabilities are issued per mounted render slot and revoked at + teardown; they do not live on the Boxel semantic runtime. +- Nested styles cannot select Host chrome or sibling slots. +- A child asking for intrinsic height cannot override a parent-allocated + fitted rectangle. +- Move and reference placement preserve stable Card identity and commit the + exact ordered index requested by the target. +- Pointer capture, drag observers, lifted planes, transition names, and + temporary ghosts are released after commit, cancel, error, and teardown. + +## Use case 9 — Split-module 3D merchandise artifact + +### Boxels introduced + +- `MerchArtifact` CardDef in Studio with safe metadata/formats. +- `merch-artifact-canvas.gts` in Lab importing Three.js and a 3MF loader. +- linked 3MF FileDef, poster ImageDef, and thumbnail formats. +- a safe editable Canvas graph with two nodes and one edge whose canonical + node state is also consumed by the Sandbox Scene. + +### Semantics exercised + +- format implementation split across modules; +- natural classifier behavior: the safe GTS is Capsule and the Three.js GTS is + Sandbox because of its browser-dependent import; +- literal `static prefersFullSandbox = true` as an explicit minimum-isolation + request where present; +- format-aware selection without pretending module-level imports are + format-local; +- `surfaceCanvas`, pointer input, allocated/intrinsic sizing, body/container + presentation, prerender placeholder, readiness spinner, and teardown; +- safe thumbnail/atom/head/fitted rendering without inline iframes; +- Canvas node drag/resize, handles, connection/reconnection, edge label, + minimap, and viewport portal; and +- Scene camera drag, wheel momentum, node motion, and one deterministic visual + effect without granting ambient DOM authority to the Capsule. + +### Visual and interactive contract + +Isolated and embedded show the interactive rotating artifact in an +origin-isolated Sandbox. Edit exposes metadata controls and a bounded preview. +Fitted, atom, head, and markdown use the safe poster/thumbnail module in a +Capsule. Prerendered format-correct HTML appears immediately while the iframe +loads; the standard spinner sits beside the realm icon until interaction is +ready. Replacement causes no content jump beyond a declared tolerance. + +### Required assertions + +- The same source module is never classified differently merely because a + different format was requested. Separate modules are the optimization seam. +- If the author imports Three.js into the main module, all executable formats + from that module use Sandbox; non-executable prerendered thumbnails remain + possible but are not hydrated as Capsule code. +- Sandbox origin, CSP/fetch policy, MessageChannel version, and height mode are + verified. +- Canvas and Scene agree on coordinate conversion while retaining independent + mounted presentation state. +- WebGL contexts, observers, animation frames, listeners, and object URLs are + released on teardown. + +## Use case 10 — Release-planning spreadsheet and explicit grants + +### Boxels introduced + +- `ReleaseCollection` in Studio. +- `ReleasePlanningSheet` in Studio, backed by the collection's canonical query + and rendered with trusted `Table` and `Cell` components. +- `LicensedTrack extends Track` in Partner. +- `ReleaseAccessPolicy`, a linked BXL policy for resource-scoped release + visibility and Commands. +- label, release-team, rights-team, finance-team, suspended-collaborator, and + guest principals, including one recursively nested release team. +- query result sections rendered through `@context.searchResultsComponent`. +- one granted Partner image and one ungranted sibling. +- rows for each Track with title, artist, ISRC, rights status, launch date, + territory, price, rating, and campaign-ready state. + +### Semantics exercised + +- `RealmField`, `CodeRefField`, and `AbsoluteCodeRefField`; +- `codeRef(here, path, name)` and injected `realmURL` identity; +- type, eq, in, contains, range, matches, any, every, and not filters; +- custom-field sort with `on`, general sort without `on`, and pagination; +- `@context.searchResultsComponent`, `getCards`, and query-backed fields as + three distinct query consumers; +- `searchable` relationship projection; +- user-selected cross-Realm links and narrow Store grants; +- Host-owned BXL projection from a finite, already-authorized relationship + snapshot; +- nested team membership, `via(Resource.Label; ...)`, capability composition, + request inputs, and explicit refusal that wins after positive eligibility; +- resource-scoped field, relationship, query, section, and Command visibility; +- inheritance across provenance boundaries; +- stable Table/Cell identities under virtualization, pinned/resized columns, + keyboard cell traversal, selection, and deterministic bulk status changes; +- trusted number, date, currency, enum/status, checkbox, and rating cell + widgets; and +- lifted cell editor and context menu focus/commit/cancel behavior. + +### Visual and interactive contract + +The release-planning spreadsheet has stable sections for new releases, +high-rated tracks, launch windows, text matches, and explicitly licensed +Partner content. Its visible columns and editors are meaningful to music +release operations rather than a generic Grid demonstration. Each result uses +its declared child format and shows title, image, and release status. Empty +authorized results show an empty state; unauthorized results show no leaked +title, count, URL, or timing-dependent placeholder. + +The same sheet is projected for several viewers. The artist sees identity, +catalog, and submission controls. A rights-team member sees territories, +licenses, and rights approval. Finance sees pricing and revenue fields without +unreleased media or internal notes. A guest sees a release locator and +request-access control only. A suspended collaborator remains a member of a +nested release team, but an explicit refusal removes internal notes and every +mutation. Switching viewers updates only authorization-dependent columns, +sections, rows, and Commands; unaffected Table/Cell and child render-slot +identity remains stable. + +### Required assertions + +- A bare `{ on: ref }` fixture is rejected by fixture validation rather than + silently passing with zero rows. +- Custom sort without `on` is rejected; `lastModified`, `createdAt`, and + `cardURL` remain valid general sorts. +- Granting one linked Partner card does not grant arbitrary search, neighboring + card access, module source, or Realm enumeration. +- A grant or nested membership on one Release resource does not grant another + Release, and `via(...)` exposes only the declared label capability. +- Denied fields, relationships, rows, totals, titles, URLs, menu items, and + Commands are absent from the render record and DOM rather than hidden after + materialization. +- Client BXL decisions can only reduce the server upper bound; forged `allow`, + stale policy/input revisions, and direct Command attempts are refused by the + Host/server. +- Explicit refusal wins over nested-team eligibility in Direct, Capsule, and + Sandbox consumers. +- Revocation invalidates only affected render/query consumers and retains + unaffected Direct/Capsule slots. +- Query refresh preserves Table selection, focus, geometry, and unrelated Cell + mount identity. + +## Use case 11 — Live production, Realm Script, async AI, and volatile modules + +### Boxels introduced + +- `ProductionConsole` in Studio. +- a typed `PublishReleaseCommand` with a run card and progress. +- a volatile Track format module edited by Monaco and by an out-of-band write. +- `CampaignImageRun` with contained stages/logs and four generated + `CampaignAsset` image links. +- a capability-scoped Realm Script that resolves release data into a validated + image-generation plan; provider IO and binary persistence remain Host + commands. +- `ReleaseReviewAnnotation` with target, typed Field/TextRange/Cell anchor, + body, author Actor, assignee, state, and a linked approval workflow. +- `ReleaseApprovalRoom`, a concrete mixed-ownership card governed for a bounded + review window by a versioned `ReleaseApprovalPolicy`: + - liner notes are a Yjs-concurrent rich-text field; + - publication state, rights approval, and approver are Command-owned; + - release date and territories are frozen after the review window opens; + - cover artwork remains an ordinary revisioned ImageDef link; + - readiness is computed; and + - discussion and `ReleaseReviewAnnotation` records are evidence, not + authority. + +### Semantics exercised + +- Command input/output/progress and Host command capabilities; +- source classification and transpilation cache by source hash; +- volatile module generation, local draft, server acknowledgement, and + last-known-good state machine; +- HMR for Capsule formats and persistent Sandbox protocol for Sandbox formats; +- error overlay, reload action, and explicit execution signage; +- source navigation independent from preview readiness; +- BXL patch result paths, deterministic scheduling, and two-client replay + convergence; +- Realm Script preview-versus-commit, JSON-schema output validation, input and + result byte limits, cancellation, and wall-clock timeout; +- an optimistic asynchronous image pipeline: resolve voice/prompt, dispatch + four provider jobs, persist each binary as it completes, link ImageDefs, + then settle indexing acknowledgements; +- partial success, out-of-order completion, cancellation, retry, idempotency, + and stale-generation rejection; and +- exact Host-tool grants and import denials rather than ambient AI, Store, + network, filesystem, or credential access; +- durable Annotation creation, assignment, reply, resolution, and query-backed + review state through typed Commands with Actor attribution; and +- ordered workflow advancement using canonical Annotation and Command state, + never UI-local comment state; +- episodic, field-scoped coordination: one current write owner per path, + minimal Policy custody, and automatic return to ordinary revisioned writes + when the approval term lapses; +- lazy Yjs epochs for the declared rich-text field, with cursor, selection, + focus, and presence carried as ephemeral awareness rather than card state; +- typed Command admission, sequencing, idempotency, and receipts for + consequential approval fields, with every accepted field patch joining the + same canonical card-revision boundary as ordinary and collaborative writes; +- atomic AI snapshot compare-and-swap while collaboration is active: settle + accepted Yjs updates, fence and close the affected epoch, install the + candidate only if its base revision is current, and start a new epoch for + connected collaborators; and +- publication's two clocks: compatible instance-data revisions follow the + source change feed while code, schema, templates, theme, and projection + policy stay pinned until republish. + +### Visual and interactive contract + +Monaco/file navigation displays source as soon as fetched and never waits for +preview classification/rendering. Valid text or CSS edits update the preview +without destroying the render slot or unrelated Store state. The first +canonical-to-volatile transition may show one loading flash; subsequent +compatible generations do not. A syntax error keeps last-known-good output and +floats the standard error panel over the bottom of the card. + +The image run shows four stable aspect-ratio placeholders, per-stage progress, +and each successful image as soon as its binary is durable. A failed variant +remains an actionable retry tile; it does not hide the three successful +results or block Monaco, playback, or navigation. + +The review Annotation is visibly attached to the release field, +release-planning Cell, or liner-notes text range it addresses. Its body, +author, assignee, status, replies, and workflow step survive navigation and +render in Capsule and Sandbox without exposing the target's ungranted +neighboring data. + +The Release Approval Room shows the ownership model rather than hiding it in a +protocol test. Two collaborators can edit liner notes and see each other's +presence. Approval controls show admitted, refused, duplicate, and completed +Command receipts. Frozen release terms remain legible but unavailable, while +an ungoverned title correction and artwork replacement remain writable. A +review Annotation can recommend approval but cannot change publication state. +When the bounded Policy term closes, its custody indicator disappears and the +formerly governed paths return to their declared ordinary behavior. + +### Required assertions + +- Matching SSE/index echoes acknowledge the active generation; they do not + reload the preview or revert to older source. +- An out-of-band Boxel CLI write to the displayed module joins the same + volatile pipeline until the card unloads. +- A manual Reload Card action deliberately remounts only the selected render + slot and resets its volatile runtime generation. +- Command authority is explicit and cannot be acquired merely by importing a + Host tool module from authored code. +- Annotation targets and anchors retain stable Boxel identities, Actor + attribution, and authorization across Direct, Capsule, and Sandbox; resolve + and reply are mutations, not local component state. +- At most one of ordinary revision writes, Yjs, Command authority, frozen + custody, or atomic snapshot installation owns a field path at a time; the + Host never infers ownership from which component happens to be mounted. +- A direct write to a Command-owned or frozen path is refused, while an + unlisted path on the same card remains writable. Duplicate or out-of-order + approval Commands produce exactly one accepted transition and stable + receipts. +- Two Yjs clients converge on the same liner-notes content; awareness never + appears in serialized card JSON, indexing, search, prerender, or PATCH. +- A stale AI snapshot cannot silently replace collaborative work. A current + snapshot changes the epoch without replacing Command-owned fields that were + outside its admitted scope. +- Annotation bodies, local messages, and federated evidence cannot mutate the + approval projection without an independently authorized local Command. +- A compatible data change reaches a published view without republishing its + executable definition. An incompatible schema change retains last-known-good + output and reports `republish-required` instead of partially rendering. +- CI uses a deterministic fake image provider with controlled completion + order; the contract test still exercises the real Realm Script, command, + binary-file, ImageDef, Store, indexing-acknowledgement, and rendering path. +- Realm Script cannot select its model, read provider credentials, issue an + ungranted request, write in preview mode, or smuggle an executable function + through its schema-validated result. +- A BXL mutation and two-client scheduled event log converge exactly once + after duplicate, delayed, and out-of-order acknowledgements. + +## Use case 12 — Multimedia production timeline + +This is the graph acceptance test. It reuses the exact Boxels from earlier +cases: + +- MusicPlayer and Track from use case 4; +- Playlist from use case 5; +- LinerNotes from use case 6; +- ReleaseEditor state from use case 7; +- Campaign PosterBoard and Frames from use case 8; +- MerchArtifact from use case 9; +- federated Partner content from use case 10; +- command/volatile state, generated CampaignAssets, and the still-queryable + CampaignImageRun from use case 11; and +- the unresolved release-review Annotation and approval workflow from use case 11. +- the active Release Approval Room, including its collaborative liner notes, + field-custody projection, Command receipts, and bounded Policy term from use + case 11. + +### Boxels introduced + +- `TimelineEntry` FieldDef with time, duration, lane, format, and linked Boxel; +- `MultimediaTimeline` CardDef in Studio; +- one video FileDef and synchronized audio/video surfaces. + +### Semantics exercised + +- heterogeneous CardDef/FieldDef/FileDef visual graph; +- repeated rendering of the same Card instance in different formats; +- nested Capsule → Host → Direct → Host → Capsule and Capsule → Host → + Sandbox paths; +- playback group synchronization, viewport synchronization, focus, pointer, + layout, presentation, intrinsic/allocated height, and slot coordination; +- simultaneous relationship/query loading and a later grant revocation; +- cycle detection, render budgets, stable identities, and teardown; +- edit and command mutations while media remains mounted; +- pointer/keyboard/paste placement of earlier Boxels, lifted metadata editing, + use/change/inspect mode switching, and one Surface-scoped view transition; + and +- generated image arrival and failed-variant retry while the timeline remains + mounted; and +- a TimelineAnchor Annotation on a cue, resolved without replacing the + Annotation card or interrupting playback; and +- concurrent liner-notes edits and an approval Command crossing distinct + nested render paths without either writer replacing the other's field-scoped + canonical patch. + +### Visual and interactive contract + +The timeline has visible time rulers and lanes for audio, video, notes, poster +composition, and 3D artifact. The use-case-4 MusicPlayer is mounted in the +audio lane and remains the controller of its media state. Seeking the timeline +updates player and video; playing either approved leader synchronizes the +group. Notes and board elements retain their authored theme. The Sandbox 3D +entry receives an allocated viewport while atom/fitted references use safe +thumbnails. Editing release metadata updates all relevant labels without +resetting playback, pan/zoom, focus, or the iframe. +The campaign lane reuses the case-11 image assets: successful variants appear +in completion order without changing their requested slot order, and retrying +the failed variant does not remount the player or the board. + +### Required assertions + +- The expected execution graph includes at least one route with five boundary + transitions and preserves the original Card/Field/File identities. +- Duplicate views share canonical Store data but have independent mounted + presentation state unless explicitly joined by a `surface*` group. +- No child can search, mutate, navigate, or read media beyond its capabilities + and explicit Store grants. +- One child error or revoked grant does not blank the timeline. +- Dragging the case-4 player from the release-planning Table into a timeline lane uses the + same typed placement command for pointer, keyboard, and paste. A lifted + editor can modify it without interrupting playback. +- Grant revocation and generated-image acknowledgements preserve unrelated + focus, playback, Table selection, Canvas viewport, and Sandbox state. +- Policy expiry, Yjs epoch transition, and approval receipts update every view + of the Release Approval Room without remounting the player, Sandbox Scene, + RichMarkdown editor, or unrelated Direct Base fields. +- After teardown there are zero active timeline surface registrations, media + subscriptions, iframe ports, Capsule component handles, styles, observers, + or animation loops. + +## Boxel semantic cross-product + +The twelve cases are the narrative fixtures. The following table is the +boring-but-load-bearing checklist generated from the Boxel skills glossary. +Every row must map to at least one fixture assertion and one boundary codec or +explicit statement that the value never crosses a boundary. + +### Base field catalog + +| Family | Required coverage | Primary use case | +| ---------------- | ---------------------------------------------------------------------------------------------------------------------------------- | ---------------------------- | +| Primitive/string | String, Number, Boolean, BigInteger, TextArea, Email, URL, PhoneNumber, EthereumAddress, Color; null/empty/zero/false distinctions | 1 | +| Time | Date, DateTime, Time, DateRange, DateTimeStamp; exact JSON formats and formatter output | 1 | +| Money/quantity | Percentage, AmountWithCurrency, Currency and nested `symbol` getter | 1, 2 | +| Geographic | Address, Country option loading, Coordinate round-trip | 2, 8 | +| Markdown | Markdown read/edit and RichMarkdown read/edit/BFM/Mermaid | 6 | +| File-backed | generic FileDef, ImageDef polymorphism, PNG, JPG, WebP, GIF, AVIF, SVG, Markdown, CSV, JSON, GTS, TS, Text | 3, 4, 6, 9 plus codec matrix | +| Metadata/schema | CodeRef, AbsoluteCodeRef, Realm, CssValue, Typography | 3, 10 | +| Enum | static, rich, dynamic, per-use override, null, invalid | 3, 7 | +| Special | Tag, PositionedCardField; Base64Image explicitly rejected for new fixtures | 3, 8 | + +The narrative cases need not render a separate large card for every file +subtype. A parameterized file-codec contract test covers all listed subtypes; +the visual graph uses representative image, audio, video, 3MF, Markdown, and +source files. + +### Field and relationship semantics + +| Semantic | Required variants | Primary use case | +| ------------------- | ------------------------------------------------------------------------------------------------------ | ---------------- | +| Field declaration | contains, containsMany, linksTo, linksToMany | 1, 2, 5 | +| Field kind | CardDef, FieldDef, FileDef, polymorphic target | 2, 3, 4 | +| Configuration | `ConfigurationInput`, static/callback, cache invalidation, inherited/per-use merge, `enumConfig`, null | 2, 3 | +| Computation | ordinary getter, computeVia, chained dependency, nested Base getter, null/error/cycle | 1, 2, 5 | +| Relationship state | unloaded, loading, present, null, broken, undefined hole, live-query reload | 5 | +| Search projection | contained always, linked opt-in with searchable path(s), non-searchable query rejection | 5, 10 | +| Inheritance | inherited fields/formats, override, child provenance, polymorphism | 3, 10 | +| Delegated rendering | `@fields`, `viewCard`, searchResultsComponent, atom/embedded/fitted recursion | 2, 5, 6, 10 | +| Component context | model, field, format, context, configuration, set | 2, 7 | + +### Query semantics + +| Query feature | Required assertion | Use case 10 section | +| ---------------------------- | --------------------------------------------------------------------- | ------------------- | +| type | adopts-from match without `on` | +| eq/in/contains/range/matches | each has a valid `on` scope and returns known rows | +| any/every/not | OR, AND, and negation preserve result identity/order | +| sort | custom field includes `on`; general metadata sort may omit it | +| page | stable page size and cursor/offset behavior used by implementation | +| refs | `codeRef(here, ...)`, `.gts` module path, injected `realmURL` | +| substitutions | `$this` and `$REALM` resolve in the owning runtime | +| consumers | searchResultsComponent, getCards, query-backed linksTo/linksToMany | +| persistence | query fields never enter card PATCH payload | +| refresh | loading state re-enters and matching acknowledgement does not remount | + +### Authorization semantics + +| Authorization feature | Required assertion | +| --------------------- | ---------------------------------------------------------------------------------------------------------- | +| server upper bound | client projection can remove an allowed capability but cannot add a denied one | +| linked BXL policy | Host resolves the authorized policy revision; authored code receives no policy evaluator | +| nested userset | recursive team membership is cycle-safe, bounded, and grants only the named resource seat | +| `via(...)` | a declared parent-resource capability projects without exposing unrelated parent data | +| composition | one capability may depend on another without leaking the intermediate policy graph | +| request input | break-glass-like input updates the projection only when all policy predicates hold | +| explicit refusal | refusal wins after positive eligibility and removes both data and Commands | +| resource scope | membership/grant on one Release does not authorize a sibling Release | +| UI projection | denied values, relationships, query rows, totals, formats, sections, menus, and Commands never materialize | +| reauthorization | every fetch, search, traversal, Command, and mutation is independently checked by the Host/server | +| lifecycle | policy, membership, principal, resource, or input changes preserve unrelated render-slot identity | + +### Presentation, Surface, and lifecycle semantics + +The suite must enumerate every shipped or planned `surface*` contract in the +architecture capability ledger. At minimum it assigns fixtures for: + +- `surfacePresentation` — header/background intent and iframe container match; +- `surfaceLayout` — intrinsic size and parent-allocated rectangles; +- `surfaceViewport` — pan, zoom, coordinate conversion, viewport observation; +- `surfacePlayback` — play/pause/seek/rate/time/error and group leadership; +- `surfaceCanvas` — 2D/WebGL target and lifecycle; +- `surfaceFocus`, `surfacePointer`, `surfaceObserve`, `surfaceStyle`, and + `surfaceSlot` — only when their contract is declared in the runtime plan. + +Capabilities belong to a mounted Surface registration, not `BoxelRuntime`. +They are scoped to a render-slot id, revocable, and reauthorized at every +mutating operation. A capability named in this document but not yet shipped is +marked `design` in the implementation ledger and cannot be counted as a pass. + +## Boundary graph rules + +These rules make the suite about composability rather than one-hop transport: + +1. Every nested render request returns to the Host router with Boxel identity, + format, parent slot, and provenance. +2. The router independently selects Direct, Capsule, or Sandbox for the child. +3. Trusted Base components execute Direct even inside a Capsule presentation; + the Capsule receives no live component constructor or Ember service. +4. A Sandbox owns its document and Glimmer runtime. It receives bounded Store + projections and capabilities over the protocol. +5. A CardDef, FieldDef, or FileDef may appear at any depth. Boundary records + distinguish kind without forcing non-card Boxels into Card JSON:API. +6. Repeated Card documents share canonical Store state, not component state. +7. Format is passed on every delegated render; it is never inferred from an + ancestor after the first selection. +8. The graph has cycle and depth budgets with a visible bounded diagnostic. +9. Runtime/cache invalidation is keyed by affected module generation, not one + global Realm revision. +10. Teardown walks the graph and releases children even after partial failure. + +## CI suite layout + +The intended deterministic fixture location is a test-only realm under the +Host test fixtures, with source organized by the twelve use cases rather than +copied from staging. The exact path should follow the existing Host fixture +convention at implementation time. + +### Layer A — protocol and codec tests + +Fast unit tests, no browser: + +- clone/reject each primitive and compound value; +- BoxelDescription formats/kind/cardInfo/theme metadata; +- BoxelRenderRecord projection and mutation sanitation; +- all Base file subtype metadata codecs; +- missing-path diagnostics; +- protocol feature/version negotiation; +- capability ids are opaque and unforgeable; +- no live constructor, Store, service, loader, DOM node, function, or Proxy is + cloneable across the boundary. + +### Layer B — semantic conformance adapter tests + +Run the same semantic contract against Direct, Capsule, and Sandbox adapters +where the fixture is eligible: + +- describe Boxel; +- instantiate Card from canonical document; +- resolve fields/configuration/getters/computeVia; +- resolve relationships and query fields; +- select and render formats; +- authorize and commit mutations; +- reload/acknowledge generations; +- release all handles. + +Sandbox-only browser dependencies are not forced through Capsule merely to +fill a matrix cell. Instead, use case 9 proves that safe and browser-dependent +modules compose at the format boundary. + +### Layer C — browser composition acceptance tests + +Run the twelve cases in order in one browser suite. Each case may assume the +fixture definitions from earlier cases but resets Store documents and runtime +registries. Browser tests assert semantics, accessible DOM, computed styles, +geometry, interactions, trace route, and lifecycle counts. + +High-value screenshot regions: + +- use case 3 themed isolated and fitted views; +- use case 6 RichMarkdown read and edit views; +- use case 8 poster board before/after pan; +- use case 9 prerender-to-Sandbox transition; +- use case 12 complete timeline. + +### Layer D — navigation/HMR acceptance tests + +Use case 11 and 12 add focused tests for: + +- immediate file-tree and recent-file navigation; +- Monaco source display independent from preview readiness; +- valid local edit, invalid edit, recovery, SSE acknowledgement, out-of-band + write, format switch, manual reload, and unload; +- one flash when entering volatile mode, no compatible-generation remounts; +- no stale generation replacing a newer local draft. + +### Layer E — soak and compatibility sampling + +This is not a per-PR blocker initially: + +- navigate all twelve cases across all meaningful formats for 30 minutes; +- open/close repeated Sandbox instances; +- change grants and themes; +- perform 100 compatible HMR generations; +- assert bounded module/runtime/template/style/handle/media growth; +- then sample the larger compatibility corpus and ten recent staging cards. + +## Pass criteria + +The suite is green only when: + +- all twelve cases pass their semantic, visual, interactive, boundary, and + lifecycle assertions; +- the expected execution graph matches exactly; +- there are no console exceptions, blank panels, raw JSON fallbacks, missing + required images, perpetual loading labels, or `Untitled` headers; +- Direct reference and Capsule/Sandbox output meet each case's declared visual + parity tolerances; +- writable controls remain writable with authority and fail closed without it; +- one case's module update does not remount unrelated Boxels; +- all handles, ports, Surface registrations, styles, and media resources are + released after teardown; and +- no existing regression test was weakened merely to accommodate the runtime. + +## Implementation order + +1. Build use cases 1–3 and the protocol/semantic adapter harness. This is the + ordinary Boxel API foundation. +2. Add use case 4 once and expose it as a reusable fixture dependency. +3. Add use cases 5–7 to prove relationships, RichMarkdown, and writes. +4. Add use cases 8–9 for Surface coordination and true browser isolation. +5. Add use cases 10–11 for grants, queries, commands, and HMR. +6. Assemble use case 12 exclusively from the earlier reusable Boxels. +7. Turn the cross-product tables into machine-readable coverage metadata and + fail CI when an API row has no test owner. +8. Keep the larger compatibility corpus as an independent discovery/soak gate; + do not substitute it for this deterministic graph suite. + +The final design test is simple: if use case 12 requires a copied player, +copied card markup, a special one-off iframe API, or a trusted Host import in +user code, the architecture has failed the composition goal even if the page +looks correct. + +## Glossary sources and maintenance rule + +The ordinary semantic checklist is derived from the author guidance shipped +with Boxel CLI, especially: + +- [the Base field catalog](../packages/boxel-cli/plugin/skills/boxel/references/base-field-catalog.md); +- [enumerations](../packages/boxel-cli/plugin/skills/boxel/references/enumerations.md); +- [query systems](../packages/boxel-cli/plugin/skills/boxel/references/query-systems.md); +- [relationship loading state](../packages/boxel-cli/plugin/skills/boxel/references/relationship-loading-state.md); +- [delegated rendering](../packages/boxel-cli/plugin/skills/boxel/references/delegated-rendering.md); and +- the main [Boxel skill glossary](../packages/boxel-cli/plugin/skills/boxel/SKILL.md). + +When that glossary adds a field family, relationship/query semantic, format, +component argument, or author-visible capability, the same change must name a +row and owning use case here. Conversely, a POC-only transport mechanism does +not enter the author glossary merely because the runtime needs it internally. diff --git a/docs/boxel-execution-runtime-coverage-audit.md b/docs/boxel-execution-runtime-coverage-audit.md new file mode 100644 index 00000000000..282fdefa7e1 --- /dev/null +++ b/docs/boxel-execution-runtime-coverage-audit.md @@ -0,0 +1,579 @@ +# Boxel execution runtime coverage audit + +This audit is the broad inventory and migration ledger. The companion +[execution runtime composition suite](boxel-execution-runtime-composition-suite.md) +is the smaller deterministic CI graph: twelve cumulative use cases covering +ordinary fields and queries through rich composition, media, Surface +capabilities, and mixed Direct/Capsule/Sandbox execution. +The +[real-example audit](boxel-execution-runtime-real-example-audit.md) checks that +single suite against fifty Boxel Labs and realm examples, including the full +foundation Surface vocabulary and asynchronous AI production. + +## Purpose + +This is the migration checklist for replacing the current realm-sandbox POC +with the execution runtime described in +[Boxel execution runtime architecture](boxel-execution-runtime-architecture.md). +The architecture may replace the internals, but it may not silently reduce the +set of cards, fields, formats, interactions, or UI behavior that the POC has +already exercised. + +This document answers four different questions that must not be collapsed: + +1. Which authored and implicit Boxel APIs do current cards use? +2. Which of those APIs have a boundary representation in the POC? +3. Which behaviors have deterministic automated proof, rather than only a + successful manual comparison? +4. Which multi-boundary execution graphs must the new architecture preserve? + +The detailed author-facing API semantics remain canonical in the +[card API compatibility ledger](realm-sandbox-card-api-compatibility.md). The +[surface capability design](realm-sandbox-surface-capabilities.md) remains the +canonical proposal for the `surface*` family. This audit is the cross-product: +it maps those APIs to actual cards, tests, execution owners, graph paths, and +completion gates. + +## Evidence vocabulary + +Every row uses one of these evidence levels: + +| Level | Meaning | +| ---------------------- | ------------------------------------------------------------------------------------------------------------------ | +| **Exact automated** | A deterministic test constructs the same API shape and asserts the relevant render, effect, or protocol result. | +| **Contract automated** | A lower-level or smaller fixture proves the individual contract, but not the full corpus card or nested graph. | +| **Manual parity** | The same Realm document was compared in staging and the sandbox Host and recorded in `COMPARISON-OBSERVATIONS.md`. | +| **Fixture only** | A representative card exists, but no current automated or recorded manual result proves the claimed behavior. | +| **Design only** | The API is proposed and has no implementation proof. | + +A capability is **not migration-complete** until it has exact automated proof +in Direct and every applicable confined runtime. Pairwise proof is also not +enough for a compositional API: it needs at least one graph test in which it is +nested behind another card or field boundary. + +## Audited sources + +This audit is based on the current `codex/code-preview-instant-reload` branch +and these concrete sources: + +- the 40 synthetic cards and five real-card probes in + `/Users/chris/boxel-workspaces/sandbox-compatibility-corpus-20260803`; +- `Acceptance | code submode | sandbox live reload`; +- `Integration | preview` and `Integration | realm sandbox iframe`; +- the realm-compartment, boundary, source-policy, import-policy, iframe + protocol, media, Store-boundary, style, HMR, and lifecycle unit suites; +- Boxel Base's `card-api.gts`, `field-support.ts`, default templates, Rich + Markdown implementation, and `CardContext`; +- the existing skill and real-workspace import audit in + [realm-sandbox-skill-import-audit.md](realm-sandbox-skill-import-audit.md). + +The synthetic corpus currently imports these major runtime families: + +| Import family | Current corpus use | +| ---------------------------------------------- | -----------------: | +| `@cardstack/base/card-api` | 43 files | +| `@cardstack/base/string` | 39 files | +| `@cardstack/base/number` | 19 files | +| `@ember/modifier` | 14 files | +| `@glimmer/tracking` | 11 files | +| `@ember/helper` | 8 files | +| `@cardstack/boxel-ui/helpers` | 5 files | +| `@cardstack/base/boolean` | 5 files | +| `ember-modifier` | 3 files | +| `@cardstack/boxel-ui/components` | 3 files | +| current Surfaces modules | 3 files | +| browser libraries such as Three.js and Leaflet | 3 files | + +The larger staging-workspace scan is more important for compatibility breadth: +3,072 source files, 1,415 distinct runtime module specifiers, and concentrated +use of Base, Boxel UI, Ember/Glimmer, commands, runtime-common, local modules, +and a smaller external-package tail. The new runtime must be driven by that +inventory, not by whichever imports happen to be convenient in a small test +Realm. + +### Automated suite ledger + +This table is the index from existing proof to the contract it protects. It is +not permission to delete narrower historical tests: the new architecture must +pass these tests or replace them with stronger assertions that retain their +original regression intent. + +| Existing suite | Contracts currently exercised | Refactor use | +| ------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | +| [`sandbox-live-reload-test.gts`](../packages/host/tests/acceptance/code-submode/sandbox-live-reload-test.gts) | cold Capsule render, authored FieldDef delegation, chained compute, relationships, all formats, Rich Markdown, recursion, HMR, warm format islands, persistent iframe, failure recovery, explicit reload | Primary end-to-end compatibility suite; extend it with the mixed-runtime graph gauntlet rather than weakening its assertions. | +| [`preview-test.gts`](../packages/host/tests/integration/components/preview-test.gts) | trust routing, opaque `getComponent()` compatibility, themes, `surfacePresentation`, nested/primitive FieldDefs, `viewCard`, head format, stable contained-field identity | Direct-versus-Capsule semantic and Host portal conformance. | +| [`realm-sandbox-iframe-test.gts`](../packages/host/tests/integration/components/realm-sandbox-iframe-test.gts) | readiness, inert type presentation, intrinsic/allocated sizing, permission updates | Parent-side Sandbox component contract; add a real child runtime test separately. | +| [`realm-sandbox-boundary-test.ts`](../packages/host/tests/unit/lib/realm-sandbox-boundary-test.ts) | cloneable records, projection, mutation sanitation, rejection of unsafe values | Seed for the canonical `BoxelRenderRecord` codec. | +| [`realm-compartment-module-runtime-test.ts`](../packages/host/tests/unit/realm-compartment-module-runtime-test.ts) | Capsule imports, evaluation, trusted identities, helpers/components, generation behavior | Seed for `BoxelRuntime` and `CapsuleComponentRuntime`. | +| source/import/URL policy unit suites | runtime classification, safe/unsafe module graph decisions, URL normalization | Convert to policy tests over `ModuleGraphDescription`; never let a query parameter choose execution. | +| iframe protocol, origin, draft, and media unit suites | exact origin, one-use channel/bootstrap, persistent draft updates, bounded image transport | Seed for the versioned Sandbox protocol and grants. | +| styles, acknowledgement, lifecycle, contextual-field unit suites | scoped CSS, server-echo acknowledgement, runtime cleanup, nested field state | Preserve as subsystem tests and add graph-level assertions for their interactions. | +| [`hydratable-card-test.gts`](../packages/host/tests/integration/components/hydratable-card-test.gts) | prerender identity, placeholder adoption, render-slot stability | Seed for `I -> H -> C/S` handoff tests. | +| Boxel UI `safe-modifier` and `surface-presentation` integration suites | current authored effect/presentation APIs and teardown | Seed for shared Direct/Capsule/Sandbox `SurfaceService` behavior specifications. | + +### Manual and corpus artifacts + +The Realm corpus adds evidence that deterministic tests cannot yet express: + +- `COMPARISON-OBSERVATIONS.md` is the append-only staging-versus-branch visual + and interaction log. Its results are **manual parity**, not CI proof. +- `CompatibilityMatrix/matrix` is a card-rendered summary grouped by syntax, + API, and boundary layer. The matrix itself exercises nested + `containsMany(FieldDef)` rendering. +- `FormatPreviewBatchOne/sample` mounts Primitive Profile, Activity Timeline, + Rich Markdown Article, Multi-format Signal, and Image Story simultaneously + in `isolated`, `embedded`, `fitted`, `atom`, `edit`, `head`, and `markdown`. + That is 35 visible delegated boundaries and must become a deterministic + graph acceptance test before the POC paths are removed. + +## Completion rule for every semantic + +No new or migrated Boxel semantic is complete until its implementation record +contains all five items: + +1. **Owner** — Store, trusted Base, Host, authored Capsule, or authored + Sandbox. +2. **Boundary representation** — immutable value, trusted identity, component + handle, effect handle, or capability request. +3. **Runtime consumers** — Direct, Capsule, Sandbox, indexing/prerender, and + code preview as applicable. +4. **Pairwise conformance** — the same authored behavior in Direct and each + applicable confined runtime. +5. **Graph conformance** — the semantic still works when its renderer is + reached through at least one additional card, field, trusted portal, or + Sandbox boundary. + +If an item is intentionally unsupported, the record must name the replacement +behavior and the diagnostic. Partial rendering of an unknown record is not a +valid fallback; retain last-known-good output or fail with a named contract +error. + +## The execution graph, not a stack of pairs + +### Nodes + +The compositional model is a graph whose nodes are render or semantic owners: + +| Node | Owns | +| ------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- | +| `H` — Host | canonical Store identity, trusted Ember owner, Host chrome, policy, capabilities, mutation validation, render-slot identity | +| `D` — Direct | trusted Base/official module execution using native Boxel and Glimmer APIs | +| `C(principal, generation)` — Capsule | authored module instances, authored getters/computeds, tracked state, captured template programs | +| `S(origin, instance, generation)` — Sandbox | a separate document, local Glimmer runtime, browser-dependent authored modules, local DOM and library state | +| `I` — Index/prerender | server materialization, indexed values, inert HTML/markdown, and last-known-good presentation | + +### Typed edges + +Every crossing must be a typed edge. “It is already in memory” is not a type. + +| Edge | Required representation | +| ----------------------- | -------------------------------------------------------------------------------------------------------- | +| `H -> C` | canonical card projection, type/field descriptions, trusted import identities, scoped capability handles | +| `C -> H` | captured component program, bounded effect request, mutation proposal, presentation update, diagnostic | +| `H -> S` | versioned bootstrap, card document/grant, format presentation, permission state, private `MessagePort` | +| `S -> H` | readiness, size/presentation, bounded fetch/effect request, mutation proposal, diagnostic | +| `H -> D` | native card instance and `CardContext`; no boundary projection is necessary | +| `H <-> I` | indexed card resource, prerendered format HTML/markdown, source/index acknowledgement | +| `H -> H` trusted portal | trusted Base component identity plus projected model/configuration and bounded authored callbacks | + +An invocation path must carry a stable trace tuple: + +```ts +interface RenderGraphTrace { + renderSlotId: string; + parentRenderSlotId?: string; + principal: string; + runtime: 'direct' | 'capsule' | 'sandbox' | 'prerender'; + cardId?: string; + typeRef: { module: string; name: string }; + format: string; + generation: number; + sourceHash?: string; +} +``` + +This is diagnostic identity, not authority. Authority remains in unforgeable +Host-owned grants associated with the render slot. + +### Graph invariants + +- A nested render always re-enters the Host policy router; a parent runtime + cannot decide that a child may run with less isolation. +- A trusted Base portal may render Host DOM, but it receives only the projected + data and callbacks declared by its contract. It does not hand its Ember owner + back to authored code. +- An iframe Sandbox uses its own local trusted Base/Glimmer runtime. Host DOM + is never transplanted into the child document and child DOM never crosses to + the Host. +- A Capsule may call a trusted helper/component/modifier identity only through + the Host component runtime. It never receives a live Host element. +- Relationships create graph edges, not embedded authority. Loading a linked + card does not widen the caller's Store grant. +- Cycles are detected by logical card/type identity, not just object identity. +- Each render request has depth, node-count, and in-flight-load budgets. A + recursive `containsMany(FieldDef)` may be deep; an accidental cycle may not + allocate unbounded components, ports, or iframes. +- Teardown is graph-aware: releasing a parent releases only descendants for + which it is the last active consumer. +- Format selection is per render node. One CardDef may use a Capsule renderer + for `atom` and a Sandbox renderer for `isolated` without changing canonical + card identity. + +## Required graph gauntlet + +These paths are the minimum compositional suite for the new architecture. +They deliberately include alternating owners rather than only one boundary. + +| ID | Path | Representative behavior | Current proof | Migration gate | +| ---- | ------------------------------------------------------- | ------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------- | +| G-01 | `H -> C` | ordinary authored isolated card | Exact automated: `COLD-INTERACT-01` | Preserve authored DOM, styles, metadata, and stable slot. | +| G-02 | `H -> C -> H -> C` | CardDef invokes trusted Base field portal which invokes an authored FieldDef template | Exact automated: `COLD-INTERACT-02` | Assert projected value, configuration, scoped CSS inheritance, and `@set`. | +| G-03 | `H -> C -> H -> C -> H -> C` | recursive `containsMany(FieldDef)` with indexed components | Exact automated: `CORPUS-03` | Assert three depths, no JSON fallback, bounded recursion, and teardown. | +| G-04 | `H -> C -> H -> C2` | `linksTo`/`linksToMany` delegates another CardDef and format | Exact automated: `COLD-INTERACT-03` | Include same-realm and separately granted cross-realm child principals. | +| G-05 | `H -> C -> H -> D -> H -> C` | authored card uses trusted Rich Markdown, which embeds an authored card | Partial: `CORPUS-02` proves Rich Markdown; embed tests prove formats separately | Add one exact nested Rich Markdown card-embed graph test. | +| G-06 | `H -> C -> H -> S` | Capsule parent delegates a browser-dependent linked card | Contract automated only | Add an exact parent/child test with readiness, intrinsic height, and parent stability. | +| G-07 | `H -> S(local Base -> authored child)` | iframe card renders Base fields and authored local components in its own document | Protocol automated; full nested child is manual-only | Add a real child-document integration test, not only parent messages. | +| G-08 | `H -> C -> H -> S -> H(write) -> Store -> C/S` | nested Sandbox edit proposes a write and all visible consumers reconcile | Pairwise write protocol exists | Add exact multi-consumer mutation and permission-revocation test. | +| G-09 | `I -> H placeholder -> S interactive` | prerendered HTML is immediate while an iframe becomes interactive | Hydratable Host card is automated; iframe handoff is incomplete | Assert format-correct placeholder, header spinner, no layout jump, then readiness. | +| G-10 | `H -> C(format A) / C(format B)` | same module and card switch between warm formats | Exact automated: `[NAV-07]` for two SES islands | Preserve card identity and bounded two-format LRU. | +| G-11 | `H -> C(atom) / S(isolated)` | compact renderer stays Capsule-safe while browser-heavy renderer uses iframe | Source-policy unit proof | Add one real split-module CardDef across both formats. | +| G-12 | `H -> C -> surface* -> H` and `H -> S -> surface* -> H` | same named capability through direct dispatcher and MessageChannel | Only `surfacePresentation` has both paths | Every shipped `surface*` capability needs this transport-equivalence test. | + +## Authored Boxel semantic checklist + +### Definitions, fields, and projections + +| Semantic used by current cards | POC boundary behavior | Evidence | Required new-architecture contract | +| --------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | +| `CardDef`, `FieldDef`, `FileDef`, `Component` | Capsule-owned classes plus Host type descriptions; native classes in Direct/Sandbox-local runtime | Contract automated | One `BoxelRuntime` description/projection contract; never pass a live authored constructor to Host. | +| `@field` | field metadata captured during Capsule evaluation | Contract automated | Preserve declaration order, inheritance, override identity, definition kind, and configuration. | +| `contains` / `containsMany` | materialized snapshot plus stable singular/indexed component capabilities | Exact automated: COLD-02, CORPUS-03 | Project values recursively; preserve array length/index/iteration and authored FieldDef renderer identity. | +| `linksTo` / `linksToMany` | relationship resource plus asynchronously prepared delegated renderer | Exact automated: COLD-03 | Relationship stays an id/grant edge; Host loads and routes each target without widening the parent grant. | +| query-backed relationships | projected results use the relationship component contract | Manual parity; existing query suites | Add exact sandbox test for loading, empty, error, membership change, and teardown. | +| `computeVia` | executes in the owning Capsule; iframe leaves use child/indexed value rather than opening an iframe for projection | Exact automated: COLD-02B and CORPUS-03 | Include dependency tracking, chained/nested FieldDefs, errors, cycles, and indexing parity. | +| authored getters | evaluated in authored runtime and exposed as projected values | Contract automated | Same dependency/error/cycle rules as `computeVia`; never evaluate authored getter in Host. | +| field `configuration` object/function | resolved against the parent instance and projected to the renderer | Contract coverage is incomplete | Add inherited, per-use merge, function, and dynamic update tests across a trusted Base field portal. | +| `searchable`, query formatting, serializers | Store/index semantics remain trusted | Existing Base/realm tests; sandbox graph not exact | New runtime consumes canonical Store/index output; it must not reimplement serializer/index rules. | +| recursive/lazy field type functions | Capsule resolves type identity without eager infinite recursion | Exact automated: CORPUS-03 | Preserve lazy resolution and logical cycle guard across runtime generations. | +| inherited fields and templates | type description includes ancestors and inherited slots | CORPUS-03 plus manual corpus | Add direct/Capsule/Sandbox conformance for inherited metadata and renderer selection. | +| polymorphic or overridden field types | opaque type state supports per-instance field overrides | Contract automated in Base; sandbox proof incomplete | Include override type refs in the boundary record and test nested render/write. | +| primitive serialization/deserialization | primitive value crosses as data; trusted field handles edit semantics | Exact primitive FieldDef integration test | Preserve `null`, empty, number/string/boolean, date, URL, currency, and file values without JSON coercion. | +| JSON:API persistence | Host serializes canonical document and strips non-card side-loaded projections | Exact boundary unit tests | All writes use one `projectCardMutation()` path; reject non-card included resources and non-cloneable values. | +| `cardInfo` | nested field with `name`, `summary`, `cardThumbnail`, `cardThumbnailURL`, `theme`, and `notes` | Contract automated; corpus has CardInfo Recipe and Theme | Treat as one nested field, not six synthetic top-level fields. Test title, description, image, theme relationship, and notes together. | +| `cardTitle`, `cardDescription`, `cardTheme`, `cardThumbnailURL` | trusted Base computed aliases over `cardInfo` | Manual parity and metadata tests | Project canonical computed presentation before Host header render; eliminate `Untitled` races. | + +### Formats and type presentation + +The set of formats must be open-ended in the new architecture. These are the +current examples, not a closed protocol enum. + +| Semantic | Current evidence | Migration checklist | +| ------------------------------------ | ------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- | +| `isolated` | Exact automated top-level render | ☐ Direct/Capsule/Sandbox; ☐ default Base fallback; ☐ prerender placeholder; ☐ error/LKG. | +| `embedded` | Exact delegated-format test | ☐ nested graph; ☐ intrinsic iframe sizing; ☐ Rich Markdown embedding policy. | +| `fitted` | Exact delegated-format test | ☐ allocated sizing; ☐ warm gallery performance; ☐ no iframe pill/farm regression. | +| `atom` | Exact delegated-format test | ☐ compact Capsule/default renderer; ☐ inert fallback for iframe-only modules. | +| `edit` | Exact delegated-format and primitive `@set` tests | ☐ writable/read-only parity; ☐ Base fallback; ☐ mutation rejection; ☐ intrinsic iframe size. | +| `head` | Exact delegated-format test | ☐ compact safe renderer or inert fallback; ☐ no iframe farm. | +| `markdown` | Exact delegated-format and fallback tests | ☐ trusted conversion fallback; ☐ authored renderer; ☐ nested directives. | +| named fitted/custom dimensions | Markdown embed integration tests | ☐ carry variant id and allocated dimensions through all runtime adapters. | +| future/custom format | Not proven | ☐ `FormatDescription` is keyed by string and unknown formats fail explicitly rather than disappearing. | +| `displayName` | Boundary/type-presentation tests | ☐ no `Untitled` settlement race; ☐ update on valid HMR generation. | +| `icon` | Trusted identity fallback exists | ☐ approved identity mapping; ☐ authored unsupported icon fallback; ☐ no component crosses iframe protocol. | +| `headerColor` | Metadata and iframe presentation path | ☐ validation parity; ☐ title-bar-only semantics; ☐ HMR update. | +| `prefersWideFormat` | Metadata and iframe presentation path | ☐ strict boolean; ☐ Host remains layout owner; ☐ HMR update. | +| `prefersFullSandbox` | Unit source/routing tests | ☐ only strengthens `isolated`/`embedded`/`edit`; ☐ cannot be set by URL; ☐ compact formats remain composable. | +| authored format in a separate module | Source-policy unit tests | ☐ defer unsafe graph only for that slot; ☐ shared module-scope use makes dependency eager; ☐ real two-file card test. | + +### Component arguments and implicit template API + +These are implicit APIs because existing cards receive them through Base's +component signature rather than importing a capability module. + +| Implicit input | Owner and boundary rule | Evidence | Migration gate | +| --------------------------------------- | ---------------------------------------------------------------------------- | ---------------------------------------------- | ------------------------------------------------------------------------------------------------ | +| `@model` | projected canonical value; authored runtime owns executable behavior | Exact across card and primitive FieldDef tests | Stable identity where possible; nested object writes cannot bypass mutation validation. | +| `@cardOrField` | inert type description/approved identity, never authored constructor in Host | Contract automated | Replace constructor introspection with `BoxelDescription`. | +| `@fields` | stable named and indexed renderer handles | Exact COLD/CORPUS tests | Singular, many, inherited, recursive, relationship, loading, and error states. | +| `@format` | string selected by Host policy | Exact format gauntlet | Open-ended, per-node, and updateable without replacing unrelated runtime state. | +| `@set` | bounded mutation callback | Exact FieldDef test | Permission, path, type, generation, and card identity revalidated by Host. | +| `@fieldName` | inert field path/name | Contract automated | Preserve nested path semantics without exposing unrelated schema. | +| `@configuration` | resolved cloneable field configuration | Incomplete | Add exact trusted Base field portal coverage. | +| `@canEdit` / `canWrite` | Host permission-derived boolean | Iframe protocol test | No writable/read-only flash; Host rechecks every write. | +| `@typeConstraint` | resolved inert CodeRef | Contract partial | Validate through owning realm without Host-importing authored code. | +| CRUD functions | Host capabilities, not function-valued card data | Direct exists; sandbox coverage incomplete | Separate named create/view/edit/save/delete operations with principal and activation checks. | +| `@context.mode` / `submode` | inert presentation hint | Contract partial | Same values across Direct/Capsule/Sandbox; never an authority decision. | +| `@context.requestRender` | trusted Base portal asks its owning render slot to update | Rich Markdown proof | Private Host/Base capability; do not project into arbitrary authored context. | +| `@context.trustedUI` | trusted loaders for CodeMirror, KaTeX, Mermaid | Exact Rich Markdown acceptance test | Private to trusted Base/catalog portals; authored code sees rendered result, not loader/service. | +| `@context.validateCodeRef` | realm-scoped validation through owning runtime | Unit/integration partial | Private trusted editor capability with bounded CodeRef result. | +| `searchResultsComponent` | trusted Host rendering portal | Existing component integration tests | Add nested authored-card use through Capsule; do not pass component class through JSON. | +| `cardComponentModifier` | trusted element tracking hook | Hydratable-card tests | Host-only; never present in authored Capsule or iframe code. | +| `markdownEmbedChooser` | trusted operator-mode UI capability | Existing editor tests | Host/Base only and absent in prerender/unsupported contexts. | +| `toolContext` / legacy `commandContext` | privileged Host tool context | Current Capsule receives no real context | Replace each needed operation with a named command capability; never proxy the context bag. | + +### Glimmer, Ember, and authored-program semantics + +| Semantic in current cards | Current handling | Evidence | Required contract | +| -------------------------------------------------------- | ----------------------------------------------------------------------------- | ----------------------------------------- | -------------------------------------------------------------------------------------------------- | +| `@tracked` / `@cached` | Capsule-local reactive state with render notifications | HMR and interaction contract tests | Persistent component instance; same update semantics; teardown stops notifications. | +| `@action` | Capsule-local method binding | Contract automated | Event handle invokes only the owning instance/generation. | +| `on` modifier | trusted modifier identity plus authored callback | Interaction tests | Host owns element/listener; callback receives projected event data or approved event contract. | +| `fn`, `get`, `concat`, `hash`, `array` | trusted helper identities | Module-runtime tests | Preserve helper semantics and nested component handles without general helper authority. | +| template-only and ordinary Glimmer components | captured program/Host component manager in Capsule; native locally in Sandbox | Manual corpus and component-runtime tests | Blocks, args, splattributes, dynamic element, and local state conformance. | +| blocks and `yield` | implicit Glimmer composition | Coverage incomplete | Add Capsule -> trusted Base portal -> authored yielded-block test, including updates and teardown. | +| dynamic component/helper/modifier lookup | allowlisted/trusted identities only | Source/module policy partial | Unknown dynamic identity fails with a named diagnostic; no ambient owner lookup. | +| `ember-modifier` custom modifiers | iframe classification unless represented by a reviewed adapter | Source-policy exact | Existing cards run unchanged in Sandbox; new `surface*` alternatives may remain Capsule. | +| `ember-concurrency` | used by real cards; facade incomplete | Import audit only | Decide Capsule-local task runtime versus Sandbox; test cancellation, restart, error, and teardown. | +| `ember-resources`, destroyables, provide/consume context | real-workspace use; incomplete | Import audit only | Specify runtime-local lifetime/context semantics and graph test before migration. | +| locale/string/number/date behavior | selected safe intrinsics preserved | Computed Flight Plan manual parity | Direct/Capsule/Sandbox conformance for locale, timezone, formatting, and deterministic indexing. | + +## `surface*` API inventory + +### What is actually shipped in the POC + +The distinction in this table is critical. The POC does **not** currently ship +the whole proposed family merely because the design document names it. + +| API | Current status | Semantics and evidence | New-runtime gate | +| ---------------------------------------- | -------------------- | ------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------- | +| `safeModifier('focus')` | Shipped opt-in | trusted focus operation; Boxel UI integration test | Decide whether retained as compatibility alias for `surfaceFocus`. | +| `safeModifier('scroll-into-view')` | Shipped opt-in | trusted bounded scroll operation; implementation exists, focused test should be added | Same as above. | +| `safeModifier('observe-size', callback)` | Shipped opt-in | frozen finite `{ width, height }`, trusted `ResizeObserver`, teardown; exact Boxel UI test | Map to `surfaceObserve('size')`; test Capsule and Sandbox transport equivalence. | +| `surfacePresentation` | Shipped opt-in | publishes validated `containerBackground`; SES integration and iframe protocol tests | Move dispatch/lifetime into `SurfaceService`; preserve the authored import. | +| `static headerColor` | Existing CardDef API | trusted card-title background metadata | Keep separate from body/container presentation. | + +### Proposed `surface*` family + +Each proposed item remains unchecked until it has an authored API, Host +dispatcher, Capsule adapter, iframe protocol adapter where applicable, +revocation, bounds, and Direct/Capsule/Sandbox conformance tests. + +| Capability | Replaces | Current corpus demand | Status | +| --------------------- | ---------------------------------------------------- | -------------------------------------------------- | ---------------------------------------------------------- | +| `surfaceRoot` | ad hoc root discovery/identity | all root-confined effects and nested surfaces | ☐ Design only | +| `surfaceLifecycle` | lifecycle-only custom modifiers | Scrabble, Tier Maker, Assistant Run, Signet | ☐ Design only | +| `surfaceObserve` | `ResizeObserver`, `IntersectionObserver` | iframe height, responsive panels, 3D/canvas sizing | ◐ `safeModifier observe-size` only | +| `surfaceFocus` | `focus()`, `scrollIntoView()` | Tier Maker, forms, keyboard workflows | ◐ `safeModifier` operations only | +| `surfacePointer` | pointer listeners/capture/drag | Tier Maker, Poster Board, maps, signature/canvas | ☐ Design only | +| `surfaceStyle` | validated dynamic style mutation | Invoice form, rating, geometry, pan/zoom | ☐ Design only | +| `surfacePresentation` | Host container/backdrop presentation | iframe double-frame/background parity | ☑ Shipped for solid/matched color | +| `surfaceTransition` | document view transitions and global names | View Transition Gallery, Tier Maker | ☐ Design only | +| `surfaceSchedule` | timers/animation scheduling | Scrabble, playback, workflows | ☐ Design only | +| `surfaceClipboard` | `navigator.clipboard` | Tier Maker/share flows | ☐ Design only | +| `surfaceHaptics` | `navigator.vibrate` | Tier Maker | ☐ Design only | +| `surfaceSlot` | portals into Host chrome | Assistant Run toolbar | ☐ Design only | +| `surfacePlayback` | cross-surface media intent/state/leader coordination | Video, Audio, 3D, playback lab | ☐ Separate design in `surface-playback-synchronization.md` | +| `surfaceViewport` | pan/zoom intent and effective viewport state | Poster Board, map, viewport relay | ☐ Design only | + +Network, Store access, Realm search, commands, secrets, and AI proxy requests +must not be hidden inside `surface*`. Their authority is a principal/data +grant, not a mounted DOM surface. + +## Browser, DOM, CSS, and media checklist + +| Behavior | Runtime decision | Current evidence | Migration gate | +| -------------------------------------- | ----------------------------------------------------------------------- | -------------------------------------------- | -------------------------------------------------------------------------------------------- | +| scoped styles | Capsule shared document with compiler scope; native in Sandbox document | style unit tests and visual canaries | Reject selectors that escape scope; ref-count styles; stable identity across HMR. | +| unscoped/global CSS | Sandbox or fail closed | source-policy unit tests | No network-bearing CSS bypass; preserve existing card in iframe. | +| dynamic inline styles | currently conservative Sandbox selection | source-policy tests | Keep Sandbox until `surfaceStyle` covers the exact property/value contract. | +| theme CSS variables | Host resolves `cardInfo.theme` and attaches bounded variables | preview integration; Themed Dashboard manual | Direct/Capsule/Sandbox visual token parity and update test. | +| inherited parent CSS/custom properties | delegated FieldDef must retain Base wrapper and inheritance | Computed Flight Plan manual parity | Exact nested FieldDef CSS conformance test. | +| native image | SES-safe markup; iframe media bridge for child-private fetch | media bridge unit tests | Relative URL resolution, cross-realm grant, content type, loading/error, edit/format switch. | +| native audio/video | SES-safe unless authored code requests browser authority | manual corpus | Add play/control/source/poster tests without iframe escalation. | +| canvas/WebGL/Three.js/3MF | Sandbox document | source policy + manual corpus | Hosted origin, asset/module policy, resize, cleanup, context loss, prerender placeholder. | +| Leaflet/maps | Sandbox document unless replaced by a narrow surface | manual corpus | Tiles/style/module policy, pointer, marker state, resize, teardown. | +| Mermaid/KaTeX/CodeMirror | trusted Base portal loads Host-vetted packages | exact Rich Markdown test | Nested Capsule -> Base portal -> authored embed graph, edit write parity. | +| top layer/popover | Sandbox while it requires document-global top-layer behavior | source-policy test + manual corpus | Correct iframe containment, focus, size, close, and teardown. | +| view transitions | Sandbox until `surfaceTransition` exists | source-policy + manual corpus | Namespaced identity and lifecycle if brought into Capsule. | + +## Iframe Sandbox protocol inventory + +| Message/semantic | Current POC | Required new-runtime treatment | +| --------------------------------------- | ----------------------------------------------------- | ------------------------------------------------------------------------------------------------- | +| exact-origin `listening` + bootstrap id | Implemented and tested | Version negotiation, nonce origin, one-use port, sibling-frame rejection. | +| `connect` | document/draft, root module, presentation, `canWrite` | Replace ad hoc payload with versioned Sandbox session/grant record. | +| `ready` | card id, revision, error, type presentation | Separate loaded, interactive, and failed states; preserve last-known-good placeholder. | +| `render` | format, height mode, field/code ref, container flag | Persistent child session; open-ended format; stable child document where compatible. | +| `permissions` | `canWrite` boolean | Update without frame remount; Host still rechecks every mutation. | +| `resize` | finite width/height | Intrinsic for isolated/embedded/edit/atom; allocated for fitted; clamp/rate-limit. | +| `surface-presentation` | validated solid container background | Generalize through `SurfaceService` without arbitrary CSS/DOM. | +| `fetch-request/response` | bounded module/media broker | Bind to declared graph and grant; no ambient credentials or arbitrary authenticated fetch. | +| `card-update/result` | full data-only card document and revision | Move to canonical mutation protocol with field/card identity and conflict semantics. | +| draft/HMR | source, URL, revision | Source-hash generation, child acknowledgement, LKG, no server-echo remount. | +| media hydration | private image transport | Extend only by content-specific reviewed capability; do not make it generic credentialed network. | +| height readiness and header spinner | implemented UI pieces | Spinner right of Realm icon; prerender remains visible until interactive; no layout jump. | + +## Corpus coverage ledger + +The corpus README remains the detailed visible-proof specification. This table +records what each card contributes to the migration, and whether the current +automated suite proves that exact card shape. + +| # | Card | Primary contract tags | Best current evidence | Missing exact proof | +| --: | ----------------------- | --------------------------------------------- | ----------------------------------------- | -------------------------------------- | +| 1 | Primitive Profile | primitives, compute, three formats, wide | Contract automated + manual parity | Exact three-format fixture | +| 2 | Nested Field Host | `contains(FieldDef)`, delegated CSS/write | Exact automated COLD-02 | Direct/Sandbox conformance | +| 3 | Activity Timeline | indexed `containsMany`, sort | Manual parity; indexed contract automated | Exact ordering/update test | +| 4 | Rich Markdown Article | trusted portal, Mermaid, editor, embeds | Exact Rich Markdown core CORPUS-02 | Full nested embed graph | +| 5 | Linked Project | links, delegated cards/formats | Exact automated COLD-03 | Cross-principal grant | +| 6 | Query Board | query-backed links | Manual parity | Exact loading/update/error test | +| 7 | Safe Interaction | tracked/action/`on` | Contract automated | Exact corpus fixture | +| 8 | Themed Dashboard | `cardInfo.theme`, CSS variables | Preview contract + manual parity | Exact theme update test | +| 9 | Browser Canvas | iframe, canvas, intrinsic height | Protocol automated + manual parity | Real child-document CI test | +| 10 | Default Template | trusted Base fallback | Contract automated + manual parity | Explicit no-sandbox timing/parity test | +| 11 | Computed Flight Plan | nested/chained compute, CSS inheritance | Exact compute contract + manual parity | Exact CSS assertions in CI | +| 12 | Recursive Discussion | recursive `containsMany(FieldDef)` | Exact automated CORPUS-03 | Cycle/budget failure case | +| 13 | Inherited Experience | inherited fields/computeds | CORPUS-03 analogous + manual | Exact inheritance fixture | +| 14 | Variant Scorecard | nested config/dynamic presentation | Manual parity | Configuration conformance | +| 15 | Sectioned Profile | DateField, DOM, anchor, iframe height | Protocol contract + manual | Exact navigation/height test | +| 16 | CardInfo Recipe | name/summary override | Metadata contract + manual | Full `cardInfo` aliases test | +| 17 | Editable Rating | icons/helpers/action/`@set`/atom | `@set` contract + manual | Exact edit/read-write test | +| 18 | Atomic Work Item | generated FieldDefs/edit/select/radio | Field relationship tests + manual | Generated type/edit fixture | +| 19 | Multi-format Signal | trusted FittedCard/icons/formats | Exact format contract + manual | Trusted component portal nesting | +| 20 | Dynamic Title Group | template-only, dynamic element, splattributes | Manual parity | Glimmer bridge conformance | +| 21 | Typed Command Lab | `Command`, typed context/progress | Command unit contract + manual | Named capability end-to-end | +| 22 | Surface Data Table | Surfaces Environment/Layout/Grid | Manual parity | New runtime portal/API decision | +| 23 | Poster Board | Surfaces, geometry, images, nested fields | Manual parity | Graph + image + mutation test | +| 24 | Workflow Studio | tracked state/actions/workflow | Contract automated + manual | Exact no-remount test | +| 25 | Image Story | realm image/CSS | Media contract + manual | Visual/layout and error test | +| 26 | Surface Command Center | Surfaces, form, metrics | Manual parity | New runtime portal/API decision | +| 27 | View Transition Gallery | document transition/CSS pseudo-elements | Source-policy + manual | Sandbox child integration | +| 28 | Fabrication Viewer | Three.js/3MF/WebGL/cleanup | Source-policy + manual | Asset/cleanup/prerender CI | +| 29 | Video Dispatch | native video/poster/sources | Manual parity | Exact SES no-escalation test | +| 30 | Audio Program | native audio/realm media | Manual parity | Exact SES no-escalation test | +| 31 | Media Cue Workflow | nested fields/tracked selection | Contract automated + manual | Playback graph integration | +| 32 | Geo Dispatch Map | Leaflet/CSS/tiles/pointer/cleanup | Manual parity | Hosted Sandbox/network test | +| 33 | Chord Conductor | UMD/Tone/WebAudio/gesture/cleanup | Manual parity | Activation/audio/teardown test | +| 34 | Flip Memory | keyboard/click/3D CSS/tracked | Contract automated + manual | Exact accessibility/state test | +| 35 | Sorted Queue | multi-key derived sort | Manual parity | Exact deterministic ordering test | +| 36 | Playback Capability Lab | leader/epoch/sequence/lease | Fixture/manual semantic proof | Real `surfacePlayback` implementation | +| 37 | Viewport Relay | pan/zoom intent/effective state | Fixture/manual semantic proof | Real `surfaceViewport` implementation | +| 38 | CSS Carousel Deck | nested fields/3D CSS/keyframes | CSS contracts + manual | Exact scoped keyframe/state test | +| 39 | Protocol Event Log | ordering/discriminated events/ack | Manual parity | Exact sequence/ack test | +| 40 | Top Layer Studio | popover/backdrop/nested control | Source-policy + manual | Sandbox focus/size integration | + +### Real-card probes + +| Card | Implicit APIs it revealed | Migration requirement | +| -------------------- | -------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | +| Scrabble Stream | lifecycle, scheduling, authenticated AI proxy, collaborative state | Keep ordinary UI in Capsule once named lifecycle/schedule/data capabilities exist; do not generalize to ambient Host access. | +| Tier Maker | modifiers, pointer/drag, focus, clipboard, haptics, transitions, dynamic styles, image media | Preserve current iframe fallback first; then migrate only covered operations to `surface*`; exact image and edit/return tests. | +| Assistant Run | Host tools, Realm runner operations, toolbar placement | `surfaceSlot` for presentation and separately reviewed command capabilities for data/effects. | +| Signet Proposal | enum factory, Markdown, canvas/signature, commands | Trusted Base enum/Markdown portals; isolate only the canvas package if it truly requires browser DOM. | +| Invoice Billing Form | nested Base FieldDefs, configuration, dynamic color style, writes | `surfaceStyle` or trusted Base portal; writable parity is mandatory. | + +## UX parity checklist + +The architecture is not complete when the card eventually renders. It must +retain the behavior users experienced on `main` and the successful parts of +the POC. + +### Interact mode + +- [ ] Trusted Base/default templates appear without starting a Capsule or + Sandbox. +- [ ] Card header title, icon, `headerColor`, theme, and wide-format state are + correct on first stable paint; no `Untitled` settlement. +- [ ] Prerendered HTML is format-correct and remains visible while a live + Capsule/Sandbox becomes interactive. +- [ ] The loading spinner appears beside the Realm icon only while the live + renderer is not interactive. +- [ ] Intrinsic and allocated sizing obey format semantics without double + frames or layout jumps. +- [ ] Edit controls are writable exactly when the Host permission says they + are; no read-only/writable flashing. +- [ ] A failed generation keeps last-known-good output and shows the standard + bottom error overlay. +- [ ] Reload Card deliberately replaces only the selected render generation. +- [ ] Execution signage reports `Direct`, `Capsule`, or `Sandbox` for the + selected format and does not imply a trust label for the whole card. + +### Code mode and HMR + +- [ ] File tree and recent files do not wait for card analysis/rendering. +- [ ] Monaco mounts when source arrives and does not wait for preview startup. +- [ ] A valid local generation updates the existing render island. +- [ ] Matching save/index/SSE echoes acknowledge that generation rather than + restoring old code or remounting the card. +- [ ] Syntax/runtime failures retain last-known-good output and standard error + UI. +- [ ] Source classification and transpilation are cached by source hash. +- [ ] Volatile state lasts at least the agreed quiet period and is shared by + Monaco, AI patches, and out-of-band Realm file writes. +- [ ] Format switching reuses the same module graph and a bounded warm-island + cache. + +### Performance and lifetime + +- [ ] Direct/Base loaders are immune to user-module churn. +- [ ] Capsule runtimes are keyed by explicit principal/generation policy and + never per card by accident. +- [ ] Sandbox documents persist across compatible format/source updates. +- [ ] Nested graph nodes share completed loads and styles by identity. +- [ ] Cross-realm navigation releases inactive runtimes, ports, observers, + timers, styles, media resources, and Store residency. +- [ ] A fitted gallery does not perform global invalidation or create an + iframe per compact child when a safe/inert renderer exists. + +## Missing coverage to add before the refactor deletes POC paths + +The following gaps are higher priority than adding more pairwise unit tests: + +1. Exact Rich Markdown graph: Capsule article -> trusted Rich Markdown portal + -> embedded Capsule card -> authored FieldDef, with Mermaid and edit. +2. Exact mixed-runtime graph: Capsule parent -> linked Sandbox child -> Host + mutation -> parent and child reconciliation. +3. Sandbox child integration: execute a real nested Base field and authored + child inside the child document in CI, not only protocol message mocks. +4. Field configuration cross-product: static, functional, inherited, + per-usage, dynamic update, and custom edit renderer. +5. Query relationship lifecycle: loading, success, empty, error, membership + update, navigation, and teardown. +6. Blocks/yields through a trusted portal, including authored callback and + contextual component identity. +7. One split-module card whose compact formats are Capsule and whose isolated + format is Sandbox. +8. Format-correct prerender placeholder -> Capsule and placeholder -> Sandbox + handoffs with stable dimensions. +9. Permission revocation during a nested edit, with a forged stale write + rejected by generation and principal. +10. Transport-equivalence harness that runs every shipped `surface*` + capability through Direct, Capsule, and Sandbox adapters from one behavior + specification. + +## New-architecture migration ledger + +This ledger should be updated in the new branch. A row may be checked only +when the old path has been replaced and its exact proof passes. + +- [ ] `BoxelRuntime` owns type description, projection, compute/getter + execution, and mutation proposals for Direct, Capsule, and Sandbox adapters. +- [ ] `BoxelRenderRecord` is the only card/field data representation consumed + by confined renderers. +- [ ] `FormatDescription` is open-ended and contains renderer, sizing, + presentation, and prerender policy. +- [ ] `CapsuleComponentRuntime` is the only Capsule-to-Glimmer bridge. +- [ ] Trusted Base components/helpers/modifiers are explicit portals with + enumerated args, blocks, callbacks, and capabilities. +- [ ] `SurfaceService` owns surface registration, generation, dispatch, + revocation, coordination, and Direct/Capsule/Sandbox adapters. +- [ ] Sandbox uses one versioned session protocol and a child-local + Boxel/Glimmer runtime; no Host object or credential crosses. +- [ ] Store grants bind principal, allowed roots/cards/operations, expiry, and + revocation to every sandbox-originated read, search, hydrate, or write. +- [ ] Render graph traces, depth/node budgets, cycle detection, and + graph-aware teardown are implemented. +- [ ] Direct, Capsule, Sandbox, prerender/index, Interact, and Code preview all + consume the same semantic records instead of rebuilding snapshots. +- [ ] The graph gauntlet and missing-coverage list above are green. +- [ ] The 40-card corpus has no red compatibility cells for three consecutive + representative expansion rounds, with every newly discovered contract + minimized into CI before the POC implementation is removed. + +## Review rule + +When a refactor changes one of these contracts, reviewers should be able to +follow a single row from authored syntax, to semantic owner, to boundary +record, to runtime adapter, to pairwise test, to nested graph test, to visible +UX expectation. If that chain cannot be followed, the semantic has not yet +been made explicit enough to safely replace the POC. diff --git a/docs/boxel-execution-runtime-minimality-review.md b/docs/boxel-execution-runtime-minimality-review.md new file mode 100644 index 00000000000..4d0107f4683 --- /dev/null +++ b/docs/boxel-execution-runtime-minimality-review.md @@ -0,0 +1,119 @@ +# Execution runtime minimality review + +Maintainer-lens review of the execution-runtime core (working tree on +`codex/boxel-execution-runtime-architecture`), answering: assuming the +protocol and implementation are correct, is this the smallest, least +blast-radius, most framework-like implementation possible — and if not, what +takes it there? + +## Verdict + +**Mergeable with conditions.** The architecture is framework-shaped where it +matters: `ModuleEvaluator` is a clean injected seam on the existing Loader; +Capsule rendering rides public `setComponentManager` / +`createTemplateFactory` / `capabilities`; the tiers share one projection +pipeline and one record assembler; the total (~11.9k production lines +including runtime-common deltas) sits inside the architecture doc's +9,000–14,000 guardrail. What blocks a clean merge is shape, not size: + +- a speculative module-invalidation chain (~110 lines) in shared + `runtime-common/loader.ts` with zero consumers today (its consumer arrives + with the HMR slice); +- ~400–500 lines of per-transport boilerplate (pending-request tables, + hand-rolled envelope validators, error projectors) wanting one shared + kernel; +- at least eight vocabularies spelled two or three times across + classifier/evaluator/policy — the `networkBearingCSS` extraction proved the + fix, its siblings were left behind; +- two unrelated riders (Percy/Vite alias; `isolated_html` query-engine + projection) that belong in their own PRs. + +Estimated net effect of the response plan: **−800 to −950 production lines, +no protocol or behavior change**, every inline vocabulary reduced to one +declared owner. + +## Findings (ranked) + +1. **F1 — dead speculative invalidation chain** (`Loader.invalidateModule` / + `directModuleDependencies` in runtime-common, `CapsuleModuleEvaluator. +invalidateModule`, `BoxelExecutionService.invalidate`) — zero consumers; + ~110 lines in the most-shared touched file. NOTE: the sandbox HMR slice + (see boxel-sandbox-hmr-extraction.md) is this chain's consumer — either + delete now and re-add with HMR, or land HMR first and keep it. +2. **F2 — empty subclass file** `capsule-runtime-registry.ts` (5 lines), plus + `evictIdle`, `identityFor`, `BoxelRenderFormat` (all caller-less). +3. **F3 — four transports hand-roll one RPC kernel**: pending-request maps, + timeouts, `failPending`/`destroy`/closed-flag, ~230 lines of structural + validators spelling `'x' in value && typeof value.x === 'string'` chains, + six copies of `asError`/`projectedError`. One `sandbox-port-rpc.ts` + (PendingRequestTable + table-driven envelope guard + shared error + projector): −250 to −350 lines. +4. **F4 — un-extracted vocabulary siblings** (each spelled ≥2 places): + document-global at-rule list, top-layer attribute names, Glimmer wire + opcodes, cssVar trusted identity, child format cascade, renderable-format + list. One table per ownership boundary: −60 to −90 lines and closes every + drift channel. +5. **F5 — trusted-import vocabulary exists three times** + (`trusted-modules.ts`, evaluator's dead `defaultTrustedImport`, the facade + install list). Make the option required; drive `installRuntimeFacades` + from a 12-row spec table: −90 to −120 lines. +6. **F6 — module-evaluator machinery in a component file**: + `rewriteDynamicImports` + `createSandboxModuleEvaluator` (~120 non-UI + lines in `boxel-sandbox-runtime.gts`, define-shell duplicating the + Loader's evaluator). Move to `lib/`; ideally the Loader's evaluator grows + an extra-bindings/source-rewrite hook so the shell exists once. +7. **F7 — SafeEvent allowlists** live apart from their protocol type; move + the `as const` arrays into `boxel-execution-protocol.ts` and derive the + type. +8. **F8 — Capsule "Host-less fallback" projection** (~150 lines) is + product-unreachable (the service always supplies a host projection). + Author decision required: it may be intentionally retained as the RP-14.4 + "child re-derives" oracle. +9. **F9 — hand-rolled string/comment masking scanner** (56 lines) exists + only as a perf prefilter before the authoritative Babel pass; a plain + word-boundary regex gate costs at worst one extra parse. +10. **F10 — `MaterializationPurpose` is 60% unproduced** (only + `host-display` / `interactive-edit` are ever created; nothing reads it). + RESOLVED as protocol-reserved, keep: the frozen branch's CI collapse + (one `could not identify card` indexing failure taking down 19/20 host + shards) is empirical evidence the indexing-vs-interactive + materialization split must be encoded before the Sandbox tier reaches + the store — see + [boxel-frozen-branch-parity-audit.md](boxel-frozen-branch-parity-audit.md) + N6. Same family: `invokeCardMethod` (~60 lines, no product callers — + the deferred BXL-command seam). +11. **F11 — duplicated pending-relationship predicate** in + `boxel-projection.ts` (two spellings, one meaning). +12. **F12 — blast radius**: shared-file touches are individually defensible + (router +3, loader-service +2, the boot-gate initializer, search-entries + +1 arg, card-renderer switch). Two riders to split into their own PRs: + the Percy/Vite alias and the `isolated_html` renderSet projection (a + server query change with its own perf considerations). Naming: the tier + adapters are `DirectBoxelRuntime`/`CapsuleBoxelRuntime`/ + `SandboxRuntimeProcess` — the third breaks the arch-doc pattern. + +## Config-extraction shortlist + +| Inline today | Proposed home | +| ---------------------------------------------------------------- | ------------------------------------------------------------- | +| Document-global CSS at-rule list (classifier + 3 policy regexes) | export from `capsule-css-policy.ts`, like `networkBearingCSS` | +| Top-layer attribute names (regex string + Set) | same module, one array | +| Glimmer wire opcodes (10; 14/24; 15/16/22/23) | one `glimmer-wire-opcodes.ts` const table | +| cssVar trusted identity (two predicates) | one exported predicate | +| Trusted framework import vocabulary (three spellings) | `trusted-modules.ts` + facade spec table | +| SafeEvent property allowlists | `boxel-execution-protocol.ts` `as const` arrays | +| Renderable-format literal list | import runtime-common `formats` | +| Child default-format cascade (renderer + evaluator) | one `childFormatCascade(format)` in the protocol module | + +## Response plan (ordered commits) + +1. **S** Prune dead surface (F1 caveat: HMR is the consumer — sequence with + the HMR slice), F2 family, purpose pruning. ~−230 lines. +2. **S** Split the two riders into their own PRs. +3. **M** Shared transport kernel (F3). ~−300 lines. +4. **M** Config-extraction commit (F4/F5/F7 tables). ~−80 lines, drift + channels closed. +5. **M** Evaluator diet (facade table, dead fallback import predicate, + `invokeCardMethod` behind the BXL seam, scanner removal). ~−250 lines. +6. **S** File hygiene (F6 move, F11 predicate, fold + `boxel-execution-policy.ts` into the router, decide F8). diff --git a/docs/boxel-execution-runtime-mutation-protocol.md b/docs/boxel-execution-runtime-mutation-protocol.md new file mode 100644 index 00000000000..06a2830f65b --- /dev/null +++ b/docs/boxel-execution-runtime-mutation-protocol.md @@ -0,0 +1,310 @@ +# Boxel execution runtime: mutation protocol + +## Status and scope + +This is a proposed companion contract for the Boxel execution runtime. It +describes how edits originating in Direct, Capsule, or Sandbox rendering become +canonical Store mutations without sending a rendered projection back as card +JSON:API. + +In this document, **Boxel means Box Element**: a CardDef, FieldDef, FileDef, or +future compatible visual building block. Realm remains the server-side data and +module location. + +This proposal covers: + +- editable projections and bounded write grants; +- serialization of primitive, contained, and linked changes; +- optimistic state, validation, acknowledgement, and rollback; +- typed Commands and long-running effects; and +- identical mutation behavior across Direct, Capsule, and Sandbox execution. + +It does not define BXL policy evaluation, provider credentials, collaborative +text internals, or the complete Realm Server authorization model. BXL +projection is specified separately in +[boxel-execution-runtime-authorization-projection.md](boxel-execution-runtime-authorization-projection.md). + +## Why mutation is not render transport in reverse + +A render record may contain computed values, expanded linked resources, +side-loaded data, presentation metadata, authorization decisions, and resolved +FieldDef state. None of those facts makes them writable card JSON:API. + +The failure mode already has a concrete signature: a UI edits a projected +record and submits the entire object, then the server rejects it because +side-loaded data is not a valid card resource. A more dangerous implementation +could silently persist presentation or foreign relationship data. + +Mutation therefore starts with a named semantic change, not a modified render +snapshot: + +```text +rendered projection + -> named field/relationship/Command intent + -> current write grant + -> serializeCardPatch() + -> canonical JSON:API attributes + relationship identifiers + -> Host admission + -> Realm Server authorization and validation + -> Store acknowledgement or bounded refusal +``` + +## Invariants + +1. The Store owns canonical card documents and relationship identity. +2. Receiving a document, render record, component handle, or Surface capability + grants no mutation authority. +3. A rendered projection is never submitted as a PATCH body. +4. Every change names its target Boxel, path, expected revision, and write + grant. +5. Computed, presentation-only, authorization, side-loaded, unknown, and + foreign values are rejected before network IO. +6. Linked relationships serialize identifiers, never expanded child records. +7. Contained values serialize only through their declared field schema. +8. Writability is stable while the same edit session and authorization + generation remain valid; the UI does not flash read-only while metadata + settles. +9. Optimistic state never becomes authority. The server independently + authorizes and validates every operation. +10. A matching SSE/index event acknowledges the active mutation; it does not + reload an older snapshot or remount the renderer. +11. A stale, denied, invalid, or conflicting mutation preserves the last known + good canonical document and returns a structured result. +12. Direct, Capsule, and Sandbox use the same mutation request and result + semantics. + +## Ownership + +| Concern | Owner | Boundary representation | +| ----------------------------------- | ---------------------------------- | ------------------------------------------------ | +| canonical document and revision | Store | stable Boxel id and document revision | +| server authorization and validation | Realm Server | admitted receipt or structured refusal | +| client authorization projection | `BoxelAuthorizationService` | available Commands and writable paths | +| edit session and optimistic overlay | Host mutation coordinator | edit-session id and generation | +| semantic PATCH serialization | trusted Boxel runtime/Base schema | canonical JSON:API patch | +| authored edit UI | Direct/Capsule/Sandbox renderer | named mutation intent | +| persistence and acknowledgement | Store + Host mutation coordinator | mutation id, canonical revision, changed paths | +| long-running effects | Host capability broker/Run records | typed Command request and durable Run/Job result | + +Mutation is not a `surface*` capability. Surface capabilities coordinate a +mounted visual root. Mutation authority is scoped to principal, target Boxel, +document revision, semantic path, operation, and authorization generation. + +## Protocol records + +The transport is cloneable and descriptive. It contains no Store method, +component callback, or ambient service. + +```ts +export interface BoxelWriteGrant { + id: string; + principal: string; + target: CardIdentifier; + documentRevision: string; + authorizationRevision: string; + writablePaths: string[]; + availableCommands: string[]; + expiresAt?: string; +} + +export type BoxelMutationIntent = + | { + kind: 'set-field'; + path: string; + value: JSONValue; + } + | { + kind: 'set-relationship'; + path: string; + value: CardIdentifier | null; + } + | { + kind: 'set-relationships'; + path: string; + value: CardIdentifier[]; + } + | { + kind: 'invoke-command'; + command: string; + input: Record; + }; + +export interface BoxelMutationRequest { + protocolVersion: number; + mutationId: string; + editSessionId: string; + generation: number; + target: CardIdentifier; + expectedRevision: string; + writeGrantId: string; + intent: BoxelMutationIntent; +} + +export type BoxelMutationResult = + | { + status: 'accepted'; + mutationId: string; + canonicalRevision: string; + changedPaths: string[]; + } + | { + status: 'rejected'; + mutationId: string; + reason: + | 'unauthorized' + | 'read-only' + | 'stale-revision' + | 'invalid-value' + | 'invalid-relationship' + | 'unknown-path' + | 'conflict'; + safeMessage?: string; + currentRevision?: string; + }; +``` + +The exact public names can be revised during implementation, but these facts +must remain explicit. An opaque `set(model)` callback does not carry enough +identity, revision, or authority to be a safe cross-boundary mutation API. + +## Canonical patch serialization + +`serializeCardPatch()` is a trusted semantic operation. It resolves the target +field against the same Boxel description used to build the render record, then +emits the smallest canonical patch. + +It must distinguish: + +| Field shape | Canonical write | +| ------------------- | ------------------------------------------------------------------ | +| primitive | declared JSON value under the canonical attribute path | +| compound/FieldDef | schema-validated contained attribute object | +| contains | contained data owned by the parent field | +| containsMany | ordered contained values with stable identity where needed | +| linksTo | one relationship identifier or null | +| linksToMany | ordered relationship identifiers | +| computed/computeVia | rejected; no canonical write path | +| query field | rejected; query definitions/results are not persisted as card data | +| presentation | rejected unless separately declared canonical card data | + +Field configuration can affect parsing, validation, labels, and controls, but +does not create a write path that the underlying schema lacks. + +## Edit-session behavior + +An edit session binds: + +- target Boxel and canonical revision; +- authorization/write-grant generation; +- projected editable fields and relationships; +- optimistic overlay generation; and +- pending mutation ids. + +The Host can show the editor as soon as source/data and the last valid grant are +available. It should not toggle writable/read-only state because a redundant +card load, template load, or sandbox classification completes. Writability +changes only when the authoritative permission, target, revision/conflict +state, or explicit mode changes. + +Local edits update an optimistic overlay immediately. Lint, server save, +indexing, Matrix acknowledgement, and SSE are separate phases. A matching +server echo settles the pending mutation and advances the canonical revision +without replacing the optimistic generation. An older or non-matching echo +cannot overwrite it. + +## Execution-tier behavior + +### Direct + +Trusted components emit the same named intent through the Host mutation +coordinator. Direct execution does not get a privileged Store-write shortcut. + +### Capsule + +Authored code sends a cloneable mutation request keyed to its execution +identity and write grant. The Host validates identity, target, path, revision, +and grant before calling trusted serialization or persistence. + +### Sandbox + +The iframe sends the same request over the versioned protocol. It receives only +the structured result and any subsequently projected canonical state. Guessing +a field path, relationship id, command name, or mutation id grants nothing. + +Nested delegated rendering does not inherit ambient write authority. Each child +slot receives only the grant appropriate to its target and projected semantic +path. A writable parent cannot mutate a linked child unless a distinct grant +permits it. + +## Commands and asynchronous work + +A Command is used when the operation is more than a canonical field or +relationship PATCH. Examples include approval transitions, image generation, +provider calls, file writes, and workflow advancement. + +The render projection exposes a descriptive Command name and input schema. On +invocation, the Host capability broker checks execution identity and the +current authorization projection; the Realm Server checks again. Long-running +work creates or updates canonical Run/Job cards. Component lifetime is not the +job lifetime. + +Partial success, progress, retry, cancellation, timeout, and duplicate +acknowledgement are durable state transitions. Remounting a card or destroying +an iframe cannot cancel or erase already admitted work unless an explicit +authorized cancellation Command does so. + +## Authorization interaction + +Authorization projection controls whether an edit affordance, writable path, +or Command is presented. Mutation admission controls whether an attempted +operation may execute now. + +Revocation between display and invocation is expected. The Host rejects the +stale write grant and the server rejects the operation. The UI refreshes the +authorization projection and retains user input as a non-authoritative draft +when it is safe to do so; it never retries under broader authority. + +## Acceptance coverage + +The cumulative composition suite must prove: + +- primitive, compound, contained, contained-many, linked, and linked-many + edits in Direct, Capsule, and Sandbox; +- null, empty, zero, false, ordering, deletion, and replacement semantics; +- read/write parity with Direct rendering and no transient read-only flash; +- computed, query, presentation, unknown, and side-loaded values are excluded; +- the exact JSON:API body contains only the intended canonical change; +- nested delegated children cannot borrow a parent or sibling grant; +- stale revision, conflict, invalid value, invalid relationship, and + authorization revocation have structured outcomes; +- optimistic UI is immediate and matching acknowledgement does not remount; +- an older SSE/index echo cannot restore old data or code; +- Command invocation is independently authorized and returns durable state; + and +- teardown releases edit sessions and protocol handles without discarding + admitted server work. + +## Implementation sequence + +1. Define mutation intent, write-grant, request, and result records. +2. Implement `serializeCardPatch()` against the canonical Boxel description. +3. Add a Host mutation coordinator with edit-session and generation identity. +4. Route existing Direct edits through it without changing visible behavior. +5. Add Capsule and Sandbox protocol adapters for the same requests. +6. Separate optimistic application, validation, persistence, indexing, and + acknowledgement state. +7. Treat matching SSE/index events as acknowledgements rather than reloads. +8. Integrate BXL authorization projection and reauthorization. +9. Add the cross-tier mutation matrix and browser interaction tests. +10. Remove whole-snapshot PATCH paths only after unchanged-card parity passes. + +## Open design decisions + +- the stable identity model for members of `containsMany` fields; +- whether simple field intents are batched per animation frame or edit + transaction; +- conflict UX for independent fields versus compound/ordered values; +- how collaborative fields publish materialized revisions into the ordinary + Store acknowledgement stream; and +- how long a safe local draft may survive authorization revocation or target + navigation. diff --git a/docs/boxel-execution-runtime-parity-plan.md b/docs/boxel-execution-runtime-parity-plan.md new file mode 100644 index 00000000000..ae909421307 --- /dev/null +++ b/docs/boxel-execution-runtime-parity-plan.md @@ -0,0 +1,459 @@ +# Execution runtime parity plan: Capsule (SES) and Sandbox (iframe) + +## Who this is for and how to use it + +This is a work order for a coding agent continuing +`codex/boxel-execution-runtime-architecture`. It supersedes ad hoc +hypothesis-chasing (the H1–H11 log in +[boxel-execution-runtime-suite-parity.md](boxel-execution-runtime-suite-parity.md)) +with a slice-ordered plan derived from a first-principles audit of the branch +at `c630781d61`. + +Read first, in this order: + +1. [boxel-execution-runtime-architecture.md](boxel-execution-runtime-architecture.md) + — the target design. It is authoritative for ownership, naming, and + boundary rules. +2. This document — the diagnosis, the work slices, and the working agreements. +3. [boxel-execution-runtime-suite-parity.md](boxel-execution-runtime-suite-parity.md) + — the behavioral log so far (Opening Night green through Capsule; Track and + Playlist red through Sandbox). + +Rules of engagement: + +- Every slice has exit criteria. Do not start slice N+1 features while slice N + exit criteria are red, except where a slice explicitly marks parallelizable + items. +- The frozen reference branch `codex/code-preview-instant-reload` (local; last + commit `3a51b51039 "Fix persistent hosted iframe capsules"`) is an oracle + for behavior and proven algorithms, never a source of orchestration code. +- The staging suite realm + `https://realms-staging.stack.cards/ctse/execution-runtime-suite/` is the + exploratory oracle. CI fixtures are the deterministic layer; the realm is + not a substitute for CI, and CI is not a substitute for looking at the + realm. + +## Diagnosis: why the recent work feels like patching holes + +The H1–H11 fixes were individually reasonable, but ten of eleven are symptoms +of three root causes. Fixing symptoms without the root causes guarantees an +unbounded stream of H12, H13, … Each root cause below names the hypotheses it +explains and the slice that retires it. + +### Root cause 1 — there is no single canonical projection (H1, H3, H4, H7, H8, H10) + +The architecture's central promise is one `BoxelRenderRecord` assembled by one +pure pipeline and consumed identically by every tier. What exists is **three +competing projection builders** that each reverse-engineer the record: + +- Direct's builder in `packages/host/app/lib/direct-boxel-runtime.ts` + (linked values as `{$boxel:{id,type}}` references; real merged + `resolvedConfiguration`; formats from the real prototype chain); +- Capsule's `snapshotFromResource` path in + `packages/host/app/lib/capsule-boxel-runtime.ts` (linked values fully + expanded from `included`; `resolvedConfiguration` always `null`; + `presentation` a different shape; formats from a hard-coded 7-format + fallback list); +- a third mutating pass, `projectTrustedBoxelSemantics` (~210 lines in + `packages/host/app/services/boxel-execution.ts:561-772`), that rewrites the + request document in place before either runtime sees it. + +Every relativeTo/cardInfo/getter/currency/absolute-URL hypothesis was a place +where two of these three disagreed. There is no test that renders the same +card through Direct and Capsule and diffs the records, so each disagreement is +discovered visually on staging and patched at the symptom site. + +### Root cause 2 — the Capsule Card API facade has no conformance oracle (H8, H9, plus the currency shim) + +`cardAPIFacade()` in `packages/host/app/lib/capsule-module-evaluator.ts` +(~1676–2105) is a from-scratch reimplementation of CardDef/FieldDef/field +decorators with no serializer, no `queryableValue`, no `computeVia` string +form, and inert trusted Base field types. Nothing pins it to real Card API +behavior, so every divergence surfaces as a rendering bug and gets a bespoke +patch. The tell is `packages/runtime-common/currency-code-symbol-map.ts`: 179 +lines of vendored ISO currency data added so one Base getter's output could be +reproduced — precisely what the architecture forbids ("The Host never +attempts to copy the function or rediscover field-specific statics such as +currency symbols"). The architecture's actual answer (H8 got halfway there): +**trusted Base semantics are materialized Host-side by the Direct runtime and +cross the boundary as data; the facade only ever needs authored semantics.** +The `esm.run/currency-code-symbol-map` loader shim in +`boxel-loader-compatibility.ts` is separately fine (it host-owns a data-only +package that deployed realms import); the vendored map's use as a semantic +substitute is not. + +### Root cause 3 — no CI rendering path through any tier (H11 and every future H) + +`BoxelExecutionRenderer` — the component every tier flows through — is never +rendered by any test. There is no Capsule DOM-output test (only bundle-shape +tests), no Sandbox child test at all (bootstrap handshake, child shell, slot +modifier: zero coverage). That is why H11 is being debugged by hand on +staging, and why the prime suspect below could sit invisible in a +one-line modifier. The reactive fix loop is a direct consequence: with no +executable conformance layer, "fix" means "make the staging page look right." + +### The H11 prime suspect (verify first, then fix) + +Static reading of the code identifies a concrete ordering defect that produces +exactly the reported symptom (iframe mounts, no authored DOM, no error): + +1. `createSandbox()` parents the iframe into a hidden **parking div** + (`services/boxel-execution.ts:373-376`) before + `sandbox-runtime-process.ts:339` sets `src`. +2. The child boots there, hands over `port2`, and the parent removes its + bootstrap listener permanently (`sandbox-runtime-process.ts:335`). +3. `getRenderSlot()` awaits `renderClient.render(...)` — the child paints + inside the 0×0 parking div and replies `ok: true`. +4. Only then does the renderer set `state.slot`, mounting the slot `
`, + and `BoxelSandboxSlotModifier` runs + `element.replaceChildren(slot.iframe)` + (`modifiers/boxel-sandbox-slot.ts:20`). +5. **Re-parenting an `