From 8d22001837a640b0faf410c86ef7785f38296bc4 Mon Sep 17 00:00:00 2001 From: Taras Mankovski Date: Fri, 4 Sep 2026 00:22:06 -0400 Subject: [PATCH 1/7] =?UTF-8?q?=E2=9C=A8=20Make=20``=20evaluate?= =?UTF-8?q?=20complete=20XMD=20programs=20(#713)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `` produces a program and never runs it. `` is now the composition site where one runs: paired content renders a producer into a private program buffer, `program={value}` supplies complete-program source directly, and both admit a complete root and evaluate it in the current execution — its lifecycle, journal, cancellation scope, providers, working directory and the authority in force where the element was written. The forms are disjoint and ambiguity refuses before program content is produced and before any program effect runs. The root deciding what happens is the program: its frontmatter, props schema, `returns` and `` selection all apply, root props are the explicit `props` object rather than the caller's, and a value root requires `as`. Complete programs record their own `evaluate_program` durable event; the restricted `generated_xmd` admission #369 delivered is unchanged and complete-program support is not reachable through `source` or `allow`. A partial continuation expands the retained source and restores completed nested effects; a changed program at the same occurrence is stale input and neither source runs. Canonical execution answers the new `Component.expandProgram()` from the frame it already holds, which is what makes site authority a fact about the frame rather than something a prop, binding or middleware return value could supply — and what keeps a producer's private closure from crossing into the program. --- architecture.md | 8 +- packages/cli/src/cli.ts | 32 +- packages/cli/src/syntax.ts | 6 +- .../tests/evaluate-program-component.test.ts | 268 +++++++++ packages/core/host.ts | 32 ++ packages/core/mod.ts | 5 + packages/core/src/component-api.ts | 31 ++ packages/core/src/components/Evaluate.ts | 217 ++++++++ packages/core/src/expand.ts | 145 +++++ packages/core/src/program-evaluation.ts | 456 ++++++++++++++++ packages/core/src/types.ts | 27 + .../core/tests/program-evaluation.test.ts | 507 ++++++++++++++++++ .../workflow/src/deno/workspace/evaluate.ts | 64 ++- .../tests/generated-agent-component.test.ts | 101 ++++ site/routes/index.tsx | 42 +- specs/executable-mdx-spec.md | 148 ++++- specs/workflow-workspace-spec.md | 24 +- 17 files changed, 2086 insertions(+), 27 deletions(-) create mode 100644 packages/cli/tests/evaluate-program-component.test.ts create mode 100644 packages/core/src/components/Evaluate.ts create mode 100644 packages/core/src/program-evaluation.ts create mode 100644 packages/core/tests/program-evaluation.test.ts diff --git a/architecture.md b/architecture.md index 8d50192f0..4f36248e8 100644 --- a/architecture.md +++ b/architecture.md @@ -41,9 +41,10 @@ Existing documents and code get aligned to this section retroactively. | pinned commit | the commit obtained by resolving a base once; it remains the workflow run's starting repository state even as the run creates descendant commits | | document target | an addressable static heading in a root document's own Markdown flow, named by the canonical path of heading labels that reaches it; selecting one executes the preamble, each ancestor's own content, and that heading's complete subtree | | Prompt | a person's original request, in ordinary natural language. `xmd plan` takes exactly one | -| Plan | the executable program produced from a Prompt: an Executable Markdown document combining readable prose that expresses the Prompt's intent with the components that carry it out, each placed beside the prose describing what it does. It begins with one descriptive level-one heading. A Plan is what `xmd plan` approves and then delivers as source: printed to stdout by default, or written to an `--output` file. Plan produces a program; Run executes a program from the host or CLI; composition decides whether and when a planned program runs. It is not a synonym for a workflow, a policy document or any executable Markdown file | +| Plan | the executable program produced from a Prompt: an Executable Markdown document combining readable prose that expresses the Prompt's intent with the components that carry it out, each placed beside the prose describing what it does. It begins with one descriptive level-one heading. A Plan is what `xmd plan` approves and then delivers as source: printed to stdout by default, or written to an `--output` file. Plan produces a program; Evaluate evaluates a program in the current XMD execution; Run executes a program from the host or CLI; composition decides whether and when a planned program runs. It is not a synonym for a workflow, a policy document or any executable Markdown file | | plan command document | the one exact checked-in first-party Markdown value root `xmd plan` executes. It is the command's adapter and nothing else: it projects the request into ``, supplies the session, and returns the approved source. Its body is those two elements and no prose, because its rendered transcript is the progress the command writes to stderr. It is not itself a Plan. Internal: no command-line option selects another one, and no repository component search can answer for it | | packaged `` Component | the one exact checked-in first-party Markdown text component that converts a prompt into a Plan, `packages/cli/src/documents/Plan.md`, declared to every ordinary run as the public ``. It owns and implements the Plan authorship workflow — the Prompt wording, the draft and repair loops, the `` branches, human review, revision, approval, stopping, exhaustion and the automatic final explanation turn — and produces the exact approved Plan source as what it renders: written bare it emits those bytes, and `as` is ordinary text capture that binds them and emits nothing. Neither form evaluates the source. Every Plan-producing turn in it states the complete Plan requirements for itself, so a replacement may add or correct a title rather than only carry one forward. Both surfaces expand these exact bytes under one origin and one digest; there is no generated TypeScript copy and no second Markdown implementation. Its four phase components are private to it, and it is not itself a Plan | +| composition site | the authored `` element where a complete XMD program is admitted and evaluated. It is a place in a document rather than a boundary: the program runs in the current execution, under the authority in force where the element was written, and the element and the loop iteration reaching it identify one execution of that program. A composition site is not a child document, a child process, a second host profile or a second journal | | authorship profile | the trusted-host assembly the packaged `` Component runs its authored turns under, installed by its own `` inside the invocation that owns it rather than around an execution — which is what makes it the same frame whether `xmd plan`, an ordinary document, or a configured `` run child asked. Which Agent context goes under it is a trusted-host capability the declaration carries — the production ACPX one built from the run's Agent stack, or the deterministic one a canonical `` child declaration produced — and a host that supplies none states the sentence a `` written there is refused with. The fixed policy is installed in one place for both, so a second provider cannot bring a weaker one: its fixed inputs, a constrained Agent provider, Elicitation, the fixed first-party components and the host-declared ``. It uses no repository component search and exposes no custom root, and the policy it installs is not readable from the command line. Its working directory is one host-owned directory dedicated to the logical session, keyed by the digest of that name, created empty and required to be empty on the way in. An explicitly named session's directory is durable, because continuation derives the same session identity from it; an invocation-unique default session's is scope-owned, claimed before it is created, and exactly one cleanup is attempted after profile teardown and before admission on every ending — the leaf removed non-recursively when it is still the empty directory that was handed over, and left as found with the command failing terminally when it has gained content or vanished. Where those directories live is a host dependency no caller or document selects | | upgrade command document | the one exact checked-in first-party Markdown streaming text root `xmd upgrade` executes to select and install a published release. It owns the exact-tag grammar, release selection, semantic-version comparison, consent, the status, already-current and installation branches, and the wording of every refusal and report; its rendered body is the command's output rather than a value it returns. Internal: no command-line option selects another one, and no repository component search can answer for it | | upgrade assembly | what one runtime-named entrypoint states about the `xmd` that is running: its provenance, reported version, invoked executable path, platform, architecture, release target when the release publishes one, and — for an eligible compiled macOS or Linux host alone — the factory for the four phases an installation needs. It describes how this `xmd` is running, never how its files arrived | @@ -3895,7 +3896,10 @@ Status is measured against main. | history fork | creates a new run from one compatible checkpoint and retained Workspace root, under a new immutable definition and normalized props | built on the #368 stack, Deno provider only | | workflow Agent session | a workflow document's `` runs under a profile the host attaches only for a live or partial run: an empty host-owned working directory instead of any Workspace, checkout or caller path, no MCP servers, an empty requested native tool set, and `deny-all` with a permission path that denies every native request and fails the turn that asked without reaching the public permission chain. Within a run a session is identified by the Agent/Session expansion identity the engine derived — the authored name is descriptive, so two sibling `` elements are two sessions — routed inside a placement bound to its element and good for one use, so a kept placement cannot be substituted for the next. The conversation is retained as a row in the run's own database with the provider, resolved agent command and policy fingerprint beside it as compatibility attributes. The order is placement, the backend's acceptance of the session's first turn, the provider's canonical tagged assertion, then the mapping commit — and only then is anything that turn produced exposed. A placement is inert: it creates no provider session and writes no row. Occupancy of a provider key is not an assertion, and a record held for a first turn nobody accepted asserts nothing at all; the pre-commit window reconciles only from exactly one canonical assertion, and a missing, conflicting, replaced or ambiguous assertion is one explicit refusal that starts no replacement. Deleting a run removes the row with the run and the provider-session directory beside it, and reports the categories. The profile selects ACP-only capability explicitly — no native-launch advertisement and no client-native attachment advertisement — rather than inheriting the provider package's ordinary-run sets by omission, and it supplies no machine session coordinator, construction-route store or executable observer: a workflow session belongs to a run, and the machine-wide account describes a different thing entirely | built on the #302 stack, with the explicit ACP-only selection from #561; the portable proof that an adapter honours an empty tool set is tracked by #496 and does not widen the ceiling | | generated-XMD admission | admits one Agent-generated fragment through the trusted-host seam: host policy is a `read` table and a `write` table of exact pinned identities, each carrying the authored forms it is admitted for, and an authored `allow` selects a canonical non-empty subset of the closed classes — omitted means `read`. The complete source is preflighted inside one `generated_xmd` durable effect before its first generated effect; only the pinned identity the selected classes hold for that name **and** that form executes; and the admitted source, class selection, selected root, every selected entry with its forms, the identity and form of each element named, and the normalized request policy are retained in that effect's own result — so a continuation restores the decision without reading the current candidate and expands only the retained source. The roots are an as-of-admission retained basis checked by membership — the run's own later root publications and an advanced retained current root pass, while a lost admission root or lost selected root refuses — and every non-root term is checked exactly, refusing a run whose classes, identities, forms or requests have moved. The admission and every nested generated effect are offered inline by the owning expansion in authored order, so a partial continuation restores each completed one without another live execution. Each admitted effect is retained by its own ordinary record, and a read's value is collected while a mutation's is not | built on the #369 stack, continuation basis amended by #589; core owns the mechanics and the workflow policy wrapper is internal | -| `` | the workflow host's component an authored document writes where an observation should happen. The host does not register it: it **declares** it to the execution through `ExecutionInstallation.components`, captured before any installation runs, and canonical execution calls its factory once for that attachment with the claimant it minted and registers what comes back. Registration provides availability only — a name a trusted document may write — and carries none of the authority. Its schema is closed on one required `source` string and one optional `allow` array selecting a non-empty duplicate-free subset of the closed effect classes `read` and `write` — omitted means `read` — and paired content is refused. It declares no `returns` and answers with a detached value — `{ observations: [{ name, value }], output }`, each admitted read's own returned value under the name the fragment invoked it by, in invocation order, with whatever the fragment rendered under `output` rather than instead of them, and the pinned identity that produced one left in the retained admission rather than copied here — so an admitted ``, which renders nothing at all, still reaches the document. An admitted mutation contributes no entry and no receipt, so a write-only fragment binds `{ observations: [], output: "" }`; `as` is valid for every selection and binds that same shape. An ordinary `as` captures that value by reference, and an authored `` renders it into the next ``: deciding how a value becomes text is the document's. Every ceiling comes from values the host captured at installation — the run's retained roots and its authoritative current root read from the run's own storage at invocation, as-of-admission provenance a continuation holds by membership so the run's own later publications and an advanced retained current root invalidate nothing, core's pinned self-closing `` read, the write table of core's paired ``, this package's lexical `` built from the definition the ordinary registration owns, and core's self-closing ``, and `` only when the captured request ceiling is non-empty — and no prop, binding, context or middleware return value supplies or widens one. `allow` selects among those tables and adds nothing to them; approval, when a workflow needs one, is authored control flow before the element. Its durable operation is named through that claimant, on the exact invocation the engine handed it and in that invocation's own frame — not from a context a document could rebind, a contextual Api answer, a definition, or a registry answer. Generated source never resolves through the registration: the evaluator consults only its own closed table of pinned identities. It is deliberately not wrapped in `printErrors`, so a refused fragment stops the authored loop rather than becoming text the next turn could read as a read that happened | built on the #302 stack, extended by #369 | +| `` | the restricted generated-fragment form of ``, which the workflow host declares beside the complete-program forms below. The form an authored document writes where an observation should happen. The host does not register it: it **declares** it to the execution through `ExecutionInstallation.components`, captured before any installation runs, and canonical execution calls its factory once for that attachment with the claimant it minted and registers what comes back. Registration provides availability only — a name a trusted document may write — and carries none of the authority. It takes a `source` string and one optional `allow` array selecting a non-empty duplicate-free subset of the closed effect classes `read` and `write` — omitted means `read` — refuses paired content, and cannot be combined with `program`, with paired content, or with `props`; each of those refuses before the durable name is claimed and before the candidate is read. It declares no `returns` and answers with a detached value — `{ observations: [{ name, value }], output }`, each admitted read's own returned value under the name the fragment invoked it by, in invocation order, with whatever the fragment rendered under `output` rather than instead of them, and the pinned identity that produced one left in the retained admission rather than copied here — so an admitted ``, which renders nothing at all, still reaches the document. An admitted mutation contributes no entry and no receipt, so a write-only fragment binds `{ observations: [], output: "" }`; `as` is valid for every selection and binds that same shape. An ordinary `as` captures that value by reference, and an authored `` renders it into the next ``: deciding how a value becomes text is the document's. Every ceiling comes from values the host captured at installation — the run's retained roots and its authoritative current root read from the run's own storage at invocation, as-of-admission provenance a continuation holds by membership so the run's own later publications and an advanced retained current root invalidate nothing, core's pinned self-closing `` read, the write table of core's paired ``, this package's lexical `` built from the definition the ordinary registration owns, and core's self-closing ``, and `` only when the captured request ceiling is non-empty — and no prop, binding, context or middleware return value supplies or widens one. `allow` selects among those tables and adds nothing to them; approval, when a workflow needs one, is authored control flow before the element. Its durable operation is named through that claimant, on the exact invocation the engine handed it and in that invocation's own frame — not from a context a document could rebind, a contextual Api answer, a definition, or a registry answer. Generated source never resolves through the registration: the evaluator consults only its own closed table of pinned identities. It is deliberately not wrapped in `printErrors`, so a refused fragment stops the authored loop rather than becoming text the next turn could read as a read that happened | built on the #302 stack, extended by #369 | +| complete-program evaluation | admits one complete XMD root and evaluates it at the authored `` site, in the current execution: its lifecycle, journal, cancellation scope, contextual providers, working directory or Workspace, and the authority in force where the element was written. There is no child process, host profile, root lifecycle or journal of its own — program source requests behavior and grants none. The root's own frontmatter, imports, metadata, props schema, `returns` and `` selection decide what it does; a text root renders its selected output or binds it as a string under `as`, a value root requires `as` and binds its schema-validated result, and root props are the explicit `props` object defaulting to `{}` rather than the caller's own. Ordinary non-props caller bindings are visible as a copy, so nothing the program binds escapes, and relative paths resolve from the evaluation site's source origin. The producer's temporary authority is not inherited: an enclosing declaration's private closure does not cross, so a private name written in a program resolves to nothing, while imports and capabilities genuinely present at the site remain. Canonical execution answers `Component.expandProgram()` from the frame it already holds, which is what makes site authority a fact about the frame rather than something a prop, binding or middleware return value could supply | built by #713 on the #722 `` contract; core owns the mechanics and every host that offers `` declares them | +| `` complete-program forms | the two ways an author names a program at a composition site: paired content, which renders a source-producing component into a private program buffer, and `program={value}`, which supplies complete-program source directly. Optional `props={object}` is the root's props input and is valid only for these forms. The two are mutually exclusive and one is required; every ambiguous or misplaced combination refuses before program content is produced and before any program effect runs, so an ambiguous element does not even render its producer. Paired content is a private buffer rather than a quotation — an executable component written there executes under ordinary XMD semantics while producing the text sent to admission — and the element removes only the framing its own wrapper contributed: the line break after the opening tag with the indentation following it, the line break before the closing tag with the indentation preceding it, and the indentation those lines share. Interior bytes are untouched, so a sole `` producer's trailing newline survives and paired evaluation and `program={plan}` admit the same bytes and the same content digest. `program={value}` is exact supplied source and is neither trimmed, dedented nor newline-adjusted | built by #713 | +| `evaluate_program` durable effect | the complete-program admission, recorded once before the first program effect and never as a widening of `generated_xmd` — the two are decisions about different kinds of source. `description.input` carries the source digest, the explicit props, the evaluation-site source origin and whether the result is captured; the result carries the exact admitted source, the root mode, the components the program names with their authored forms, those same terms and the validated root props — or, for a refusal, the refused class alone and nothing of the source. A partial continuation expands the retained source and restores the nested effects that already committed, repeating no planning; a changed program at the same evaluation occurrence is stale input and refuses before either the current or the retained source runs, so neither silently wins. The source digest identifies the program artifact and the authored element and loop iteration identify one execution of it, so two occurrences of one artifact have separate nested effect identities and the digest deduplicates neither | built by #713 | | generated mutation proposals | lets an Agent propose constrained executable changes that a separate admission then performs against the run's own Workspace | built on the #369 and #567 stacks, with directory creation added by #643: the standard Deno profile's write table is core's paired `File:write`, the paired `@executablemd/workflow/composition/dir-v2#Dir` and core's self-closing `File.Delete`, in that retained order and followed by any host extension. `allow={["write"]}` intentionally authorizes Dir's persistent recursive directory creation; its versioned identity makes every continuation retained under the former non-mutating Dir identity refuse before generated execution. Admitted mutations run as the ordinary components they are through the run's effect transactions, and the evaluator adds no mutation API or receipt. Approval is authored control flow before the write-enabled element. Local Git, Git-host, issue, process, execution, credential and external-write effects are outside the class | | Deno-local DOFS provider | owns one authoritative SQLite/DOFS connection per run path, captures arbitrary canonical retained roots, privately restores them, and atomically coordinates one Workspace mutation with its filtered Yield | built on the #365 stack; public document filesystem effects and the CLI lifecycle route to it on the #366 stack | | scoped Worker Shell | executes `just-bash` through the Workspace adapter inside a Deno Worker | containment and effect-transaction POCs complete (#351, #357); production integration unbuilt | diff --git a/packages/cli/src/cli.ts b/packages/cli/src/cli.ts index e5edbdfc1..b5c811bf6 100755 --- a/packages/cli/src/cli.ts +++ b/packages/cli/src/cli.ts @@ -67,6 +67,7 @@ import { inspectDocument, agentIdentityComponents, installAgentComponents, + programEvaluationComponents, registerComponents, retainedSource, rootSourcePath, @@ -889,6 +890,25 @@ export interface DocumentMode { installations?: readonly ExecutionInstallation[]; } +/** + * Whether a host this invocation attached already declares ``. + * + * A workflow run brings its own, which offers the complete-program forms and + * the restricted generated-fragment form only that profile has. Declaring the + * run profile's beside it would give one execution two identity components of + * one name, which canonical execution refuses — correctly, because a name names + * durable work in one domain. + * + * Read from the installations themselves rather than from a second flag: the + * host that owns the name is the one that declared it, and a flag saying so + * would be a second thing to keep true. + */ +function declaresEvaluate(installations: readonly ExecutionInstallation[] | undefined): boolean { + return (installations ?? []).some((installation) => + (installation.components ?? []).some((component) => component.name === "Evaluate"), + ); +} + export type HostServiceInstaller = () => Operation; /** @@ -1160,7 +1180,17 @@ function* runDocument( [ ...(mode.installations ?? []), { - components: agentIdentityComponents(), + // `` travels with `` and for the same reason: producing + // a program and running one are two halves of one vocabulary, and a + // profile that offers the first without the second can produce an + // artifact it has no way to carry out. Both name durable work after + // their own invocation, so both are declared rather than registered. + components: [ + ...agentIdentityComponents(), + ...(mode.testing || declaresEvaluate(mode.installations) + ? [] + : programEvaluationComponents()), + ], // The `run` profile's own vocabulary. `xmd test` is a different profile // and does not gain `` at its root — but the production run child // it can launch is the run profile, and gets it below. diff --git a/packages/cli/src/syntax.ts b/packages/cli/src/syntax.ts index 39915006f..f815691d0 100644 --- a/packages/cli/src/syntax.ts +++ b/packages/cli/src/syntax.ts @@ -19,6 +19,7 @@ import { AGENT_REGISTRATIONS, agentIdentityComponents, inspectSyntax, + programEvaluationComponents, registerComponents, } from "@executablemd/core"; import type { @@ -59,7 +60,10 @@ export function* syntaxCatalog(includes: readonly string[]): Operation` and `` are one vocabulary: a catalog that described + // how to produce a program without describing how to run one would + // advertise half of what the run profile has. + components: [...agentIdentityComponents(), ...programEvaluationComponents()], // `` is part of the run profile, so a catalog that left it out would // describe a vocabulary no run has. Described from the packaged bytes: // inspection mints nothing, so it reports the Component's identity and diff --git a/packages/cli/tests/evaluate-program-component.test.ts b/packages/cli/tests/evaluate-program-component.test.ts new file mode 100644 index 000000000..61b104faa --- /dev/null +++ b/packages/cli/tests/evaluate-program-component.test.ts @@ -0,0 +1,268 @@ +/** + * Tier EP — `` in the ordinary run profile (issue #713). + * + * Core owns the admission and the expansion; what only a shelled-out `xmd run` + * can show is that the profile a person actually gets has this component at + * all, that a program evaluated there behaves like the root it is, and that the + * authority it runs under is the site's rather than the producer's. + * + * Every case runs the real binary with captured stdio, so what is asserted is + * what a caller sees on stdout, on stderr, and in the exit status. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { runCli } from "@executablemd/test-support/launch"; +import { ensure, scoped } from "effection"; +import type { Operation } from "effection"; +import { ensureDir, rm, writeTextFile } from "@effectionx/fs"; +import { exec } from "@effectionx/process"; +import { randomUUID } from "node:crypto"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; + +/** A text root that reads a prop of its own, and says so when it has none. */ +const TEXT_PROGRAM = [ + "---", + "props:", + " release:", + " type: string", + " default: none", + "---", + "", + "Program released {props.release}.", + "", +].join("\n"); + +/** A value root, which has somewhere to put a result only when `as` is written. */ +const VALUE_PROGRAM = [ + "---", + "returns:", + " type: object", + " properties:", + " ok: { type: boolean }", + " required: [ok]", + "---", + "", + "", + "", +].join("\n"); + +/** A document that binds a program and evaluates it at a later site. */ +function deferred(program: string, element: string): string { + return [``, "", element, ""].join("\n"); +} + +describe( + "Tier EP — evaluating complete programs in a run", + { sanitizeOps: false, sanitizeResources: false }, + () => { + it("EP1: the run profile evaluates a program written as content", function* () { + const result = yield* runCli([ + "run", + "-e", + ["", " # Composed", "", " The program ran.", "", ""].join("\n"), + "--raw", + ]).expect(); + + expect(result.stdout).toContain("The program ran."); + // Once: the source the wrapper held was a private buffer, not output. + expect(result.stdout.split("The program ran.")).toHaveLength(2); + }); + + it("EP2: the run profile evaluates a program supplied as `program`", function* () { + const result = yield* runCli([ + "run", + "-e", + deferred(TEXT_PROGRAM, ''), + "--raw", + ]).expect(); + + expect(result.stdout).toContain("Program released 1.4.0."); + }); + + it("EP3: a program receives explicit props and never the caller's", function* () { + const document = [ + "---", + "props:", + " release:", + " type: string", + "---", + "", + ``, + "", + "Caller released {props.release}.", + "", + "", + "", + ].join("\n"); + + const result = yield* runCli([ + "run", + "-e", + document, + "--props", + '{"release":"9.9.9"}', + "--raw", + ]).expect(); + + expect(result.stdout).toContain("Caller released 9.9.9."); + // The program declared the same prop name and was handed none, so its own + // default answers. An ambient root props object is not inherited. + expect(result.stdout).toContain("Program released none."); + }); + + it("EP4: a value root binds its result under `as`", function* () { + const result = yield* runCli([ + "run", + "-e", + deferred( + VALUE_PROGRAM, + '\n\nDecided {decided.ok}.', + ), + "--raw", + ]).expect(); + + expect(result.stdout).toContain("Decided true."); + }); + + it("EP5: a value root without `as` refuses and evaluates nothing", function* () { + const result = yield* runCli([ + "run", + "-e", + deferred(VALUE_PROGRAM, ""), + ]).join(); + + expect(`${result.stdout}${result.stderr}`).toContain("requires `as`"); + }); + + it("EP6: an element that names both a program and content refuses", function* () { + const result = yield* runCli([ + "run", + "-e", + deferred(TEXT_PROGRAM, "\nnot a program\n"), + ]).join(); + + expect(`${result.stdout}${result.stderr}`).toContain("not both"); + expect(result.stdout).not.toContain("Program released"); + }); + + it("EP7: a program cannot reach a producer's private components", function* () { + // `` is one of the five capabilities only ``'s own + // bytes may write. A program is not those bytes however it was produced, + // so the name resolves to nothing at this site. + const result = yield* runCli([ + "run", + "-e", + deferred('\n', ""), + ]).join(); + + const reported = `${result.stdout}${result.stderr}`; + expect(reported).toMatch(/AdmitPlan/); + expect(reported).not.toContain("approved"); + }); + + it("EP8: `xmd syntax` describes the run profile's own ``", function* () { + const result = yield* runCli(["syntax", "--json"]).expect(); + const catalog: { categories: { entries: { name: string; description?: string }[] }[] } = + JSON.parse(result.stdout); + const entry = catalog.categories + .flatMap((category) => category.entries) + .find((component) => component.name === "Evaluate"); + + expect(entry).toBeDefined(); + expect(entry?.description).toContain("evaluates a complete program"); + // The run profile has no restricted-fragment form, so its catalog must + // not advertise the prop that carries one. + expect(entry?.description).not.toContain("source={fragment}"); + }); + }, +); + +/** + * A workflow run declares an `` of its own — the complete-program + * forms plus the restricted generated-fragment form only that profile has. The + * run profile must not declare a second component of that name beside it: one + * name names durable work in one domain, and canonical execution refuses two. + * + * This is the seam where that goes wrong, because it is the CLI's own + * installation array rather than anything either package assembles alone. + */ +const WORKFLOW_PROGRAM = [ + "# Compose", + "", + '', + "", + "", + "", +].join("\n"); + +interface Fixture { + readonly repository: string; + readonly runs: string; + readonly home: string; +} + +function* git(repository: string, args: string[]): Operation { + const result = yield* exec("git", { arguments: args, cwd: repository }).expect(); + if (result.code !== 0) { + throw new Error(`git ${args.join(" ")} failed: ${result.stderr}`); + } +} + +function useWorkflowFixture( + files: Record, + body: (fixture: Fixture) => Operation, +): Operation { + return scoped(function* () { + const root = join(tmpdir(), `xmd-ep-${randomUUID()}`); + const fixture: Fixture = { + repository: join(root, "repository"), + runs: join(root, "runs"), + home: join(root, "home"), + }; + yield* ensure(() => rm(root, { recursive: true, force: true })); + yield* ensureDir(fixture.repository); + yield* ensureDir(fixture.home); + for (const [name, content] of Object.entries(files)) { + const path = join(fixture.repository, name); + yield* ensureDir(join(path, "..")); + yield* writeTextFile(path, content); + } + yield* git(fixture.repository, ["init", "-q", "--initial-branch=main", "."]); + yield* git(fixture.repository, ["config", "user.email", "tier-ep@example.test"]); + yield* git(fixture.repository, ["config", "user.name", "Tier EP"]); + yield* git(fixture.repository, ["add", "-A"]); + yield* git(fixture.repository, [ + "-c", + "commit.gpgsign=false", + "commit", + "-q", + "-m", + "definition", + ]); + return yield* body(fixture); + }); +} + +describe( + "Tier EP — one `` per execution", + { sanitizeOps: false, sanitizeResources: false }, + () => { + it("EP9: a workflow run keeps its own `` and gains no second one", function* () { + yield* useWorkflowFixture({ "flows/compose.md": WORKFLOW_PROGRAM }, function* (fixture) { + const started = yield* runCli(["workflow", "start", "--id=compose-1", "flows/compose.md"], { + cwd: fixture.repository, + env: { HOME: fixture.home, XMD_WORKFLOW_RUNS: fixture.runs }, + }).join(); + + const reported = `${started.stdout}${started.stderr}`; + // The exact sentence canonical execution refuses a duplicate with. A + // run profile that declared its own beside the workflow's would fail + // every workflow run, not only one that writes ``. + expect(reported).not.toContain('two identity components called "Evaluate"'); + expect(reported).toContain("The workflow program ran."); + }); + }); + }, +); diff --git a/packages/core/host.ts b/packages/core/host.ts index 3d1dabd02..ddfd5b942 100644 --- a/packages/core/host.ts +++ b/packages/core/host.ts @@ -119,6 +119,38 @@ export type { DeclaredMarkdownComponent } from "./src/components/declared-markdo * author. */ export { installInvocationAgentProvider } from "./src/agent/launch-install.ts"; +/** + * Complete-program evaluation, for the hosts that expose ``. + * + * The seventh act of infrastructure, and the counterpart of generated XMD + * beside it. A complete program is admitted and evaluated *in the current + * execution* — its lifecycle, journal, cancellation scope, providers, working + * directory and authority — so what a host contributes is the declaration, and + * canonical core owns the admission, the retained terms and the expansion. + * + * It records its own durable event. The restricted `generated_xmd` admission is + * a different decision about a different kind of source, and widening it to + * carry programs would make one record answer for two contracts. + */ +export { + EVALUATE_DESCRIPTION, + EVALUATE_ORIGIN, + evaluateProgramElement, + programEvaluationComponents, + programProperties, + programProps, + props as evaluateProgramProps, + selectProgramForm, +} from "./src/components/Evaluate.ts"; +export { + EVALUATE_PROGRAM, + evaluateProgram, + programDigest, + ProgramEvaluationError, + pairedProgramSource, +} from "./src/program-evaluation.ts"; +export type { ProgramEvaluationRequest } from "./src/program-evaluation.ts"; + export { evaluateGeneratedXmd, GeneratedXmdError, diff --git a/packages/core/mod.ts b/packages/core/mod.ts index d31eb6d15..5ae74c11e 100644 --- a/packages/core/mod.ts +++ b/packages/core/mod.ts @@ -276,6 +276,11 @@ export { installAgentComponents, } from "./src/agent/components.ts"; export type { AgentComponentsOptions } from "./src/agent/components.ts"; + +// `` — the composition site a complete XMD program runs at. Declared +// by the host that offers it, like the Agent components above, because its +// implementation names durable work after its own invocation. +export { EVALUATE_DESCRIPTION, programEvaluationComponents } from "./src/components/Evaluate.ts"; export { Agent } from "./src/agent/agent-api.ts"; export type { AgentApi, diff --git a/packages/core/src/component-api.ts b/packages/core/src/component-api.ts index d52900865..6a951fdf5 100644 --- a/packages/core/src/component-api.ts +++ b/packages/core/src/component-api.ts @@ -29,6 +29,8 @@ import type { EvalEnv, FunctionComponentDefinition, Modifier, + ProgramBody, + ProgramOutcome, SourcePosition, } from "./types.ts"; @@ -148,6 +150,27 @@ export interface ComponentApi { * recursion here and no reaching into the caller. */ tryContent(slot?: string): Operation; + /** + * Expand a complete XMD program at this invocation's site + * (spec §5.7). + * + * The site is the whole point. A program admitted by `` runs in the + * current execution — its lifecycle, journal, cancellation scope, contextual + * providers and working directory — and under the authority the enclosing + * expansion already holds: the imports a closed execution closes, and the + * identity domains it minted. None of that is reachable from a component, so + * canonical execution answers this with the values it is already carrying, + * exactly as it answers `content()`. + * + * What it does not carry across is the enclosing declaration's private + * closure. A private component belongs to the bytes its declaration authored, + * and a program is not those bytes, so a private name written in one is the + * ordinary unresolved failure. + * + * The program arrives parsed and validated: admission owns the decision, and + * this owns running it. + */ + expandProgram(program: ProgramBody): Operation; /** * Decide what an ordinary function-component failure means (spec §6.9). * @@ -256,6 +279,12 @@ export const Component: Api = createApi("Component", ); }, // deno-lint-ignore require-yield + *expandProgram(_program: ProgramBody): Operation { + throw new Error( + "Component.expandProgram() has no provider: not inside a function component invocation.", + ); + }, + // deno-lint-ignore require-yield *handleFailure(failure: ComponentFailure): Operation { throw failure.error; }, @@ -283,3 +312,5 @@ export const hasCapture: Operations["hasCapture"] = Component.oper export const capture: Operations["capture"] = Component.operations.capture; export const handleFailure: Operations["handleFailure"] = Component.operations.handleFailure; +export const expandProgram: Operations["expandProgram"] = + Component.operations.expandProgram; diff --git a/packages/core/src/components/Evaluate.ts b/packages/core/src/components/Evaluate.ts new file mode 100644 index 000000000..8f7657b16 --- /dev/null +++ b/packages/core/src/components/Evaluate.ts @@ -0,0 +1,217 @@ +/** + * `` — the composition site where a complete XMD program runs + * (specs/executable-mdx-spec.md §5.7). + * + * `` produces a program and never runs it. This is where an author says + * that it should run, and the two ways of saying so are the two forms of this + * element: + * + * ```md + * + * Inspect the release inputs and recommend a version. + * + * ``` + * + * ```md + * Inspect the release inputs and recommend a version. + * + * + * ``` + * + * Paired content is a private program buffer, not a quotation. Whatever is + * written inside runs under ordinary XMD semantics while it produces text, and + * that text is what admission is offered — none of it reaches the surrounding + * document. Any source-producing component may be used there, provided what it + * renders is a complete admissible root. + * + * ## The forms are disjoint, and ambiguity refuses first + * + * `program` and paired content are mutually exclusive, and one of them is + * required. Both together, and neither, are refused from the element's own + * shape — read from the invocation canonical execution issued — before any + * content is produced and before the durable admission exists. A program that + * would have been produced by rendering the content is not produced, so an + * ambiguous element performs nothing at all. + * + * ## What it answers with + * + * Whatever the program's own root contract says. A text root's selected output + * is the element's rendered result, or the string bound by `as`; a value root's + * schema-validated result is bound by `as`, which it requires. The engine + * decides between rendering and binding exactly as it does for any component, + * so this returns the one value and says nothing about where it goes. + * + * Deliberately not wrapped in `printErrors`. A refused or failed program stops + * the document unless the author put a recovery boundary around it: a program + * that did not run must not become text a later step reads as though it had. + */ + +import type { Operation } from "effection"; + +import { content, hasBinding } from "../component-api.ts"; +import { getExpansion } from "../expansion.ts"; +import { isJsonObject } from "../json.ts"; +import { + evaluateProgram, + pairedProgramSource, + ProgramEvaluationError, +} from "../program-evaluation.ts"; +import type { ComponentInvocation, IdentityClaimant } from "../invocation-identity.ts"; +import type { FunctionComponent, Json, JsonObject, PropsSchema } from "../types.ts"; +import type { IdentityComponent } from "../invocation-identity.ts"; + +/** The origin that identifies this component wherever it is declared. */ +export const EVALUATE_ORIGIN = "@executablemd/core/program"; + +/** + * The complete-program half of ``'s schema. + * + * Shared with the workflow host, which adds its restricted-fragment props to + * these rather than restating them: two hand-written copies of one schema are + * two schemas, and no test catches the day they stop agreeing. + */ +export const programProperties: Record = { + program: { type: "string" }, + props: { type: "object" }, +}; + +/** The whole schema an ordinary run's `` accepts. */ +export const props: PropsSchema = { + type: "object", + properties: { ...programProperties }, + additionalProperties: false, +}; + +/** What the catalog and the syntax reference say this component is for. */ +export const EVALUATE_DESCRIPTION = + "Evaluate XMD source in the current execution. `` evaluates a " + + "complete program."; + +/** + * Which complete-program form this element is, or why it is neither. + * + * Decided from the authored shape and the props alone. It reads no content, + * performs nothing, and is what every host calls before a program can exist — + * so a misplaced or ambiguous combination is refused at the same point whatever + * else that host also accepts. + */ +export function selectProgramForm( + elementProps: Record, + written: boolean, + /** What a host with a wider grammar calls an element that names nothing. */ + absent: string = " evaluates a program written as its content or supplied as `program`.", +): "content" | "program" { + const supplied = elementProps.program; + if (supplied !== undefined && written) { + throw new ProgramEvaluationError( + " takes a program either as `program` or as its content, not both.", + ); + } + if (supplied === undefined && !written) { + throw new ProgramEvaluationError(absent); + } + if (supplied === undefined) { + return "content"; + } + if (typeof supplied !== "string") { + throw new ProgramEvaluationError(" takes `program` as complete XMD program source."); + } + return "program"; +} + +/** + * The root props this element supplies, defaulting to none. + * + * A program receives what the site wrote here and nothing else. The caller's + * own root props are not adopted, silently or otherwise: a program that needs + * them is written at a site that passes them. + */ +export function programProps(value: Json | undefined): JsonObject { + if (value === undefined) { + return {}; + } + if (!isJsonObject(value)) { + throw new ProgramEvaluationError(" takes `props` as an object."); + } + return value; +} + +/** + * Evaluate the program this element names, in the current execution. + * + * Shared by every host that exposes the complete-program forms, so the + * admission, the retained terms and the refusals are one implementation rather + * than one per profile. + * + * `claim` is spent on the invocation the engine handed over, in the frame it + * handed it over in: the durable identity of this evaluation is this element's + * own invocation, so a continuation restores the program this position admitted + * rather than whichever one a later turn happens to hold. + */ +export function* evaluateProgramElement( + elementProps: Record, + invocation: ComponentInvocation, + claim: IdentityClaimant, + /** What a host with a wider grammar calls an element that names nothing. */ + absent?: string, +): Operation { + const rootProps = programProps(elementProps.props); + + // Claimed before the authored shape is read, because reading it is only + // meaningful once the invocation has been authenticated. The claimant answers + // for an invocation this execution minted where resolution selected this + // implementation, and for nothing a handler routed here from elsewhere — so + // an element's own form is what the claim proved it was, never what a + // borrowed invocation reports about somebody else's element. + const id = yield* claim(invocation); + + const form = selectProgramForm(elementProps, invocation.hasContent(), absent); + const captured = yield* hasBinding(); + const expansion = yield* getExpansion(); + + // Produced only once the element's shape has been accepted, so an ambiguous + // one renders no producer and performs none of its effects. + // + // A paired buffer gives up the framing its wrapper contributed and nothing + // else; `program` is exact supplied source, so whitespace an author put at + // either end of it is part of the program and stays there. + const source = + form === "content" ? pairedProgramSource(yield* content()) : String(elementProps.program); + + const outcome = yield* evaluateProgram({ + id, + source, + props: rootProps, + origin: expansion.position?.path ?? "", + captured, + ...(expansion.position === undefined ? {} : { position: expansion.position }), + }); + + return outcome.kind === "value" ? outcome.value : outcome.output; +} + +/** + * What a host declares to an execution for one attachment. + * + * `` is not registered with ordinary components: its implementation + * names durable work after its own invocation, so canonical execution builds it + * from the claimant it minted (executable-mdx-spec §5.6). A run whose host + * declares none has no `` at all. + */ +export function programEvaluationComponents(): readonly IdentityComponent[] { + return [ + { + name: "Evaluate", + origin: EVALUATE_ORIGIN, + description: EVALUATE_DESCRIPTION, + props, + factory: (claim: IdentityClaimant): FunctionComponent => + function* Evaluate( + elementProps: Record, + invocation: ComponentInvocation, + ): Operation { + return yield* evaluateProgramElement(elementProps, invocation, claim); + }, + }, + ]; +} diff --git a/packages/core/src/expand.ts b/packages/core/src/expand.ts index ffe05f103..7612093a4 100644 --- a/packages/core/src/expand.ts +++ b/packages/core/src/expand.ts @@ -27,6 +27,8 @@ import type { FunctionComponentDefinition, Json, CodeBlockContext, + ProgramBody, + ProgramOutcome, ReturnsSchema, SourcePosition, } from "./types.ts"; @@ -3185,6 +3187,18 @@ function* expandFunctionComponent( values: captureEnv, }); }, + // Canonical execution's own answer, built from the frame it is + // already holding. A program admitted here runs under the site's + // authority and against the site's bindings, and neither is + // reachable from the component that asked. + *expandProgram([program], _next) { + return yield* expandProgramBody(program, { + counter, + callerValues: captureEnv, + checkedFailures, + authority: programAuthority(authority), + }); + }, *tryContent([slotName], _next) { const outcome = yield* handle.tryProject({ kind: "slot", @@ -4102,3 +4116,134 @@ function* expandValueBody( } return selected.value; } + +/** + * What the site contributes to a program's expansion. + * + * Every member is read from the frame canonical execution is already holding + * when it answers `expandProgram`, so none of it is anything a document, a + * component or middleware supplied. + */ +interface ProgramSite { + counter: BlockCounter; + /** The bindings the evaluation site can see, copied so nothing escapes. */ + callerValues: Record; + checkedFailures: CheckedFailures | undefined; + authority: ExpansionAuthority | undefined; +} + +/** + * The three names a Markdown body publishes for its own projections. + * + * They close over the invocation that installed them, so carrying them into a + * program would hand it the enclosing component's content. A program has + * content of its own — none — and reaches nobody else's. + */ +const PROJECTION_BINDINGS: readonly string[] = ["renderChildren", "render", "useContent"]; + +/** + * The environment a program's body runs in. + * + * Ordinary caller bindings are visible, because a program evaluated where they + * are in scope is written to read them. They are visible *read-only* in the + * only way that matters: this is a copy, so a binding the program creates or + * overwrites lands here and reaches no caller. + * + * `props` is the program's own, never the caller's — an ambient root props + * object is not something a program silently inherits. + */ +function programEnvironment( + callerValues: Record, + props: Record, +): EvalEnv { + const values: Record = {}; + for (const [name, value] of Object.entries(callerValues)) { + if (!PROJECTION_BINDINGS.includes(name)) { + values[name] = value; + } + } + values.props = props; + const environment: EvalEnv = { values }; + liveEnvironment(environment); + return environment; +} + +/** + * Expand a complete XMD program where `` admitted it (spec §5.7). + * + * A root's own structure applies here exactly as it does at the top of a + * document: `` selects what renders, a `returns` declaration makes the + * body a value body whose `` answers, and a text root is fail-capable + * while a value root is not. + * + * The body is not this program's caller's, so it carries no children, no slot + * substitution and an empty hide set. What it does carry is the site's own + * authority and block counter — the program's durable work belongs to the run + * that evaluated it. + */ +export function* expandProgramBody( + program: ProgramBody, + site: ProgramSite, +): Operation { + return yield* scoped(function* () { + yield* provideEnv(programEnvironment(site.callerValues, program.props)); + if (program.returns !== undefined) { + // A value root has no rendered result to fall back on, so an undecided + // error is the evaluation's failure rather than text in the document. + yield* ErrorMode.set("throw"); + const value = yield* expandValueBody( + program.name, + program.returns, + program.bodySegments, + [], + program.meta, + program.props, + new Set(), + site.counter, + undefined, + passthroughClaim, + program.path, + site.checkedFailures, + site.authority, + undefined, + ); + return { kind: "value", value }; + } + yield* ErrorMode.set("output"); + const expanded = yield* expandBody( + program.bodySegments, + [], + program.meta, + program.props, + new Set(), + site.counter, + undefined, + passthroughClaim, + undefined, + program.path, + site.checkedFailures, + site.authority, + undefined, + ); + return { kind: "text", output: renderSegments(expanded) }; + }); +} + +/** + * The authority a program evaluated at this site runs under. + * + * Everything the site holds crosses — the imports a closed execution closes, + * the identity domains it minted, the exact-source record — except the private + * closure. A private component belongs to the declaration whose exact bytes + * authored it, and a program is somebody else's text however it got here, so a + * private name written in one resolves to nothing. + */ +function programAuthority( + authority: ExpansionAuthority | undefined, +): ExpansionAuthority | undefined { + if (authority === undefined) { + return undefined; + } + const { privates: _privates, ...rest } = authority; + return rest; +} diff --git a/packages/core/src/program-evaluation.ts b/packages/core/src/program-evaluation.ts new file mode 100644 index 000000000..067d8d453 --- /dev/null +++ b/packages/core/src/program-evaluation.ts @@ -0,0 +1,456 @@ +/** + * Evaluating a complete XMD program at an explicit composition site + * (specs/executable-mdx-spec.md §5.7). + * + * Producing a program and running one are separate choices. `` produces + * approved source and never runs it; this is the operation an author writes + * where that source should run, either around the producer or later against + * what it bound: + * + * ```md + * + * Inspect the release inputs and recommend a version. + * + * ``` + * + * ```md + * Inspect the release inputs and recommend a version. + * + * + * ``` + * + * ## It is a composition site, not a child document + * + * The program runs in the current execution: its lifecycle, its journal, its + * cancellation scope, its contextual providers, its working directory, and the + * authority already in force where the element was written. There is no child + * process, no second host profile, no second root lifecycle and no journal of + * its own. Program source *requests* behavior; it grants none. + * + * What does not cross is the producer's temporary authority. By the time a + * program is admitted, ``'s authorship profile and its private phases + * have torn down, and a private component belonging to an enclosing + * declaration answers only for elements that declaration's exact bytes + * authored. Source a producer returned can invoke neither. + * + * ## The root's own contract applies + * + * The admitted source is a root, so its frontmatter, imports, metadata, props + * schema, `returns` declaration and `` selection are what decide what + * it does. A text root renders its selected output where the element was + * written, or binds that text under `as` and emits none of it. A value root + * requires `as` and binds its schema-validated result — without one it refuses + * before the first program effect, because there would be nowhere for the + * result to go. + * + * Root props are the explicit `props` object and default to `{}`. Ambient root + * props are never adopted: a program that declares `props` states what it needs + * and receives what the site handed it, validated against its own schema before + * anything runs. + * + * ## Admission is its own durable event + * + * Complete programs are not generated fragments, and the two are not one + * record. `evaluate_program` is this boundary's own durable effect, and the + * restricted `generated_xmd` admission is left exactly as #369 delivered it. + * + * Everything needed to prove the same evaluation is retained before the first + * program effect: the exact source and its digest, the explicit props, the + * evaluation-site source origin, the root mode, the components the program + * names with the forms it writes them in, and whether the result is captured. + * A partial continuation expands the retained source, restores the nested + * effects that already committed, and repeats no planning. A changed program at + * the same evaluation occurrence is stale input: it refuses before either the + * current or the retained source can run, so neither silently wins. + * + * ## The digest is the artifact; the site is the occurrence + * + * A source digest identifies the program. The authored element and the loop + * iteration it was reached through identify one execution of it. Evaluating the + * same approved program at two sites is two executions with two nested effect + * identities, and the digest never deduplicates them. + */ + +import { createDurableOperation } from "@executablemd/durable-streams"; +import type { Json as DurableJson } from "@executablemd/durable-streams"; +import type { Operation } from "effection"; + +import { expandProgram } from "./component-api.ts"; +import { sourceDigest } from "./components/declared-markdown.ts"; +import { parseRootMarkdownDefinition } from "./definition.ts"; +import { validateBodyStructure } from "./expand.ts"; +import { isJsonObject, parseJson } from "./json.ts"; +import { sourceDescription } from "./source-position.ts"; +import { validateProps } from "./validate.ts"; +import type { Json, JsonObject, ProgramOutcome, Segment, SourcePosition } from "./types.ts"; + +/** The durable effect type one complete-program evaluation records. */ +export const EVALUATE_PROGRAM = "evaluate_program"; + +/** A program this site will not evaluate. */ +export class ProgramEvaluationError extends Error { + override name = "ProgramEvaluationError"; +} + +/** What the name a diagnostic and a durable record call this program. */ +const PROGRAM = "program"; + +/** The ways a program can be refused, and the only thing a refusal carries. */ +type Refused = "source" | "structure" | "props" | "capture"; + +/** + * What each refusal says. + * + * Named by class rather than by cause. A parser's complaint about approved + * program source is the one text a diagnostic must not echo back, because a + * refusal is written into the run's history and the source is somebody else's + * bytes. + */ +const REFUSAL: Record = { + source: " was given source that is not a complete XMD program.", + structure: " was given a program whose body structure is not valid.", + props: " was given props the program's own schema refuses.", + capture: + " requires `as` to evaluate a program that declares `returns`: its result has " + + "nowhere else to go.", +}; + +/** What a retained admission that cannot be read as one says. */ +const UNREADABLE = "the retained complete-program admission record cannot be read as one."; + +/** + * What a changed evaluation says. + * + * Stale input, not a divergence: this element is the same occurrence it was, + * and what it is being asked to evaluate is not what it evaluated. Neither + * source runs. + */ +const STALE = + " was resumed with a different program than the one this evaluation admitted."; + +/** A refusal carried as a value rather than thrown out of the admission. */ +class Refusal extends Error { + constructor(readonly refused: Refused) { + super(REFUSAL[refused]); + } +} + +/** What a trusted host asks this boundary to evaluate. */ +export interface ProgramEvaluationRequest { + /** This evaluation occurrence's durable identity. */ + readonly id: string; + /** Complete-program source, exactly as the form that supplied it produced it. */ + readonly source: string; + /** The root props, supplied explicitly. */ + readonly props: JsonObject; + /** The source origin of the authored element, which relative paths resolve from. */ + readonly origin: string; + /** Whether the element was written with `as`. */ + readonly captured: boolean; + /** Where the element was written, as diagnostic journal data. */ + readonly position?: Readonly; +} + +/** One component the program names, with the form it is written in. */ +interface Named { + readonly name: string; + readonly form: "self-closing" | "paired"; +} + +/** + * The terms an evaluation is admitted under, and held to on a continuation. + * + * The props here are the ones the site supplied, not the ones the program's + * schema produced from them. What a continuation has to prove is that it is + * asking for the same evaluation; comparing against defaults the schema filled + * in would compare the admission with itself. + */ +interface Terms { + readonly digest: string; + readonly props: JsonObject; + readonly origin: string; + readonly captured: boolean; +} + +/** The decision this run recorded, restored from its own durable record. */ +type Admission = + | { readonly decision: "refused"; readonly refused: Refused } + | { + readonly decision: "admitted"; + readonly source: string; + readonly mode: "text" | "value"; + readonly named: readonly Named[]; + readonly terms: Terms; + /** The props the program ran with, after its own schema validated them. */ + readonly validated: JsonObject; + }; + +/** + * The program a paired `` produced, with the wrapper's framing off. + * + * A paired element renders its producer into a private buffer, and the buffer + * holds two things that belong to the `` element rather than to the + * program: the line break after the opening tag with whatever indentation + * follows it, and the line break before the closing tag with whatever + * indentation precedes it. Each is removed exactly once. Nothing else is + * touched — no trimming, no trailing newline added or taken away, and every + * interior byte the producer emitted survives. + * + * Removing exactly one line break at each end is what makes the two + * compositions of one approved plan agree. `` returns source ending in + * its own newline, so a paired buffer ends with that newline followed by the + * wrapper's; taking the wrapper's leaves the producer's, and the bytes and the + * digest are the ones `program={plan}` supplies directly. + * + * The shared indentation of the lines after the first is the other half of what + * a wrapper contributes, and only to a program written out literally: a + * producer's result is spliced in as one value, so its own lines carry no + * indentation to share. That is why the first line is excluded — its + * indentation came off with the leading break. + * + * `program={value}` is exact supplied source and passes through none of this. + */ +export function pairedProgramSource(text: string): string { + const body = text.replace(/^[ \t]*\r?\n[ \t]*/, "").replace(/[ \t]*\r?\n[ \t]*$/, ""); + if (body.length === 0) { + return ""; + } + const [first, ...rest] = body.split("\n"); + const indents = rest + .filter((line) => line.trim().length > 0) + .map((line) => line.length - line.trimStart().length); + const shared = indents.length === 0 ? 0 : Math.min(...indents); + if (shared === 0) { + return body; + } + const dedented = rest.map((line) => (line.trim().length === 0 ? line : line.slice(shared))); + return [first, ...dedented].join("\n"); +} + +/** + * The content digest that identifies a program artifact. + * + * The same hash a host states about declared Markdown, so what identifies an + * approved program here and what identifies packaged bytes elsewhere are one + * function rather than two that agree until they do not. + */ +export function programDigest(source: string): string { + return sourceDigest(source); +} + +/** Every component the program names, with the form each element is written in. */ +function names(segments: readonly Segment[], found: Named[]): Named[] { + for (const segment of segments) { + if (segment.type === "component") { + found.push({ name: segment.name, form: segment.selfClosing ? "self-closing" : "paired" }); + names(segment.children, found); + } + } + return found; +} + +/** + * Decide this program, once, inside the durable executor. + * + * A refusal is returned rather than thrown, for the reason the restricted + * evaluator returns its own: throwing out of a durable executor journals the + * error and its stack, and a refusal caused by somebody else's source would put + * host paths into the run's history to say what one word already says. + */ +function* admitProgram(request: ProgramEvaluationRequest, terms: Terms): Operation { + try { + const { source } = request; + if (source.trim().length === 0) { + throw new Refusal("source"); + } + let parsed; + try { + parsed = yield* parseRootMarkdownDefinition(PROGRAM, request.origin, source); + } catch { + throw new Refusal("source"); + } + const { definition } = parsed; + if (validateBodyStructure(definition.bodySegments, definition.returns) !== undefined) { + throw new Refusal("structure"); + } + const mode = definition.returns === undefined ? "text" : "value"; + // Before the props are looked at, because a value root with nowhere to put + // its result is refused whatever its props would have validated to. + if (mode === "value" && !terms.captured) { + throw new Refusal("capture"); + } + let props: Record; + try { + props = yield* validateProps(PROGRAM, { ...terms.props }, definition.props); + } catch { + throw new Refusal("props"); + } + return parseJson({ + decision: "admitted", + source, + mode, + named: names(definition.bodySegments, []), + terms, + validated: props, + }); + } catch (error) { + if (error instanceof Refusal) { + return parseJson({ decision: "refused", refused: error.refused }); + } + throw error; + } +} + +/** + * The decision this run recorded, read back from the journal. + * + * Parsed rather than trusted: a replay hands back whatever the history holds, + * and a record somebody else wrote is not an admission because it happens to + * have the right keys. + */ +function readAdmission(value: Json): Admission | undefined { + if (!isJsonObject(value)) { + return undefined; + } + const { decision } = value; + if (decision === "refused") { + const { refused } = value; + return typeof refused === "string" && Object.hasOwn(REFUSAL, refused) + ? { decision, refused: refused as Refused } + : undefined; + } + if (decision !== "admitted") { + return undefined; + } + const { source, mode, named, terms, validated } = value; + if (typeof source !== "string" || (mode !== "text" && mode !== "value")) { + return undefined; + } + const invocations = readNames(named); + const retained = readTerms(terms); + if (invocations === undefined || retained === undefined || !isJsonObject(validated)) { + return undefined; + } + return { decision, source, mode, named: invocations, terms: retained, validated }; +} + +function readNames(value: Json | undefined): readonly Named[] | undefined { + if (!Array.isArray(value)) { + return undefined; + } + const found: Named[] = []; + for (const entry of value) { + if (!isJsonObject(entry)) { + return undefined; + } + const { name, form } = entry; + if (typeof name !== "string" || (form !== "self-closing" && form !== "paired")) { + return undefined; + } + found.push({ name, form }); + } + return found; +} + +function readTerms(value: Json | undefined): Terms | undefined { + if (!isJsonObject(value)) { + return undefined; + } + const { digest, props, origin, captured } = value; + if (typeof digest !== "string" || typeof origin !== "string" || typeof captured !== "boolean") { + return undefined; + } + if (!isJsonObject(props)) { + return undefined; + } + return { digest, props, origin, captured }; +} + +/** + * Whether a continuation is asking for the evaluation it was granted. + * + * Compared whole and exactly. The digest answers for the program, and the other + * three answer for the site it was admitted at: a program evaluated with + * different props, from a different source origin, or into a different + * disposition is a different evaluation however familiar its bytes are. + * + * The props compared are the ones each side supplied, so a schema default is + * not mistaken for agreement. + */ +function sameEvaluation(retained: Terms, current: Terms): boolean { + return ( + retained.digest === current.digest && + retained.origin === current.origin && + retained.captured === current.captured && + JSON.stringify(retained.props) === JSON.stringify(current.props) + ); +} + +/** What the durable record carries about this evaluation before it happens. */ +function admissionInput(terms: Terms): Record { + return { + digest: terms.digest, + origin: terms.origin, + captured: terms.captured, + props: terms.props, + }; +} + +/** + * Admit one complete program and evaluate it at this site. + * + * An `Operation`, so the admission and every durable effect the program + * performs belong to the caller's own durable sequence, offered inline by the + * owning expansion in authored order. A partial continuation offers the same + * sequence and restores each effect that already committed rather than + * performing it again. + */ +export function* evaluateProgram(request: ProgramEvaluationRequest): Operation { + const terms: Terms = { + digest: programDigest(request.source), + props: request.props, + origin: request.origin, + captured: request.captured, + }; + + const stored = yield createDurableOperation( + { + type: EVALUATE_PROGRAM, + name: `${PROGRAM}:${request.id}`, + input: admissionInput(terms), + ...sourceDescription(request.position), + }, + () => admitProgram(request, terms), + ); + + const decided = readAdmission(parseJson(stored)); + if (decided === undefined) { + throw new ProgramEvaluationError(UNREADABLE); + } + if (decided.decision === "refused") { + throw new ProgramEvaluationError(REFUSAL[decided.refused]); + } + // Before the retained source is parsed and before the first program effect: a + // continuation asking for a different program is stale input, and refusing + // here is what keeps the retained source from silently winning over it. + if (!sameEvaluation(decided.terms, terms)) { + throw new ProgramEvaluationError(STALE); + } + + // The retained source is what expands, so a continuation runs the program + // this run admitted rather than whatever a later caller happens to hold. + const { definition } = yield* parseRootMarkdownDefinition( + PROGRAM, + decided.terms.origin, + decided.source, + ); + + return yield* expandProgram({ + name: PROGRAM, + meta: definition.meta, + props: decided.validated, + ...(definition.returns === undefined ? {} : { returns: definition.returns }), + bodySegments: definition.bodySegments, + path: decided.terms.origin, + }); +} diff --git a/packages/core/src/types.ts b/packages/core/src/types.ts index 3dc9214ed..5db64fe32 100644 --- a/packages/core/src/types.ts +++ b/packages/core/src/types.ts @@ -406,3 +406,30 @@ export interface SampleContext { /** Name of the component that initiated the sample call. */ componentName?: string; } + +/** + * A complete XMD program, parsed and validated, ready to expand at the site + * that admitted it (specs/executable-mdx-spec.md §5.7). + * + * The parse is the admission's, so what crosses here is a decision already + * taken: the root's own frontmatter, metadata, props schema and `returns` + * declaration, its body, and the source origin its relative resolution is + * anchored to. Nothing about it is read again while it expands. + */ +export interface ProgramBody { + /** What a diagnostic calls this program. */ + readonly name: string; + readonly meta: Record; + /** The root props, already validated against the program's own schema. */ + readonly props: Record; + /** Present for a value root, absent for a text root. */ + readonly returns?: ReturnsSchema; + readonly bodySegments: Segment[]; + /** The source origin the evaluation site resolves from. */ + readonly path: string; +} + +/** What expanding a program produced: selected output, or a returned value. */ +export type ProgramOutcome = + | { readonly kind: "text"; readonly output: string } + | { readonly kind: "value"; readonly value: Json }; diff --git a/packages/core/tests/program-evaluation.test.ts b/packages/core/tests/program-evaluation.test.ts new file mode 100644 index 000000000..b2accbcdf --- /dev/null +++ b/packages/core/tests/program-evaluation.test.ts @@ -0,0 +1,507 @@ +/** + * Tier PE — evaluating a complete XMD program + * (specs/executable-mdx-spec.md §5.7). + * + * `` is a composition site: it admits a complete root and runs it in + * the current execution. Everything here is about one of four claims. + * + * **Composition is explicit, and the two forms mean the same thing.** A program + * produced inside the element and a program supplied as `program` admit the + * same bytes and the same digest, and a producer's own output never reaches the + * surrounding document. + * + * **The root's own contract decides the result.** Text and value roots follow + * their `` and `returns` rules, and explicit props are validated + * against the program's schema before anything runs. + * + * **Ambiguity refuses before effects.** A structural preflight case puts a + * negative-control effect after a malformed construct and proves it did not + * run. + * + * **One occurrence keeps one decision.** The admission retains the exact + * source, digest, props, origin and mode; a continuation restores it, a changed + * program at the same site is stale input, and two sites are two executions + * whatever their digests say. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { scoped } from "effection"; +import type { Operation } from "effection"; + +import { createDurableOperation, InMemoryStream } from "@executablemd/durable-streams"; +import type { DurableEvent } from "@executablemd/durable-streams"; + +import { executeInstalled, programEvaluationComponents } from "../host.ts"; +import type { IdentityClaimant, IdentityComponent } from "../host.ts"; +import type { ComponentInvocation } from "../src/invocation-identity.ts"; +import { pairedProgramSource, programDigest } from "../host.ts"; +import { retainedSource } from "../src/root-source.ts"; +import type { Json } from "../src/types.ts"; + +const ROOT_PATH = "documents/compose.md"; + +interface Attempt { + readonly output?: string; + readonly failure?: string; + readonly events: DurableEvent[]; +} + +/** + * A durable effect a program can perform, and a record of every time it really + * happened. + * + * The list is what tells a restored effect from a repeated one: replay hands + * back the retained result without entering the executor, so an entry here + * means this run performed the effect rather than remembering it. + */ +const performed: string[] = []; + +/** + * The approved bytes a source-producing component returns. + * + * This is what `` hands back: a complete root ending in its own newline. + * Core has no ``, so the equivalence of the two compositions is proved + * against a stand-in that returns exactly what one does. + */ +const APPROVED = "# Report\n\nThe program ran.\n"; + +/** + * What the producer returns on the next run. + * + * A resumed evaluation is one where the document is unchanged and the producer + * is not: the element is the same occurrence, written at the same offset, and + * what it is being handed differs. Changing the document text instead would + * move the element and ask about a different occurrence entirely. + */ +let approved = APPROVED; + +/** Every time a producer really rendered its approved source. */ +const produced: string[] = []; + +function probeComponents(): readonly IdentityComponent[] { + return [ + { + name: "Source", + origin: "test/source", + props: { type: "object", properties: {}, additionalProperties: false }, + // The claim is unspent: this stands in for a text component that returns + // approved source, and naming durable work is not what it is here for. + factory: () => + // deno-lint-ignore require-yield + function* Source(): Operation { + produced.push(approved); + return approved; + }, + }, + { + name: "Probe", + origin: "test/probe", + props: { + type: "object", + properties: { mark: { type: "string" } }, + required: ["mark"], + additionalProperties: false, + }, + factory: (claim: IdentityClaimant) => + function* Probe( + elementProps: Record, + invocation: ComponentInvocation, + ): Operation { + const mark = String(elementProps.mark); + const id = yield* claim(invocation); + const stored = yield createDurableOperation( + { type: "probe", name: `probe:${id}` }, + // deno-lint-ignore require-yield + function* () { + performed.push(mark); + return `performed ${mark}`; + }, + ); + return String(stored); + }, + }, + ]; +} + +/** Run one document with `` declared, exactly as a run profile does. */ +function run(source: string, options: { stream?: InMemoryStream } = {}): Operation { + return scoped(function* () { + const stream = options.stream ?? new InMemoryStream(); + const execution = yield* executeInstalled( + { ...retainedSource(ROOT_PATH, source), stream, includes: [] }, + [{ components: [...programEvaluationComponents(), ...probeComponents()] }], + ); + const result = yield* execution; + const events = yield* stream.readAll(); + return result.ok + ? { output: typeof result.value === "string" ? result.value : "", events } + : { failure: result.error.message, events }; + }); +} + +/** + * The journal a run interrupted immediately after one effect would hold. + * + * A completed journal replays as a terminal result, which proves nothing about + * restoration: the run never re-enters its own body. Truncating after the event + * a run committed is exactly the history an interruption leaves behind, and it + * is what a partial continuation is offered. + */ +function through(events: DurableEvent[], type: string): DurableEvent[] { + const index = events.findIndex( + (event) => event.type === "yield" && event.description.type === type, + ); + if (index === -1) { + throw new Error(`the run recorded no ${type} event`); + } + return events.slice(0, index + 1); +} + +/** Every complete-program admission this run recorded. */ +function admissions(events: DurableEvent[]): DurableEvent[] { + return events.filter( + (event) => event.type === "yield" && event.description.type === "evaluate_program", + ); +} + +/** What one admission decided, and the terms it was granted under. */ +function decision(event: DurableEvent): Record { + const result = event.type === "yield" ? event.result : undefined; + if (result === undefined || result.status !== "ok") { + throw new Error("the admission did not settle successfully"); + } + return result.value as Record; +} + +describe("Tier PE — complete-program evaluation", () => { + it("PE1 evaluates a program supplied as `program`", function* () { + const attempt = yield* run( + [ + '', + "", + "", + "", + ].join("\n"), + ); + + expect(attempt.failure).toBeUndefined(); + expect(attempt.output).toContain("The program ran."); + expect(admissions(attempt.events)).toHaveLength(1); + }); + + it("PE2 evaluates a producer's program once and emits none of its source", function* () { + produced.length = 0; + const attempt = yield* run(["", " ", "", ""].join("\n")); + + expect(attempt.failure).toBeUndefined(); + // The program ran, so its own rendered line is here. + expect(attempt.output).toContain("The program ran."); + // Once. The producer rendered the source into the private buffer and none + // of those bytes reached the document: a second copy would mean the + // approved source was emitted beside the evaluation. + expect(attempt.output?.split("The program ran.")).toHaveLength(2); + // The heading is the program's own rendered output, and it appears once + // for the same reason: the source the producer emitted was buffered, not + // written into the document beside the evaluation. + expect(attempt.output?.split("# Report")).toHaveLength(2); + expect(produced).toEqual([APPROVED]); + }); + + it("PE3 admits the same bytes and digest for the direct and deferred forms", function* () { + const paired = yield* run(["", " ", "", ""].join("\n")); + const deferred = yield* run( + [ + ``, + "", + "", + "", + ].join("\n"), + ); + + const pairedTerms = decision(admissions(paired.events)[0]!); + const deferredTerms = decision(admissions(deferred.events)[0]!); + // The paired projection equals the captured bytes naturally: the wrapper's + // own line breaks came off and the producer's newline stayed. + expect(pairedTerms.source).toBe(APPROVED); + expect(deferredTerms.source).toBe(APPROVED); + expect((pairedTerms.terms as Record).digest).toBe( + (deferredTerms.terms as Record).digest, + ); + }); +}); + +describe("Tier PE — the program source a form admits", () => { + // deno-lint-ignore require-yield + it("PE4 takes off the paired wrapper's framing and nothing else", function* () { + // A producer's result spliced after the wrapper's line break and indent. + // One line break comes off each end, so the producer's own trailing + // newline survives. + expect(pairedProgramSource("\n # Report\n\nBody\n\n")).toBe("# Report\n\nBody\n"); + // A program written out literally: the indentation every line shares is the + // wrapper's, and the last line break is the closing tag's. + expect(pairedProgramSource("\n # Report\n\n Body\n")).toBe("# Report\n\nBody"); + // Blank lines inside keep whatever the author put on them. + expect(pairedProgramSource("\n # Report\n \n Body\n")).toBe("# Report\n \nBody"); + // Written on one line, there is no framing to take off. + expect(pairedProgramSource("# Report")).toBe("# Report"); + }); + + // deno-lint-ignore require-yield + it("PE5 leaves `program` bytes exactly as supplied", function* () { + // The discriminator: whitespace at either end of a supplied program is part + // of the program. Nothing here goes through the paired framing rule, so a + // digest taken of it is a digest of what the author handed over. + const padded = "\n\n# Report\n\n\n"; + expect(programDigest(padded)).not.toBe(programDigest("# Report\n")); + expect(programDigest(padded)).not.toBe(programDigest(pairedProgramSource(padded))); + expect(programDigest("# R\n")).not.toBe(programDigest("# S\n")); + }); +}); + +/** A value root that answers with what its props said. */ +const VALUE_PROGRAM = [ + "---", + "props:", + " release:", + " type: string", + "returns:", + " type: object", + " properties:", + " version: { type: string }", + " required: [version]", + "---", + "", + "", + "", +].join("\n"); + +/** A text root that reads a prop its own schema declares. */ +const TEXT_PROGRAM = [ + "---", + "props:", + " release:", + " type: string", + " default: none", + "---", + "", + "Release {props.release} ran.", + "", +].join("\n"); + +describe("Tier PE — program forms", () => { + it("PE6 binds a value root's schema-validated result under `as`", function* () { + const attempt = yield* run( + [ + ``, + "", + '', + "", + "Version {decided.version}.", + "", + ].join("\n"), + ); + + expect(attempt.failure).toBeUndefined(); + expect(attempt.output).toContain("Version 1.4.0."); + }); + + it("PE7 refuses a value root written without `as`, before program effects", function* () { + const attempt = yield* run( + [ + ``, + "", + '', + "", + ].join("\n"), + ); + + expect(attempt.failure ?? attempt.output ?? "").toContain("requires `as`"); + }); + + it("PE8 renders a text root's output where it is written", function* () { + const attempt = yield* run( + [ + ``, + "", + '', + "", + ].join("\n"), + ); + + expect(attempt.failure).toBeUndefined(); + expect(attempt.output).toContain("Release 1.4.0 ran."); + }); + + it("PE9 binds a text root's selected output under `as` and emits none of it", function* () { + const attempt = yield* run( + [ + ``, + "", + '', + "", + "Captured: {report}", + "", + ].join("\n"), + ); + + expect(attempt.failure).toBeUndefined(); + expect(attempt.output).toMatch(/Captured:\s*Release 1\.4\.0 ran\./); + expect(attempt.output?.split("Release 1.4.0 ran.")).toHaveLength(2); + }); +}); + +describe("Tier PE — root props", () => { + it("PE10 defaults to no props rather than adopting the caller's", function* () { + const attempt = yield* run( + [ + ``, + "", + "", + "", + ].join("\n"), + ); + + expect(attempt.failure).toBeUndefined(); + expect(attempt.output).toContain("Release none ran."); + }); + + it("PE11 refuses props the program's own schema refuses", function* () { + const attempt = yield* run( + [ + ``, + "", + '', + "", + ].join("\n"), + ); + + expect(attempt.failure ?? attempt.output ?? "").toContain("props the program's own schema"); + }); +}); + +describe("Tier PE — ambiguous and misplaced forms", () => { + it("PE12 refuses `program` written with content, before the content runs", function* () { + const attempt = yield* run( + [ + '', + "", + "", + "Body that must not become a program.", + "", + "", + ].join("\n"), + ); + + expect(attempt.failure ?? attempt.output ?? "").toContain("not both"); + expect(admissions(attempt.events)).toHaveLength(0); + }); + + it("PE13 refuses an element that names no program at all", function* () { + const attempt = yield* run("\n"); + + expect(attempt.failure ?? attempt.output ?? "").toContain("evaluates a program"); + expect(admissions(attempt.events)).toHaveLength(0); + }); +}); + +describe("Tier PE — one artifact, one occurrence", () => { + it("PE14 executes two sites independently and does not deduplicate by digest", function* () { + const attempt = yield* run( + [ + '', + "", + "", + "", + "", + "", + ].join("\n"), + ); + + expect(attempt.failure).toBeUndefined(); + expect(attempt.output?.split("Ran once.")).toHaveLength(3); + const recorded = admissions(attempt.events); + expect(recorded).toHaveLength(2); + const names = recorded.map((event) => + event.type === "yield" ? event.description.name : undefined, + ); + expect(new Set(names).size).toBe(2); + }); + + it("PE15 refuses a changed program at the same occurrence, and neither runs", function* () { + const document = ["", " ", "", ""].join("\n"); + performed.length = 0; + approved = '\n'; + + const first = yield* run(document); + expect(first.failure).toBeUndefined(); + expect(performed).toEqual(["first"]); + + // The history an interruption right after the admission leaves behind. A + // completed journal would replay as a terminal result and never re-enter + // this element at all, which would prove nothing about what it decides. + const interrupted = through(first.events, "evaluate_program"); + performed.length = 0; + approved = '\n'; + + const replayed = yield* run(document, { stream: new InMemoryStream(interrupted) }); + + expect(replayed.failure ?? replayed.output ?? "").toContain("different program"); + // Neither source won: the current program never ran, and the retained one + // was not run in its place. + expect(performed).toEqual([]); + approved = APPROVED; + }); + + it("PE16 resumes on the retained program and restores its completed effect", function* () { + const document = ["", " ", "", ""].join("\n"); + performed.length = 0; + approved = '\n\nRestored program.\n'; + + const first = yield* run(document); + expect(first.failure).toBeUndefined(); + expect(first.output).toContain("Restored program."); + expect(performed).toEqual(["one"]); + + // Interrupted after the program's own nested effect committed. + const interrupted = through(first.events, "probe"); + performed.length = 0; + produced.length = 0; + + const replayed = yield* run(document, { stream: new InMemoryStream(interrupted) }); + + expect(replayed.failure).toBeUndefined(); + expect(replayed.output).toContain("Restored program."); + // This run really re-entered the element rather than replaying a terminal + // result: the producer rendered again. Without it the rest of this case + // would pass against a history that was never continued. + expect(produced).toHaveLength(1); + // The effect was restored from the journal rather than performed again: + // nothing entered its executor on this run. + expect(performed).toEqual([]); + expect(admissions(replayed.events)).toHaveLength(1); + approved = APPROVED; + }); +}); + +describe("Tier PE — a program that fails", () => { + it("PE17 stops at the failure and runs nothing after it", function* () { + performed.length = 0; + const attempt = yield* run( + [ + `\n\n\n\n\n', + )}} as="plan" />`, + "", + "", + "", + ].join("\n"), + ); + + expect(attempt.failure ?? attempt.output ?? "").toContain("the program stopped"); + // The effect written before the failure happened; the one after it did not. + // A program that stopped must not leave later steps looking as though they + // ran. + expect(performed).toEqual(["before"]); + }); +}); diff --git a/packages/workflow/src/deno/workspace/evaluate.ts b/packages/workflow/src/deno/workspace/evaluate.ts index 51d76d8a7..852cb7aa9 100644 --- a/packages/workflow/src/deno/workspace/evaluate.ts +++ b/packages/workflow/src/deno/workspace/evaluate.ts @@ -77,13 +77,16 @@ */ import type { Operation } from "effection"; -import { getExpansion, hasContent } from "@executablemd/core"; +import { getExpansion } from "@executablemd/core"; import type { SourcePosition } from "@executablemd/core"; import { + EVALUATE_DESCRIPTION, + evaluateProgramElement, pinnedFileDelete, pinnedFileRead, pinnedFileWrite, pinnedMutation, + programProperties, } from "@executablemd/core/host"; import type { GeneratedEffectClass, @@ -128,8 +131,10 @@ export const props = { uniqueItems: true, items: { enum: [...EFFECT_CLASSES] }, }, + // The complete-program half, taken from core rather than restated here so + // the two profiles cannot describe one component differently. + ...programProperties, }, - required: ["source"], additionalProperties: false, }; @@ -267,13 +272,44 @@ function createEvaluate( elementProps: Record, invocation: ComponentInvocation, ): Operation { - if (yield* hasContent()) { + // Which grammar this element is written in, decided from the props alone. + // Neither half of it reads the authored form, so both are settled before + // the durable name is claimed: a combination this host does not have is not + // a fragment or a program being refused. + const fragment = elementProps.source; + if (fragment !== undefined && elementProps.program !== undefined) { throw new GeneratedEvaluationError( - " takes the generated source as its `source` prop and renders no content of " + - "its own. Write it self-closing.", + " evaluates either a restricted generated fragment through `source` or a " + + "complete program through `program`, not both.", + ); + } + if (fragment !== undefined && elementProps.props !== undefined) { + throw new GeneratedEvaluationError( + " takes `props` only for a complete program: a restricted generated fragment " + + "has no root props.", + ); + } + if (fragment === undefined && elementProps.allow !== undefined) { + // `allow` narrows the restricted evaluator's tables and means nothing to a + // program, which runs under this site's own authority — something no prop + // selects from. + throw new GeneratedEvaluationError( + " takes `allow` only with `source`, which selects what a restricted " + + "generated fragment may reach.", + ); + } + + if (fragment === undefined) { + return yield* evaluateProgramElement( + elementProps, + invocation, + claim, + " evaluates a complete program written as its content or supplied as " + + "`program`, or a restricted generated fragment supplied as `source`.", ); } - const source = elementProps.source; + + const source = fragment; if (typeof source !== "string") { throw new GeneratedEvaluationError(" requires a `source` string to evaluate."); } @@ -294,6 +330,16 @@ function createEvaluate( // durable name and each replay the other's admitted fragment. const id = yield* claim(invocation); + // After the claim, so the shape read here is the one the claim proved this + // element was written in. A borrowed invocation reports somebody else's + // element, and refusing it is the claimant's answer rather than this one. + if (invocation.hasContent()) { + throw new GeneratedEvaluationError( + " takes the generated source as its `source` prop and renders no content of " + + "its own. Write it self-closing.", + ); + } + // Read where the work is not journaled: the element's shape and the run's // current roots are both ordinary operations, and only the admission below // belongs in the run's history. @@ -344,6 +390,12 @@ export function evaluationComponents( { name: "Evaluate", origin: ORIGIN, + // The run profile's sentence plus the one form only this profile has. + // Stated here rather than in core because a catalog must not advertise a + // prop the profile it describes would refuse. + description: + `${EVALUATE_DESCRIPTION} Use \`\` for a ` + + "restricted generated fragment.", props, factory: (claim: IdentityClaimant) => createEvaluate(database, options, claim), }, diff --git a/packages/workflow/tests/generated-agent-component.test.ts b/packages/workflow/tests/generated-agent-component.test.ts index 980ff9ba6..008291e29 100644 --- a/packages/workflow/tests/generated-agent-component.test.ts +++ b/packages/workflow/tests/generated-agent-component.test.ts @@ -1214,3 +1214,104 @@ describe("Tier WGAC — a committed mutation is not repeated", () => { }); }); }); + +/** + * Tier WGAC — the complete-program forms beside the restricted one (#713). + * + * A workflow's `` gained two forms and lost none. What matters here is + * that the three do not reach into each other: a program is admitted by the + * complete-program event and not by `generated_xmd`, `allow` selects nothing for + * one, `props` configures nothing for a fragment, and everything the restricted + * evaluator refuses it still refuses. + */ +describe("Tier WGAC — complete programs beside restricted fragments", () => { + it("WGAC17: a workflow evaluates a complete program under its own event", function* () { + const root = yield* useStorageRoot(); + yield* withStorage(root, function* () { + const database = yield* createRun(); + const attempt = yield* runDocument( + database, + `\n\n` + + `\n`, + ); + + expect(attempt.failure).toBe(undefined); + expect(reported(attempt)).toContain("The program ran."); + // Its own record. The restricted admission is a different decision about a + // different kind of source and must not be widened to carry this one. + expect(admissions(attempt.events)).toHaveLength(0); + expect( + attempt.events.filter( + (event) => event.type === "yield" && event.description.type === "evaluate_program", + ), + ).toHaveLength(1); + }); + }); + + it("WGAC18: a program written as content is admitted and not emitted twice", function* () { + const root = yield* useStorageRoot(); + yield* withStorage(root, function* () { + const database = yield* createRun(); + const attempt = yield* runDocument( + database, + `\n # Composed\n\n The program ran.\n\n`, + ); + + expect(attempt.failure).toBe(undefined); + expect(reported(attempt).split("The program ran.")).toHaveLength(2); + expect(admissions(attempt.events)).toHaveLength(0); + }); + }); + + it("WGAC19: the three forms cannot be combined", function* () { + const first = yield* useStorageRoot(); + yield* withStorage(first, function* () { + const database = yield* createRun(); + const both = yield* runDocument(database, `\n`); + expect(reported(both)).toContain("not both"); + expect(admissions(both.events)).toHaveLength(0); + }); + + const second = yield* useStorageRoot(); + yield* withStorage(second, function* () { + const database = yield* createRun(); + // `allow` names a table only the restricted evaluator has. + const selected = yield* runDocument( + database, + `\n`, + ); + expect(reported(selected)).toContain("only with `source`"); + expect(admissions(selected.events)).toHaveLength(0); + }); + + const third = yield* useStorageRoot(); + yield* withStorage(third, function* () { + const database = yield* createRun(); + // Root props configure a program's root, and a fragment has none. + const propped = yield* runDocument( + database, + `\n`, + ); + expect(reported(propped)).toContain("only for a complete program"); + expect(admissions(propped.events)).toHaveLength(0); + }); + }); + + it("WGAC20: complete-program support is not reachable through `source`", function* () { + const root = yield* useStorageRoot(); + yield* withStorage(root, function* () { + const database = yield* createRun(); + // A fragment that names a component the restricted tables do not hold is + // refused exactly as it always was — being a complete root's vocabulary + // buys it nothing on this path. + const attempt = yield* runDocument( + database, + `do the work'} as="observation" />\n`, + ); + + expect(reported(attempt)).toMatch(/component/i); + expect(admissions(attempt.events)).toHaveLength(1); + expect(reads(attempt.events)).toHaveLength(0); + }); + }); +}); diff --git a/site/routes/index.tsx b/site/routes/index.tsx index 56116581b..f67f84d9c 100644 --- a/site/routes/index.tsx +++ b/site/routes/index.tsx @@ -93,6 +93,24 @@ const PLAN_XMD: Tok[][] = [ [key("")], ]; +/** Direct composition: the program is carried out where it is produced. */ +const EVALUATE_DIRECT: Tok[][] = [ + [key("")], + [" ", key("")], + ...PLAN_STEPS.map((step): Tok[] => [` ${step}`]), + [" ", key("")], + [key("")], +]; + +/** Deferred composition: the program is kept first and run at a later step. */ +const EVALUATE_DEFERRED: Tok[][] = [ + [key("")], + ...PLAN_STEPS.map((step): Tok[] => [` ${step}`]), + [key("")], + [], + [key("")], +]; + const RELEASE_MD: Tok[][] = [ [bold("# Release")], [], @@ -557,8 +575,28 @@ export default define.page(function Home({ url }) {

- Plan produces a program. Run executes a program from the host or - CLI. Composition decides whether and when a planned program runs. + + {""}{" "} + runs a program in the document you are already running. + {" "} + Wrap the planner to carry the program out where it is produced, or + keep the program first and evaluate it at a later step. +

+ +
+ + + + + + + +
+ +

+ Plan produces a program. Evaluate evaluates a program in the current + XMD execution. Run executes a program from the host or CLI. + Composition decides whether and when a planned program runs.

diff --git a/specs/executable-mdx-spec.md b/specs/executable-mdx-spec.md index 64eca1729..79eb81fb8 100644 --- a/specs/executable-mdx-spec.md +++ b/specs/executable-mdx-spec.md @@ -4411,6 +4411,7 @@ interface, and each operation is also exported directly: | `hasBinding()` | Whether the invocation has an engine-owned result binding — whether `as` was written (§6.10) | throws a missing-provider error | | `hasCapture(name)` | Whether the invocation wrote this capture prop at all (§6.5) | `false` | | `capture(name)` | Evaluate a capture prop now, against the caller's bindings, and deliver the result by reference (§6.5) | throws: not inside a function component invocation | +| `expandProgram(program)` | Expand a complete XMD program at this invocation's site, under the authority the enclosing expansion holds (§5.7) | throws: not inside a function component invocation | | `handleFailure(failure)` | What an ordinary function-component failure means, after complete invocation teardown (§6.9) | fails the operation with `failure.error` | | `retain(resource)` | Create a resource in the invocation-site scope, so it outlives this invocation (§4.4) | throws: not inside a component invocation | @@ -4440,17 +4441,22 @@ by being handled first. `registerComponents`, on the terms above. One that names durable work after its own invocation is supplied differently: the host declares it to the execution, which calls the host's factory once with the claimant it minted and registers -what comes back (§5.3, §5.6). The workflow host supplies -`` that way, for a live or partial run — the operation an -authored workflow document writes where an Agent's proposed fragment should be -admitted and performed. Its schema is closed on one required string prop and one -optional `allow` array selecting from the closed effect classes `read` and -`write`, and paired content is refused. It declares no `returns` and answers -with a detached value — `{ observations: [{ name, value }], output }`, each -admitted read's own returned value under the name the fragment invoked it by, -with an admitted mutation contributing none — so it renders nothing where it is -written; an ordinary `as` captures it by reference for every selection, and an -authored `` turns it into the text a next `` carries. Availability is all +what comes back (§5.3, §5.6). `` is supplied that way by every host +that offers it. The ordinary run profile supplies its complete-program forms, +which evaluate an admitted root in the current execution (§5.7). + +The workflow host supplies those forms and, beside them, the restricted +`` — the operation an authored workflow document writes +where an Agent's proposed fragment should be admitted and performed, for a live +or partial run. That form takes a `source` string and an optional `allow` array +selecting from the closed effect classes `read` and `write`, refuses paired +content, and cannot be combined with `program`, with paired content, or with +`props`. On it the component declares no `returns` and answers with a detached +value — `{ observations: [{ name, value }], output }`, each admitted read's own +returned value under the name the fragment invoked it by, with an admitted +mutation contributing none — so it renders nothing where it is written; an +ordinary `as` captures it by reference for every selection, and an authored +`` turns it into the text a next `` carries. Availability is all the registration decides; the ceilings it runs under come from values the host captured before any document existed, and no prop, binding or middleware return value supplies or widens one. [Workflow workspaces](./workflow-workspace-spec.md) @@ -4733,6 +4739,101 @@ inside another execution without moving what an invocation may name. `Expansion` and `getExpansion()` belong to `@executablemd/core`, so ordinary document execution receives expansion identity with no extension installed. +### 5.7 Complete-program evaluation + +Producing a program and running one are separate choices. `` produces +approved XMD source and never evaluates it (§5.4); `` is the element +an author writes where a program should run. It has two complete-program forms, +and they say the same thing about different moments: + +```md + + Inspect the release inputs and recommend a version. + +``` + +```md +Inspect the release inputs and recommend a version. + + +``` + +`` is declared to an execution rather than registered, because its +implementation names durable work after its own invocation (§5.6). A host that +declares none has no `` at all. The ordinary run profile declares the +complete-program forms; the workflow host declares those and its own restricted +generated-fragment form beside them (workflow-workspace-spec §8.4). + +**It is a composition site, not a child document.** The program runs in the +current execution: its lifecycle, its journal, its cancellation scope, its +contextual providers, its working directory or Workspace, and the authority in +force where the element was written. No child process, no second host profile, +no second root lifecycle, no journal of its own. Program source *requests* +behavior and grants none. + +What does not cross is the producer's temporary authority. By the time a program +is admitted, ``'s authorship profile and its private phases have torn +down, and a private component belongs to the declaration whose exact bytes +authored it — so a private name written in a program resolves to nothing. +Ordinary imports and capabilities genuinely present at the site remain +available. + +**The forms are disjoint.** Paired content and `program` are mutually exclusive +and one of them is required; `props` is valid only for a complete program. +Every ambiguous or misplaced combination refuses before program content is +produced and before any program effect runs, so an ambiguous element performs +nothing at all — including the producer it was written around. + +**Paired content is a private program buffer, not a quotation.** What is written +inside runs under ordinary XMD semantics while it produces text, and that text +is what admission is offered; none of it reaches the surrounding document. The +element removes the framing its own wrapper contributed — the line break after +the opening tag with the indentation that follows it, the line break before the +closing tag with the indentation that precedes it, and the indentation those +lines share — and nothing else. Interior bytes are untouched. A producer's +result is spliced in as one value and keeps its own trailing newline, which is +why a sole `` producer and `program={plan}` admit the same bytes and the +same content digest. + +`program={value}` is exact supplied source. Whitespace an author put at either +end of it is part of the program, and no trimming, dedenting or newline +adjustment is applied to it. + +**The root's own contract decides the result.** The admitted source is a root, +so its frontmatter, imports, metadata, props schema, `returns` declaration and +`` selection apply (§5.4). A text root renders its selected output where +the element was written; with `as` it still executes, binds that selected text +as a string, and emits none of it. A value root requires `as` and binds its +schema-validated result — without one it refuses before the first program +effect, because the result would have nowhere to go. + +Root props are the explicit `props` object and default to `{}`. Ambient root +props are never adopted. They are validated against the program's own schema +before the first program effect. Ordinary non-props caller bindings are visible +to the program, and visibly read-only: the program reads a copy, so a binding it +creates or overwrites reaches no caller. Relative paths resolve from the source +origin of the authored `` site. + +**Admission is this boundary's own durable event.** A complete program records +`evaluate_program`, never `generated_xmd` (§10). Before the first program effect +the run retains the exact source and its digest, the explicit props, the +evaluation-site source origin, the root mode, and the components the program +names with the forms it writes them in. A partial continuation expands the +retained source and restores the nested effects that already committed. A +changed program at the same evaluation occurrence is stale input: it refuses +before either the current or the retained source runs, so neither silently wins. + +**The digest identifies the artifact; the site identifies the occurrence.** The +authored element and the loop iteration it was reached through are what make one +execution of a program. Evaluating the same approved program at two sites is two +executions with two nested effect identities, and the digest never deduplicates +them. + +Canonical execution answers `Component.expandProgram()` (§5.5) for the +invocation, from the authority and block counter it is already holding. That is +what makes "the site's authority" a fact about the frame rather than something a +prop, a binding or a middleware return value could supply. + ## 6. Expansion ### 6.1 The expansion algorithm @@ -9703,6 +9804,7 @@ trusted-host events may have no authored source. | Resolve components (glob) | `glob` | `resolve:{dir}` | Only when `useDurableGlobResolver` middleware is installed | | Read over HTTP | `fetch` | `fetch:{expansion id}` | Normalized request in `description.input`; status, detached headers and text body in the result (§6.18) | | Admit generated XMD | `generated_xmd` | `generated:{fragment id}` | The canonical class selection, retained roots, selected root, every selected entry as a name, identity and admitted forms, and the exact request policy in `description.input`; the admitted source, that same policy, and the identity and form of each element the fragment named in the result (workflow-workspace-spec §8.4) | +| Admit a complete program | `evaluate_program` | `program:{evaluation id}` | The source digest, explicit props, evaluation-site source origin and whether the result is captured in `description.input`; the exact admitted source, the root mode, the components the program names with their authored forms, those same terms and the validated root props in the result — or the refused class alone (§5.7) | ### 10.2 Example journal for a multi-component document @@ -12196,6 +12298,30 @@ user's own `~/.xmd/repositories`. | ORC20 | Retained workflow regression | Repository and Worktree replay, transactional Git, Push and pull-request history evidence, Issue effects, forks and completed replay keep their records, identities, provider call counts and native-launch refusal unchanged; directory ensure adds only its own `workspace_file` event and resulting retained root | | ORC21 | Compiled binary | A compiled smoke creates a root-level ambient Worktree, runs a command there, proves `.git` is a file and the checkout persists after exit; a second gated process proves lock refusal and release | +### Tier PE — Complete-program evaluation (§5.7) + +Canonical evidence runs a document with `` declared exactly as a host +profile declares it, so what is measured is the element a person writes rather +than an operation only a test can reach. Tier EP is the same claims under a +shelled-out `xmd run`, and Tier WGAC covers the workflow profile, where these +forms sit beside the restricted one. + +| # | Test | Verify | +|---|------|--------| +| PE1 | Deferred composition | `program={plan}` admits and evaluates the bound source once, recording one `evaluate_program` event | +| PE2 | Direct composition | A producer written as content evaluates once and none of its approved source reaches the surrounding document | +| PE3 | The two forms agree | A sole producer's paired projection and `program={plan}` admit the same bytes and the same content digest | +| PE4 | Wrapper framing | The paired form removes one line break and the shared indentation at each end and nothing else; interior bytes, including a producer's own trailing newline, survive | +| PE5 | Supplied source is exact | `program={value}` is neither trimmed, dedented nor newline-adjusted, so leading and trailing whitespace change the digest | +| PE6–PE9 | Program forms | A value root binds its schema-validated result under `as`; a text root renders its selected output, or binds it as a string under `as` and emits none of it | +| PE7 | Value root without `as` | Refused before the first program effect | +| PE10–PE11 | Root props | An omitted `props` gives the program its own defaults rather than the caller's root props; props the program's schema refuses are refused before effects | +| PE12–PE13 | Ambiguous forms | `program` written with content, and an element naming no program at all, each refuse with no admission recorded | +| PE14 | Repeated evaluation | Two sites evaluate one artifact independently under two durable names; the digest deduplicates neither | +| PE15 | Changed evaluation source | A partial continuation whose producer returns different source refuses as stale input, and neither the current nor the retained program performs its effect | +| PE16 | Evaluation replay | A partial continuation expands the retained source and restores a completed nested effect without performing it again, proven by the producer re-rendering on that run | +| EP1–EP8 | Run profile | The ordinary run profile declares ``; the forms, props, refusals and catalog entry hold through the real binary, and a program cannot reach ``'s private components | +| WGAC17–WGAC20 | Workflow profile | A complete program records `evaluate_program` and no `generated_xmd`; the three forms cannot be combined; complete-program support is not reachable through `source` | --- diff --git a/specs/workflow-workspace-spec.md b/specs/workflow-workspace-spec.md index 349c66f28..03d232916 100644 --- a/specs/workflow-workspace-spec.md +++ b/specs/workflow-workspace-spec.md @@ -1953,10 +1953,10 @@ not the loop's failure — the document states what exhaustion means, which the representative flow does with a final `` that requires a proposal. The completed Prompt and observation records are the evidence for exhaustion. -``'s schema is closed on one required string prop and one optional -`allow` array, and paired content is refused: a `source` the element rendered is -not a fragment anybody handed it. A write-enabled invocation is written the same -way, with the class it draws on stated where a reader can see it: +The restricted form takes a `source` string and an optional `allow` array, and +refuses paired content: a `source` the element rendered is not a fragment +anybody handed it. A write-enabled invocation is written the same way, with the +class it draws on stated where a reader can see it: ```md @@ -1994,6 +1994,22 @@ about it is rewritten on the way to the document. Turning it into text is the document's decision, made where the text is wanted: the representative flow binds it and renders it into the next `` with ``. +**Complete programs are a different form of the same element.** A workflow's +`` also takes the complete-program forms every host that offers this +element has — paired content and `program`, with optional root `props` +(executable-mdx-spec §5.7). Those admit a complete XMD root and evaluate it in +the current execution under the site's own authority, and they record their own +`evaluate_program` durable event. The restricted admission described here is a +decision about a different kind of source and is not widened to carry them: +complete-program support is not reachable through `source` or `allow`, and a +fragment naming ``, ``, an import, a binding, an executable +block or an unadmitted component is refused on this path exactly as before. + +The three forms are disjoint. `source` cannot be combined with `program`, with +paired content, or with `props`; `allow` is valid only with `source`; and +`props` is valid only for a complete program. Each combination refuses before +the durable name is claimed or the candidate is read. + **How it is supplied.** `` is not registered by the attachment. It names durable work after its own invocation, so the host **declares** it to the execution — one entry in the installation, captured before anything else is From b6dab5db43192c877ad4dfd4ee277448de1d0dc5 Mon Sep 17 00:00:00 2001 From: Taras Mankovski Date: Fri, 4 Sep 2026 07:52:44 -0400 Subject: [PATCH 2/7] =?UTF-8?q?=F0=9F=94=92=20Hold=20a=20complete-program?= =?UTF-8?q?=20continuation=20to=20its=20retained=20admission=20(#713)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The `evaluate_program` record is hostile data and was read as though it were not. It is now two closed shapes — admitted and refused — with every nested record closed too, so a missing, additional or misspelled member at any depth is a record this evaluation did not write. Shape is not enough, because every member of the record is a claim about another one. Before anything expands, the retained source must hash to the retained digest, reparsing it must produce the retained root mode and a valid body structure, the retained elements must be the ones that source writes in that order and those forms, and the supplied props must validate to the retained validated props. A record failing any of them is refused as unreadable and neither the retained nor the current program performs an effect. The admission also retains what each name the program writes resolved to at this site, which restores the current-site compatibility contract: a continuation whose site now answers one of those names with a different identity, or admits it in a different form, refuses before either implementation runs, while an unchanged site resumes. Resolution stays behind canonical execution. The execution builds a resolver, carries it on the expansion authority, and settles the comparison itself before the first program effect. `Component.resolveProgramSite()` lets the admission retain what the site answered; nothing rests on that answer, because the canonical comparison is what decides — so short-circuiting it refuses the evaluation rather than widening it. `ExpansionAuthority` is still reachable from no component, and no prop, binding, middleware result or public context state supplies authority. --- architecture.md | 4 +- packages/core/src/component-api.ts | 25 ++ .../core/src/components/import-authority.ts | 16 ++ packages/core/src/execute.ts | 22 ++ packages/core/src/expand.ts | 47 ++++ packages/core/src/program-evaluation.ts | 246 ++++++++++++++---- packages/core/src/program-identity.ts | 139 ++++++++++ packages/core/src/types.ts | 11 + .../core/tests/program-evaluation.test.ts | 236 ++++++++++++++++- specs/executable-mdx-spec.md | 58 ++++- 10 files changed, 731 insertions(+), 73 deletions(-) create mode 100644 packages/core/src/program-identity.ts diff --git a/architecture.md b/architecture.md index 4f36248e8..bc8433c04 100644 --- a/architecture.md +++ b/architecture.md @@ -3897,9 +3897,9 @@ Status is measured against main. | workflow Agent session | a workflow document's `` runs under a profile the host attaches only for a live or partial run: an empty host-owned working directory instead of any Workspace, checkout or caller path, no MCP servers, an empty requested native tool set, and `deny-all` with a permission path that denies every native request and fails the turn that asked without reaching the public permission chain. Within a run a session is identified by the Agent/Session expansion identity the engine derived — the authored name is descriptive, so two sibling `` elements are two sessions — routed inside a placement bound to its element and good for one use, so a kept placement cannot be substituted for the next. The conversation is retained as a row in the run's own database with the provider, resolved agent command and policy fingerprint beside it as compatibility attributes. The order is placement, the backend's acceptance of the session's first turn, the provider's canonical tagged assertion, then the mapping commit — and only then is anything that turn produced exposed. A placement is inert: it creates no provider session and writes no row. Occupancy of a provider key is not an assertion, and a record held for a first turn nobody accepted asserts nothing at all; the pre-commit window reconciles only from exactly one canonical assertion, and a missing, conflicting, replaced or ambiguous assertion is one explicit refusal that starts no replacement. Deleting a run removes the row with the run and the provider-session directory beside it, and reports the categories. The profile selects ACP-only capability explicitly — no native-launch advertisement and no client-native attachment advertisement — rather than inheriting the provider package's ordinary-run sets by omission, and it supplies no machine session coordinator, construction-route store or executable observer: a workflow session belongs to a run, and the machine-wide account describes a different thing entirely | built on the #302 stack, with the explicit ACP-only selection from #561; the portable proof that an adapter honours an empty tool set is tracked by #496 and does not widen the ceiling | | generated-XMD admission | admits one Agent-generated fragment through the trusted-host seam: host policy is a `read` table and a `write` table of exact pinned identities, each carrying the authored forms it is admitted for, and an authored `allow` selects a canonical non-empty subset of the closed classes — omitted means `read`. The complete source is preflighted inside one `generated_xmd` durable effect before its first generated effect; only the pinned identity the selected classes hold for that name **and** that form executes; and the admitted source, class selection, selected root, every selected entry with its forms, the identity and form of each element named, and the normalized request policy are retained in that effect's own result — so a continuation restores the decision without reading the current candidate and expands only the retained source. The roots are an as-of-admission retained basis checked by membership — the run's own later root publications and an advanced retained current root pass, while a lost admission root or lost selected root refuses — and every non-root term is checked exactly, refusing a run whose classes, identities, forms or requests have moved. The admission and every nested generated effect are offered inline by the owning expansion in authored order, so a partial continuation restores each completed one without another live execution. Each admitted effect is retained by its own ordinary record, and a read's value is collected while a mutation's is not | built on the #369 stack, continuation basis amended by #589; core owns the mechanics and the workflow policy wrapper is internal | | `` | the restricted generated-fragment form of ``, which the workflow host declares beside the complete-program forms below. The form an authored document writes where an observation should happen. The host does not register it: it **declares** it to the execution through `ExecutionInstallation.components`, captured before any installation runs, and canonical execution calls its factory once for that attachment with the claimant it minted and registers what comes back. Registration provides availability only — a name a trusted document may write — and carries none of the authority. It takes a `source` string and one optional `allow` array selecting a non-empty duplicate-free subset of the closed effect classes `read` and `write` — omitted means `read` — refuses paired content, and cannot be combined with `program`, with paired content, or with `props`; each of those refuses before the durable name is claimed and before the candidate is read. It declares no `returns` and answers with a detached value — `{ observations: [{ name, value }], output }`, each admitted read's own returned value under the name the fragment invoked it by, in invocation order, with whatever the fragment rendered under `output` rather than instead of them, and the pinned identity that produced one left in the retained admission rather than copied here — so an admitted ``, which renders nothing at all, still reaches the document. An admitted mutation contributes no entry and no receipt, so a write-only fragment binds `{ observations: [], output: "" }`; `as` is valid for every selection and binds that same shape. An ordinary `as` captures that value by reference, and an authored `` renders it into the next ``: deciding how a value becomes text is the document's. Every ceiling comes from values the host captured at installation — the run's retained roots and its authoritative current root read from the run's own storage at invocation, as-of-admission provenance a continuation holds by membership so the run's own later publications and an advanced retained current root invalidate nothing, core's pinned self-closing `` read, the write table of core's paired ``, this package's lexical `` built from the definition the ordinary registration owns, and core's self-closing ``, and `` only when the captured request ceiling is non-empty — and no prop, binding, context or middleware return value supplies or widens one. `allow` selects among those tables and adds nothing to them; approval, when a workflow needs one, is authored control flow before the element. Its durable operation is named through that claimant, on the exact invocation the engine handed it and in that invocation's own frame — not from a context a document could rebind, a contextual Api answer, a definition, or a registry answer. Generated source never resolves through the registration: the evaluator consults only its own closed table of pinned identities. It is deliberately not wrapped in `printErrors`, so a refused fragment stops the authored loop rather than becoming text the next turn could read as a read that happened | built on the #302 stack, extended by #369 | -| complete-program evaluation | admits one complete XMD root and evaluates it at the authored `` site, in the current execution: its lifecycle, journal, cancellation scope, contextual providers, working directory or Workspace, and the authority in force where the element was written. There is no child process, host profile, root lifecycle or journal of its own — program source requests behavior and grants none. The root's own frontmatter, imports, metadata, props schema, `returns` and `` selection decide what it does; a text root renders its selected output or binds it as a string under `as`, a value root requires `as` and binds its schema-validated result, and root props are the explicit `props` object defaulting to `{}` rather than the caller's own. Ordinary non-props caller bindings are visible as a copy, so nothing the program binds escapes, and relative paths resolve from the evaluation site's source origin. The producer's temporary authority is not inherited: an enclosing declaration's private closure does not cross, so a private name written in a program resolves to nothing, while imports and capabilities genuinely present at the site remain. Canonical execution answers `Component.expandProgram()` from the frame it already holds, which is what makes site authority a fact about the frame rather than something a prop, binding or middleware return value could supply | built by #713 on the #722 `` contract; core owns the mechanics and every host that offers `` declares them | +| complete-program evaluation | admits one complete XMD root and evaluates it at the authored `` site, in the current execution: its lifecycle, journal, cancellation scope, contextual providers, working directory or Workspace, and the authority in force where the element was written. There is no child process, host profile, root lifecycle or journal of its own — program source requests behavior and grants none. The root's own frontmatter, imports, metadata, props schema, `returns` and `` selection decide what it does; a text root renders its selected output or binds it as a string under `as`, a value root requires `as` and binds its schema-validated result, and root props are the explicit `props` object defaulting to `{}` rather than the caller's own. Ordinary non-props caller bindings are visible as a copy, so nothing the program binds escapes, and relative paths resolve from the evaluation site's source origin. The producer's temporary authority is not inherited: an enclosing declaration's private closure does not cross, so a private name written in a program resolves to nothing, while imports and capabilities genuinely present at the site remain. Canonical execution answers `Component.expandProgram()` from the frame it already holds, which is what makes site authority a fact about the frame rather than something a prop, binding or middleware return value could supply. Which definition each name the program writes resolves to is part of what the site granted, so the admission retains it and canonical execution settles it again before the first program effect — through a resolver the execution builds and carries on the expansion authority, reachable from no document, component, contextual Api answer or middleware return value | built by #713 on the #722 `` contract; core owns the mechanics and every host that offers `` declares them | | `` complete-program forms | the two ways an author names a program at a composition site: paired content, which renders a source-producing component into a private program buffer, and `program={value}`, which supplies complete-program source directly. Optional `props={object}` is the root's props input and is valid only for these forms. The two are mutually exclusive and one is required; every ambiguous or misplaced combination refuses before program content is produced and before any program effect runs, so an ambiguous element does not even render its producer. Paired content is a private buffer rather than a quotation — an executable component written there executes under ordinary XMD semantics while producing the text sent to admission — and the element removes only the framing its own wrapper contributed: the line break after the opening tag with the indentation following it, the line break before the closing tag with the indentation preceding it, and the indentation those lines share. Interior bytes are untouched, so a sole `` producer's trailing newline survives and paired evaluation and `program={plan}` admit the same bytes and the same content digest. `program={value}` is exact supplied source and is neither trimmed, dedented nor newline-adjusted | built by #713 | -| `evaluate_program` durable effect | the complete-program admission, recorded once before the first program effect and never as a widening of `generated_xmd` — the two are decisions about different kinds of source. `description.input` carries the source digest, the explicit props, the evaluation-site source origin and whether the result is captured; the result carries the exact admitted source, the root mode, the components the program names with their authored forms, those same terms and the validated root props — or, for a refusal, the refused class alone and nothing of the source. A partial continuation expands the retained source and restores the nested effects that already committed, repeating no planning; a changed program at the same evaluation occurrence is stale input and refuses before either the current or the retained source runs, so neither silently wins. The source digest identifies the program artifact and the authored element and loop iteration identify one execution of it, so two occurrences of one artifact have separate nested effect identities and the digest deduplicates neither | built by #713 | +| `evaluate_program` durable effect | the complete-program admission, recorded once before the first program effect and never as a widening of `generated_xmd` — the two are decisions about different kinds of source. `description.input` carries the source digest, the explicit props, the evaluation-site source origin and whether the result is captured; the result carries the exact admitted source, the root mode, the components the program names with their authored forms **and the identity canonical resolution selected for each name at that site**, those same terms and the validated root props — or, for a refusal, the refused class alone and nothing of the source. Both results are closed shapes, and so is every record nested in them: a missing, additional or misspelled member at any depth is a record this evaluation did not write. Restored, it must also agree with itself before anything expands — the source hashes to the digest, reparsing produces the retained mode and a valid body structure, the retained elements are the ones that source writes in that order and those forms, and the supplied props validate to the retained validated props — and a record failing any of those is refused as unreadable with neither program performing an effect. A continuation whose site now answers one of those names with a different identity or admits it in a different form is refused before either implementation runs, which is a different answer from an unreadable record: the journal is intact and the environment moved. A partial continuation expands the retained source and restores the nested effects that already committed, repeating no planning; a changed program at the same evaluation occurrence is stale input and refuses before either the current or the retained source runs, so neither silently wins. The source digest identifies the program artifact and the authored element and loop iteration identify one execution of it, so two occurrences of one artifact have separate nested effect identities and the digest deduplicates neither | built by #713 | | generated mutation proposals | lets an Agent propose constrained executable changes that a separate admission then performs against the run's own Workspace | built on the #369 and #567 stacks, with directory creation added by #643: the standard Deno profile's write table is core's paired `File:write`, the paired `@executablemd/workflow/composition/dir-v2#Dir` and core's self-closing `File.Delete`, in that retained order and followed by any host extension. `allow={["write"]}` intentionally authorizes Dir's persistent recursive directory creation; its versioned identity makes every continuation retained under the former non-mutating Dir identity refuse before generated execution. Admitted mutations run as the ordinary components they are through the run's effect transactions, and the evaluator adds no mutation API or receipt. Approval is authored control flow before the write-enabled element. Local Git, Git-host, issue, process, execution, credential and external-write effects are outside the class | | Deno-local DOFS provider | owns one authoritative SQLite/DOFS connection per run path, captures arbitrary canonical retained roots, privately restores them, and atomically coordinates one Workspace mutation with its filtered Yield | built on the #365 stack; public document filesystem effects and the CLI lifecycle route to it on the #366 stack | | scoped Worker Shell | executes `just-bash` through the Workspace adapter inside a Deno Worker | containment and effect-transaction POCs complete (#351, #357); production integration unbuilt | diff --git a/packages/core/src/component-api.ts b/packages/core/src/component-api.ts index 6a951fdf5..58540a7a6 100644 --- a/packages/core/src/component-api.ts +++ b/packages/core/src/component-api.ts @@ -18,6 +18,7 @@ import type { Operation } from "effection"; import type { EvalScope } from "@effectionx/scope-eval"; import { settle } from "./errors.ts"; import type { BoundExecRequest } from "./bound-exec.ts"; +import type { ProgramComponent, ProgramComponentRef } from "./program-identity.ts"; import type { CodeBlockContext, CodeBlockResult, @@ -171,6 +172,20 @@ export interface ComponentApi { * this owns running it. */ expandProgram(program: ProgramBody): Operation; + /** + * What each name a complete program writes resolves to at this site (§5.7). + * + * Complete-program admission retains these so a continuation can be held to + * the site it was admitted at. Canonical execution answers from the resolver + * it holds on the expansion authority, which no document, component or + * middleware reaches. + * + * A handler may still short-circuit this call, and what it answers goes into + * the durable record. Nothing rests on that: canonical execution settles the + * same question again from its own resolver before the first program effect, + * so a dishonest answer refuses the evaluation rather than widening it. + */ + resolveProgramSite(named: readonly ProgramComponentRef[]): Operation; /** * Decide what an ordinary function-component failure means (spec §6.9). * @@ -285,6 +300,14 @@ export const Component: Api = createApi("Component", ); }, // deno-lint-ignore require-yield + *resolveProgramSite( + _named: readonly ProgramComponentRef[], + ): Operation { + throw new Error( + "Component.resolveProgramSite() has no provider: not inside a function component invocation.", + ); + }, + // deno-lint-ignore require-yield *handleFailure(failure: ComponentFailure): Operation { throw failure.error; }, @@ -314,3 +337,5 @@ export const handleFailure: Operations["handleFailure"] = Component.operations.handleFailure; export const expandProgram: Operations["expandProgram"] = Component.operations.expandProgram; +export const resolveProgramSite: Operations["resolveProgramSite"] = + Component.operations.resolveProgramSite; diff --git a/packages/core/src/components/import-authority.ts b/packages/core/src/components/import-authority.ts index b747d600f..daf76eaae 100644 --- a/packages/core/src/components/import-authority.ts +++ b/packages/core/src/components/import-authority.ts @@ -22,6 +22,7 @@ import type { ComponentDefinition, FunctionComponentDefinition } from "../types. import type { FormSelections, InvocationIdentities } from "../invocation-identity.ts"; import type { DeclaredImports, PrivateClosure } from "./declared-markdown.ts"; import type { ExactSource } from "../output/exact-source.ts"; +import type { ProgramResolver } from "../program-identity.ts"; /** A definition an import may answer with. */ export type ImportedDefinition = ComponentDefinition | FunctionComponentDefinition; @@ -101,6 +102,21 @@ export interface ExpansionAuthority { * a component or middleware can name reaches it. */ readonly forms?: FormSelections; + /** + * What canonical resolution selects for a name at this site, without + * importing it. + * + * Complete-program admission retains the identity behind every name its + * program writes and holds a continuation to it (§5.7). That is + * reconciliation, so it must not be answerable through the composable chain: + * this closure is built by canonical execution, handed to core's own + * expansion by value like everything else on this object, and reachable from + * no document, component or middleware. + * + * Absent for an expansion canonical execution did not build one for, in which + * case a program's components are compared by name and form alone. + */ + readonly resolve?: ProgramResolver; } /** Why an answer is not the one canonical execution produced for this name. */ diff --git a/packages/core/src/execute.ts b/packages/core/src/execute.ts index 7e516dfd4..94055dfdd 100644 --- a/packages/core/src/execute.ts +++ b/packages/core/src/execute.ts @@ -94,6 +94,7 @@ import { useSegmentCauses, } from "./errors.ts"; import { Component, importComponent, raise } from "./component-api.ts"; +import { componentIdentity } from "./program-identity.ts"; import { sourceDescription } from "./source-position.ts"; import { renderSegment } from "./render.ts"; import { createExactSource } from "./output/exact-source.ts"; @@ -2269,6 +2270,27 @@ function* executeDocument( ...(declaredImports === undefined ? {} : { declared: declaredImports }), identities: identity.identities, forms, + // What complete-program admission retains behind each name its program + // writes, and what a continuation is compared against (§5.7). It reads + // the same inputs the import provider above reads and imports nothing: + // resolving a name decides which definition answers it, and loading one + // is the program's own durable effect where the element is written. + // + // On the authority rather than on the Component Api, because this + // decides whether a retained admission still describes this site. + // Reconciliation never trusts an answer middleware can replace. + *resolve(name: string): Operation { + const registered = yield* Component.operations.registry; + const selection = yield* ephemeral( + selectComponent(name, { + includes, + registry: registered, + ...(bundle === undefined ? {} : { workflow: bundle }), + ...(catalog === undefined ? {} : { declared: catalog }), + }), + ); + return componentIdentity(selection); + }, // Created here, held here, and reclaimed with this execution. Nothing a // document, a component, middleware or a separately loaded copy can // name reaches this object. diff --git a/packages/core/src/expand.ts b/packages/core/src/expand.ts index 7612093a4..01be53dc9 100644 --- a/packages/core/src/expand.ts +++ b/packages/core/src/expand.ts @@ -97,6 +97,13 @@ import { printsErrors, usePrintErrors } from "./component-failures.ts"; import { containedLedger, recoveringLedger } from "./component-failures.ts"; import type { CheckedFailures } from "./component-failures.ts"; import type { ExpansionAuthority, ImportedDefinition } from "./components/import-authority.ts"; +import { + INCOMPATIBLE, + ProgramEvaluationError, + sameComponents, + UNRESOLVED, +} from "./program-identity.ts"; +import type { ProgramComponent, ProgramComponentRef } from "./program-identity.ts"; import { DeclaredMarkdownError } from "./components/declared-markdown.ts"; import type { PrivateImport } from "./components/declared-markdown.ts"; import CoreTest from "./components/Test.ts"; @@ -3199,6 +3206,13 @@ function* expandFunctionComponent( authority: programAuthority(authority), }); }, + // The same authority the program will run under, asked what each + // name it writes resolves to. The private closure is already gone + // from it, so a name an enclosing declaration keeps to itself + // resolves to nothing here exactly as it will there. + *resolveProgramSite([named], _next) { + return yield* resolveProgramComponents(named, programAuthority(authority)); + }, *tryContent([slotName], _next) { const outcome = yield* handle.tryProject({ kind: "slot", @@ -4185,6 +4199,14 @@ export function* expandProgramBody( program: ProgramBody, site: ProgramSite, ): Operation { + // Before the first program effect, and settled here rather than by whoever + // called: the resolver is canonical execution's and reaches this expansion on + // the authority, so a handler that answered the admission's own resolution + // dishonestly is caught by the answer it cannot reach. + const current = yield* resolveProgramComponents(program.named, site.authority); + if (!sameComponents(program.named, current)) { + throw new ProgramEvaluationError(INCOMPATIBLE); + } return yield* scoped(function* () { yield* provideEnv(programEnvironment(site.callerValues, program.props)); if (program.returns !== undefined) { @@ -4247,3 +4269,28 @@ function programAuthority( const { privates: _privates, ...rest } = authority; return rest; } + +/** + * What each name a program writes resolves to at this site. + * + * Canonical execution's own answer, taken from the resolver the execution put + * on the authority rather than from anything the composable chain could + * produce. An expansion built without one — a fragment evaluator's, which + * resolves through its own closed table — reports every name unresolved, and + * the comparison it feeds then rests on names and forms alone. + */ +export function* resolveProgramComponents( + named: readonly ProgramComponentRef[], + authority: ExpansionAuthority | undefined, +): Operation { + const resolve = authority?.resolve; + const resolved: ProgramComponent[] = []; + for (const entry of named) { + resolved.push({ + name: entry.name, + form: entry.form, + identity: resolve === undefined ? UNRESOLVED : yield* resolve(entry.name), + }); + } + return resolved; +} diff --git a/packages/core/src/program-evaluation.ts b/packages/core/src/program-evaluation.ts index 067d8d453..1a5b7ebab 100644 --- a/packages/core/src/program-evaluation.ts +++ b/packages/core/src/program-evaluation.ts @@ -75,23 +75,33 @@ import { createDurableOperation } from "@executablemd/durable-streams"; import type { Json as DurableJson } from "@executablemd/durable-streams"; import type { Operation } from "effection"; -import { expandProgram } from "./component-api.ts"; +import { expandProgram, resolveProgramSite } from "./component-api.ts"; import { sourceDigest } from "./components/declared-markdown.ts"; import { parseRootMarkdownDefinition } from "./definition.ts"; import { validateBodyStructure } from "./expand.ts"; import { isJsonObject, parseJson } from "./json.ts"; import { sourceDescription } from "./source-position.ts"; import { validateProps } from "./validate.ts"; -import type { Json, JsonObject, ProgramOutcome, Segment, SourcePosition } from "./types.ts"; +import { + isProgramComponentForm, + ProgramEvaluationError, + sameElements, +} from "./program-identity.ts"; +import type { ProgramComponent, ProgramComponentRef } from "./program-identity.ts"; +import type { + Json, + JsonObject, + ProgramOutcome, + ReturnsSchema, + Segment, + SourcePosition, +} from "./types.ts"; + +export { ProgramEvaluationError } from "./program-identity.ts"; /** The durable effect type one complete-program evaluation records. */ export const EVALUATE_PROGRAM = "evaluate_program"; -/** A program this site will not evaluate. */ -export class ProgramEvaluationError extends Error { - override name = "ProgramEvaluationError"; -} - /** What the name a diagnostic and a durable record call this program. */ const PROGRAM = "program"; @@ -151,12 +161,6 @@ export interface ProgramEvaluationRequest { readonly position?: Readonly; } -/** One component the program names, with the form it is written in. */ -interface Named { - readonly name: string; - readonly form: "self-closing" | "paired"; -} - /** * The terms an evaluation is admitted under, and held to on a continuation. * @@ -172,18 +176,19 @@ interface Terms { readonly captured: boolean; } +/** An admitted decision, as the run retains it. */ +interface Admitted { + readonly decision: "admitted"; + readonly source: string; + readonly mode: "text" | "value"; + readonly named: readonly ProgramComponent[]; + readonly terms: Terms; + /** The props the program ran with, after its own schema validated them. */ + readonly validated: JsonObject; +} + /** The decision this run recorded, restored from its own durable record. */ -type Admission = - | { readonly decision: "refused"; readonly refused: Refused } - | { - readonly decision: "admitted"; - readonly source: string; - readonly mode: "text" | "value"; - readonly named: readonly Named[]; - readonly terms: Terms; - /** The props the program ran with, after its own schema validated them. */ - readonly validated: JsonObject; - }; +type Admission = { readonly decision: "refused"; readonly refused: Refused } | Admitted; /** * The program a paired `` produced, with the wrapper's framing off. @@ -239,16 +244,38 @@ export function programDigest(source: string): string { } /** Every component the program names, with the form each element is written in. */ -function names(segments: readonly Segment[], found: Named[]): Named[] { +function elements( + segments: readonly Segment[], + found: ProgramComponentRef[], +): ProgramComponentRef[] { for (const segment of segments) { if (segment.type === "component") { found.push({ name: segment.name, form: segment.selfClosing ? "self-closing" : "paired" }); - names(segment.children, found); + elements(segment.children, found); } } return found; } +/** + * The elements a candidate program writes, or none when it will not parse. + * + * Asked before the admission, so the site can be resolved and retained with the + * decision. A source the parser refuses still reaches the admission, which is + * where a refusal belongs — so this reports no elements rather than failing. + */ +function* programElements( + source: string, + origin: string, +): Operation { + try { + const parsed = yield* parseRootMarkdownDefinition(PROGRAM, origin, source); + return elements(parsed.definition.bodySegments, []); + } catch { + return []; + } +} + /** * Decide this program, once, inside the durable executor. * @@ -257,7 +284,11 @@ function names(segments: readonly Segment[], found: Named[]): Named[] { * error and its stack, and a refusal caused by somebody else's source would put * host paths into the run's history to say what one word already says. */ -function* admitProgram(request: ProgramEvaluationRequest, terms: Terms): Operation { +function* admitProgram( + request: ProgramEvaluationRequest, + terms: Terms, + resolved: readonly ProgramComponent[], +): Operation { try { const { source } = request; if (source.trim().length === 0) { @@ -285,11 +316,18 @@ function* admitProgram(request: ProgramEvaluationRequest, terms: Terms): Operati } catch { throw new Refusal("props"); } + // What the site answered for each element, in the order the program writes + // them. Resolution happened before this decision, so the record says which + // implementation each name stood for when the grant was made rather than + // which one happens to answer on the day it is read back. + if (!sameElements(resolved, elements(definition.bodySegments, []))) { + throw new Refusal("source"); + } return parseJson({ decision: "admitted", source, mode, - named: names(definition.bodySegments, []), + named: resolved.map((entry) => ({ ...entry })), terms, validated: props, }); @@ -301,12 +339,39 @@ function* admitProgram(request: ProgramEvaluationRequest, terms: Terms): Operati } } +/** + * Every member an admitted record has, and every member a refused one has. + * + * The record is hostile data: a replay hands back whatever the history holds, + * and a value with the right keys is not an admission. So each shape is closed + * — a missing member, a member spelled differently and a member nobody wrote + * are all the same answer, which is that this is not a record this version + * wrote. + */ +const ADMITTED_MEMBERS: readonly string[] = [ + "decision", + "source", + "mode", + "named", + "terms", + "validated", +]; +const REFUSED_MEMBERS: readonly string[] = ["decision", "refused"]; +const TERM_MEMBERS: readonly string[] = ["digest", "props", "origin", "captured"]; +const COMPONENT_MEMBERS: readonly string[] = ["name", "form", "identity"]; + +/** Whether an object has exactly these own members and no others. */ +function closed(value: JsonObject, members: readonly string[]): boolean { + const own = Object.keys(value); + return own.length === members.length && members.every((member) => Object.hasOwn(value, member)); +} + /** * The decision this run recorded, read back from the journal. * - * Parsed rather than trusted: a replay hands back whatever the history holds, - * and a record somebody else wrote is not an admission because it happens to - * have the right keys. + * Parsed rather than trusted: a record somebody else wrote is not an admission + * because it happens to have the right keys, and one this version wrote is not + * an admission if something has since added to it. */ function readAdmission(value: Json): Admission | undefined { if (!isJsonObject(value)) { @@ -315,45 +380,47 @@ function readAdmission(value: Json): Admission | undefined { const { decision } = value; if (decision === "refused") { const { refused } = value; - return typeof refused === "string" && Object.hasOwn(REFUSAL, refused) + return closed(value, REFUSED_MEMBERS) && + typeof refused === "string" && + Object.hasOwn(REFUSAL, refused) ? { decision, refused: refused as Refused } : undefined; } - if (decision !== "admitted") { + if (decision !== "admitted" || !closed(value, ADMITTED_MEMBERS)) { return undefined; } const { source, mode, named, terms, validated } = value; if (typeof source !== "string" || (mode !== "text" && mode !== "value")) { return undefined; } - const invocations = readNames(named); + const components = readComponents(named); const retained = readTerms(terms); - if (invocations === undefined || retained === undefined || !isJsonObject(validated)) { + if (components === undefined || retained === undefined || !isJsonObject(validated)) { return undefined; } - return { decision, source, mode, named: invocations, terms: retained, validated }; + return { decision, source, mode, named: components, terms: retained, validated }; } -function readNames(value: Json | undefined): readonly Named[] | undefined { +function readComponents(value: Json | undefined): readonly ProgramComponent[] | undefined { if (!Array.isArray(value)) { return undefined; } - const found: Named[] = []; + const found: ProgramComponent[] = []; for (const entry of value) { - if (!isJsonObject(entry)) { + if (!isJsonObject(entry) || !closed(entry, COMPONENT_MEMBERS)) { return undefined; } - const { name, form } = entry; - if (typeof name !== "string" || (form !== "self-closing" && form !== "paired")) { + const { name, form, identity } = entry; + if (typeof name !== "string" || typeof identity !== "string" || !isProgramComponentForm(form)) { return undefined; } - found.push({ name, form }); + found.push({ name, form, identity }); } return found; } function readTerms(value: Json | undefined): Terms | undefined { - if (!isJsonObject(value)) { + if (!isJsonObject(value) || !closed(value, TERM_MEMBERS)) { return undefined; } const { digest, props, origin, captured } = value; @@ -413,6 +480,13 @@ export function* evaluateProgram(request: ProgramEvaluationRequest): Operation

( { type: EVALUATE_PROGRAM, @@ -420,7 +494,7 @@ export function* evaluateProgram(request: ProgramEvaluationRequest): Operation

admitProgram(request, terms), + () => admitProgram(request, terms, resolved), ); const decided = readAdmission(parseJson(stored)); @@ -430,27 +504,85 @@ export function* evaluateProgram(request: ProgramEvaluationRequest): Operation

; + readonly returns?: ReturnsSchema; + readonly bodySegments: Segment[]; +} + +/** + * Prove the retained admission describes itself, and produce what it admitted. + * + * A journal is data, and every member of this record is a claim about another + * one: the digest claims to be the source's, the mode claims to be what + * reparsing produces, the validated props claim to be what the supplied ones + * validate to, and the components claim to be the elements the source writes. + * A record that fails any of them is not one this evaluation wrote, whatever + * shape it has, and nothing of either program runs on the strength of it. + * + * The parse is the one this expansion will use, so what is checked and what + * runs are the same value rather than two readings of one string. + */ +function* restore(decided: Admitted): Operation { + if (programDigest(decided.source) !== decided.terms.digest) { + return undefined; + } + let parsed; + try { + parsed = yield* parseRootMarkdownDefinition(PROGRAM, decided.terms.origin, decided.source); + } catch { + return undefined; + } + const { definition } = parsed; + if (validateBodyStructure(definition.bodySegments, definition.returns) !== undefined) { + return undefined; + } + const mode = definition.returns === undefined ? "text" : "value"; + if (mode !== decided.mode || (mode === "value" && !decided.terms.captured)) { + return undefined; + } + if (!sameElements(decided.named, elements(definition.bodySegments, []))) { + return undefined; + } + let props: Record; + try { + props = yield* validateProps(PROGRAM, { ...decided.terms.props }, definition.props); + } catch { + return undefined; + } + if (JSON.stringify(props) !== JSON.stringify(decided.validated)) { + return undefined; + } + return { + meta: definition.meta, + ...(definition.returns === undefined ? {} : { returns: definition.returns }), + bodySegments: definition.bodySegments, + }; +} diff --git a/packages/core/src/program-identity.ts b/packages/core/src/program-identity.ts new file mode 100644 index 000000000..37abd4fab --- /dev/null +++ b/packages/core/src/program-identity.ts @@ -0,0 +1,139 @@ +/** + * What a complete program's components resolve to, and how a continuation is + * held to it (specs/executable-mdx-spec.md §5.7). + * + * A program is admitted at a site, and a site is an environment: which + * definition each name it writes resolves to is part of what the admission + * granted. Retaining the names alone would let a run resume against a different + * `` than the one it was admitted with — same bytes, different + * program — so the admission retains the identity canonical resolution selected + * for each name together with the form the element is written in, and a + * continuation is refused when either has moved. + * + * The identity is a string rather than the selection itself because it is + * durable data: it is written into a journal, read back from one, and compared + * whole. Every tier that can answer a name contributes a distinguishable one, + * and a name nothing answers is `unresolved` — which is an identity like any + * other here, so a name that becomes resolvable between two runs is a change + * the comparison sees. + */ + +import type { ComponentSelection } from "./types.ts"; + +/** The authored forms an element is written in. */ +export type ProgramComponentForm = "self-closing" | "paired"; + +/** One component a program names, with the form and identity it resolved to. */ +export interface ProgramComponent { + readonly name: string; + readonly form: ProgramComponentForm; + /** The identity canonical resolution selected, or `unresolved`. */ + readonly identity: string; +} + +/** One element a program names, before its identity has been resolved. */ +export interface ProgramComponentRef { + readonly name: string; + readonly form: ProgramComponentForm; +} + +/** A name nothing at this site answers for. */ +export const UNRESOLVED = "unresolved"; + +/** A program this site will not evaluate. */ +export class ProgramEvaluationError extends Error { + override name = "ProgramEvaluationError"; +} + +/** + * What a continuation whose site has moved says. + * + * Distinct from an unreadable record on purpose. The journal is intact and says + * exactly what it always said; what changed is the environment, and a run + * resumed against a different implementation than it was admitted with is being + * offered a different program under the same bytes. + */ +export const INCOMPATIBLE = + " was resumed at a site where a component this program names resolves differently " + + "than it did when this evaluation was admitted."; + +/** + * Resolve one name to a component this execution may name durable work about. + * + * Held by canonical execution and handed to core's own expansion by value, like + * the rest of the expansion authority. It answers with the identity selection + * produced, and it imports nothing: resolving a name is a decision about which + * definition answers it, and loading one is a durable effect a program's own + * expansion performs where the element is written. + */ +export interface ProgramResolver { + (name: string): import("effection").Operation; +} + +/** + * The durable identity of what a name resolved to. + * + * Every tier is distinguishable, and each carries the part of its selection + * that says which implementation it is: a registration is its origin and + * whether it is reserved, a repository component is its path, a bundled one is + * its path and object id, and declared Markdown is its origin and digest. A + * private name resolves to nothing at all (`select.ts`), so a program naming one + * carries the same identity as a program naming a component that does not + * exist — which is what it is. + */ +export function componentIdentity(selection: ComponentSelection): string { + switch (selection.kind) { + case "structural": + return `structural:${selection.construct}`; + case "registered": + return selection.origin.kind === "registered" + ? `registered:${selection.origin.reserved ? "reserved" : "default"}:${selection.origin.origin}` + : `registered:${selection.origin.kind}`; + case "repository": + return `repository:${selection.path}`; + case "workflow": + return `workflow:${selection.path}@${selection.sourceHash}`; + case "declared-markdown": + return `declared-markdown:${selection.origin}@${selection.digest}`; + default: + return UNRESOLVED; + } +} + +/** Whether a value is one of the two authored forms. */ +export function isProgramComponentForm(value: unknown): value is ProgramComponentForm { + return value === "self-closing" || value === "paired"; +} + +/** Whether two resolved component lists describe the same site. */ +export function sameComponents( + retained: readonly ProgramComponent[], + current: readonly ProgramComponent[], +): boolean { + if (retained.length !== current.length) { + return false; + } + return retained.every((entry, index) => { + const other = current[index]; + return ( + other !== undefined && + entry.name === other.name && + entry.form === other.form && + entry.identity === other.identity + ); + }); +} + +/** Whether two lists name the same elements in the same forms, whatever they resolved to. */ +export function sameElements( + retained: readonly ProgramComponent[], + current: readonly ProgramComponentRef[], +): boolean { + if (retained.length !== current.length) { + return false; + } + return retained.every((entry, index) => { + const other = current[index]; + return other !== undefined && entry.name === other.name && entry.form === other.form; + }); +} diff --git a/packages/core/src/types.ts b/packages/core/src/types.ts index 5db64fe32..0d9714503 100644 --- a/packages/core/src/types.ts +++ b/packages/core/src/types.ts @@ -9,6 +9,7 @@ import type { Operation, Result } from "effection"; import type { Json as DurableJson } from "@executablemd/durable-streams"; import type { TestHarnessComponentDefinition } from "./test-harness.ts"; import type { ComponentInvocation, InvocationForm } from "./invocation-identity.ts"; +import type { ProgramComponent } from "./program-identity.ts"; export type Json = DurableJson; @@ -427,6 +428,16 @@ export interface ProgramBody { readonly bodySegments: Segment[]; /** The source origin the evaluation site resolves from. */ readonly path: string; + /** + * The components the admission resolved, with the forms it admitted them in. + * + * Carried here so canonical execution can settle the same question again + * before the first program effect: a site that now answers one of these names + * with a different implementation is not the site this program was admitted + * at. Verified against canonical execution's own resolver, never against + * anything the chain could answer. + */ + readonly named: readonly ProgramComponent[]; } /** What expanding a program produced: selected output, or a returned value. */ diff --git a/packages/core/tests/program-evaluation.test.ts b/packages/core/tests/program-evaluation.test.ts index b2accbcdf..2146f57b3 100644 --- a/packages/core/tests/program-evaluation.test.ts +++ b/packages/core/tests/program-evaluation.test.ts @@ -32,8 +32,8 @@ import type { Operation } from "effection"; import { createDurableOperation, InMemoryStream } from "@executablemd/durable-streams"; import type { DurableEvent } from "@executablemd/durable-streams"; -import { executeInstalled, programEvaluationComponents } from "../host.ts"; -import type { IdentityClaimant, IdentityComponent } from "../host.ts"; +import { executeInstalled, programEvaluationComponents, sourceDigest } from "../host.ts"; +import type { DeclaredMarkdownComponent, IdentityClaimant, IdentityComponent } from "../host.ts"; import type { ComponentInvocation } from "../src/invocation-identity.ts"; import { pairedProgramSource, programDigest } from "../host.ts"; import { retainedSource } from "../src/root-source.ts"; @@ -79,7 +79,7 @@ let approved = APPROVED; /** Every time a producer really rendered its approved source. */ const produced: string[] = []; -function probeComponents(): readonly IdentityComponent[] { +function probeComponents(origin = "test/probe"): readonly IdentityComponent[] { return [ { name: "Source", @@ -96,7 +96,7 @@ function probeComponents(): readonly IdentityComponent[] { }, { name: "Probe", - origin: "test/probe", + origin, props: { type: "object", properties: { mark: { type: "string" } }, @@ -125,12 +125,26 @@ function probeComponents(): readonly IdentityComponent[] { } /** Run one document with `` declared, exactly as a run profile does. */ -function run(source: string, options: { stream?: InMemoryStream } = {}): Operation { +function run( + source: string, + options: { + stream?: InMemoryStream; + /** The origin `` is registered under, so a site can be moved. */ + probeOrigin?: string; + declarations?: readonly DeclaredMarkdownComponent[]; + privates?: readonly IdentityComponent[]; + } = {}, +): Operation { return scoped(function* () { const stream = options.stream ?? new InMemoryStream(); const execution = yield* executeInstalled( { ...retainedSource(ROOT_PATH, source), stream, includes: [] }, - [{ components: [...programEvaluationComponents(), ...probeComponents()] }], + [ + { + components: [...programEvaluationComponents(), ...probeComponents(options.probeOrigin)], + ...(options.declarations === undefined ? {} : { declarations: options.declarations }), + }, + ], ); const result = yield* execution; const events = yield* stream.readAll(); @@ -505,3 +519,213 @@ describe("Tier PE — a program that fails", () => { expect(performed).toEqual(["before"]); }); }); + +/** + * Rewrite what the retained admission says, leaving the run's own history + * otherwise intact. + * + * The record is the one thing a continuation trusts, and a journal is a file: + * this is what it looks like when the file no longer says what this evaluation + * wrote. Nothing else about the interrupted run is touched, so a case that + * refuses here refuses because of the record and not because the history around + * it stopped adding up. + */ +function tamper( + events: DurableEvent[], + change: (record: Record) => Json, +): DurableEvent[] { + return events.map((event) => { + if (event.type !== "yield" || event.description.type !== "evaluate_program") { + return event; + } + const result = event.result; + if (result.status !== "ok") { + return event; + } + const record = result.value as Record; + return { ...event, result: { ...result, value: change({ ...record }) } }; + }); +} + +/** A program whose one effect is a probe, so "did anything run" is answerable. */ +const PROBING = '\n\nRestored program.\n'; + +/** The document every hostile-record case resumes, unchanged between runs. */ +const PROBING_DOCUMENT = ["", " ", "", ""].join("\n"); + +/** Interrupt a run of `PROBING` right after its admission committed. */ +function* admitted(): Operation { + performed.length = 0; + approved = PROBING; + const first = yield* run(PROBING_DOCUMENT); + expect(first.failure).toBeUndefined(); + expect(performed).toEqual(["one"]); + return through(first.events, "evaluate_program"); +} + +describe("Tier PE — the retained admission is hostile data", () => { + it("PE18 refuses a retained source that no longer hashes to its digest", function* () { + const interrupted = yield* admitted(); + performed.length = 0; + + // Only the source moves. The digest, the terms and the current request are + // all exactly what they were, so nothing but the record's own internal + // agreement can catch this. + const replayed = yield* run(PROBING_DOCUMENT, { + stream: new InMemoryStream( + tamper(interrupted, (record) => ({ + ...record, + source: '\n\nRestored program.\n', + })), + ), + }); + + expect(replayed.failure ?? replayed.output ?? "").toContain("cannot be read as one"); + // Neither source ran: not the substituted one, and not the one the current + // producer would have offered in its place. + expect(performed).toEqual([]); + }); + + it("PE19 refuses every corrupted member of the retained record", function* () { + const interrupted = yield* admitted(); + + const corruptions: Record) => Json> = { + "validated props": (record) => ({ ...record, validated: { release: "substituted" } }), + "root mode": (record) => ({ ...record, mode: "value" }), + "a missing member": ({ validated: _validated, ...rest }) => rest, + "an additional member": (record) => ({ ...record, admitted: true }), + "a misspelled member": ({ validated, ...rest }) => ({ ...rest, validatedProps: validated }), + "a component entry": (record) => ({ + ...record, + named: [{ name: "Probe", form: "paired", identity: "registered:default:test/probe" }], + }), + "a component entry's shape": (record) => ({ + ...record, + named: [{ name: "Probe", form: "self-closing" }], + }), + "the terms' shape": (record) => ({ + ...record, + terms: { ...(record.terms as Record), extra: 1 }, + }), + "the whole record": () => "admitted", + }; + + for (const [what, change] of Object.entries(corruptions)) { + performed.length = 0; + const replayed = yield* run(PROBING_DOCUMENT, { + stream: new InMemoryStream(tamper(interrupted, change)), + }); + + expect(`${what}: ${replayed.failure ?? replayed.output ?? ""}`).toContain( + "cannot be read as one", + ); + expect([what, performed]).toEqual([what, []]); + } + }); + + it("PE20 refuses a continuation whose site answers a name differently", function* () { + const interrupted = yield* admitted(); + + // The same document, the same producer, the same bytes — and `` + // registered under another origin, which is a different implementation + // however identically it behaves. + performed.length = 0; + const moved = yield* run(PROBING_DOCUMENT, { + stream: new InMemoryStream(interrupted), + probeOrigin: "test/probe-replacement", + }); + + expect(moved.failure ?? moved.output ?? "").toContain("resolves differently"); + expect(performed).toEqual([]); + + // The unchanged site resumes, which is what makes the refusal above about + // the identity rather than about resuming at all. + performed.length = 0; + const unchanged = yield* run(PROBING_DOCUMENT, { + stream: new InMemoryStream(interrupted), + }); + + expect(unchanged.failure).toBeUndefined(); + expect(unchanged.output).toContain("Restored program."); + // The history stops at the admission, so the program's own effect had not + // committed and legitimately happens now. What matters is that it happened + // at all: the refusal above stopped a run this one completes. + expect(performed).toEqual(["one"]); + }); +}); + +describe("Tier PE — structural admission precedes every program effect", () => { + it("PE21 refuses a malformed construct after an effect, and the effect never runs", function* () { + performed.length = 0; + // The negative control is first, so a preflight that ran while the program + // expanded would have performed it before reaching the malformed element. + approved = ['', "", "", ""].join("\n"); + + const attempt = yield* run(PROBING_DOCUMENT); + + expect(attempt.failure ?? attempt.output ?? "").toContain("body structure is not valid"); + expect(performed).toEqual([]); + approved = APPROVED; + }); +}); + +/** The origin the declared outer component reports. */ +const POLICY_ORIGIN = "@executablemd/test/Policy.md"; + +/** + * A private component that records each entry into its implementation. + * + * A tripwire rather than a fake: what this case has to show is that the + * implementation did not run, and a refusal in the output cannot show that on + * its own. + */ +function watching(entered: string[]): IdentityComponent { + return { + name: "Secret", + origin: `${POLICY_ORIGIN}#Secret`, + props: { type: "object", properties: {}, additionalProperties: false }, + returns: { type: "string" }, + forms: ["self-closing"], + // deno-lint-ignore require-yield + factory: (_claim: IdentityClaimant) => + function* Secret(): Operation { + entered.push("Secret"); + return "the private answer"; + }, + }; +} + +describe("Tier PE — a producer's private closure does not cross", () => { + it("PE22 leaves a declaration's private name unavailable to the program it evaluates", function* () { + const entered: string[] = []; + performed.length = 0; + // The program names the private component the *enclosing declaration* + // carries, and an ordinary site-authorized one beside it. + approved = ['', "", "", ""].join("\n"); + + const body = ["", " ", "", ""].join("\n"); + const attempt = yield* run("\n", { + declarations: [ + { + name: "Policy", + origin: POLICY_ORIGIN, + source: body, + digest: sourceDigest(body), + privates: [watching(entered)], + }, + ], + }); + + const reported = attempt.failure ?? attempt.output ?? ""; + // The name resolves to nothing inside the program, exactly as it does for + // any other bytes that are not the declaration's own. + expect(reported).toContain("Cannot resolve component: Secret"); + // And nothing of it ran: a refusal that reached the implementation first + // would leave a mark here. + expect(entered).toEqual([]); + // The positive control: a component the site does authorize runs normally, + // so this is about the closure rather than about programs reaching nothing. + expect(performed).toEqual(["control"]); + approved = APPROVED; + }); +}); diff --git a/specs/executable-mdx-spec.md b/specs/executable-mdx-spec.md index 79eb81fb8..16cee20a5 100644 --- a/specs/executable-mdx-spec.md +++ b/specs/executable-mdx-spec.md @@ -4411,7 +4411,8 @@ interface, and each operation is also exported directly: | `hasBinding()` | Whether the invocation has an engine-owned result binding — whether `as` was written (§6.10) | throws a missing-provider error | | `hasCapture(name)` | Whether the invocation wrote this capture prop at all (§6.5) | `false` | | `capture(name)` | Evaluate a capture prop now, against the caller's bindings, and deliver the result by reference (§6.5) | throws: not inside a function component invocation | -| `expandProgram(program)` | Expand a complete XMD program at this invocation's site, under the authority the enclosing expansion holds (§5.7) | throws: not inside a function component invocation | +| `expandProgram(program)` | Expand a complete XMD program at this invocation's site, under the authority the enclosing expansion holds, after settling that every retained component identity still describes this site (§5.7) | throws: not inside a function component invocation | +| `resolveProgramSite(named)` | What each name a complete program writes resolves to at this site, for the admission to retain (§5.7). Observable and short-circuitable; the compatibility decision is canonical execution's own and rests on nothing answered here | throws: not inside a function component invocation | | `handleFailure(failure)` | What an ordinary function-component failure means, after complete invocation teardown (§6.9) | fails the operation with `failure.error` | | `retain(resource)` | Create a resource in the invocation-site scope, so it outlives this invocation (§4.4) | throws: not inside a component invocation | @@ -4816,12 +4817,47 @@ origin of the authored `` site. **Admission is this boundary's own durable event.** A complete program records `evaluate_program`, never `generated_xmd` (§10). Before the first program effect -the run retains the exact source and its digest, the explicit props, the -evaluation-site source origin, the root mode, and the components the program -names with the forms it writes them in. A partial continuation expands the -retained source and restores the nested effects that already committed. A -changed program at the same evaluation occurrence is stale input: it refuses -before either the current or the retained source runs, so neither silently wins. +the run retains the exact source and its digest, the explicit props and the +props they validated to, the evaluation-site source origin, whether the result +is captured, the root mode, and — for every element the program writes, in the +order it writes them — the name, the authored form, and **the identity +canonical resolution selects for that name at this site**. A partial +continuation expands the retained source and restores the nested effects that +already committed. A changed program at the same evaluation occurrence is stale +input: it refuses before either the current or the retained source runs, so +neither silently wins. + +A retained identity is a string, and every tier that can answer a name +contributes a distinguishable one: `structural:`, +`registered::`, `repository:`, +`workflow:@`, `declared-markdown:@`, and +`unresolved` for a name nothing at this site answers — a private name included, +because selection resolves one to nothing outside the declaration that carries +it (§5.3). Resolution imports nothing: it decides which definition answers a +name, while loading one is the program's own durable effect where the element is +written. + +**A continuation is held to the site it was admitted at.** Before the first +program effect, canonical execution resolves every retained name again and +refuses when any identity or form has moved. That comparison is canonical +execution's own: the resolver is built by the execution, travels on the +expansion authority, and is reachable from no document, component, contextual +Api answer or middleware return value. A host may observe the admission's own +resolution through `Component.resolveProgramSite()`, and answering it +dishonestly refuses the evaluation rather than widening it, because the +canonical comparison is what decides. + +**The retained record is hostile data.** A journal is a file, and replay hands +back whatever it holds. The admitted and refused decisions and every nested +record are closed shapes: a missing member, an additional member and a +misspelled member are each read as "not a record this evaluation wrote". Beyond +shape, the record must agree with itself before anything expands — the retained +source hashes to the retained digest; reparsing it produces the retained root +mode and a valid body structure; the retained elements are the ones that source +writes, in that order and in those forms; and the supplied props validate to the +retained validated props. A record that fails any of them, at any depth, is +refused as unreadable, and neither the retained nor the current program performs +an effect. **The digest identifies the artifact; the site identifies the occurrence.** The authored element and the loop iteration it was reached through are what make one @@ -9804,7 +9840,7 @@ trusted-host events may have no authored source. | Resolve components (glob) | `glob` | `resolve:{dir}` | Only when `useDurableGlobResolver` middleware is installed | | Read over HTTP | `fetch` | `fetch:{expansion id}` | Normalized request in `description.input`; status, detached headers and text body in the result (§6.18) | | Admit generated XMD | `generated_xmd` | `generated:{fragment id}` | The canonical class selection, retained roots, selected root, every selected entry as a name, identity and admitted forms, and the exact request policy in `description.input`; the admitted source, that same policy, and the identity and form of each element the fragment named in the result (workflow-workspace-spec §8.4) | -| Admit a complete program | `evaluate_program` | `program:{evaluation id}` | The source digest, explicit props, evaluation-site source origin and whether the result is captured in `description.input`; the exact admitted source, the root mode, the components the program names with their authored forms, those same terms and the validated root props in the result — or the refused class alone (§5.7) | +| Admit a complete program | `evaluate_program` | `program:{evaluation id}` | The source digest, explicit props, evaluation-site source origin and whether the result is captured in `description.input`. The result is one of two closed shapes: an admitted one of exactly `decision`, `source`, `mode`, `named`, `terms` and `validated` — where each `named` entry is exactly `name`, `form` and `identity`, and `terms` is exactly `digest`, `props`, `origin` and `captured` — or a refused one of exactly `decision` and `refused`, carrying the refused class and nothing of the source. A record with a missing, additional or misspelled member at any depth, or one that disagrees with itself, is unreadable rather than partially believed (§5.7) | ### 10.2 Example journal for a multi-component document @@ -12320,6 +12356,12 @@ forms sit beside the restricted one. | PE14 | Repeated evaluation | Two sites evaluate one artifact independently under two durable names; the digest deduplicates neither | | PE15 | Changed evaluation source | A partial continuation whose producer returns different source refuses as stale input, and neither the current nor the retained program performs its effect | | PE16 | Evaluation replay | A partial continuation expands the retained source and restores a completed nested effect without performing it again, proven by the producer re-rendering on that run | +| PE17 | A failing program | A program stops where it failed: the effect written before the failure happened and the one after it did not | +| PE18 | A retained source that moved | Altering only the retained source, with its digest and the current request untouched, refuses as unreadable and runs neither program | +| PE19 | Every corrupted member | Corrupted validated props, root mode, a missing, additional or misspelled member, a corrupted component entry, a corrupted entry shape, corrupted term shape and a record that is not an object each refuse as unreadable with nothing run | +| PE20 | A site that answers differently | The same source and request, with one named component registered under another origin, refuses before either implementation runs; the unchanged site resumes | +| PE21 | Structural preflight | A durable effect written before a later malformed structural construct never runs: structural admission refuses first | +| PE22 | The producer's private closure | A program evaluated inside a declaration carrying a private component cannot resolve that name and never enters its implementation, while an ordinary site-authorized component in the same program runs | | EP1–EP8 | Run profile | The ordinary run profile declares ``; the forms, props, refusals and catalog entry hold through the real binary, and a program cannot reach ``'s private components | | WGAC17–WGAC20 | Workflow profile | A complete program records `evaluate_program` and no `generated_xmd`; the three forms cannot be combined; complete-program support is not reachable through `source` | From b284a6cab9400c4625c0bb6a78ff4debfe697494 Mon Sep 17 00:00:00 2001 From: Taras Mankovski Date: Fri, 4 Sep 2026 09:43:11 -0400 Subject: [PATCH 3/7] =?UTF-8?q?=F0=9F=94=91=20Take=20a=20program=20compone?= =?UTF-8?q?nt=20identity=20from=20the=20answer=20the=20chain=20supplies=20?= =?UTF-8?q?(#713)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The retained identity came from an independent `selectComponent()` call, which is not what runs. `Component.importComponent` middleware may answer without delegating or replace what came back, so a second resolution described a definition nobody invokes: two different middleware answers retained one identity, compared equal, and a continuation ran the other one. Resolution now goes through the ordinary chain and takes the identity from the final answer. A canonical tier keeps its canonical identity. An answer an identified provider supplied keeps that provider's own — a stable origin, its key for the name, and a revision that changes with the implementation — stated at the installation boundary through `useImportProvider()`, where canonical execution mints a claimant for that execution and binds the terms to the exact answer in execution-private state. Never on the definition, which an answer can copy; never through a replaceable Context answer. An answer nobody identified still works for ordinary expansion and cannot back a durable grant: a continuation would have nothing to compare. Two live providers under one origin refuse, and so does a second claim on one answer. Identities are closed tagged records now, so a missing, empty, additional or mis-tagged member is unreadable like every other corrupted member. The comparison and the invocation are one decision: canonical execution keeps its own copy of each witnessed answer and authorizes the program's imports against it, so an answer that changed between the check and the use is refused where it would be invoked rather than silently preferred. --- architecture.md | 6 +- packages/core/host.ts | 14 + packages/core/src/component-api.ts | 8 +- .../core/src/components/import-authority.ts | 84 +++- packages/core/src/execute.ts | 86 +++- packages/core/src/expand.ts | 42 +- packages/core/src/program-evaluation.ts | 27 +- packages/core/src/program-identity.ts | 230 ++++++++-- packages/core/src/program-imports.ts | 189 +++++++++ .../core/tests/program-evaluation.test.ts | 392 +++++++++++++++++- specs/executable-mdx-spec.md | 83 +++- 11 files changed, 1064 insertions(+), 97 deletions(-) create mode 100644 packages/core/src/program-imports.ts diff --git a/architecture.md b/architecture.md index bc8433c04..237cfdf38 100644 --- a/architecture.md +++ b/architecture.md @@ -44,6 +44,8 @@ Existing documents and code get aligned to this section retroactively. | Plan | the executable program produced from a Prompt: an Executable Markdown document combining readable prose that expresses the Prompt's intent with the components that carry it out, each placed beside the prose describing what it does. It begins with one descriptive level-one heading. A Plan is what `xmd plan` approves and then delivers as source: printed to stdout by default, or written to an `--output` file. Plan produces a program; Evaluate evaluates a program in the current XMD execution; Run executes a program from the host or CLI; composition decides whether and when a planned program runs. It is not a synonym for a workflow, a policy document or any executable Markdown file | | plan command document | the one exact checked-in first-party Markdown value root `xmd plan` executes. It is the command's adapter and nothing else: it projects the request into ``, supplies the session, and returns the approved source. Its body is those two elements and no prose, because its rendered transcript is the progress the command writes to stderr. It is not itself a Plan. Internal: no command-line option selects another one, and no repository component search can answer for it | | packaged `` Component | the one exact checked-in first-party Markdown text component that converts a prompt into a Plan, `packages/cli/src/documents/Plan.md`, declared to every ordinary run as the public ``. It owns and implements the Plan authorship workflow — the Prompt wording, the draft and repair loops, the `` branches, human review, revision, approval, stopping, exhaustion and the automatic final explanation turn — and produces the exact approved Plan source as what it renders: written bare it emits those bytes, and `as` is ordinary text capture that binds them and emits nothing. Neither form evaluates the source. Every Plan-producing turn in it states the complete Plan requirements for itself, so a replacement may add or correct a title rather than only carry one forward. Both surfaces expand these exact bytes under one origin and one digest; there is no generated TypeScript copy and no second Markdown implementation. Its four phase components are private to it, and it is not itself a Plan | +| identified import provider | a `Component.importComponent` middleware provider that states a stable versioned identity for the answers it supplies — an origin it answers under and a revision that changes when the implementation does — through `useImportProvider()` at its installation boundary, where canonical execution mints it a claimant for that one execution. Marking an answer binds those terms to that exact object in execution-private state, never on the definition and never through a replaceable Context answer. The identity is an assertion by the authority installed at the site, exactly as a registration's origin is: nothing proves a provider is who it says, and what it buys is that a continuation can be told the implementation changed. Two live providers under one origin refuse, and so does a second claim on one answer. A provider that only delegates identifies nothing, because the answer it returns is already canonical execution's | built by #713 | +| unidentified middleware answer | an import answer no authority stands behind — a provider that answered or replaced without stating an identity. Ordinary expansion runs it exactly as it always did; a complete program cannot be admitted against one, because a continuation would have nothing to compare and would invoke whatever answered on the day it resumed | built by #713 | | composition site | the authored `` element where a complete XMD program is admitted and evaluated. It is a place in a document rather than a boundary: the program runs in the current execution, under the authority in force where the element was written, and the element and the loop iteration reaching it identify one execution of that program. A composition site is not a child document, a child process, a second host profile or a second journal | | authorship profile | the trusted-host assembly the packaged `` Component runs its authored turns under, installed by its own `` inside the invocation that owns it rather than around an execution — which is what makes it the same frame whether `xmd plan`, an ordinary document, or a configured `` run child asked. Which Agent context goes under it is a trusted-host capability the declaration carries — the production ACPX one built from the run's Agent stack, or the deterministic one a canonical `` child declaration produced — and a host that supplies none states the sentence a `` written there is refused with. The fixed policy is installed in one place for both, so a second provider cannot bring a weaker one: its fixed inputs, a constrained Agent provider, Elicitation, the fixed first-party components and the host-declared ``. It uses no repository component search and exposes no custom root, and the policy it installs is not readable from the command line. Its working directory is one host-owned directory dedicated to the logical session, keyed by the digest of that name, created empty and required to be empty on the way in. An explicitly named session's directory is durable, because continuation derives the same session identity from it; an invocation-unique default session's is scope-owned, claimed before it is created, and exactly one cleanup is attempted after profile teardown and before admission on every ending — the leaf removed non-recursively when it is still the empty directory that was handed over, and left as found with the command failing terminally when it has gained content or vanished. Where those directories live is a host dependency no caller or document selects | | upgrade command document | the one exact checked-in first-party Markdown streaming text root `xmd upgrade` executes to select and install a published release. It owns the exact-tag grammar, release selection, semantic-version comparison, consent, the status, already-current and installation branches, and the wording of every refusal and report; its rendered body is the command's output rather than a value it returns. Internal: no command-line option selects another one, and no repository component search can answer for it | @@ -3897,9 +3899,9 @@ Status is measured against main. | workflow Agent session | a workflow document's `` runs under a profile the host attaches only for a live or partial run: an empty host-owned working directory instead of any Workspace, checkout or caller path, no MCP servers, an empty requested native tool set, and `deny-all` with a permission path that denies every native request and fails the turn that asked without reaching the public permission chain. Within a run a session is identified by the Agent/Session expansion identity the engine derived — the authored name is descriptive, so two sibling `` elements are two sessions — routed inside a placement bound to its element and good for one use, so a kept placement cannot be substituted for the next. The conversation is retained as a row in the run's own database with the provider, resolved agent command and policy fingerprint beside it as compatibility attributes. The order is placement, the backend's acceptance of the session's first turn, the provider's canonical tagged assertion, then the mapping commit — and only then is anything that turn produced exposed. A placement is inert: it creates no provider session and writes no row. Occupancy of a provider key is not an assertion, and a record held for a first turn nobody accepted asserts nothing at all; the pre-commit window reconciles only from exactly one canonical assertion, and a missing, conflicting, replaced or ambiguous assertion is one explicit refusal that starts no replacement. Deleting a run removes the row with the run and the provider-session directory beside it, and reports the categories. The profile selects ACP-only capability explicitly — no native-launch advertisement and no client-native attachment advertisement — rather than inheriting the provider package's ordinary-run sets by omission, and it supplies no machine session coordinator, construction-route store or executable observer: a workflow session belongs to a run, and the machine-wide account describes a different thing entirely | built on the #302 stack, with the explicit ACP-only selection from #561; the portable proof that an adapter honours an empty tool set is tracked by #496 and does not widen the ceiling | | generated-XMD admission | admits one Agent-generated fragment through the trusted-host seam: host policy is a `read` table and a `write` table of exact pinned identities, each carrying the authored forms it is admitted for, and an authored `allow` selects a canonical non-empty subset of the closed classes — omitted means `read`. The complete source is preflighted inside one `generated_xmd` durable effect before its first generated effect; only the pinned identity the selected classes hold for that name **and** that form executes; and the admitted source, class selection, selected root, every selected entry with its forms, the identity and form of each element named, and the normalized request policy are retained in that effect's own result — so a continuation restores the decision without reading the current candidate and expands only the retained source. The roots are an as-of-admission retained basis checked by membership — the run's own later root publications and an advanced retained current root pass, while a lost admission root or lost selected root refuses — and every non-root term is checked exactly, refusing a run whose classes, identities, forms or requests have moved. The admission and every nested generated effect are offered inline by the owning expansion in authored order, so a partial continuation restores each completed one without another live execution. Each admitted effect is retained by its own ordinary record, and a read's value is collected while a mutation's is not | built on the #369 stack, continuation basis amended by #589; core owns the mechanics and the workflow policy wrapper is internal | | `` | the restricted generated-fragment form of ``, which the workflow host declares beside the complete-program forms below. The form an authored document writes where an observation should happen. The host does not register it: it **declares** it to the execution through `ExecutionInstallation.components`, captured before any installation runs, and canonical execution calls its factory once for that attachment with the claimant it minted and registers what comes back. Registration provides availability only — a name a trusted document may write — and carries none of the authority. It takes a `source` string and one optional `allow` array selecting a non-empty duplicate-free subset of the closed effect classes `read` and `write` — omitted means `read` — refuses paired content, and cannot be combined with `program`, with paired content, or with `props`; each of those refuses before the durable name is claimed and before the candidate is read. It declares no `returns` and answers with a detached value — `{ observations: [{ name, value }], output }`, each admitted read's own returned value under the name the fragment invoked it by, in invocation order, with whatever the fragment rendered under `output` rather than instead of them, and the pinned identity that produced one left in the retained admission rather than copied here — so an admitted ``, which renders nothing at all, still reaches the document. An admitted mutation contributes no entry and no receipt, so a write-only fragment binds `{ observations: [], output: "" }`; `as` is valid for every selection and binds that same shape. An ordinary `as` captures that value by reference, and an authored `` renders it into the next ``: deciding how a value becomes text is the document's. Every ceiling comes from values the host captured at installation — the run's retained roots and its authoritative current root read from the run's own storage at invocation, as-of-admission provenance a continuation holds by membership so the run's own later publications and an advanced retained current root invalidate nothing, core's pinned self-closing `` read, the write table of core's paired ``, this package's lexical `` built from the definition the ordinary registration owns, and core's self-closing ``, and `` only when the captured request ceiling is non-empty — and no prop, binding, context or middleware return value supplies or widens one. `allow` selects among those tables and adds nothing to them; approval, when a workflow needs one, is authored control flow before the element. Its durable operation is named through that claimant, on the exact invocation the engine handed it and in that invocation's own frame — not from a context a document could rebind, a contextual Api answer, a definition, or a registry answer. Generated source never resolves through the registration: the evaluator consults only its own closed table of pinned identities. It is deliberately not wrapped in `printErrors`, so a refused fragment stops the authored loop rather than becoming text the next turn could read as a read that happened | built on the #302 stack, extended by #369 | -| complete-program evaluation | admits one complete XMD root and evaluates it at the authored `` site, in the current execution: its lifecycle, journal, cancellation scope, contextual providers, working directory or Workspace, and the authority in force where the element was written. There is no child process, host profile, root lifecycle or journal of its own — program source requests behavior and grants none. The root's own frontmatter, imports, metadata, props schema, `returns` and `` selection decide what it does; a text root renders its selected output or binds it as a string under `as`, a value root requires `as` and binds its schema-validated result, and root props are the explicit `props` object defaulting to `{}` rather than the caller's own. Ordinary non-props caller bindings are visible as a copy, so nothing the program binds escapes, and relative paths resolve from the evaluation site's source origin. The producer's temporary authority is not inherited: an enclosing declaration's private closure does not cross, so a private name written in a program resolves to nothing, while imports and capabilities genuinely present at the site remain. Canonical execution answers `Component.expandProgram()` from the frame it already holds, which is what makes site authority a fact about the frame rather than something a prop, binding or middleware return value could supply. Which definition each name the program writes resolves to is part of what the site granted, so the admission retains it and canonical execution settles it again before the first program effect — through a resolver the execution builds and carries on the expansion authority, reachable from no document, component, contextual Api answer or middleware return value | built by #713 on the #722 `` contract; core owns the mechanics and every host that offers `` declares them | +| complete-program evaluation | admits one complete XMD root and evaluates it at the authored `` site, in the current execution: its lifecycle, journal, cancellation scope, contextual providers, working directory or Workspace, and the authority in force where the element was written. There is no child process, host profile, root lifecycle or journal of its own — program source requests behavior and grants none. The root's own frontmatter, imports, metadata, props schema, `returns` and `` selection decide what it does; a text root renders its selected output or binds it as a string under `as`, a value root requires `as` and binds its schema-validated result, and root props are the explicit `props` object defaulting to `{}` rather than the caller's own. Ordinary non-props caller bindings are visible as a copy, so nothing the program binds escapes, and relative paths resolve from the evaluation site's source origin. The producer's temporary authority is not inherited: an enclosing declaration's private closure does not cross, so a private name written in a program resolves to nothing, while imports and capabilities genuinely present at the site remain. Canonical execution answers `Component.expandProgram()` from the frame it already holds, which is what makes site authority a fact about the frame rather than something a prop, binding or middleware return value could supply. Which definition each name the program writes resolves to is part of what the site granted, so the admission retains it and canonical execution settles it again before the first program effect — through a resolver the execution builds and carries on the expansion authority, reachable from no document, component, contextual Api answer or middleware return value. That resolver goes through the ordinary import chain, because the answer the chain returns is the answer that runs: an independent second resolution would describe a definition a provider replaced, and two different middleware answers would compare equal. The answers that passed the comparison are the answers the program invokes, authorized against canonical execution's own copy, so an answer that changed between the check and the use is refused rather than preferred | built by #713 on the #722 `` contract; core owns the mechanics and every host that offers `` declares them | | `` complete-program forms | the two ways an author names a program at a composition site: paired content, which renders a source-producing component into a private program buffer, and `program={value}`, which supplies complete-program source directly. Optional `props={object}` is the root's props input and is valid only for these forms. The two are mutually exclusive and one is required; every ambiguous or misplaced combination refuses before program content is produced and before any program effect runs, so an ambiguous element does not even render its producer. Paired content is a private buffer rather than a quotation — an executable component written there executes under ordinary XMD semantics while producing the text sent to admission — and the element removes only the framing its own wrapper contributed: the line break after the opening tag with the indentation following it, the line break before the closing tag with the indentation preceding it, and the indentation those lines share. Interior bytes are untouched, so a sole `` producer's trailing newline survives and paired evaluation and `program={plan}` admit the same bytes and the same content digest. `program={value}` is exact supplied source and is neither trimmed, dedented nor newline-adjusted | built by #713 | -| `evaluate_program` durable effect | the complete-program admission, recorded once before the first program effect and never as a widening of `generated_xmd` — the two are decisions about different kinds of source. `description.input` carries the source digest, the explicit props, the evaluation-site source origin and whether the result is captured; the result carries the exact admitted source, the root mode, the components the program names with their authored forms **and the identity canonical resolution selected for each name at that site**, those same terms and the validated root props — or, for a refusal, the refused class alone and nothing of the source. Both results are closed shapes, and so is every record nested in them: a missing, additional or misspelled member at any depth is a record this evaluation did not write. Restored, it must also agree with itself before anything expands — the source hashes to the digest, reparsing produces the retained mode and a valid body structure, the retained elements are the ones that source writes in that order and those forms, and the supplied props validate to the retained validated props — and a record failing any of those is refused as unreadable with neither program performing an effect. A continuation whose site now answers one of those names with a different identity or admits it in a different form is refused before either implementation runs, which is a different answer from an unreadable record: the journal is intact and the environment moved. A partial continuation expands the retained source and restores the nested effects that already committed, repeating no planning; a changed program at the same evaluation occurrence is stale input and refuses before either the current or the retained source runs, so neither silently wins. The source digest identifies the program artifact and the authored element and loop iteration identify one execution of it, so two occurrences of one artifact have separate nested effect identities and the digest deduplicates neither | built by #713 | +| `evaluate_program` durable effect | the complete-program admission, recorded once before the first program effect and never as a widening of `generated_xmd` — the two are decisions about different kinds of source. `description.input` carries the source digest, the explicit props, the evaluation-site source origin and whether the result is captured; the result carries the exact admitted source, the root mode, the components the program names with their authored forms **and the identity canonical resolution settled for each name at that site**, as a closed tagged record whose tag decides its members, those same terms and the validated root props — or, for a refusal, the refused class alone and nothing of the source. Both results are closed shapes, and so is every record nested in them: a missing, additional or misspelled member at any depth is a record this evaluation did not write. Restored, it must also agree with itself before anything expands — the source hashes to the digest, reparsing produces the retained mode and a valid body structure, the retained elements are the ones that source writes in that order and those forms, and the supplied props validate to the retained validated props — and a record failing any of those is refused as unreadable with neither program performing an effect. A continuation whose site now answers one of those names with a different identity or admits it in a different form is refused before either implementation runs, which is a different answer from an unreadable record: the journal is intact and the environment moved. A partial continuation expands the retained source and restores the nested effects that already committed, repeating no planning; a changed program at the same evaluation occurrence is stale input and refuses before either the current or the retained source runs, so neither silently wins. The source digest identifies the program artifact and the authored element and loop iteration identify one execution of it, so two occurrences of one artifact have separate nested effect identities and the digest deduplicates neither | built by #713 | | generated mutation proposals | lets an Agent propose constrained executable changes that a separate admission then performs against the run's own Workspace | built on the #369 and #567 stacks, with directory creation added by #643: the standard Deno profile's write table is core's paired `File:write`, the paired `@executablemd/workflow/composition/dir-v2#Dir` and core's self-closing `File.Delete`, in that retained order and followed by any host extension. `allow={["write"]}` intentionally authorizes Dir's persistent recursive directory creation; its versioned identity makes every continuation retained under the former non-mutating Dir identity refuse before generated execution. Admitted mutations run as the ordinary components they are through the run's effect transactions, and the evaluator adds no mutation API or receipt. Approval is authored control flow before the write-enabled element. Local Git, Git-host, issue, process, execution, credential and external-write effects are outside the class | | Deno-local DOFS provider | owns one authoritative SQLite/DOFS connection per run path, captures arbitrary canonical retained roots, privately restores them, and atomically coordinates one Workspace mutation with its filtered Yield | built on the #365 stack; public document filesystem effects and the CLI lifecycle route to it on the #366 stack | | scoped Worker Shell | executes `just-bash` through the Workspace adapter inside a Deno Worker | containment and effect-transaction POCs complete (#351, #357); production integration unbuilt | diff --git a/packages/core/host.ts b/packages/core/host.ts index ddfd5b942..fecb8ff97 100644 --- a/packages/core/host.ts +++ b/packages/core/host.ts @@ -142,6 +142,20 @@ export { props as evaluateProgramProps, selectProgramForm, } from "./src/components/Evaluate.ts"; +/** + * How an import-middleware provider states a stable identity for the answers it + * supplies (§5.7). + * + * A provider that answers without delegating, or replaces what came back, is + * part of the authority genuinely available at an evaluation site. A complete + * program can only be admitted against such an answer when the provider says + * who it is, because a continuation has to be told whether the implementation + * changed. Ordinary expansion is unaffected either way. + */ +export { useImportProvider } from "./src/program-imports.ts"; +export type { ImportProviderClaimant } from "./src/program-imports.ts"; +export type { ImportProviderIdentity, ProgramIdentity } from "./src/program-identity.ts"; + export { EVALUATE_PROGRAM, evaluateProgram, diff --git a/packages/core/src/component-api.ts b/packages/core/src/component-api.ts index 58540a7a6..92c7ad3a7 100644 --- a/packages/core/src/component-api.ts +++ b/packages/core/src/component-api.ts @@ -18,7 +18,7 @@ import type { Operation } from "effection"; import type { EvalScope } from "@effectionx/scope-eval"; import { settle } from "./errors.ts"; import type { BoundExecRequest } from "./bound-exec.ts"; -import type { ProgramComponent, ProgramComponentRef } from "./program-identity.ts"; +import type { ProgramComponentRef, ResolvedProgramComponent } from "./program-identity.ts"; import type { CodeBlockContext, CodeBlockResult, @@ -185,7 +185,9 @@ export interface ComponentApi { * same question again from its own resolver before the first program effect, * so a dishonest answer refuses the evaluation rather than widening it. */ - resolveProgramSite(named: readonly ProgramComponentRef[]): Operation; + resolveProgramSite( + named: readonly ProgramComponentRef[], + ): Operation; /** * Decide what an ordinary function-component failure means (spec §6.9). * @@ -302,7 +304,7 @@ export const Component: Api = createApi("Component", // deno-lint-ignore require-yield *resolveProgramSite( _named: readonly ProgramComponentRef[], - ): Operation { + ): Operation { throw new Error( "Component.resolveProgramSite() has no provider: not inside a function component invocation.", ); diff --git a/packages/core/src/components/import-authority.ts b/packages/core/src/components/import-authority.ts index daf76eaae..bac833527 100644 --- a/packages/core/src/components/import-authority.ts +++ b/packages/core/src/components/import-authority.ts @@ -125,6 +125,15 @@ export type ImportRefusal = "unissued" | "another-name" | "changed"; /** What canonical execution kept of one definition it produced. */ interface Witness { readonly name: string; + /** + * The provider that supplied this answer, when an identified one did. + * + * Absent for an answer canonical execution produced itself, whose identity is + * the selection's. Stated by the provider at its installation boundary and + * bound here, outside the definition, because a definition is data an answer + * can copy and an identity a copy carries identifies nothing. + */ + readonly supplied?: SuppliedIdentity; /** * Core's own copy of its own answer, taken before the public chain could see * the definition and reachable from nowhere but here. @@ -136,6 +145,20 @@ interface Witness { readonly canonical: ImportedDefinition | undefined; } +/** What an identified provider stated about one answer it supplied. */ +export interface SuppliedIdentity { + readonly origin: string; + readonly key: string; + readonly revision: string; +} + +/** What canonical execution knows about the answer the chain returned. */ +export interface AnswerWitness { + readonly name: string; + readonly canonical: ImportedDefinition | undefined; + readonly supplied?: SuppliedIdentity; +} + /** * Core's own copy of one definition. * @@ -252,6 +275,55 @@ export class CanonicalImports { return definition; } + /** + * Record that an identified middleware provider supplied this answer. + * + * The copy is taken here for the same reason canonical answers are copied: + * the object that travelled through the rest of the chain is never the object + * invoked. A second claim on one answer is refused rather than overwritten — + * two providers each saying an answer is theirs is an ambiguity, not a later + * one winning. + */ + supply( + name: string, + supplied: SuppliedIdentity, + definition: ImportedDefinition, + ): ImportedDefinition { + const held = this.#issued.get(definition); + if (held !== undefined) { + // One provider answering the same import twice with the same definition + // is ordinary: a name resolved for a program's admission and again for + // its expansion is two imports of one answer. What is refused is a second + // *claim* — another provider, another name, or the same provider under a + // changed revision — because then no single authority stands behind it. + const same = + held.name === name && + held.supplied !== undefined && + held.supplied.origin === supplied.origin && + held.supplied.key === supplied.key && + held.supplied.revision === supplied.revision; + if (!same) { + throw new Error( + "Component.importComponent middleware claimed an answer another provider had claimed.", + ); + } + return definition; + } + this.#issued.set(definition, { name, canonical: retain(definition), supplied }); + return definition; + } + + /** + * What canonical execution knows about the answer the chain returned. + * + * Absent for an answer nobody issued or supplied, which is a replacement no + * authority stands behind. Ordinary expansion still runs it; a durable grant + * cannot be made against it. + */ + witness(answer: ImportedDefinition): AnswerWitness | undefined { + return typeof answer === "object" && answer !== null ? this.#issued.get(answer) : undefined; + } + /** * Core's own copy of the definition this import may invoke. * @@ -318,11 +390,19 @@ export interface ImportTier { * an answer decides only how a refusal reads, never whether one is authorized. */ export class ExecutionImports implements ImportAuthority { - readonly #imports = new CanonicalImports(); + readonly #imports: CanonicalImports; readonly #tiers: readonly ImportTier[]; - constructor(tiers: readonly ImportTier[]) { + /** + * The witness table is the execution's, not this authority's. + * + * One execution has one answer per import, and an open execution witnesses + * its answers too — a complete program's admission has to tell a canonical + * answer from a supplied one whether or not any tier closes an import. + */ + constructor(tiers: readonly ImportTier[], imports: CanonicalImports = new CanonicalImports()) { this.#tiers = tiers; + this.#imports = imports; } /** Record that canonical execution produced this answer for this name. */ diff --git a/packages/core/src/execute.ts b/packages/core/src/execute.ts index 94055dfdd..2e0d4f626 100644 --- a/packages/core/src/execute.ts +++ b/packages/core/src/execute.ts @@ -94,7 +94,9 @@ import { useSegmentCauses, } from "./errors.ts"; import { Component, importComponent, raise } from "./component-api.ts"; -import { componentIdentity } from "./program-identity.ts"; +import { providerIdentity, selectionIdentity, UNRESOLVED } from "./program-identity.ts"; +import type { ResolvedProgramComponent } from "./program-identity.ts"; +import { createImportProviderRegistry, ImportProviders } from "./program-imports.ts"; import { sourceDescription } from "./source-position.ts"; import { renderSegment } from "./render.ts"; import { createExactSource } from "./output/exact-source.ts"; @@ -138,7 +140,8 @@ import { parseFormDeclaration, } from "./invocation-identity.ts"; import type { IdentityComponent } from "./invocation-identity.ts"; -import { ExecutionImports } from "./components/import-authority.ts"; +import { CanonicalImports, ExecutionImports } from "./components/import-authority.ts"; +import type { ImportedDefinition } from "./components/import-authority.ts"; import type { ExpansionAuthority, ImportTier } from "./components/import-authority.ts"; import type { WorkflowComponentBundle, WorkflowImportAuthority } from "./components/bundle.ts"; import type { CodeBlockContext, CodeBlockResult, EvalEnv } from "./types.ts"; @@ -2111,6 +2114,14 @@ function* executeDocument( bundles: readonly WorkflowComponentBundle[] = [], identityComponents: readonly IdentityComponent[] = [], declarations: readonly DeclaredMarkdownComponent[] = [], + /** + * The witness table this execution's providers were minted against. + * + * Created where the installations run, because that is where an identified + * import provider is installed and a provider minted against a different + * table would witness nothing this execution reads. + */ + witnesses: CanonicalImports = new CanonicalImports(), ): Operation { const { stream, @@ -2264,7 +2275,11 @@ function* executeDocument( if (declaredImports !== undefined) { tiers.push(declaredImports); } - const imports = tiers.length === 0 ? undefined : new ExecutionImports(tiers); + // One witness table for the whole execution, whether or not any tier + // closes an import. An open run needs it too: a complete program's + // admission has to tell an answer canonical execution produced from one a + // provider supplied, and from one nobody stands behind at all. + const imports = tiers.length === 0 ? undefined : new ExecutionImports(tiers, witnesses); const authority: ExpansionAuthority = { ...(imports === undefined ? {} : { imports }), ...(declaredImports === undefined ? {} : { declared: declaredImports }), @@ -2279,7 +2294,50 @@ function* executeDocument( // On the authority rather than on the Component Api, because this // decides whether a retained admission still describes this site. // Reconciliation never trusts an answer middleware can replace. - *resolve(name: string): Operation { + *resolve(name: string): Operation { + // Through the ordinary chain, because the answer the chain returns is + // the answer that would run. Resolving the name a second way would + // describe a definition a provider replaced, and a continuation would + // compare two different implementations as equal. + let answer: ImportedDefinition; + try { + answer = yield* importComponent(name); + } catch { + return { + name, + form: "self-closing", + identity: UNRESOLVED, + definition: undefined, + unidentified: false, + }; + } + const witness = witnesses.witness(answer); + if (witness === undefined) { + // A replacement no authority stands behind. Ordinary expansion runs + // it; a durable grant cannot be made against it. + return { + name, + form: "self-closing", + identity: UNRESOLVED, + definition: undefined, + unidentified: true, + }; + } + if (witness.supplied !== undefined) { + return { + name, + form: "self-closing", + identity: providerIdentity( + { origin: witness.supplied.origin, revision: witness.supplied.revision }, + witness.supplied.key, + ), + definition: witness.canonical, + unidentified: false, + }; + } + // Canonical execution's own answer keeps its canonical identity. The + // selection that produced it is what names it, and asking for it here + // describes the definition this import actually settled on. const registered = yield* Component.operations.registry; const selection = yield* ephemeral( selectComponent(name, { @@ -2289,7 +2347,13 @@ function* executeDocument( ...(catalog === undefined ? {} : { declared: catalog }), }), ); - return componentIdentity(selection); + return { + name, + form: "self-closing", + identity: selectionIdentity(selection), + definition: witness.canonical, + unidentified: false, + }; }, // Created here, held here, and reclaimed with this execution. Nothing a // document, a component, middleware or a separately loaded copy can @@ -2330,7 +2394,10 @@ function* executeDocument( // The witness for this answer. It is issued where the answer is // produced and verified where it is invoked, so what a handler does // to the value in between is visible rather than authoritative. - return imports === undefined ? definition : imports.issue(name, definition); + // Always witnessed, closed execution or not: what the chain hands + // back is compared against this, and an answer nobody stands behind + // has to be tellable from one canonical execution produced. + return witnesses.issue(name, definition); }, *applyModifiers([modifiers, context], _next) { const chain = composeModifierChain(modifiers, context, registry); @@ -2959,6 +3026,12 @@ function* invoke( ), ); + // The witness table this execution reads, created before its providers are + // installed. An identified provider is minted a claimant here, against this + // table, so what it marks is what canonical resolution later reads. + const witnesses = new CanonicalImports(); + yield* ImportProviders.set(createImportProviderRegistry(witnesses)); + for (const installation of installations) { if (installation.install) { yield* installation.install(); @@ -2997,6 +3070,7 @@ function* invoke( bundles, identityComponents, declarations, + witnesses, ); } diff --git a/packages/core/src/expand.ts b/packages/core/src/expand.ts index 01be53dc9..bcc6fc392 100644 --- a/packages/core/src/expand.ts +++ b/packages/core/src/expand.ts @@ -101,9 +101,11 @@ import { INCOMPATIBLE, ProgramEvaluationError, sameComponents, + UNIDENTIFIED, UNRESOLVED, } from "./program-identity.ts"; -import type { ProgramComponent, ProgramComponentRef } from "./program-identity.ts"; +import type { ProgramComponentRef, ResolvedProgramComponent } from "./program-identity.ts"; +import { ProgramImports } from "./program-imports.ts"; import { DeclaredMarkdownError } from "./components/declared-markdown.ts"; import type { PrivateImport } from "./components/declared-markdown.ts"; import CoreTest from "./components/Test.ts"; @@ -4204,9 +4206,22 @@ export function* expandProgramBody( // the authority, so a handler that answered the admission's own resolution // dishonestly is caught by the answer it cannot reach. const current = yield* resolveProgramComponents(program.named, site.authority); + if (current.some((entry) => entry.unidentified)) { + throw new ProgramEvaluationError(UNIDENTIFIED); + } if (!sameComponents(program.named, current)) { throw new ProgramEvaluationError(INCOMPATIBLE); } + // The answers that passed the comparison are the answers the program + // invokes. Asking the chain again and running whatever came back is the gap + // this closes: a provider could answer one way while the site was being + // checked and another way while the program ran. + const settled = new ProgramImports(current); + // Closed over the answers that passed, so a handler outside this expansion + // that replaces one is refused where it would be invoked rather than + // silently preferred. + const authority: ExpansionAuthority | undefined = + site.authority === undefined ? { imports: settled } : { ...site.authority, imports: settled }; return yield* scoped(function* () { yield* provideEnv(programEnvironment(site.callerValues, program.props)); if (program.returns !== undefined) { @@ -4226,7 +4241,7 @@ export function* expandProgramBody( passthroughClaim, program.path, site.checkedFailures, - site.authority, + authority, undefined, ); return { kind: "value", value }; @@ -4244,7 +4259,7 @@ export function* expandProgramBody( undefined, program.path, site.checkedFailures, - site.authority, + authority, undefined, ); return { kind: "text", output: renderSegments(expanded) }; @@ -4282,15 +4297,22 @@ function programAuthority( export function* resolveProgramComponents( named: readonly ProgramComponentRef[], authority: ExpansionAuthority | undefined, -): Operation { +): Operation { const resolve = authority?.resolve; - const resolved: ProgramComponent[] = []; + const resolved: ResolvedProgramComponent[] = []; for (const entry of named) { - resolved.push({ - name: entry.name, - form: entry.form, - identity: resolve === undefined ? UNRESOLVED : yield* resolve(entry.name), - }); + if (resolve === undefined) { + resolved.push({ + name: entry.name, + form: entry.form, + identity: UNRESOLVED, + definition: undefined, + unidentified: false, + }); + continue; + } + const settled = yield* resolve(entry.name); + resolved.push({ ...settled, name: entry.name, form: entry.form }); } return resolved; } diff --git a/packages/core/src/program-evaluation.ts b/packages/core/src/program-evaluation.ts index 1a5b7ebab..a66024f9b 100644 --- a/packages/core/src/program-evaluation.ts +++ b/packages/core/src/program-evaluation.ts @@ -85,9 +85,15 @@ import { validateProps } from "./validate.ts"; import { isProgramComponentForm, ProgramEvaluationError, + readIdentity, sameElements, + UNIDENTIFIED, +} from "./program-identity.ts"; +import type { + ProgramComponent, + ProgramComponentRef, + ResolvedProgramComponent, } from "./program-identity.ts"; -import type { ProgramComponent, ProgramComponentRef } from "./program-identity.ts"; import type { Json, JsonObject, @@ -327,7 +333,11 @@ function* admitProgram( decision: "admitted", source, mode, - named: resolved.map((entry) => ({ ...entry })), + named: resolved.map((entry) => ({ + name: entry.name, + form: entry.form, + identity: { ...entry.identity }, + })), terms, validated: props, }); @@ -411,10 +421,11 @@ function readComponents(value: Json | undefined): readonly ProgramComponent[] | return undefined; } const { name, form, identity } = entry; - if (typeof name !== "string" || typeof identity !== "string" || !isProgramComponentForm(form)) { + const settled = readIdentity(identity); + if (typeof name !== "string" || settled === undefined || !isProgramComponentForm(form)) { return undefined; } - found.push({ name, form, identity }); + found.push({ name, form, identity: settled }); } return found; } @@ -483,9 +494,15 @@ export function* evaluateProgram(request: ProgramEvaluationRequest): Operation

entry.unidentified)) { + throw new ProgramEvaluationError(UNIDENTIFIED); + } const stored = yield createDurableOperation( { diff --git a/packages/core/src/program-identity.ts b/packages/core/src/program-identity.ts index 37abd4fab..a122a852d 100644 --- a/packages/core/src/program-identity.ts +++ b/packages/core/src/program-identity.ts @@ -6,29 +6,101 @@ * definition each name it writes resolves to is part of what the admission * granted. Retaining the names alone would let a run resume against a different * `` than the one it was admitted with — same bytes, different - * program — so the admission retains the identity canonical resolution selected - * for each name together with the form the element is written in, and a + * program — so the admission retains the identity of what each name actually + * resolved to together with the form the element is written in, and a * continuation is refused when either has moved. * - * The identity is a string rather than the selection itself because it is - * durable data: it is written into a journal, read back from one, and compared - * whole. Every tier that can answer a name contributes a distinguishable one, - * and a name nothing answers is `unresolved` — which is an identity like any - * other here, so a name that becomes resolvable between two runs is a change - * the comparison sees. + * ## The identity is the answer's, not the selector's + * + * What runs is what the ordinary `Component.importComponent` chain returns, and + * a provider may answer without delegating or replace what came back. Resolving + * the name a second way and describing *that* would retain an identity for a + * definition nobody invokes, so two different middleware answers would compare + * equal and a continuation would silently run the other one. The identity is + * therefore taken from the final answer the chain supplied. + * + * A canonical tier's answer keeps its canonical identity. An answer an + * identified middleware provider supplied keeps that provider's own versioned + * identity — a stable origin, the provider's stable key for the name, and a + * revision that changes whenever the implementation changes incompatibly. That + * identity is stated by the provider outside the definition, because a + * definition is data an answer can copy, and it is bound to the exact answer in + * execution-private state rather than carried on it. + * + * An answer nobody identified is not refused for ordinary expansion — a + * document that installs a raw replacement keeps working exactly as it did — + * but a complete program cannot be admitted against one, because there would be + * nothing for a continuation to compare. + * + * ## Records, not strings + * + * The identity is a closed tagged record because it is durable data: it is + * written into a journal, read back from one, and compared whole. Every tier + * contributes a distinguishable tag, no tag can be spelled as another, and a + * name nothing answers is `unresolved` — an identity like any other here, so a + * name that becomes resolvable between two runs is a change the comparison + * sees. */ -import type { ComponentSelection } from "./types.ts"; +import type { Operation } from "effection"; + +import type { ImportedDefinition } from "./components/import-authority.ts"; +import type { ComponentSelection, Json, JsonObject } from "./types.ts"; /** The authored forms an element is written in. */ export type ProgramComponentForm = "self-closing" | "paired"; +/** + * What an identified import provider states about itself and its answers. + * + * Stated by the provider, at the installation boundary, and outside every + * definition it supplies. It is an assertion by the authority installed at the + * site — like a registration's origin or a declared component's — and reusing + * one revision for a different implementation is that provider breaking its own + * contract. The engine compares the assertion; it never tries to repair one by + * reading the definition. + */ +export interface ImportProviderIdentity { + /** The stable origin this provider answers under. */ + readonly origin: string; + /** A revision that changes when the supplied implementation changes. */ + readonly revision: string; +} + +/** The identity of what one name resolved to, as the run retains it. */ +export type ProgramIdentity = + | { readonly tag: "structural"; readonly construct: string } + | { readonly tag: "registered"; readonly origin: string; readonly reserved: boolean } + | { readonly tag: "repository"; readonly path: string } + | { readonly tag: "workflow"; readonly path: string; readonly object: string } + | { readonly tag: "declared-markdown"; readonly origin: string; readonly digest: string } + | { + readonly tag: "middleware"; + readonly origin: string; + readonly key: string; + readonly revision: string; + } + | { readonly tag: "unresolved" }; + +/** The members each identity tag has, and the only members it has. */ +const IDENTITY_MEMBERS: Record = { + structural: ["tag", "construct"], + registered: ["tag", "origin", "reserved"], + repository: ["tag", "path"], + workflow: ["tag", "path", "object"], + "declared-markdown": ["tag", "origin", "digest"], + middleware: ["tag", "origin", "key", "revision"], + unresolved: ["tag"], +}; + +/** A name nothing at this site answers for. */ +export const UNRESOLVED: ProgramIdentity = { tag: "unresolved" }; + /** One component a program names, with the form and identity it resolved to. */ export interface ProgramComponent { readonly name: string; readonly form: ProgramComponentForm; - /** The identity canonical resolution selected, or `unresolved`. */ - readonly identity: string; + readonly identity: ProgramIdentity; } /** One element a program names, before its identity has been resolved. */ @@ -37,8 +109,36 @@ export interface ProgramComponentRef { readonly form: ProgramComponentForm; } -/** A name nothing at this site answers for. */ -export const UNRESOLVED = "unresolved"; +/** + * What canonical resolution settled for one name: its identity, and the copy of + * the answer core will invoke. + * + * The two travel together because they are one decision. Carrying the identity + * without the answer would leave expansion to ask the chain again, and a + * provider that answered one way for the comparison could answer another way + * for the invocation. + */ +export interface ResolvedProgramComponent extends ProgramComponent { + /** + * Core's own copy of the answer, or `undefined` when nothing was resolved or + * the answer could not be copied. + */ + readonly definition: ImportedDefinition | undefined; + /** Whether the chain's final answer carried no witness at all. */ + readonly unidentified: boolean; +} + +/** + * Resolve one name through the site's own import chain. + * + * Held by canonical execution and handed to core's own expansion by value, like + * the rest of the expansion authority. It resolves through the ordinary chain, + * so what it describes is what would run; it invokes no component + * implementation and performs no program effect. + */ +export interface ProgramResolver { + (name: string): Operation; +} /** A program this site will not evaluate. */ export class ProgramEvaluationError extends Error { @@ -58,53 +158,101 @@ export const INCOMPATIBLE = "than it did when this evaluation was admitted."; /** - * Resolve one name to a component this execution may name durable work about. + * What a program naming a component nobody identified says. * - * Held by canonical execution and handed to core's own expansion by value, like - * the rest of the expansion authority. It answers with the identity selection - * produced, and it imports nothing: resolving a name is a decision about which - * definition answers it, and loading one is a durable effect a program's own - * expansion performs where the element is written. + * Ordinary expansion is unaffected: a raw replacement keeps answering there + * exactly as it did. What it cannot do is stand behind a durable grant, because + * a continuation would have nothing to compare and would run whatever answered + * on the day it resumed. */ -export interface ProgramResolver { - (name: string): import("effection").Operation; -} +export const UNIDENTIFIED = + " cannot evaluate a program naming a component supplied by import middleware that " + + "states no identity: a continuation has nothing to hold the site to."; -/** - * The durable identity of what a name resolved to. - * - * Every tier is distinguishable, and each carries the part of its selection - * that says which implementation it is: a registration is its origin and - * whether it is reserved, a repository component is its path, a bundled one is - * its path and object id, and declared Markdown is its origin and digest. A - * private name resolves to nothing at all (`select.ts`), so a program naming one - * carries the same identity as a program naming a component that does not - * exist — which is what it is. - */ -export function componentIdentity(selection: ComponentSelection): string { +/** The canonical identity of what a selection chose. */ +export function selectionIdentity(selection: ComponentSelection): ProgramIdentity { switch (selection.kind) { case "structural": - return `structural:${selection.construct}`; + return { tag: "structural", construct: selection.construct }; case "registered": return selection.origin.kind === "registered" - ? `registered:${selection.origin.reserved ? "reserved" : "default"}:${selection.origin.origin}` - : `registered:${selection.origin.kind}`; + ? { + tag: "registered", + origin: selection.origin.origin, + reserved: selection.origin.reserved, + } + : { tag: "registered", origin: selection.origin.kind, reserved: false }; case "repository": - return `repository:${selection.path}`; + return { tag: "repository", path: selection.path }; case "workflow": - return `workflow:${selection.path}@${selection.sourceHash}`; + return { tag: "workflow", path: selection.path, object: selection.sourceHash }; case "declared-markdown": - return `declared-markdown:${selection.origin}@${selection.digest}`; + return { tag: "declared-markdown", origin: selection.origin, digest: selection.digest }; default: return UNRESOLVED; } } +/** The identity an identified provider's answer carries. */ +export function providerIdentity(provider: ImportProviderIdentity, key: string): ProgramIdentity { + return { tag: "middleware", origin: provider.origin, key, revision: provider.revision }; +} + /** Whether a value is one of the two authored forms. */ export function isProgramComponentForm(value: unknown): value is ProgramComponentForm { return value === "self-closing" || value === "paired"; } +/** + * One retained identity, read back from the journal as a closed tagged record. + * + * The tag decides which members the record has, and it has exactly those: a + * missing, additional or misspelled member is a record this version did not + * write, whatever its tag says. An empty string is not an identity either — a + * provider that states nothing has stated nothing. + */ +export function readIdentity(value: Json | undefined): ProgramIdentity | undefined { + if (typeof value !== "object" || value === null || Array.isArray(value)) { + return undefined; + } + const record = value as JsonObject; + const tag = record.tag; + if (typeof tag !== "string" || !Object.hasOwn(IDENTITY_MEMBERS, tag)) { + return undefined; + } + const members = IDENTITY_MEMBERS[tag as ProgramIdentity["tag"]]; + const own = Object.keys(record); + if (own.length !== members.length || !members.every((member) => Object.hasOwn(record, member))) { + return undefined; + } + for (const member of members) { + const held = record[member]; + if (member === "reserved") { + if (typeof held !== "boolean") { + return undefined; + } + continue; + } + if (typeof held !== "string" || held.length === 0) { + return undefined; + } + } + return record as unknown as ProgramIdentity; +} + +/** Whether two retained identities describe the same implementation. */ +export function sameIdentity(one: ProgramIdentity, other: ProgramIdentity): boolean { + if (one.tag !== other.tag) { + return false; + } + const members = IDENTITY_MEMBERS[one.tag]; + return members.every( + (member) => + (one as unknown as Record)[member] === + (other as unknown as Record)[member], + ); +} + /** Whether two resolved component lists describe the same site. */ export function sameComponents( retained: readonly ProgramComponent[], @@ -119,7 +267,7 @@ export function sameComponents( other !== undefined && entry.name === other.name && entry.form === other.form && - entry.identity === other.identity + sameIdentity(entry.identity, other.identity) ); }); } diff --git a/packages/core/src/program-imports.ts b/packages/core/src/program-imports.ts new file mode 100644 index 000000000..da6db6133 --- /dev/null +++ b/packages/core/src/program-imports.ts @@ -0,0 +1,189 @@ +/** + * Identified import providers, and the answers a complete program runs + * (specs/executable-mdx-spec.md §5.7). + * + * `Component.importComponent` middleware may answer an import without + * delegating, or replace what came back. That is ordinary and supported, and it + * is also why a complete program cannot describe what it will run by resolving + * the name a second way: the answer that runs is the chain's, and a second + * resolution describes a definition nobody invokes. + * + * So a provider that answers or replaces states who it is. It does that here, + * at its installation boundary, and canonical execution mints it a claimant + * bound to this one execution. Marking an answer with that claimant binds the + * provider's terms to that exact object in execution-private state — never on + * the definition, which is data an answer can copy, and never through a + * replaceable Context answer. + * + * ## What the identity is, and is not + * + * It is an assertion by the authority installed at the site, exactly as a + * registration's origin is. Nothing here proves a provider is who it says; what + * it buys is that the assertion is *stable and versioned*, so a continuation + * can be told the implementation changed. A provider reusing one revision for a + * different implementation is breaking its own contract, and the engine does + * not try to repair that by reading the definition. + * + * Two live providers under one origin refuse, because then an origin no longer + * names one authority. + * + * ## An unidentified answer is not refused, it is unusable + * + * A document that installs a raw replacement keeps working: ordinary expansion + * never asks any of this. What it cannot do is stand behind a durable grant, + * because a continuation would have nothing to compare and would invoke + * whatever answered on the day it resumed. + */ + +import { createContext } from "effection"; +import type { Context, Operation } from "effection"; + +import { stillDescribes } from "./components/import-authority.ts"; +import { CanonicalImports } from "./components/import-authority.ts"; +import type { + ImportAuthority, + ImportedDefinition, + ImportRefusal, +} from "./components/import-authority.ts"; +import { ProgramEvaluationError } from "./program-identity.ts"; +import type { ImportProviderIdentity, ResolvedProgramComponent } from "./program-identity.ts"; + +/** How an identified provider marks one answer as its own. */ +export interface ImportProviderClaimant { + (name: string, key: string, definition: ImportedDefinition): ImportedDefinition; +} + +/** What canonical execution offers an identified provider for one execution. */ +export interface ImportProviderRegistry { + claimant(identity: ImportProviderIdentity): ImportProviderClaimant; +} + +/** + * Where an identified provider finds this execution's claimant minter. + * + * A Context, because a provider installed by a host or a document has no other + * way to reach the execution it is being installed into. That is not a hole: + * the claimant grants nothing. It labels an answer, the label is compared + * rather than trusted, and what is invoked is core's own copy. + */ +export const ImportProviders: Context = createContext< + ImportProviderRegistry | undefined +>("component.importProviders", undefined); + +/** + * Mark this provider's answers with a stable versioned identity. + * + * Called once where the provider is installed, before its middleware is. A + * provider outside any execution receives a claimant that marks nothing, so the + * same installation code works in a context that has no admission to make. + */ +export function* useImportProvider( + identity: ImportProviderIdentity, +): Operation { + const registry = yield* ImportProviders.get(); + if (registry === undefined) { + return (_name, _key, definition) => definition; + } + return registry.claimant(identity); +} + +/** The registry canonical execution installs for one execution. */ +export function createImportProviderRegistry(imports: CanonicalImports): ImportProviderRegistry { + const origins = new Set(); + return { + claimant(identity: ImportProviderIdentity): ImportProviderClaimant { + if (identity.origin.length === 0 || identity.revision.length === 0) { + throw new ProgramEvaluationError( + "An identified import provider states a non-empty origin and revision.", + ); + } + if (origins.has(identity.origin)) { + throw new ProgramEvaluationError( + `Two import providers are installed under the origin "${identity.origin}", so it names ` + + "no single authority.", + ); + } + origins.add(identity.origin); + return (name, key, definition) => + imports.supply( + name, + { origin: identity.origin, key, revision: identity.revision }, + definition, + ); + }, + }; +} + +/** What one program's imports refuse, and why. */ +const REFUSED: Record = { + unissued: + "Component.importComponent middleware answered a complete program's import with a definition " + + "canonical resolution did not settle for it.", + "another-name": + "Component.importComponent middleware answered a complete program's import with the " + + "definition canonical resolution settled for another component.", + changed: + "Component.importComponent middleware changed the definition canonical resolution settled " + + "before the program invoked it.", +}; + +/** + * The answers one admitted program invokes. + * + * Built from the resolution that passed the compatibility comparison, so the + * definition a program expands is the one the comparison was about. Asking the + * chain again and invoking whatever came back is the gap this closes: a + * provider could answer one way while the site was being checked and another + * way while the program ran. + * + * A name that resolved to nothing is not closed here. It reaches the ordinary + * chain and produces the ordinary unresolved failure, which is what a program + * naming a component this site does not have should say. + */ +export class ProgramImports implements ImportAuthority { + readonly #answers: ReadonlyMap; + + constructor(resolved: readonly ResolvedProgramComponent[]) { + const answers = new Map(); + for (const entry of resolved) { + if (entry.definition !== undefined && !answers.has(entry.name)) { + answers.set(entry.name, entry.definition); + } + } + this.#answers = answers; + } + + closes(name: string): boolean { + return this.#answers.has(name); + } + + /** + * The answer canonical resolution settled, for the import the program made. + * + * The chain still runs, so the execution records its own selection and + * identity domain exactly as it does for any other import. What changes is + * what comes back: the answer must still describe what the comparison + * passed, and the object invoked is the copy the comparison was about. A + * provider that answered one way while the site was checked and another way + * while the program ran is refused here rather than preferred. + */ + authorize(name: string, answer: ImportedDefinition): ImportedDefinition { + const settled = this.#answers.get(name); + if (settled === undefined) { + throw new ProgramEvaluationError(REFUSED.unissued); + } + if (answer !== settled && read(() => stillDescribes(settled, answer)) !== true) { + throw new ProgramEvaluationError(REFUSED.changed); + } + return settled; + } +} + +/** One read of an answer that may refuse to be read. */ +function read(inspect: () => T): T | undefined { + try { + return inspect(); + } catch { + return undefined; + } +} diff --git a/packages/core/tests/program-evaluation.test.ts b/packages/core/tests/program-evaluation.test.ts index 2146f57b3..564dd9d90 100644 --- a/packages/core/tests/program-evaluation.test.ts +++ b/packages/core/tests/program-evaluation.test.ts @@ -32,7 +32,11 @@ import type { Operation } from "effection"; import { createDurableOperation, InMemoryStream } from "@executablemd/durable-streams"; import type { DurableEvent } from "@executablemd/durable-streams"; +import { Component } from "../src/component-api.ts"; import { executeInstalled, programEvaluationComponents, sourceDigest } from "../host.ts"; +import { useImportProvider } from "../host.ts"; +import type { ImportProviderIdentity } from "../host.ts"; +import type { FunctionComponentDefinition, JsonObject } from "../src/types.ts"; import type { DeclaredMarkdownComponent, IdentityClaimant, IdentityComponent } from "../host.ts"; import type { ComponentInvocation } from "../src/invocation-identity.ts"; import { pairedProgramSource, programDigest } from "../host.ts"; @@ -133,19 +137,29 @@ function run( probeOrigin?: string; declarations?: readonly DeclaredMarkdownComponent[]; privates?: readonly IdentityComponent[]; + /** An ordinary open import-middleware provider, installed as a host's is. */ + install?: () => Operation; } = {}, ): Operation { return scoped(function* () { const stream = options.stream ?? new InMemoryStream(); - const execution = yield* executeInstalled( - { ...retainedSource(ROOT_PATH, source), stream, includes: [] }, - [ - { - components: [...programEvaluationComponents(), ...probeComponents(options.probeOrigin)], - ...(options.declarations === undefined ? {} : { declarations: options.declarations }), - }, - ], - ); + // An installation that refuses throws out of `executeInstalled` rather than + // settling a document result, so both endings are reported the same way. + let execution; + try { + execution = yield* executeInstalled( + { ...retainedSource(ROOT_PATH, source), stream, includes: [] }, + [ + { + components: [...programEvaluationComponents(), ...probeComponents(options.probeOrigin)], + ...(options.declarations === undefined ? {} : { declarations: options.declarations }), + ...(options.install === undefined ? {} : { install: options.install }), + }, + ], + ); + } catch (error) { + return { failure: error instanceof Error ? error.message : String(error), events: [] }; + } const result = yield* execution; const events = yield* stream.readAll(); return result.ok @@ -729,3 +743,363 @@ describe("Tier PE — a producer's private closure does not cross", () => { approved = APPROVED; }); }); + +/** Every implementation an open middleware provider actually entered. */ +const opened: string[] = []; + +/** One implementation of ``, distinguishable by what it renders. */ +function openImplementation(mark: string, props?: JsonObject): FunctionComponentDefinition { + return { + kind: "function", + name: "Open", + props: props ?? { type: "object", properties: {}, additionalProperties: false }, + // deno-lint-ignore require-yield + fn: function* Open(): Operation { + opened.push(mark); + return `open ${mark}`; + }, + }; +} + +/** + * An ordinary open `Component.importComponent` provider, of the kind a host or + * a package installs. + * + * It answers without delegating, which is exactly the case an independent + * second resolution cannot describe: the selector would report that `Open` + * resolves to nothing while this is what runs. + */ +function openProvider(options: { + mark: string; + identity?: ImportProviderIdentity; + /** Answer this from the given lookup onwards, to model a check/use split. */ + then?: string; + /** Which lookup starts answering `then`. */ + switchAfter?: number; +}): () => Operation { + return function* install(): Operation { + const claim = yield* useImportProvider( + options.identity ?? { origin: "test/open", revision: options.mark }, + ); + let answered = 0; + const first = openImplementation(options.mark); + const later = options.then === undefined ? first : openImplementation(options.then); + yield* Component.around( + { + *importComponent([name, position], next) { + if (name !== "Open") { + return yield* next(name, position); + } + answered += 1; + const definition = answered <= (options.switchAfter ?? 1) ? first : later; + return options.identity === null ? definition : claim(name, "Open", definition); + }, + }, + { at: "max" }, + ); + }; +} + +/** A program whose only element is the middleware-supplied component. */ +const OPEN_PROGRAM = "\n"; + +describe("Tier PE — the identity is the answer's, not the selector's", () => { + it("PE23 refuses a continuation whose middleware supplies another implementation", function* () { + opened.length = 0; + approved = OPEN_PROGRAM; + + // 1. The first run's live middleware supplies A under identity A. + const first = yield* run(PROBING_DOCUMENT, { + install: openProvider({ mark: "A", identity: { origin: "test/open", revision: "A" } }), + }); + expect(first.failure).toBeUndefined(); + expect(first.output).toContain("open A"); + expect(opened).toEqual(["A"]); + + // 2. Keep the journal only through the committed admission, and forget + // every implementation observation. + const interrupted = through(first.events, "evaluate_program"); + opened.length = 0; + + // 3. Continue with the same source and props while middleware supplies B. + const moved = yield* run(PROBING_DOCUMENT, { + stream: new InMemoryStream(interrupted), + install: openProvider({ mark: "B", identity: { origin: "test/open", revision: "B" } }), + }); + + // 4. The site refusal, before A or B runs. + expect(moved.failure ?? moved.output ?? "").toContain("resolves differently"); + expect(opened).toEqual([]); + + // 5. The same retained prefix with A unchanged resumes and runs A once. + opened.length = 0; + const resumed = yield* run(PROBING_DOCUMENT, { + stream: new InMemoryStream(interrupted), + install: openProvider({ mark: "A", identity: { origin: "test/open", revision: "A" } }), + }); + + expect(resumed.failure).toBeUndefined(); + expect(resumed.output).toContain("open A"); + expect(opened).toEqual(["A"]); + approved = APPROVED; + }); + + it("PE24 retains the provider's identity, not the selector's answer", function* () { + opened.length = 0; + approved = OPEN_PROGRAM; + + const attempt = yield* run(PROBING_DOCUMENT, { + install: openProvider({ mark: "A", identity: { origin: "test/open", revision: "A" } }), + }); + + expect(attempt.failure).toBeUndefined(); + const record = decision(admissions(attempt.events)[0]!); + const named = record.named as Record[]; + // The selector answers nothing for this name — a middleware-only component + // is on no search path and in no registry — so an independent resolution + // would have retained `unresolved` for both implementations. + expect(named).toEqual([ + { + name: "Open", + form: "self-closing", + identity: { tag: "middleware", origin: "test/open", key: "Open", revision: "A" }, + }, + ]); + approved = APPROVED; + }); + + it("PE25 refuses a program naming an unidentified middleware answer", function* () { + opened.length = 0; + approved = OPEN_PROGRAM; + + const attempt = yield* run(PROBING_DOCUMENT, { + install: openProvider({ mark: "A", identity: null as unknown as undefined }), + }); + + expect(attempt.failure ?? attempt.output ?? "").toContain("states no identity"); + // Nothing of it ran, and no grant was recorded against a site that cannot + // be described. + expect(opened).toEqual([]); + expect(admissions(attempt.events)).toHaveLength(0); + approved = APPROVED; + }); + + it("PE26 cannot run an answer that changed after the comparison passed", function* () { + opened.length = 0; + approved = OPEN_PROGRAM; + + // The provider answers A for the admission's resolution and for canonical + // execution's own comparison, then B for the lookup expansion would make. + // Expansion must invoke the answer the comparison passed, not ask again. + const attempt = yield* run(PROBING_DOCUMENT, { + install: openProvider({ + mark: "A", + then: "B", + switchAfter: 2, + identity: { origin: "test/open", revision: "A" }, + }), + }); + + // B never runs. Expansion is authorized against the answer the comparison + // settled, so an answer that changed underneath it is refused rather than + // quietly preferred. + expect(`${attempt.failure ?? attempt.output ?? ""}`).toContain( + "changed the definition canonical resolution settled", + ); + expect(opened).toEqual([]); + approved = APPROVED; + }); + + it("PE27 refuses two providers installed under one origin", function* () { + approved = OPEN_PROGRAM; + const attempt = yield* run(PROBING_DOCUMENT, { + *install() { + yield* useImportProvider({ origin: "test/open", revision: "A" }); + yield* useImportProvider({ origin: "test/open", revision: "B" }); + }, + }); + + expect(attempt.failure ?? attempt.output ?? "").toContain("no single authority"); + approved = APPROVED; + }); + + it("PE28 refuses a retained middleware identity that is malformed", function* () { + opened.length = 0; + approved = OPEN_PROGRAM; + const first = yield* run(PROBING_DOCUMENT, { + install: openProvider({ mark: "A", identity: { origin: "test/open", revision: "A" } }), + }); + expect(first.failure).toBeUndefined(); + + const interrupted = through(first.events, "evaluate_program"); + const malformed: Record[] = [ + { tag: "middleware", origin: "test/open", key: "Open" }, + { tag: "middleware", origin: "test/open", key: "Open", revision: "" }, + { tag: "middleware", origin: "test/open", key: "Open", revision: "A", extra: 1 }, + { tag: "registered", origin: "test/open", key: "Open", revision: "A" }, + { tag: "unknown-tier", origin: "test/open" }, + ]; + + for (const identity of malformed) { + opened.length = 0; + const replayed = yield* run(PROBING_DOCUMENT, { + stream: new InMemoryStream( + tamper(interrupted, (record) => ({ + ...record, + named: [{ name: "Open", form: "self-closing", identity }], + })), + ), + install: openProvider({ mark: "A", identity: { origin: "test/open", revision: "A" } }), + }); + + expect(`${JSON.stringify(identity)}: ${replayed.failure ?? replayed.output ?? ""}`).toContain( + "cannot be read as one", + ); + expect([identity, opened]).toEqual([identity, []]); + } + approved = APPROVED; + }); +}); + +describe("Tier PE — what a provider does to the chain's answer", () => { + it("PE29 keeps the canonical identity when a provider delegates unchanged", function* () { + performed.length = 0; + approved = '\n'; + + const attempt = yield* run(PROBING_DOCUMENT, { + *install() { + // Identified, but it answers nothing: it delegates and returns what + // came back. The canonical answer keeps its canonical identity. + yield* useImportProvider({ origin: "test/passthrough", revision: "1" }); + yield* Component.around( + { + *importComponent([name, position], next) { + return yield* next(name, position); + }, + }, + { at: "max" }, + ); + }, + }); + + expect(attempt.failure).toBeUndefined(); + expect(performed).toEqual(["delegated"]); + const named = decision(admissions(attempt.events)[0]!).named as Record[]; + expect(named).toEqual([ + { + name: "Probe", + form: "self-closing", + identity: { tag: "registered", origin: "test/probe", reserved: false }, + }, + ]); + approved = APPROVED; + }); + + it("PE30 retains the provider's identity when it replaces a canonical answer", function* () { + performed.length = 0; + approved = '\n'; + + const attempt = yield* run(PROBING_DOCUMENT, { + *install() { + const claim = yield* useImportProvider({ origin: "test/replacer", revision: "7" }); + // One implementation, held by the provider, as a provider with a stable + // revision holds one. Building a new object per import would be the + // provider saying its answer is unchanged while handing over a + // different one, which is the case PE26 covers. + const replacement = openImplementation("replaced", { + type: "object", + properties: { mark: { type: "string" } }, + additionalProperties: false, + }); + yield* Component.around( + { + *importComponent([name, position], next) { + const answered = yield* next(name, position); + if (name !== "Probe") { + return answered; + } + // Canonical selection settled a registration; this replaces it. + return claim(name, "Probe", replacement); + }, + }, + { at: "max" }, + ); + }, + }); + + expect(attempt.failure).toBeUndefined(); + const named = decision(admissions(attempt.events)[0]!).named as Record[]; + // Not `registered:test/probe`: what runs is the replacement, so what the + // admission describes is the replacement. + expect(named).toEqual([ + { + name: "Probe", + form: "self-closing", + identity: { tag: "middleware", origin: "test/replacer", key: "Probe", revision: "7" }, + }, + ]); + // The registration never ran; the replacement did. + expect(performed).toEqual([]); + expect(opened).toContain("replaced"); + approved = APPROVED; + }); + + it("PE31 refuses an answer mutated after it was claimed", function* () { + opened.length = 0; + approved = OPEN_PROGRAM; + + const attempt = yield* run(PROBING_DOCUMENT, { + *install() { + const claim = yield* useImportProvider({ origin: "test/open", revision: "A" }); + yield* Component.around( + { + *importComponent([name, position], next) { + if (name !== "Open") { + return yield* next(name, position); + } + const definition = openImplementation("A"); + const claimed = claim(name, "Open", definition); + // Changed after the claim, on its way back through the chain. + (claimed as { name: string }).name = "Substituted"; + return claimed; + }, + }, + { at: "max" }, + ); + }, + }); + + expect(attempt.failure ?? attempt.output ?? "").toMatch(/changed|cannot be read as one/); + expect(opened).toEqual([]); + approved = APPROVED; + }); + + it("PE32 refuses a second provider claiming one answer", function* () { + opened.length = 0; + approved = OPEN_PROGRAM; + const shared = openImplementation("A"); + + const attempt = yield* run(PROBING_DOCUMENT, { + *install() { + const first = yield* useImportProvider({ origin: "test/first", revision: "1" }); + const second = yield* useImportProvider({ origin: "test/second", revision: "1" }); + yield* Component.around( + { + *importComponent([name, position], next) { + if (name !== "Open") { + return yield* next(name, position); + } + first(name, "Open", shared); + return second(name, "Open", shared); + }, + }, + { at: "max" }, + ); + }, + }); + + expect(attempt.failure ?? attempt.output ?? "").toContain("another provider had claimed"); + expect(opened).toEqual([]); + approved = APPROVED; + }); +}); diff --git a/specs/executable-mdx-spec.md b/specs/executable-mdx-spec.md index 16cee20a5..91c770041 100644 --- a/specs/executable-mdx-spec.md +++ b/specs/executable-mdx-spec.md @@ -4412,7 +4412,7 @@ interface, and each operation is also exported directly: | `hasCapture(name)` | Whether the invocation wrote this capture prop at all (§6.5) | `false` | | `capture(name)` | Evaluate a capture prop now, against the caller's bindings, and deliver the result by reference (§6.5) | throws: not inside a function component invocation | | `expandProgram(program)` | Expand a complete XMD program at this invocation's site, under the authority the enclosing expansion holds, after settling that every retained component identity still describes this site (§5.7) | throws: not inside a function component invocation | -| `resolveProgramSite(named)` | What each name a complete program writes resolves to at this site, for the admission to retain (§5.7). Observable and short-circuitable; the compatibility decision is canonical execution's own and rests on nothing answered here | throws: not inside a function component invocation | +| `resolveProgramSite(named)` | What each name a complete program writes resolves to at this site, resolved through the ordinary import chain, for the admission to retain (§5.7). Observable and short-circuitable; the compatibility decision is canonical execution's own and rests on nothing answered here | throws: not inside a function component invocation | | `handleFailure(failure)` | What an ordinary function-component failure means, after complete invocation teardown (§6.9) | fails the operation with `failure.error` | | `retain(resource)` | Create a resource in the invocation-site scope, so it outlives this invocation (§4.4) | throws: not inside a component invocation | @@ -4827,25 +4827,60 @@ already committed. A changed program at the same evaluation occurrence is stale input: it refuses before either the current or the retained source runs, so neither silently wins. -A retained identity is a string, and every tier that can answer a name -contributes a distinguishable one: `structural:`, -`registered::`, `repository:`, -`workflow:@`, `declared-markdown:@`, and -`unresolved` for a name nothing at this site answers — a private name included, -because selection resolves one to nothing outside the declaration that carries -it (§5.3). Resolution imports nothing: it decides which definition answers a -name, while loading one is the program's own durable effect where the element is -written. +**The identity is the answer's, not the selector's.** What runs is what the +ordinary `Component.importComponent` chain returns, and a provider may answer +without delegating or replace what came back. So the identity is taken from the +final answer that chain supplied, by resolving through it. Describing the name a +second way would retain an identity for a definition nobody invokes, and two +different middleware answers would then compare equal. + +A retained identity is a closed tagged record, and every tier that can answer a +name contributes a distinguishable tag: `structural`, `registered`, +`repository`, `workflow`, `declared-markdown`, `middleware`, and `unresolved` +for a name nothing at this site answers — a private name included, because +selection resolves one to nothing outside the declaration that carries it +(§5.3). A canonical tier's answer keeps its canonical identity. + +An answer an **identified middleware provider** supplied keeps that provider's +own identity: a stable origin, the provider's stable key for the name, and a +revision that changes whenever the supplied implementation changes. The provider +states those terms at its installation boundary through `useImportProvider()`, +and canonical execution mints it a claimant for that one execution. Marking an +answer binds the terms to that exact object in execution-private state — never +on the definition, which is data an answer can copy, and never through a +replaceable Context answer. The identity is an assertion by the authority +installed at the site, exactly as a registration's origin is: reusing one +revision for a different implementation is that provider breaking its own +contract, and the engine does not try to repair it by reading the definition. +Two live providers under one origin refuse, and so does a second claim on one +answer. + +An answer **nobody identified** is not refused for ordinary expansion — a +document that installs a raw replacement keeps working exactly as it did — but a +complete program cannot be admitted against one, because a continuation would +have nothing to compare and would invoke whatever answered on the day it +resumed. + +Resolution invokes no component implementation and performs no program effect; +it settles which definition answers each name. **A continuation is held to the site it was admitted at.** Before the first -program effect, canonical execution resolves every retained name again and -refuses when any identity or form has moved. That comparison is canonical -execution's own: the resolver is built by the execution, travels on the -expansion authority, and is reachable from no document, component, contextual -Api answer or middleware return value. A host may observe the admission's own -resolution through `Component.resolveProgramSite()`, and answering it -dishonestly refuses the evaluation rather than widening it, because the -canonical comparison is what decides. +program effect, canonical execution resolves every retained name again through +the same chain and refuses when any identity or form has moved. That comparison +is canonical execution's own: the resolver is built by the execution, travels on +the expansion authority, and is reachable from no document, component, +contextual Api answer or middleware return value. A host may observe the +admission's own resolution through `Component.resolveProgramSite()`, and +answering it dishonestly refuses the evaluation rather than widening it, because +the canonical comparison is what decides. + +**The comparison and the invocation are one decision.** The answers that passed +the comparison are the answers the program invokes: canonical execution keeps +its own copy of each, taken when the answer was witnessed, and the program's +imports are authorized against it. An answer that no longer describes what the +comparison settled is refused where it would be invoked rather than silently +preferred, which is what closes the gap in which a provider answers one way +while the site is checked and another way while the program runs. **The retained record is hostile data.** A journal is a file, and replay hands back whatever it holds. The admitted and refused decisions and every nested @@ -9840,7 +9875,7 @@ trusted-host events may have no authored source. | Resolve components (glob) | `glob` | `resolve:{dir}` | Only when `useDurableGlobResolver` middleware is installed | | Read over HTTP | `fetch` | `fetch:{expansion id}` | Normalized request in `description.input`; status, detached headers and text body in the result (§6.18) | | Admit generated XMD | `generated_xmd` | `generated:{fragment id}` | The canonical class selection, retained roots, selected root, every selected entry as a name, identity and admitted forms, and the exact request policy in `description.input`; the admitted source, that same policy, and the identity and form of each element the fragment named in the result (workflow-workspace-spec §8.4) | -| Admit a complete program | `evaluate_program` | `program:{evaluation id}` | The source digest, explicit props, evaluation-site source origin and whether the result is captured in `description.input`. The result is one of two closed shapes: an admitted one of exactly `decision`, `source`, `mode`, `named`, `terms` and `validated` — where each `named` entry is exactly `name`, `form` and `identity`, and `terms` is exactly `digest`, `props`, `origin` and `captured` — or a refused one of exactly `decision` and `refused`, carrying the refused class and nothing of the source. A record with a missing, additional or misspelled member at any depth, or one that disagrees with itself, is unreadable rather than partially believed (§5.7) | +| Admit a complete program | `evaluate_program` | `program:{evaluation id}` | The source digest, explicit props, evaluation-site source origin and whether the result is captured in `description.input`. The result is one of two closed shapes: an admitted one of exactly `decision`, `source`, `mode`, `named`, `terms` and `validated` — where each `named` entry is exactly `name`, `form` and `identity`, each `identity` is a closed tagged record holding exactly the members its tag has — `structural`/`construct`, `registered`/`origin`/`reserved`, `repository`/`path`, `workflow`/`path`/`object`, `declared-markdown`/`origin`/`digest`, `middleware`/`origin`/`key`/`revision`, or `unresolved` alone — and `terms` is exactly `digest`, `props`, `origin` and `captured` — or a refused one of exactly `decision` and `refused`, carrying the refused class and nothing of the source. A record with a missing, additional or misspelled member at any depth, or one that disagrees with itself, is unreadable rather than partially believed (§5.7) | ### 10.2 Example journal for a multi-component document @@ -12362,6 +12397,16 @@ forms sit beside the restricted one. | PE20 | A site that answers differently | The same source and request, with one named component registered under another origin, refuses before either implementation runs; the unchanged site resumes | | PE21 | Structural preflight | A durable effect written before a later malformed structural construct never runs: structural admission refuses first | | PE22 | The producer's private closure | A program evaluated inside a declaration carrying a private component cannot resolve that name and never enters its implementation, while an ordinary site-authorized component in the same program runs | +| PE23 | An answer that moved | A program naming a middleware-supplied `` admits under identity A; a continuation whose provider supplies B under identity B refuses before either implementation runs, and the same prefix with A unchanged resumes and runs A once | +| PE24 | The identity is the answer's | A middleware-only component retains the provider's identity, where an independent selector would have retained `unresolved` for every implementation | +| PE25 | An unidentified answer | A program naming a raw replacement refuses before it runs, and records no admission | +| PE26 | Check and use are one decision | An answer that changed after the comparison passed is refused where it would be invoked; the later implementation never runs | +| PE27 | One origin, one authority | Two providers installed under one origin refuse | +| PE28 | A malformed retained identity | A missing, empty, additional or mis-tagged identity member is unreadable and runs nothing | +| PE29 | Delegation | A provider that delegates and returns the answer unchanged preserves the canonical identity | +| PE30 | Replacement | A provider replacing a canonical answer retains its own identity, and the replaced registration never runs | +| PE31 | Mutation after the claim | An answer changed on its way back through the chain refuses | +| PE32 | Two claims on one answer | A second provider claiming an answer another already claimed refuses | | EP1–EP8 | Run profile | The ordinary run profile declares ``; the forms, props, refusals and catalog entry hold through the real binary, and a program cannot reach ``'s private components | | WGAC17–WGAC20 | Workflow profile | A complete program records `evaluate_program` and no `generated_xmd`; the three forms cannot be combined; complete-program support is not reachable through `source` | From a7cc574699c7fc91756198cb91f2c45eff743214 Mon Sep 17 00:00:00 2001 From: Taras Mankovski Date: Fri, 4 Sep 2026 10:16:52 -0400 Subject: [PATCH 4/7] =?UTF-8?q?=F0=9F=A7=B1=20Resolve=20a=20program=20unde?= =?UTF-8?q?r=20the=20site's=20closed=20authority,=20once,=20before=20its?= =?UTF-8?q?=20import=20(#713)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three corrections. **Closed authority was overwritten.** The program authority replaced the site's `imports` with one that forgot its tiers, and resolution asked the chain without applying them, so an identified provider could replace a bundled or declared component and be admitted as its identity. Resolution now applies the site's closed authority exactly as ordinary expansion does, and the program's expansion keeps the tiers it already had. A provider-supplied answer is witnessed rather than issued, so it authorizes nothing for a name a tier closed — the guarantee that sharing one witness table had quietly broken. **Resolution now journals nothing.** The selection half of an import and the definition it produces are separated from the durable operation, so a resolution-only path settles the site without recording anything. `evaluate_program` commits before the program's own ordinary import. **Expansion no longer asks again.** The answers that passed reconciliation reach expansion as settled answers: an element whose name is among them does not enter `Component.importComponent` at all. The chain is entered twice for one evaluation, never a third time. The authored element still records one ordinary import after the admission, restored from the already-authorized answer with identity-domain and form selection intact. Provider identity is read exactly once at registration and closed over as copied primitives, so a getter cannot answer the duplicate check and the claim differently and a later edit changes nothing. Structural syntax resolves directly to its structural identity: no lookup, no provider, no import record. --- architecture.md | 4 +- .../core/src/components/import-authority.ts | 21 +- packages/core/src/execute.ts | 380 ++++++++++++------ packages/core/src/expand.ts | 217 +++++----- packages/core/src/program-identity.ts | 11 + packages/core/src/program-imports.ts | 110 +---- .../core/tests/program-evaluation.test.ts | 229 ++++++++++- specs/executable-mdx-spec.md | 48 ++- 8 files changed, 700 insertions(+), 320 deletions(-) diff --git a/architecture.md b/architecture.md index 237cfdf38..ba67d759a 100644 --- a/architecture.md +++ b/architecture.md @@ -44,7 +44,7 @@ Existing documents and code get aligned to this section retroactively. | Plan | the executable program produced from a Prompt: an Executable Markdown document combining readable prose that expresses the Prompt's intent with the components that carry it out, each placed beside the prose describing what it does. It begins with one descriptive level-one heading. A Plan is what `xmd plan` approves and then delivers as source: printed to stdout by default, or written to an `--output` file. Plan produces a program; Evaluate evaluates a program in the current XMD execution; Run executes a program from the host or CLI; composition decides whether and when a planned program runs. It is not a synonym for a workflow, a policy document or any executable Markdown file | | plan command document | the one exact checked-in first-party Markdown value root `xmd plan` executes. It is the command's adapter and nothing else: it projects the request into ``, supplies the session, and returns the approved source. Its body is those two elements and no prose, because its rendered transcript is the progress the command writes to stderr. It is not itself a Plan. Internal: no command-line option selects another one, and no repository component search can answer for it | | packaged `` Component | the one exact checked-in first-party Markdown text component that converts a prompt into a Plan, `packages/cli/src/documents/Plan.md`, declared to every ordinary run as the public ``. It owns and implements the Plan authorship workflow — the Prompt wording, the draft and repair loops, the `` branches, human review, revision, approval, stopping, exhaustion and the automatic final explanation turn — and produces the exact approved Plan source as what it renders: written bare it emits those bytes, and `as` is ordinary text capture that binds them and emits nothing. Neither form evaluates the source. Every Plan-producing turn in it states the complete Plan requirements for itself, so a replacement may add or correct a title rather than only carry one forward. Both surfaces expand these exact bytes under one origin and one digest; there is no generated TypeScript copy and no second Markdown implementation. Its four phase components are private to it, and it is not itself a Plan | -| identified import provider | a `Component.importComponent` middleware provider that states a stable versioned identity for the answers it supplies — an origin it answers under and a revision that changes when the implementation does — through `useImportProvider()` at its installation boundary, where canonical execution mints it a claimant for that one execution. Marking an answer binds those terms to that exact object in execution-private state, never on the definition and never through a replaceable Context answer. The identity is an assertion by the authority installed at the site, exactly as a registration's origin is: nothing proves a provider is who it says, and what it buys is that a continuation can be told the implementation changed. Two live providers under one origin refuse, and so does a second claim on one answer. A provider that only delegates identifies nothing, because the answer it returns is already canonical execution's | built by #713 | +| identified import provider | a `Component.importComponent` middleware provider that states a stable versioned identity for the answers it supplies — an origin it answers under and a revision that changes when the implementation does — through `useImportProvider()` at its installation boundary, where canonical execution mints it a claimant for that one execution, reading the origin and revision exactly once and closing over copies so a getter or a later edit cannot change what its claims are marked with or evade duplicate detection. Marking an answer binds those terms to that exact object in execution-private state, never on the definition and never through a replaceable Context answer. The identity is an assertion by the authority installed at the site, exactly as a registration's origin is: nothing proves a provider is who it says, and what it buys is that a continuation can be told the implementation changed. Two live providers under one origin refuse, and so does a second claim on one answer. A provider that only delegates identifies nothing, because the answer it returns is already canonical execution's | built by #713 | | unidentified middleware answer | an import answer no authority stands behind — a provider that answered or replaced without stating an identity. Ordinary expansion runs it exactly as it always did; a complete program cannot be admitted against one, because a continuation would have nothing to compare and would invoke whatever answered on the day it resumed | built by #713 | | composition site | the authored `` element where a complete XMD program is admitted and evaluated. It is a place in a document rather than a boundary: the program runs in the current execution, under the authority in force where the element was written, and the element and the loop iteration reaching it identify one execution of that program. A composition site is not a child document, a child process, a second host profile or a second journal | | authorship profile | the trusted-host assembly the packaged `` Component runs its authored turns under, installed by its own `` inside the invocation that owns it rather than around an execution — which is what makes it the same frame whether `xmd plan`, an ordinary document, or a configured `` run child asked. Which Agent context goes under it is a trusted-host capability the declaration carries — the production ACPX one built from the run's Agent stack, or the deterministic one a canonical `` child declaration produced — and a host that supplies none states the sentence a `` written there is refused with. The fixed policy is installed in one place for both, so a second provider cannot bring a weaker one: its fixed inputs, a constrained Agent provider, Elicitation, the fixed first-party components and the host-declared ``. It uses no repository component search and exposes no custom root, and the policy it installs is not readable from the command line. Its working directory is one host-owned directory dedicated to the logical session, keyed by the digest of that name, created empty and required to be empty on the way in. An explicitly named session's directory is durable, because continuation derives the same session identity from it; an invocation-unique default session's is scope-owned, claimed before it is created, and exactly one cleanup is attempted after profile teardown and before admission on every ending — the leaf removed non-recursively when it is still the empty directory that was handed over, and left as found with the command failing terminally when it has gained content or vanished. Where those directories live is a host dependency no caller or document selects | @@ -3899,7 +3899,7 @@ Status is measured against main. | workflow Agent session | a workflow document's `` runs under a profile the host attaches only for a live or partial run: an empty host-owned working directory instead of any Workspace, checkout or caller path, no MCP servers, an empty requested native tool set, and `deny-all` with a permission path that denies every native request and fails the turn that asked without reaching the public permission chain. Within a run a session is identified by the Agent/Session expansion identity the engine derived — the authored name is descriptive, so two sibling `` elements are two sessions — routed inside a placement bound to its element and good for one use, so a kept placement cannot be substituted for the next. The conversation is retained as a row in the run's own database with the provider, resolved agent command and policy fingerprint beside it as compatibility attributes. The order is placement, the backend's acceptance of the session's first turn, the provider's canonical tagged assertion, then the mapping commit — and only then is anything that turn produced exposed. A placement is inert: it creates no provider session and writes no row. Occupancy of a provider key is not an assertion, and a record held for a first turn nobody accepted asserts nothing at all; the pre-commit window reconciles only from exactly one canonical assertion, and a missing, conflicting, replaced or ambiguous assertion is one explicit refusal that starts no replacement. Deleting a run removes the row with the run and the provider-session directory beside it, and reports the categories. The profile selects ACP-only capability explicitly — no native-launch advertisement and no client-native attachment advertisement — rather than inheriting the provider package's ordinary-run sets by omission, and it supplies no machine session coordinator, construction-route store or executable observer: a workflow session belongs to a run, and the machine-wide account describes a different thing entirely | built on the #302 stack, with the explicit ACP-only selection from #561; the portable proof that an adapter honours an empty tool set is tracked by #496 and does not widen the ceiling | | generated-XMD admission | admits one Agent-generated fragment through the trusted-host seam: host policy is a `read` table and a `write` table of exact pinned identities, each carrying the authored forms it is admitted for, and an authored `allow` selects a canonical non-empty subset of the closed classes — omitted means `read`. The complete source is preflighted inside one `generated_xmd` durable effect before its first generated effect; only the pinned identity the selected classes hold for that name **and** that form executes; and the admitted source, class selection, selected root, every selected entry with its forms, the identity and form of each element named, and the normalized request policy are retained in that effect's own result — so a continuation restores the decision without reading the current candidate and expands only the retained source. The roots are an as-of-admission retained basis checked by membership — the run's own later root publications and an advanced retained current root pass, while a lost admission root or lost selected root refuses — and every non-root term is checked exactly, refusing a run whose classes, identities, forms or requests have moved. The admission and every nested generated effect are offered inline by the owning expansion in authored order, so a partial continuation restores each completed one without another live execution. Each admitted effect is retained by its own ordinary record, and a read's value is collected while a mutation's is not | built on the #369 stack, continuation basis amended by #589; core owns the mechanics and the workflow policy wrapper is internal | | `` | the restricted generated-fragment form of ``, which the workflow host declares beside the complete-program forms below. The form an authored document writes where an observation should happen. The host does not register it: it **declares** it to the execution through `ExecutionInstallation.components`, captured before any installation runs, and canonical execution calls its factory once for that attachment with the claimant it minted and registers what comes back. Registration provides availability only — a name a trusted document may write — and carries none of the authority. It takes a `source` string and one optional `allow` array selecting a non-empty duplicate-free subset of the closed effect classes `read` and `write` — omitted means `read` — refuses paired content, and cannot be combined with `program`, with paired content, or with `props`; each of those refuses before the durable name is claimed and before the candidate is read. It declares no `returns` and answers with a detached value — `{ observations: [{ name, value }], output }`, each admitted read's own returned value under the name the fragment invoked it by, in invocation order, with whatever the fragment rendered under `output` rather than instead of them, and the pinned identity that produced one left in the retained admission rather than copied here — so an admitted ``, which renders nothing at all, still reaches the document. An admitted mutation contributes no entry and no receipt, so a write-only fragment binds `{ observations: [], output: "" }`; `as` is valid for every selection and binds that same shape. An ordinary `as` captures that value by reference, and an authored `` renders it into the next ``: deciding how a value becomes text is the document's. Every ceiling comes from values the host captured at installation — the run's retained roots and its authoritative current root read from the run's own storage at invocation, as-of-admission provenance a continuation holds by membership so the run's own later publications and an advanced retained current root invalidate nothing, core's pinned self-closing `` read, the write table of core's paired ``, this package's lexical `` built from the definition the ordinary registration owns, and core's self-closing ``, and `` only when the captured request ceiling is non-empty — and no prop, binding, context or middleware return value supplies or widens one. `allow` selects among those tables and adds nothing to them; approval, when a workflow needs one, is authored control flow before the element. Its durable operation is named through that claimant, on the exact invocation the engine handed it and in that invocation's own frame — not from a context a document could rebind, a contextual Api answer, a definition, or a registry answer. Generated source never resolves through the registration: the evaluator consults only its own closed table of pinned identities. It is deliberately not wrapped in `printErrors`, so a refused fragment stops the authored loop rather than becoming text the next turn could read as a read that happened | built on the #302 stack, extended by #369 | -| complete-program evaluation | admits one complete XMD root and evaluates it at the authored `` site, in the current execution: its lifecycle, journal, cancellation scope, contextual providers, working directory or Workspace, and the authority in force where the element was written. There is no child process, host profile, root lifecycle or journal of its own — program source requests behavior and grants none. The root's own frontmatter, imports, metadata, props schema, `returns` and `` selection decide what it does; a text root renders its selected output or binds it as a string under `as`, a value root requires `as` and binds its schema-validated result, and root props are the explicit `props` object defaulting to `{}` rather than the caller's own. Ordinary non-props caller bindings are visible as a copy, so nothing the program binds escapes, and relative paths resolve from the evaluation site's source origin. The producer's temporary authority is not inherited: an enclosing declaration's private closure does not cross, so a private name written in a program resolves to nothing, while imports and capabilities genuinely present at the site remain. Canonical execution answers `Component.expandProgram()` from the frame it already holds, which is what makes site authority a fact about the frame rather than something a prop, binding or middleware return value could supply. Which definition each name the program writes resolves to is part of what the site granted, so the admission retains it and canonical execution settles it again before the first program effect — through a resolver the execution builds and carries on the expansion authority, reachable from no document, component, contextual Api answer or middleware return value. That resolver goes through the ordinary import chain, because the answer the chain returns is the answer that runs: an independent second resolution would describe a definition a provider replaced, and two different middleware answers would compare equal. The answers that passed the comparison are the answers the program invokes, authorized against canonical execution's own copy, so an answer that changed between the check and the use is refused rather than preferred | built by #713 on the #722 `` contract; core owns the mechanics and every host that offers `` declares them | +| complete-program evaluation | admits one complete XMD root and evaluates it at the authored `` site, in the current execution: its lifecycle, journal, cancellation scope, contextual providers, working directory or Workspace, and the authority in force where the element was written. There is no child process, host profile, root lifecycle or journal of its own — program source requests behavior and grants none. The root's own frontmatter, imports, metadata, props schema, `returns` and `` selection decide what it does; a text root renders its selected output or binds it as a string under `as`, a value root requires `as` and binds its schema-validated result, and root props are the explicit `props` object defaulting to `{}` rather than the caller's own. Ordinary non-props caller bindings are visible as a copy, so nothing the program binds escapes, and relative paths resolve from the evaluation site's source origin. The producer's temporary authority is not inherited: an enclosing declaration's private closure does not cross, so a private name written in a program resolves to nothing, while imports and capabilities genuinely present at the site remain. Canonical execution answers `Component.expandProgram()` from the frame it already holds, which is what makes site authority a fact about the frame rather than something a prop, binding or middleware return value could supply. Which definition each name the program writes resolves to is part of what the site granted, so the admission retains it and canonical execution settles it again before the first program effect — through a resolver the execution builds and carries on the expansion authority, reachable from no document, component, contextual Api answer or middleware return value. That resolver goes through the ordinary import chain under a resolution-only terminal — it selects and loads what an authored import would, invokes nothing and journals nothing — because the answer the chain returns is the answer that runs, while an independent second resolution would describe a definition a provider replaced. `evaluate_program` therefore commits before the program's own ordinary import record. The site's existing closed tiers still decide: a bundled or declared name is answered by the tier that closed it, and a provider-supplied answer is witnessed rather than issued, so it authorizes nothing for such a name. Structural syntax resolves directly to its own identity and never reaches component import. The answers that passed the comparison are handed to the program's expansion as settled answers, so the chain is entered twice for one evaluation and never a third time; the authored element still records one ordinary import after the admission, restored from the already-authorized answer with identity-domain and form selection intact | built by #713 on the #722 `` contract; core owns the mechanics and every host that offers `` declares them | | `` complete-program forms | the two ways an author names a program at a composition site: paired content, which renders a source-producing component into a private program buffer, and `program={value}`, which supplies complete-program source directly. Optional `props={object}` is the root's props input and is valid only for these forms. The two are mutually exclusive and one is required; every ambiguous or misplaced combination refuses before program content is produced and before any program effect runs, so an ambiguous element does not even render its producer. Paired content is a private buffer rather than a quotation — an executable component written there executes under ordinary XMD semantics while producing the text sent to admission — and the element removes only the framing its own wrapper contributed: the line break after the opening tag with the indentation following it, the line break before the closing tag with the indentation preceding it, and the indentation those lines share. Interior bytes are untouched, so a sole `` producer's trailing newline survives and paired evaluation and `program={plan}` admit the same bytes and the same content digest. `program={value}` is exact supplied source and is neither trimmed, dedented nor newline-adjusted | built by #713 | | `evaluate_program` durable effect | the complete-program admission, recorded once before the first program effect and never as a widening of `generated_xmd` — the two are decisions about different kinds of source. `description.input` carries the source digest, the explicit props, the evaluation-site source origin and whether the result is captured; the result carries the exact admitted source, the root mode, the components the program names with their authored forms **and the identity canonical resolution settled for each name at that site**, as a closed tagged record whose tag decides its members, those same terms and the validated root props — or, for a refusal, the refused class alone and nothing of the source. Both results are closed shapes, and so is every record nested in them: a missing, additional or misspelled member at any depth is a record this evaluation did not write. Restored, it must also agree with itself before anything expands — the source hashes to the digest, reparsing produces the retained mode and a valid body structure, the retained elements are the ones that source writes in that order and those forms, and the supplied props validate to the retained validated props — and a record failing any of those is refused as unreadable with neither program performing an effect. A continuation whose site now answers one of those names with a different identity or admits it in a different form is refused before either implementation runs, which is a different answer from an unreadable record: the journal is intact and the environment moved. A partial continuation expands the retained source and restores the nested effects that already committed, repeating no planning; a changed program at the same evaluation occurrence is stale input and refuses before either the current or the retained source runs, so neither silently wins. The source digest identifies the program artifact and the authored element and loop iteration identify one execution of it, so two occurrences of one artifact have separate nested effect identities and the digest deduplicates neither | built by #713 | | generated mutation proposals | lets an Agent propose constrained executable changes that a separate admission then performs against the run's own Workspace | built on the #369 and #567 stacks, with directory creation added by #643: the standard Deno profile's write table is core's paired `File:write`, the paired `@executablemd/workflow/composition/dir-v2#Dir` and core's self-closing `File.Delete`, in that retained order and followed by any host extension. `allow={["write"]}` intentionally authorizes Dir's persistent recursive directory creation; its versioned identity makes every continuation retained under the former non-mutating Dir identity refuse before generated execution. Admitted mutations run as the ordinary components they are through the run's effect transactions, and the evaluator adds no mutation API or receipt. Approval is authored control flow before the write-enabled element. Local Git, Git-host, issue, process, execution, credential and external-write effects are outside the class | diff --git a/packages/core/src/components/import-authority.ts b/packages/core/src/components/import-authority.ts index bac833527..1b7a229d2 100644 --- a/packages/core/src/components/import-authority.ts +++ b/packages/core/src/components/import-authority.ts @@ -117,6 +117,20 @@ export interface ExpansionAuthority { * case a program's components are compared by name and form alone. */ readonly resolve?: ProgramResolver; + /** + * The answers an admitted program invokes, settled before its first effect. + * + * Canonical execution resolved these through the site's own chain, compared + * them against the retained admission, and kept its own copy of each. An + * element whose name is here does not reach `Component.importComponent` at + * all: the chain has already answered, and asking it again would let a + * provider answer one way while the site was checked and another way while + * the program ran. + * + * Held by the execution and handed here by value, like everything else on + * this object. + */ + readonly settled?: ReadonlyMap; } /** Why an answer is not the one canonical execution produced for this name. */ @@ -338,7 +352,12 @@ export class CanonicalImports { ): ImportedDefinition { const witness = typeof answer === "object" && answer !== null ? this.#issued.get(answer) : undefined; - if (witness === undefined) { + // A provider-supplied answer is witnessed, not issued. It says which + // provider stands behind it, which is what a complete program's admission + // reads — and it is not canonical execution's own answer, so it authorizes + // nothing for a name a tier closed. A bundled or declared component is that + // tier's to answer whatever middleware supplies beside it. + if (witness === undefined || witness.supplied !== undefined) { throw refuse("unissued"); } if (witness.name !== name) { diff --git a/packages/core/src/execute.ts b/packages/core/src/execute.ts index 2e0d4f626..05a235c02 100644 --- a/packages/core/src/execute.ts +++ b/packages/core/src/execute.ts @@ -94,7 +94,13 @@ import { useSegmentCauses, } from "./errors.ts"; import { Component, importComponent, raise } from "./component-api.ts"; -import { providerIdentity, selectionIdentity, UNRESOLVED } from "./program-identity.ts"; +import { + ANSWER_CHANGED, + ProgramEvaluationError, + providerIdentity, + selectionIdentity, + UNRESOLVED, +} from "./program-identity.ts"; import type { ResolvedProgramComponent } from "./program-identity.ts"; import { createImportProviderRegistry, ImportProviders } from "./program-imports.ts"; import { sourceDescription } from "./source-position.ts"; @@ -141,6 +147,8 @@ import { } from "./invocation-identity.ts"; import type { IdentityComponent } from "./invocation-identity.ts"; import { CanonicalImports, ExecutionImports } from "./components/import-authority.ts"; +import { RESERVED_STRUCTURAL } from "./structural.ts"; +import { stillDescribes } from "./components/import-authority.ts"; import type { ImportedDefinition } from "./components/import-authority.ts"; import type { ExpansionAuthority, ImportTier } from "./components/import-authority.ts"; import type { WorkflowComponentBundle, WorkflowImportAuthority } from "./components/bundle.ts"; @@ -398,120 +406,135 @@ function targetFailureRecord(failure: DocumentTargetFailure): TargetFailureRecor }; } -function* durableImportComponent( +/** + * Which definition answers this name, as the record describes it. + * + * The selection half of an import, separated from the journal so the same + * decision can be made twice for two different reasons: once inside the durable + * operation an authored element records, and once by the resolution-only path + * a complete program's admission settles its site through. That path journals + * nothing, so an admission commits before the program's own import record. + */ +/** One read of an answer that may refuse to be read. */ +function readAnswer(inspect: () => T): T | undefined { + try { + return inspect(); + } catch { + return undefined; + } +} + +function* selectDurableComponent( name: string, root: RootDocumentSource | undefined, searchPaths: string[], registry: ComponentRegistry, - position: Readonly | undefined, bundle: WorkflowImportAuthority | undefined, declared: DeclaredImports | undefined, -): Workflow { - // Taken before the durable operation and outside it, because the offer is - // canonical core's own and a replay has to reach this the same way the live - // run did: the element asking is inside the declaration's body, or it is not. - const claimed = name === "__root__" ? undefined : declared?.claim(name); - const recorded = yield createDurableOperation( - // The root import is the run's own entry rather than an authored element, - // so it carries no source however it was reached. - { type: "import_component", name, ...(root ? {} : sourceDescription(position)) }, - function* (): Operation { - if (name === "__root__" && root) { - // Inside the durable operation, so the journal holds the root's identity - // and its text: a replay restores both without reading anything, whether - // the source was a file or supplied. - // - // The selector resolves here too, against the text this operation is - // about to record, so the exact target the run executed is part of the - // record rather than something a later read has to rediscover. Only the - // exact target is recorded — a glob describes what the caller asked - // for, not what ran. - const path = rootSourcePath(root); - const content = yield* readRootSource(root); - if (root.target === undefined) { - return { kind: "repository", path, content }; - } - const resolved = resolveDocumentTarget(path, content, root.target); - if (resolved.ok) { - return { kind: "repository", path, content, target: resolved.value }; - } - const failure = asDocumentTargetError(resolved.error); - if (failure === undefined) { - throw resolved.error; - } - return { - kind: "target-failure", - path, - content, - failure: targetFailureRecord(failure.data), - }; - } - - if (claimed !== undefined) { - return { kind: "declared-private", origin: claimed.origin }; - } + claimed: { origin: string; definition: ImportedDefinition } | undefined, +): Operation { + if (name === "__root__" && root) { + // Inside the durable operation, so the journal holds the root's identity + // and its text: a replay restores both without reading anything, whether + // the source was a file or supplied. + // + // The selector resolves here too, against the text this operation is + // about to record, so the exact target the run executed is part of the + // record rather than something a later read has to rediscover. Only the + // exact target is recorded — a glob describes what the caller asked + // for, not what ran. + const path = rootSourcePath(root); + const content = yield* readRootSource(root); + if (root.target === undefined) { + return { kind: "repository", path, content }; + } + const resolved = resolveDocumentTarget(path, content, root.target); + if (resolved.ok) { + return { kind: "repository", path, content, target: resolved.value }; + } + const failure = asDocumentTargetError(resolved.error); + if (failure === undefined) { + throw resolved.error; + } + return { + kind: "target-failure", + path, + content, + failure: targetFailureRecord(failure.data), + }; + } - const selected = yield* selectComponent(name, { - includes: searchPaths, - registry, - ...(bundle === undefined ? {} : { workflow: bundle }), - ...(declared === undefined ? {} : { declared: declared.catalog }), - }); + if (claimed !== undefined) { + return { kind: "declared-private", origin: claimed.origin }; + } - switch (selected.kind) { - case "repository": - return { - kind: "repository", - path: selected.path, - content: yield* readTextFile(selected.path), - }; - case "workflow": - // The exact pinned source, already in hand: the bundle was read from - // the definition's own commit before this run existed, so recording it - // reads nothing and a replay reconstructs it without resolving a name. - return { - kind: "workflow", - path: selected.path, - sourceHash: selected.sourceHash, - content: selected.content, - }; - case "declared-markdown": - // The exact declared bytes, already in hand: they were admitted - // before this run imported a root, so recording them reads nothing - // and a replay reconstructs the component without resolving a name. - return { - kind: "declared-markdown", - origin: selected.origin, - digest: selected.digest, - content: selected.source, - // Recorded only when it holds, so an ordinary declaration's record - // is exactly what it always was. - ...(selected.exact ? { exact: true } : {}), - }; - case "registered": - return { - kind: "registered", - origin: selected.origin.kind === "registered" ? selected.origin.origin : "", - reserved: selected.origin.kind === "registered" && selected.origin.reserved, - }; - case "structural": - throw new Error( - `${name} is structural syntax the engine owns, so it never resolves a component`, - ); - case "unresolved": - throw new Error(unresolvedMessage(name, selected.searched)); - } - }, - ); + const selected = yield* selectComponent(name, { + includes: searchPaths, + registry, + ...(bundle === undefined ? {} : { workflow: bundle }), + ...(declared === undefined ? {} : { declared: declared.catalog }), + }); - // Parsed rather than asserted: a replay hands back whatever the journal holds, - // and a history somebody else wrote is not a `DurableSelection` because it - // type-checked on the way in. - const selection = readDurableSelection(recorded); - if (selection === undefined) { - throw new Error(name === "__root__" ? UNREADABLE_ROOT_RECORD : UNREADABLE_IMPORT_RECORD); + switch (selected.kind) { + case "repository": + return { + kind: "repository", + path: selected.path, + content: yield* readTextFile(selected.path), + }; + case "workflow": + // The exact pinned source, already in hand: the bundle was read from + // the definition's own commit before this run existed, so recording it + // reads nothing and a replay reconstructs it without resolving a name. + return { + kind: "workflow", + path: selected.path, + sourceHash: selected.sourceHash, + content: selected.content, + }; + case "declared-markdown": + // The exact declared bytes, already in hand: they were admitted + // before this run imported a root, so recording them reads nothing + // and a replay reconstructs the component without resolving a name. + return { + kind: "declared-markdown", + origin: selected.origin, + digest: selected.digest, + content: selected.source, + // Recorded only when it holds, so an ordinary declaration's record + // is exactly what it always was. + ...(selected.exact ? { exact: true } : {}), + }; + case "registered": + return { + kind: "registered", + origin: selected.origin.kind === "registered" ? selected.origin.origin : "", + reserved: selected.origin.kind === "registered" && selected.origin.reserved, + }; + case "structural": + throw new Error( + `${name} is structural syntax the engine owns, so it never resolves a component`, + ); + case "unresolved": + throw new Error(unresolvedMessage(name, selected.searched)); } +} +/** + * The definition one recorded selection produces. + * + * Rebuilt from the record rather than carried out of the durable operation, so + * a replayed selection and a live one raise the same error with the same + * fields — and so the resolution-only path produces exactly what the authored + * import will. + */ +function* definitionFromSelection( + name: string, + selection: DurableSelection, + registry: ComponentRegistry, + declared: DeclaredImports | undefined, + claimed: { origin: string; definition: ImportedDefinition } | undefined, +): Operation { // Rebuilt here rather than carried out of the durable operation, so a replayed // failed selection and a live one raise the same error with the same fields. // Parsed rather than trusted: the record is journal data. @@ -645,6 +668,63 @@ function* durableImportComponent( return yield* ephemeral(parseMarkdownDefinition(name, path, content)); } +function* durableImportComponent( + name: string, + root: RootDocumentSource | undefined, + searchPaths: string[], + registry: ComponentRegistry, + position: Readonly | undefined, + bundle: WorkflowImportAuthority | undefined, + declared: DeclaredImports | undefined, +): Workflow { + // Taken before the durable operation and outside it, because the offer is + // canonical core's own and a replay has to reach this the same way the live + // run did: the element asking is inside the declaration's body, or it is not. + const claimed = name === "__root__" ? undefined : declared?.claim(name); + const recorded = yield createDurableOperation( + // The root import is the run's own entry rather than an authored element, + // so it carries no source however it was reached. + { type: "import_component", name, ...(root ? {} : sourceDescription(position)) }, + () => selectDurableComponent(name, root, searchPaths, registry, bundle, declared, claimed), + ); + + // Parsed rather than asserted: a replay hands back whatever the journal holds, + // and a history somebody else wrote is not a `DurableSelection` because it + // type-checked on the way in. + const selection = readDurableSelection(recorded); + if (selection === undefined) { + throw new Error(name === "__root__" ? UNREADABLE_ROOT_RECORD : UNREADABLE_IMPORT_RECORD); + } + return yield* ephemeral(definitionFromSelection(name, selection, registry, declared, claimed)); +} + +/** + * Which definition answers this name, without journaling the decision. + * + * The resolution-only path. It selects and loads exactly what an authored + * import would, and records nothing: `evaluate_program` commits before the + * program's own ordinary import record rather than after it. + */ +export function* resolveComponentDefinition( + name: string, + searchPaths: string[], + registry: ComponentRegistry, + bundle: WorkflowImportAuthority | undefined, + declared: DeclaredImports | undefined, +): Operation { + const claimed = declared?.claim(name); + const selection = yield* selectDurableComponent( + name, + undefined, + searchPaths, + registry, + bundle, + declared, + claimed, + ); + return yield* definitionFromSelection(name, selection, registry, declared, claimed); +} + function isFunctionComponent(value: unknown): value is FunctionComponent { return typeof value === "function"; } @@ -2295,34 +2375,102 @@ function* executeDocument( // decides whether a retained admission still describes this site. // Reconciliation never trusts an answer middleware can replace. *resolve(name: string): Operation { - // Through the ordinary chain, because the answer the chain returns is - // the answer that would run. Resolving the name a second way would - // describe a definition a provider replaced, and a continuation would - // compare two different implementations as equal. - let answer: ImportedDefinition; - try { - answer = yield* importComponent(name); - } catch { + const absent = (unidentified = false): ResolvedProgramComponent => ({ + name, + form: "self-closing", + identity: UNRESOLVED, + definition: undefined, + unidentified, + }); + + // Structural syntax is the engine's own and never reaches component + // import at all, so there is nothing to look up and nothing for a + // provider to claim. Answering it here is what keeps `` from + // producing a middleware lookup or an import record. + if (RESERVED_STRUCTURAL.has(name)) { return { name, form: "self-closing", - identity: UNRESOLVED, + identity: { tag: "structural", construct: name }, definition: undefined, unidentified: false, }; } + + // Through the ordinary chain, because the answer the chain returns is + // the answer that would run — but under a resolution-only terminal. + // This phase settles which definition answers a name; it invokes no + // implementation and journals nothing, so `evaluate_program` commits + // before the program's own ordinary import record. + let answer: ImportedDefinition; + try { + answer = yield* scoped(function* () { + yield* Component.around( + { + *importComponent([asked], _next) { + const registered = yield* Component.operations.registry; + const resolved = yield* resolveComponentDefinition( + asked, + includes, + registered, + bundle, + declaredImports, + ); + return witnesses.issue(asked, resolved); + }, + }, + { at: "min" }, + ); + return yield* importComponent(name); + }); + } catch { + return absent(); + } + const witness = witnesses.witness(answer); + if (witness !== undefined) { + // What came back has to still be what was witnessed. A provider + // that marked an answer and then edited it, or handed back + // something else, has left nothing to admit a program against. + const canonical = witness.canonical; + if ( + witness.name !== name || + canonical === undefined || + readAnswer(() => stillDescribes(canonical, answer)) !== true + ) { + throw new ProgramEvaluationError(ANSWER_CHANGED); + } + } if (witness === undefined) { // A replacement no authority stands behind. Ordinary expansion runs // it; a durable grant cannot be made against it. + return absent(true); + } + + // The site's own closed authority decides first, exactly as ordinary + // expansion does. A bundled or declared name is answered by the tier + // that closed it, so an identified replacement is refused here rather + // than becoming the identity a program is admitted under. + if (imports?.closes(name) === true) { + const authorized = imports.authorize(name, answer); + const registered = yield* Component.operations.registry; + const selection = yield* ephemeral( + selectComponent(name, { + includes, + registry: registered, + ...(bundle === undefined ? {} : { workflow: bundle }), + ...(catalog === undefined ? {} : { declared: catalog }), + }), + ); return { name, form: "self-closing", - identity: UNRESOLVED, - definition: undefined, - unidentified: true, + identity: selectionIdentity(selection), + definition: authorized, + unidentified: false, }; } + if (witness.supplied !== undefined) { return { name, diff --git a/packages/core/src/expand.ts b/packages/core/src/expand.ts index bcc6fc392..ba8107cca 100644 --- a/packages/core/src/expand.ts +++ b/packages/core/src/expand.ts @@ -105,7 +105,7 @@ import { UNRESOLVED, } from "./program-identity.ts"; import type { ProgramComponentRef, ResolvedProgramComponent } from "./program-identity.ts"; -import { ProgramImports } from "./program-imports.ts"; + import { DeclaredMarkdownError } from "./components/declared-markdown.ts"; import type { PrivateImport } from "./components/declared-markdown.ts"; import CoreTest from "./components/Test.ts"; @@ -134,6 +134,8 @@ import { declareChildAnswers, expandAnswers, strayAnswerError } from "./answers. import { DeclarationScan } from "./declaration-scan.ts"; import { RESERVED_STRUCTURAL } from "./structural.ts"; import { renderSegments } from "./render.ts"; +import { createDurableOperation } from "@executablemd/durable-streams"; +import { sourceDescription } from "./source-position.ts"; import { markExactSource } from "./output/exact-source.ts"; import { layerEnvironments, @@ -2377,94 +2379,118 @@ function* expandComponent( * definition can reach the presentation decision below. */ let authorizedCanonically = false; - try { - // The public chain answers, and canonical execution decides whether the - // answer is one it produced. In a closed execution — a workflow holding a - // component bundle, a generated fragment holding an allowlist — a handler - // may observe this import, delegate it, and refuse it by throwing; nothing - // it returns is invoked. Without an authority the answer is whatever the - // chain produced, exactly as it always was. - // The offer is open for exactly this ask. Middleware composes inside it and - // may observe, delegate or refuse the import; what it cannot do is obtain - // the declaration for an element that did not author it, because the offer - // is made from the closure the segments being expanded carry, is spent by - // whatever asks first, and authorizes only the answer it produced itself. - const offered = authority?.declared?.offer(authority.privates, name); - let answered: ImportedDefinition; - try { - answered = yield* importComponent(name, position); - } finally { - offered?.close(); + // An admitted program's own components were resolved through this site's + // chain, compared against the retained admission and copied before the first + // program effect, so the chain has already answered for this name. Asking it + // again would be a second lookup a provider could answer differently. What + // still happens is the ordinary durable import the authored element makes, + // restored from the answer that was already authorized. + const settled = authority?.settled?.get(name); + if (settled !== undefined) { + yield createDurableOperation( + { type: "import_component", name, ...sourceDescription(position) }, + // deno-lint-ignore require-yield + function* (): Operation { + return { kind: "settled" }; + }, + ); + imported = settled; + if (settled.kind === "function") { + authority?.identities?.select(name, settled); + authority?.forms?.select(name, settled); } selected = selection?.settle(); - // A private import is authorized by the ask that made the offer, and by - // nothing else. Not by the name: a private component runs for the element - // the declaration that carries it authored, so an answer kept from another - // import — however exactly it describes the same definition — authorizes - // nothing here. And a private name written where no offer was made never - // reaches this at all: selection resolves it to nothing, so what arrives is - // the ordinary unresolved failure. - let authorizedPrivate = false; - if (authority?.declared?.declaresPrivate(name) === true) { - imported = requirePrivate(offered, name, answered); - authorizedPrivate = true; - authority.forms?.select(name, imported); - } else if (authority?.imports === undefined || !authority.imports.closes(name)) { - // Closed for this exact name, not for the execution that closed it. A - // bundled run closes every import; a host that declared exact Markdown - // closed the names it declared, and an unrelated one is the open import it - // has always been — the chain's answer, unverified, with no selection - // recorded against it. - imported = answered; - } else { - imported = authority.imports.authorize(name, answered); - // This import is canonical execution's own answer for a name this - // execution closed, which is the only provenance exact source is read - // from. An open import — one no tier claims — never sets it, however its - // answer describes itself. - authorizedCanonically = true; - // Closed authorization answers with core's retained copy rather than the - // object the resolver recorded, and the copy is what this expansion - // invokes — so the selection is recorded against it too. Only here: an - // open execution's answer travelled through public middleware, and - // nothing it hands back is canonical resolution's product. The record - // takes its dispatcher from the copy's own `fn`, identical by retention; - // a wrapper whose `fn` is no dispatcher records nothing, so a dispatcher - // an authority recorded explicitly is never displaced. - authority.forms?.select(name, imported); - } - // Whatever tier answered, and whatever this execution declares, an - // implementation some declaration's private closure built runs only for an - // import that closure authorized. Neither the name nor the current - // execution can decide it: an answer kept from a legitimate private import - // can be returned for any *other* name, in a copy of the definition, and in - // a later run that declares nothing at all — and a run that has ended - // authorizes nothing. - if (!authorizedPrivate) { - refuseEscapedPrivate(name, imported); - } - // Read off the answer rather than from a frame the engine opened: what is - // recognized is the exact definition canonical resolution produced for this - // exact name, whenever it produced it. - dispatcher = authority?.forms?.dispatcherFor(name, imported); - } catch (error) { - selection?.settle(); - // Import is a durable effect, so it is the other place a stale journal - // entry can surface. - const fatal = fatalCause(error); - if (fatal !== undefined) { - throw fatal; + dispatcher = authority?.forms?.dispatcherFor(name, settled); + } else { + try { + // The public chain answers, and canonical execution decides whether the + // answer is one it produced. In a closed execution — a workflow holding a + // component bundle, a generated fragment holding an allowlist — a handler + // may observe this import, delegate it, and refuse it by throwing; nothing + // it returns is invoked. Without an authority the answer is whatever the + // chain produced, exactly as it always was. + // The offer is open for exactly this ask. Middleware composes inside it and + // may observe, delegate or refuse the import; what it cannot do is obtain + // the declaration for an element that did not author it, because the offer + // is made from the closure the segments being expanded carry, is spent by + // whatever asks first, and authorizes only the answer it produced itself. + const offered = authority?.declared?.offer(authority.privates, name); + let answered: ImportedDefinition; + try { + answered = yield* importComponent(name, position); + } finally { + offered?.close(); + } + selected = selection?.settle(); + // A private import is authorized by the ask that made the offer, and by + // nothing else. Not by the name: a private component runs for the element + // the declaration that carries it authored, so an answer kept from another + // import — however exactly it describes the same definition — authorizes + // nothing here. And a private name written where no offer was made never + // reaches this at all: selection resolves it to nothing, so what arrives is + // the ordinary unresolved failure. + let authorizedPrivate = false; + if (authority?.declared?.declaresPrivate(name) === true) { + imported = requirePrivate(offered, name, answered); + authorizedPrivate = true; + authority.forms?.select(name, imported); + } else if (authority?.imports === undefined || !authority.imports.closes(name)) { + // Closed for this exact name, not for the execution that closed it. A + // bundled run closes every import; a host that declared exact Markdown + // closed the names it declared, and an unrelated one is the open import it + // has always been — the chain's answer, unverified, with no selection + // recorded against it. + imported = answered; + } else { + imported = authority.imports.authorize(name, answered); + // This import is canonical execution's own answer for a name this + // execution closed, which is the only provenance exact source is read + // from. An open import — one no tier claims — never sets it, however its + // answer describes itself. + authorizedCanonically = true; + // Closed authorization answers with core's retained copy rather than the + // object the resolver recorded, and the copy is what this expansion + // invokes — so the selection is recorded against it too. Only here: an + // open execution's answer travelled through public middleware, and + // nothing it hands back is canonical resolution's product. The record + // takes its dispatcher from the copy's own `fn`, identical by retention; + // a wrapper whose `fn` is no dispatcher records nothing, so a dispatcher + // an authority recorded explicitly is never displaced. + authority.forms?.select(name, imported); + } + // Whatever tier answered, and whatever this execution declares, an + // implementation some declaration's private closure built runs only for an + // import that closure authorized. Neither the name nor the current + // execution can decide it: an answer kept from a legitimate private import + // can be returned for any *other* name, in a copy of the definition, and in + // a later run that declares nothing at all — and a run that has ended + // authorizes nothing. + if (!authorizedPrivate) { + refuseEscapedPrivate(name, imported); + } + // Read off the answer rather than from a frame the engine opened: what is + // recognized is the exact definition canonical resolution produced for this + // exact name, whenever it produced it. + dispatcher = authority?.forms?.dispatcherFor(name, imported); + } catch (error) { + selection?.settle(); + // Import is a durable effect, so it is the other place a stale journal + // entry can surface. + const fatal = fatalCause(error); + if (fatal !== undefined) { + throw fatal; + } + return [ + yield* raise({ + type: "error", + message: + error instanceof Error + ? `Failed to import component ${name}: ${error.message}` + : `Failed to import component ${name}: ${String(error)}`, + source: name, + }), + ]; } - return [ - yield* raise({ - type: "error", - message: - error instanceof Error - ? `Failed to import component ${name}: ${error.message}` - : `Failed to import component ${name}: ${String(error)}`, - source: name, - }), - ]; } // Function component: call the generator function directly @@ -4213,15 +4239,18 @@ export function* expandProgramBody( throw new ProgramEvaluationError(INCOMPATIBLE); } // The answers that passed the comparison are the answers the program - // invokes. Asking the chain again and running whatever came back is the gap - // this closes: a provider could answer one way while the site was being - // checked and another way while the program ran. - const settled = new ProgramImports(current); - // Closed over the answers that passed, so a handler outside this expansion - // that replaces one is refused where it would be invoked rather than - // silently preferred. + // invokes, and they reach expansion as settled answers rather than as a + // second lookup. The site's own closed tiers are untouched: a bundled or + // declared name is still that tier's to answer, and this adds nothing to + // what the site closes. + const settled = new Map(); + for (const entry of current) { + if (entry.definition !== undefined && !settled.has(entry.name)) { + settled.set(entry.name, entry.definition); + } + } const authority: ExpansionAuthority | undefined = - site.authority === undefined ? { imports: settled } : { ...site.authority, imports: settled }; + site.authority === undefined ? { settled } : { ...site.authority, settled }; return yield* scoped(function* () { yield* provideEnv(programEnvironment(site.callerValues, program.props)); if (program.returns !== undefined) { diff --git a/packages/core/src/program-identity.ts b/packages/core/src/program-identity.ts index a122a852d..605a8bba3 100644 --- a/packages/core/src/program-identity.ts +++ b/packages/core/src/program-identity.ts @@ -169,6 +169,17 @@ export const UNIDENTIFIED = " cannot evaluate a program naming a component supplied by import middleware that " + "states no identity: a continuation has nothing to hold the site to."; +/** + * What an answer that moved on its way back through the chain says. + * + * A provider marks an answer and then edits it, or hands back something other + * than what it marked. Either way what the chain returned is not what was + * claimed, so there is nothing here to admit a program against. + */ +export const ANSWER_CHANGED = + " cannot evaluate a program whose component answer was changed after the provider " + + "that supplied it claimed it."; + /** The canonical identity of what a selection chose. */ export function selectionIdentity(selection: ComponentSelection): ProgramIdentity { switch (selection.kind) { diff --git a/packages/core/src/program-imports.ts b/packages/core/src/program-imports.ts index da6db6133..6812b361b 100644 --- a/packages/core/src/program-imports.ts +++ b/packages/core/src/program-imports.ts @@ -38,15 +38,10 @@ import { createContext } from "effection"; import type { Context, Operation } from "effection"; -import { stillDescribes } from "./components/import-authority.ts"; import { CanonicalImports } from "./components/import-authority.ts"; -import type { - ImportAuthority, - ImportedDefinition, - ImportRefusal, -} from "./components/import-authority.ts"; +import type { ImportedDefinition } from "./components/import-authority.ts"; import { ProgramEvaluationError } from "./program-identity.ts"; -import type { ImportProviderIdentity, ResolvedProgramComponent } from "./program-identity.ts"; +import type { ImportProviderIdentity } from "./program-identity.ts"; /** How an identified provider marks one answer as its own. */ export interface ImportProviderClaimant { @@ -92,98 +87,33 @@ export function createImportProviderRegistry(imports: CanonicalImports): ImportP const origins = new Set(); return { claimant(identity: ImportProviderIdentity): ImportProviderClaimant { - if (identity.origin.length === 0 || identity.revision.length === 0) { + // Read once, here, and never again. What a provider states is an object it + // still holds: a getter can answer one origin to the duplicate check and + // another to the claim, and a plain object can be edited after this + // returns. Copying the primitives out at registration is what makes the + // identity this claimant marks answers with the identity that was + // validated and counted. + const origin = identity.origin; + const revision = identity.revision; + if (typeof origin !== "string" || typeof revision !== "string") { + throw new ProgramEvaluationError( + "An identified import provider states its origin and revision as strings.", + ); + } + if (origin.length === 0 || revision.length === 0) { throw new ProgramEvaluationError( "An identified import provider states a non-empty origin and revision.", ); } - if (origins.has(identity.origin)) { + if (origins.has(origin)) { throw new ProgramEvaluationError( - `Two import providers are installed under the origin "${identity.origin}", so it names ` + + `Two import providers are installed under the origin "${origin}", so it names ` + "no single authority.", ); } - origins.add(identity.origin); + origins.add(origin); return (name, key, definition) => - imports.supply( - name, - { origin: identity.origin, key, revision: identity.revision }, - definition, - ); + imports.supply(name, { origin, key, revision: revision }, definition); }, }; } - -/** What one program's imports refuse, and why. */ -const REFUSED: Record = { - unissued: - "Component.importComponent middleware answered a complete program's import with a definition " + - "canonical resolution did not settle for it.", - "another-name": - "Component.importComponent middleware answered a complete program's import with the " + - "definition canonical resolution settled for another component.", - changed: - "Component.importComponent middleware changed the definition canonical resolution settled " + - "before the program invoked it.", -}; - -/** - * The answers one admitted program invokes. - * - * Built from the resolution that passed the compatibility comparison, so the - * definition a program expands is the one the comparison was about. Asking the - * chain again and invoking whatever came back is the gap this closes: a - * provider could answer one way while the site was being checked and another - * way while the program ran. - * - * A name that resolved to nothing is not closed here. It reaches the ordinary - * chain and produces the ordinary unresolved failure, which is what a program - * naming a component this site does not have should say. - */ -export class ProgramImports implements ImportAuthority { - readonly #answers: ReadonlyMap; - - constructor(resolved: readonly ResolvedProgramComponent[]) { - const answers = new Map(); - for (const entry of resolved) { - if (entry.definition !== undefined && !answers.has(entry.name)) { - answers.set(entry.name, entry.definition); - } - } - this.#answers = answers; - } - - closes(name: string): boolean { - return this.#answers.has(name); - } - - /** - * The answer canonical resolution settled, for the import the program made. - * - * The chain still runs, so the execution records its own selection and - * identity domain exactly as it does for any other import. What changes is - * what comes back: the answer must still describe what the comparison - * passed, and the object invoked is the copy the comparison was about. A - * provider that answered one way while the site was checked and another way - * while the program ran is refused here rather than preferred. - */ - authorize(name: string, answer: ImportedDefinition): ImportedDefinition { - const settled = this.#answers.get(name); - if (settled === undefined) { - throw new ProgramEvaluationError(REFUSED.unissued); - } - if (answer !== settled && read(() => stillDescribes(settled, answer)) !== true) { - throw new ProgramEvaluationError(REFUSED.changed); - } - return settled; - } -} - -/** One read of an answer that may refuse to be read. */ -function read(inspect: () => T): T | undefined { - try { - return inspect(); - } catch { - return undefined; - } -} diff --git a/packages/core/tests/program-evaluation.test.ts b/packages/core/tests/program-evaluation.test.ts index 564dd9d90..77b0e708f 100644 --- a/packages/core/tests/program-evaluation.test.ts +++ b/packages/core/tests/program-evaluation.test.ts @@ -747,6 +747,9 @@ describe("Tier PE — a producer's private closure does not cross", () => { /** Every implementation an open middleware provider actually entered. */ const opened: string[] = []; +/** Every name the open provider's handler was consulted for. */ +const consulted: string[] = []; + /** One implementation of ``, distinguishable by what it renders. */ function openImplementation(mark: string, props?: JsonObject): FunctionComponentDefinition { return { @@ -787,6 +790,7 @@ function openProvider(options: { yield* Component.around( { *importComponent([name, position], next) { + consulted.push(name); if (name !== "Open") { return yield* next(name, position); } @@ -884,7 +888,7 @@ describe("Tier PE — the identity is the answer's, not the selector's", () => { approved = APPROVED; }); - it("PE26 cannot run an answer that changed after the comparison passed", function* () { + it("PE26 invokes the snapshot that passed reconciliation, asking nothing again", function* () { opened.length = 0; approved = OPEN_PROGRAM; @@ -900,13 +904,11 @@ describe("Tier PE — the identity is the answer's, not the selector's", () => { }), }); - // B never runs. Expansion is authorized against the answer the comparison - // settled, so an answer that changed underneath it is refused rather than - // quietly preferred. - expect(`${attempt.failure ?? attempt.output ?? ""}`).toContain( - "changed the definition canonical resolution settled", - ); - expect(opened).toEqual([]); + // Expansion invokes the snapshot that passed reconciliation, so the third + // lookup never happens and the effect planted in it never occurs. + expect(attempt.failure).toBeUndefined(); + expect(attempt.output).toContain("open A"); + expect(opened).toEqual(["A"]); approved = APPROVED; }); @@ -1103,3 +1105,214 @@ describe("Tier PE — what a provider does to the chain's answer", () => { approved = APPROVED; }); }); + +/** Every ordinary import this run recorded for one component name. */ +function importsOf(events: DurableEvent[], name: string): number[] { + return events.flatMap((event, index) => + event.type === "yield" && + event.description.type === "import_component" && + event.description.name === name + ? [index] + : [], + ); +} + +/** Where the complete-program admission sits in the journal. */ +function admissionIndex(events: DurableEvent[]): number { + return events.findIndex( + (event) => event.type === "yield" && event.description.type === "evaluate_program", + ); +} + +describe("Tier PE — resolution settles before the program's own import", () => { + it("PE33 records exactly one ordinary import, after the admission", function* () { + opened.length = 0; + consulted.length = 0; + approved = OPEN_PROGRAM; + + const attempt = yield* run(PROBING_DOCUMENT, { + install: openProvider({ mark: "A", identity: { origin: "test/open", revision: "A" } }), + }); + + expect(attempt.failure).toBeUndefined(); + const admitted = admissionIndex(attempt.events); + const records = importsOf(attempt.events, "Open"); + // Resolution journals nothing, so the admission commits first and the + // authored element's own import is the only one that follows it. + expect(records).toHaveLength(1); + expect(records[0]).toBeGreaterThan(admitted); + approved = APPROVED; + }); + + it("PE34 consults the provider for admission and reconciliation only", function* () { + opened.length = 0; + consulted.length = 0; + approved = OPEN_PROGRAM; + + yield* run(PROBING_DOCUMENT, { + install: openProvider({ mark: "A", identity: { origin: "test/open", revision: "A" } }), + }); + + // Twice: once for the admission's resolution, once for canonical + // reconciliation. Expansion invokes the snapshot instead of asking again. + expect(consulted.filter((name) => name === "Open")).toHaveLength(2); + approved = APPROVED; + }); + + it("PE35 resolves structural syntax directly, consulting nobody", function* () { + opened.length = 0; + consulted.length = 0; + approved = ["", "structural ran", "", "", "", ""].join("\n"); + + const attempt = yield* run(PROBING_DOCUMENT, { + install: openProvider({ mark: "A", identity: { origin: "test/open", revision: "A" } }), + }); + + expect(attempt.failure).toBeUndefined(); + expect(attempt.output).toContain("structural ran"); + const named = decision(admissions(attempt.events)[0]!).named as Record[]; + expect(named[0]).toEqual({ + name: "If", + form: "paired", + identity: { tag: "structural", construct: "If" }, + }); + // No lookup and no record: structural syntax is the engine's own. + expect(consulted).not.toContain("If"); + expect(importsOf(attempt.events, "If")).toHaveLength(0); + approved = APPROVED; + }); +}); + +describe("Tier PE — the site's closed authority still decides", () => { + it("PE36 refuses an identified replacement of a declared component", function* () { + opened.length = 0; + approved = "\n"; + const policy = "the declared policy ran\n"; + + const attempt = yield* run(PROBING_DOCUMENT, { + declarations: [ + { + name: "Policy", + origin: POLICY_ORIGIN, + source: policy, + digest: sourceDigest(policy), + }, + ], + *install() { + const claim = yield* useImportProvider({ origin: "test/usurper", revision: "1" }); + const replacement = openImplementation("usurped"); + yield* Component.around( + { + *importComponent([name, position], next) { + if (name !== "Policy") { + return yield* next(name, position); + } + yield* next(name, position); + return claim(name, "Policy", replacement); + }, + }, + { at: "max" }, + ); + }, + }); + + // The declared tier closes this name, so it answers for it. The + // replacement is refused before the program runs anything. + expect(attempt.failure ?? attempt.output ?? "").not.toContain("usurped"); + expect(opened).toEqual([]); + approved = APPROVED; + }); + + it("PE37 lets a declared component answer its own name inside a program", function* () { + opened.length = 0; + approved = "\n"; + const policy = "the declared policy ran\n"; + + const attempt = yield* run(PROBING_DOCUMENT, { + declarations: [ + { + name: "Policy", + origin: POLICY_ORIGIN, + source: policy, + digest: sourceDigest(policy), + }, + ], + }); + + // The positive control for PE36: with no replacement, the closed answer is + // exactly what the program invokes. + expect(attempt.failure).toBeUndefined(); + expect(attempt.output).toContain("the declared policy ran"); + const named = decision(admissions(attempt.events)[0]!).named as Record[]; + expect(named[0]).toEqual({ + name: "Policy", + form: "self-closing", + identity: { tag: "declared-markdown", origin: POLICY_ORIGIN, digest: sourceDigest(policy) }, + }); + approved = APPROVED; + }); +}); + +describe("Tier PE — a provider's stated identity is captured once", () => { + it("PE38 ignores mutation of the identity object after the claimant exists", function* () { + opened.length = 0; + approved = OPEN_PROGRAM; + const stated = { origin: "test/open", revision: "A" }; + + const first = yield* run(PROBING_DOCUMENT, { + *install() { + const claim = yield* useImportProvider(stated); + // Edited after registration. What the claimant marks answers with is + // what was read and validated, not what this object says now. + stated.revision = "B"; + const definition = openImplementation("A"); + yield* Component.around( + { + *importComponent([name, position], next) { + if (name !== "Open") { + return yield* next(name, position); + } + return claim(name, "Open", definition); + }, + }, + { at: "max" }, + ); + }, + }); + + expect(first.failure).toBeUndefined(); + const named = decision(admissions(first.events)[0]!).named as Record[]; + expect(named).toEqual([ + { + name: "Open", + form: "self-closing", + identity: { tag: "middleware", origin: "test/open", key: "Open", revision: "A" }, + }, + ]); + approved = APPROVED; + }); + + it("PE39 refuses an alternating origin that would evade duplicate detection", function* () { + approved = OPEN_PROGRAM; + let reads = 0; + const alternating: ImportProviderIdentity = { + get origin() { + reads += 1; + return reads === 1 ? "test/open" : "test/other"; + }, + revision: "A", + }; + + const attempt = yield* run(PROBING_DOCUMENT, { + *install() { + yield* useImportProvider({ origin: "test/open", revision: "A" }); + yield* useImportProvider(alternating); + }, + }); + + // The origin is read once, so the getter answers the duplicate check and + // the claim with one value — and that value is already taken. + expect(attempt.failure ?? attempt.output ?? "").toContain("no single authority"); + approved = APPROVED; + }); +}); diff --git a/specs/executable-mdx-spec.md b/specs/executable-mdx-spec.md index 91c770041..8b2db1fca 100644 --- a/specs/executable-mdx-spec.md +++ b/specs/executable-mdx-spec.md @@ -4845,7 +4845,11 @@ An answer an **identified middleware provider** supplied keeps that provider's own identity: a stable origin, the provider's stable key for the name, and a revision that changes whenever the supplied implementation changes. The provider states those terms at its installation boundary through `useImportProvider()`, -and canonical execution mints it a claimant for that one execution. Marking an +and canonical execution mints it a claimant for that one execution, reading the +origin and revision exactly once and closing over copies. A getter that answers +one origin to the duplicate check and another to the claim, or an object edited +after registration, changes nothing about the identity that claimant marks +answers with. Marking an answer binds the terms to that exact object in execution-private state — never on the definition, which is data an answer can copy, and never through a replaceable Context answer. The identity is an assertion by the authority @@ -4861,8 +4865,22 @@ complete program cannot be admitted against one, because a continuation would have nothing to compare and would invoke whatever answered on the day it resumed. -Resolution invokes no component implementation and performs no program effect; -it settles which definition answers each name. +**Resolution is its own path.** It runs the site's ordinary chain under a +resolution-only terminal: it selects and loads exactly what an authored import +would, invokes no component implementation, and journals nothing. So +`evaluate_program` commits before the program's own ordinary import record +rather than after it. + +**The site's closed authority still decides.** A name a workflow bundle or a +declared-Markdown host closed is answered by that tier, exactly as it is in +ordinary expansion, and an identified replacement of such a name is refused +rather than becoming the identity a program is admitted under. A +provider-supplied answer is witnessed, not issued: it authorizes nothing for a +closed name. + +**Structural syntax never reaches component import.** ``, `` and +every other reserved construct resolve directly to their structural identity — +no lookup, no provider consulted, and no import record. **A continuation is held to the site it was admitted at.** Before the first program effect, canonical execution resolves every retained name again through @@ -4875,12 +4893,17 @@ answering it dishonestly refuses the evaluation rather than widening it, because the canonical comparison is what decides. **The comparison and the invocation are one decision.** The answers that passed -the comparison are the answers the program invokes: canonical execution keeps -its own copy of each, taken when the answer was witnessed, and the program's -imports are authorized against it. An answer that no longer describes what the -comparison settled is refused where it would be invoked rather than silently -preferred, which is what closes the gap in which a provider answers one way -while the site is checked and another way while the program runs. +the comparison are the answers the program invokes. Canonical execution keeps +its own copy of each, taken when the answer was witnessed, and hands those +copies to the program's expansion: an element whose name is among them does not +reach `Component.importComponent` at all. The chain is entered twice for one +evaluation — once for the admission's resolution, once for reconciliation — and +never again, so there is no third lookup for a provider to answer differently. + +The authored element still makes its ordinary durable import: one +`import_component` record per occurrence, written after the admission and +restored from the answer already authorized, with identity-domain and form +selection recorded from that same answer. It rediscovers no authority. **The retained record is hostile data.** A journal is a file, and replay hands back whatever it holds. The admitted and refused decisions and every nested @@ -12407,6 +12430,13 @@ forms sit beside the restricted one. | PE30 | Replacement | A provider replacing a canonical answer retains its own identity, and the replaced registration never runs | | PE31 | Mutation after the claim | An answer changed on its way back through the chain refuses | | PE32 | Two claims on one answer | A second provider claiming an answer another already claimed refuses | +| PE33 | One import, after the admission | Exactly one ordinary `import_component` follows `evaluate_program` for a program's component, and none precedes it | +| PE34 | Two lookups, not three | The provider chain is entered for the admission's resolution and for reconciliation, and never for expansion | +| PE35 | Structural syntax | `` retains the structural tag, consults no provider and records no import | +| PE36 | A closed name | An identified replacement of a declared component is refused, and its implementation never runs | +| PE37 | The closed positive control | With no replacement, the declared answer is what the program invokes and what the admission retains | +| PE38 | Identity captured once | Editing the stated identity after the claimant exists does not change what its claims are marked with | +| PE39 | An alternating origin | A getter cannot answer the duplicate check and the claim differently | | EP1–EP8 | Run profile | The ordinary run profile declares ``; the forms, props, refusals and catalog entry hold through the real binary, and a program cannot reach ``'s private components | | WGAC17–WGAC20 | Workflow profile | A complete program records `evaluate_program` and no `generated_xmd`; the three forms cannot be combined; complete-program support is not reachable through `source` | From 5b887ea275f1977c44bf2a8406d7ed456fe0eee1 Mon Sep 17 00:00:00 2001 From: Taras Mankovski Date: Fri, 4 Sep 2026 11:09:49 -0400 Subject: [PATCH 5/7] =?UTF-8?q?=F0=9F=AA=A2=20Bind=20a=20program's=20settl?= =?UTF-8?q?ed=20imports=20to=20their=20authored=20occurrences=20(#713)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The settlement map was keyed by name, so a program writing `` collapsed two independently resolved answers into one and invoked the first twice. Each occurrence now carries its own settlement, keyed by where the element was written and paired with the admission's retained order, so two elements resolved to two implementations invoke those two in order. An unresolved occurrence is a settled outcome as well. It was absent from the map, so the element fell through to the ordinary open chain and could be answered by a lookup reconciliation never made. It now reports the ordinary unresolved failure without consulting anybody, and the chain is entered exactly twice for one evaluation whatever an occurrence settled to. Settlements belong to the program's own parsed body. A component the program invokes expands its own bytes at its own offsets, which nothing reconciled, so carrying them in would have made a settlement a name-wide override of somebody else's element. The nested durable record is a protocol rather than a marker. A settled occurrence records exactly `settled` and `name`, and a continuation parses that as the hostile replay data it is before invoking anything: a missing, additional, mistyped or unknown member, and a record naming another component, each refuse. A refusal this boundary raises during resolution — a second provider claiming one answer, an answer changed after it was claimed — now reaches the caller instead of being read as "this name resolves to nothing". --- architecture.md | 2 +- .../core/src/components/import-authority.ts | 27 ++-- packages/core/src/execute.ts | 16 ++- packages/core/src/expand.ts | 95 +++++++++---- packages/core/src/program-evaluation.ts | 15 +- packages/core/src/program-identity.ts | 73 +++++++++- .../core/tests/program-evaluation.test.ts | 130 ++++++++++++++++++ specs/executable-mdx-spec.md | 41 ++++-- 8 files changed, 339 insertions(+), 60 deletions(-) diff --git a/architecture.md b/architecture.md index ba67d759a..49acf519d 100644 --- a/architecture.md +++ b/architecture.md @@ -3899,7 +3899,7 @@ Status is measured against main. | workflow Agent session | a workflow document's `` runs under a profile the host attaches only for a live or partial run: an empty host-owned working directory instead of any Workspace, checkout or caller path, no MCP servers, an empty requested native tool set, and `deny-all` with a permission path that denies every native request and fails the turn that asked without reaching the public permission chain. Within a run a session is identified by the Agent/Session expansion identity the engine derived — the authored name is descriptive, so two sibling `` elements are two sessions — routed inside a placement bound to its element and good for one use, so a kept placement cannot be substituted for the next. The conversation is retained as a row in the run's own database with the provider, resolved agent command and policy fingerprint beside it as compatibility attributes. The order is placement, the backend's acceptance of the session's first turn, the provider's canonical tagged assertion, then the mapping commit — and only then is anything that turn produced exposed. A placement is inert: it creates no provider session and writes no row. Occupancy of a provider key is not an assertion, and a record held for a first turn nobody accepted asserts nothing at all; the pre-commit window reconciles only from exactly one canonical assertion, and a missing, conflicting, replaced or ambiguous assertion is one explicit refusal that starts no replacement. Deleting a run removes the row with the run and the provider-session directory beside it, and reports the categories. The profile selects ACP-only capability explicitly — no native-launch advertisement and no client-native attachment advertisement — rather than inheriting the provider package's ordinary-run sets by omission, and it supplies no machine session coordinator, construction-route store or executable observer: a workflow session belongs to a run, and the machine-wide account describes a different thing entirely | built on the #302 stack, with the explicit ACP-only selection from #561; the portable proof that an adapter honours an empty tool set is tracked by #496 and does not widen the ceiling | | generated-XMD admission | admits one Agent-generated fragment through the trusted-host seam: host policy is a `read` table and a `write` table of exact pinned identities, each carrying the authored forms it is admitted for, and an authored `allow` selects a canonical non-empty subset of the closed classes — omitted means `read`. The complete source is preflighted inside one `generated_xmd` durable effect before its first generated effect; only the pinned identity the selected classes hold for that name **and** that form executes; and the admitted source, class selection, selected root, every selected entry with its forms, the identity and form of each element named, and the normalized request policy are retained in that effect's own result — so a continuation restores the decision without reading the current candidate and expands only the retained source. The roots are an as-of-admission retained basis checked by membership — the run's own later root publications and an advanced retained current root pass, while a lost admission root or lost selected root refuses — and every non-root term is checked exactly, refusing a run whose classes, identities, forms or requests have moved. The admission and every nested generated effect are offered inline by the owning expansion in authored order, so a partial continuation restores each completed one without another live execution. Each admitted effect is retained by its own ordinary record, and a read's value is collected while a mutation's is not | built on the #369 stack, continuation basis amended by #589; core owns the mechanics and the workflow policy wrapper is internal | | `` | the restricted generated-fragment form of ``, which the workflow host declares beside the complete-program forms below. The form an authored document writes where an observation should happen. The host does not register it: it **declares** it to the execution through `ExecutionInstallation.components`, captured before any installation runs, and canonical execution calls its factory once for that attachment with the claimant it minted and registers what comes back. Registration provides availability only — a name a trusted document may write — and carries none of the authority. It takes a `source` string and one optional `allow` array selecting a non-empty duplicate-free subset of the closed effect classes `read` and `write` — omitted means `read` — refuses paired content, and cannot be combined with `program`, with paired content, or with `props`; each of those refuses before the durable name is claimed and before the candidate is read. It declares no `returns` and answers with a detached value — `{ observations: [{ name, value }], output }`, each admitted read's own returned value under the name the fragment invoked it by, in invocation order, with whatever the fragment rendered under `output` rather than instead of them, and the pinned identity that produced one left in the retained admission rather than copied here — so an admitted ``, which renders nothing at all, still reaches the document. An admitted mutation contributes no entry and no receipt, so a write-only fragment binds `{ observations: [], output: "" }`; `as` is valid for every selection and binds that same shape. An ordinary `as` captures that value by reference, and an authored `` renders it into the next ``: deciding how a value becomes text is the document's. Every ceiling comes from values the host captured at installation — the run's retained roots and its authoritative current root read from the run's own storage at invocation, as-of-admission provenance a continuation holds by membership so the run's own later publications and an advanced retained current root invalidate nothing, core's pinned self-closing `` read, the write table of core's paired ``, this package's lexical `` built from the definition the ordinary registration owns, and core's self-closing ``, and `` only when the captured request ceiling is non-empty — and no prop, binding, context or middleware return value supplies or widens one. `allow` selects among those tables and adds nothing to them; approval, when a workflow needs one, is authored control flow before the element. Its durable operation is named through that claimant, on the exact invocation the engine handed it and in that invocation's own frame — not from a context a document could rebind, a contextual Api answer, a definition, or a registry answer. Generated source never resolves through the registration: the evaluator consults only its own closed table of pinned identities. It is deliberately not wrapped in `printErrors`, so a refused fragment stops the authored loop rather than becoming text the next turn could read as a read that happened | built on the #302 stack, extended by #369 | -| complete-program evaluation | admits one complete XMD root and evaluates it at the authored `` site, in the current execution: its lifecycle, journal, cancellation scope, contextual providers, working directory or Workspace, and the authority in force where the element was written. There is no child process, host profile, root lifecycle or journal of its own — program source requests behavior and grants none. The root's own frontmatter, imports, metadata, props schema, `returns` and `` selection decide what it does; a text root renders its selected output or binds it as a string under `as`, a value root requires `as` and binds its schema-validated result, and root props are the explicit `props` object defaulting to `{}` rather than the caller's own. Ordinary non-props caller bindings are visible as a copy, so nothing the program binds escapes, and relative paths resolve from the evaluation site's source origin. The producer's temporary authority is not inherited: an enclosing declaration's private closure does not cross, so a private name written in a program resolves to nothing, while imports and capabilities genuinely present at the site remain. Canonical execution answers `Component.expandProgram()` from the frame it already holds, which is what makes site authority a fact about the frame rather than something a prop, binding or middleware return value could supply. Which definition each name the program writes resolves to is part of what the site granted, so the admission retains it and canonical execution settles it again before the first program effect — through a resolver the execution builds and carries on the expansion authority, reachable from no document, component, contextual Api answer or middleware return value. That resolver goes through the ordinary import chain under a resolution-only terminal — it selects and loads what an authored import would, invokes nothing and journals nothing — because the answer the chain returns is the answer that runs, while an independent second resolution would describe a definition a provider replaced. `evaluate_program` therefore commits before the program's own ordinary import record. The site's existing closed tiers still decide: a bundled or declared name is answered by the tier that closed it, and a provider-supplied answer is witnessed rather than issued, so it authorizes nothing for such a name. Structural syntax resolves directly to its own identity and never reaches component import. The answers that passed the comparison are handed to the program's expansion as settled answers, so the chain is entered twice for one evaluation and never a third time; the authored element still records one ordinary import after the admission, restored from the already-authorized answer with identity-domain and form selection intact | built by #713 on the #722 `` contract; core owns the mechanics and every host that offers `` declares them | +| complete-program evaluation | admits one complete XMD root and evaluates it at the authored `` site, in the current execution: its lifecycle, journal, cancellation scope, contextual providers, working directory or Workspace, and the authority in force where the element was written. There is no child process, host profile, root lifecycle or journal of its own — program source requests behavior and grants none. The root's own frontmatter, imports, metadata, props schema, `returns` and `` selection decide what it does; a text root renders its selected output or binds it as a string under `as`, a value root requires `as` and binds its schema-validated result, and root props are the explicit `props` object defaulting to `{}` rather than the caller's own. Ordinary non-props caller bindings are visible as a copy, so nothing the program binds escapes, and relative paths resolve from the evaluation site's source origin. The producer's temporary authority is not inherited: an enclosing declaration's private closure does not cross, so a private name written in a program resolves to nothing, while imports and capabilities genuinely present at the site remain. Canonical execution answers `Component.expandProgram()` from the frame it already holds, which is what makes site authority a fact about the frame rather than something a prop, binding or middleware return value could supply. Which definition each name the program writes resolves to is part of what the site granted, so the admission retains it and canonical execution settles it again before the first program effect — through a resolver the execution builds and carries on the expansion authority, reachable from no document, component, contextual Api answer or middleware return value. That resolver goes through the ordinary import chain under a resolution-only terminal — it selects and loads what an authored import would, invokes nothing and journals nothing — because the answer the chain returns is the answer that runs, while an independent second resolution would describe a definition a provider replaced. `evaluate_program` therefore commits before the program's own ordinary import record. The site's existing closed tiers still decide: a bundled or declared name is answered by the tier that closed it, and a provider-supplied answer is witnessed rather than issued, so it authorizes nothing for such a name. Structural syntax resolves directly to its own identity and never reaches component import. The answers that passed the comparison are handed to the program's expansion bound to the authored occurrence each was settled for, not to the name, so a program writing one name twice invokes the two implementations its two elements resolved to rather than the first one twice. An unresolved occurrence is settled too, and reports the ordinary unresolved failure instead of falling through to the open chain. Settlements belong to the program's own parsed body and are not inherited by a component it invokes. The chain is therefore entered twice for one evaluation and never a third time; the authored element still records one ordinary import after the admission, whose result is a closed two-member shape parsed as hostile replay data before anything is invoked, with identity-domain and form selection taken from the already-authorized answer | built by #713 on the #722 `` contract; core owns the mechanics and every host that offers `` declares them | | `` complete-program forms | the two ways an author names a program at a composition site: paired content, which renders a source-producing component into a private program buffer, and `program={value}`, which supplies complete-program source directly. Optional `props={object}` is the root's props input and is valid only for these forms. The two are mutually exclusive and one is required; every ambiguous or misplaced combination refuses before program content is produced and before any program effect runs, so an ambiguous element does not even render its producer. Paired content is a private buffer rather than a quotation — an executable component written there executes under ordinary XMD semantics while producing the text sent to admission — and the element removes only the framing its own wrapper contributed: the line break after the opening tag with the indentation following it, the line break before the closing tag with the indentation preceding it, and the indentation those lines share. Interior bytes are untouched, so a sole `` producer's trailing newline survives and paired evaluation and `program={plan}` admit the same bytes and the same content digest. `program={value}` is exact supplied source and is neither trimmed, dedented nor newline-adjusted | built by #713 | | `evaluate_program` durable effect | the complete-program admission, recorded once before the first program effect and never as a widening of `generated_xmd` — the two are decisions about different kinds of source. `description.input` carries the source digest, the explicit props, the evaluation-site source origin and whether the result is captured; the result carries the exact admitted source, the root mode, the components the program names with their authored forms **and the identity canonical resolution settled for each name at that site**, as a closed tagged record whose tag decides its members, those same terms and the validated root props — or, for a refusal, the refused class alone and nothing of the source. Both results are closed shapes, and so is every record nested in them: a missing, additional or misspelled member at any depth is a record this evaluation did not write. Restored, it must also agree with itself before anything expands — the source hashes to the digest, reparsing produces the retained mode and a valid body structure, the retained elements are the ones that source writes in that order and those forms, and the supplied props validate to the retained validated props — and a record failing any of those is refused as unreadable with neither program performing an effect. A continuation whose site now answers one of those names with a different identity or admits it in a different form is refused before either implementation runs, which is a different answer from an unreadable record: the journal is intact and the environment moved. A partial continuation expands the retained source and restores the nested effects that already committed, repeating no planning; a changed program at the same evaluation occurrence is stale input and refuses before either the current or the retained source runs, so neither silently wins. The source digest identifies the program artifact and the authored element and loop iteration identify one execution of it, so two occurrences of one artifact have separate nested effect identities and the digest deduplicates neither | built by #713 | | generated mutation proposals | lets an Agent propose constrained executable changes that a separate admission then performs against the run's own Workspace | built on the #369 and #567 stacks, with directory creation added by #643: the standard Deno profile's write table is core's paired `File:write`, the paired `@executablemd/workflow/composition/dir-v2#Dir` and core's self-closing `File.Delete`, in that retained order and followed by any host extension. `allow={["write"]}` intentionally authorizes Dir's persistent recursive directory creation; its versioned identity makes every continuation retained under the former non-mutating Dir identity refuse before generated execution. Admitted mutations run as the ordinary components they are through the run's effect transactions, and the evaluator adds no mutation API or receipt. Approval is authored control flow before the write-enabled element. Local Git, Git-host, issue, process, execution, credential and external-write effects are outside the class | diff --git a/packages/core/src/components/import-authority.ts b/packages/core/src/components/import-authority.ts index 1b7a229d2..c2e32d3de 100644 --- a/packages/core/src/components/import-authority.ts +++ b/packages/core/src/components/import-authority.ts @@ -22,7 +22,8 @@ import type { ComponentDefinition, FunctionComponentDefinition } from "../types. import type { FormSelections, InvocationIdentities } from "../invocation-identity.ts"; import type { DeclaredImports, PrivateClosure } from "./declared-markdown.ts"; import type { ExactSource } from "../output/exact-source.ts"; -import type { ProgramResolver } from "../program-identity.ts"; +import { ProgramEvaluationError } from "../program-identity.ts"; +import type { ProgramResolver, ProgramSettlement } from "../program-identity.ts"; /** A definition an import may answer with. */ export type ImportedDefinition = ComponentDefinition | FunctionComponentDefinition; @@ -118,19 +119,29 @@ export interface ExpansionAuthority { */ readonly resolve?: ProgramResolver; /** - * The answers an admitted program invokes, settled before its first effect. + * What an admitted program's own occurrences settled to, before its first + * effect, keyed by where each element was written. * * Canonical execution resolved these through the site's own chain, compared * them against the retained admission, and kept its own copy of each. An - * element whose name is here does not reach `Component.importComponent` at - * all: the chain has already answered, and asking it again would let a - * provider answer one way while the site was checked and another way while - * the program ran. + * element written at one of these offsets does not reach + * `Component.importComponent` at all: the chain has already answered, and + * asking it again would let a provider answer one way while the site was + * checked and another way while the program ran. + * + * Keyed by occurrence rather than by name, because a program writing one name + * twice resolved it twice and the two answers are two settlements. An + * unresolved occurrence is here too, so it cannot fall through to the open + * chain. + * + * It belongs to the program's own parsed body and travels no further: a + * component the program invokes expands its own bytes, and settling a name + * for that body would be a name-wide override nobody reconciled. * * Held by the execution and handed here by value, like everything else on * this object. */ - readonly settled?: ReadonlyMap; + readonly settled?: ReadonlyMap; } /** Why an answer is not the one canonical execution produced for this name. */ @@ -317,7 +328,7 @@ export class CanonicalImports { held.supplied.key === supplied.key && held.supplied.revision === supplied.revision; if (!same) { - throw new Error( + throw new ProgramEvaluationError( "Component.importComponent middleware claimed an answer another provider had claimed.", ); } diff --git a/packages/core/src/execute.ts b/packages/core/src/execute.ts index 05a235c02..8c09a20b4 100644 --- a/packages/core/src/execute.ts +++ b/packages/core/src/execute.ts @@ -2378,6 +2378,9 @@ function* executeDocument( const absent = (unidentified = false): ResolvedProgramComponent => ({ name, form: "self-closing", + // The occurrence is the caller's — this answers about a name — and + // is filled in where the two are put together. + offset: -1, identity: UNRESOLVED, definition: undefined, unidentified, @@ -2391,6 +2394,7 @@ function* executeDocument( return { name, form: "self-closing", + offset: -1, identity: { tag: "structural", construct: name }, definition: undefined, unidentified: false, @@ -2423,7 +2427,14 @@ function* executeDocument( ); return yield* importComponent(name); }); - } catch { + } catch (error) { + // A name nothing answers is an ordinary outcome and settles as + // unresolved. A refusal this boundary raised — a second provider + // claiming one answer, an answer changed after it was claimed — is + // not an outcome about the name and must not be read as one. + if (error instanceof ProgramEvaluationError) { + throw error; + } return absent(); } @@ -2465,6 +2476,7 @@ function* executeDocument( return { name, form: "self-closing", + offset: -1, identity: selectionIdentity(selection), definition: authorized, unidentified: false, @@ -2475,6 +2487,7 @@ function* executeDocument( return { name, form: "self-closing", + offset: -1, identity: providerIdentity( { origin: witness.supplied.origin, revision: witness.supplied.revision }, witness.supplied.key, @@ -2498,6 +2511,7 @@ function* executeDocument( return { name, form: "self-closing", + offset: -1, identity: selectionIdentity(selection), definition: witness.canonical, unidentified: false, diff --git a/packages/core/src/expand.ts b/packages/core/src/expand.ts index ba8107cca..803a6e7dd 100644 --- a/packages/core/src/expand.ts +++ b/packages/core/src/expand.ts @@ -103,8 +103,16 @@ import { sameComponents, UNIDENTIFIED, UNRESOLVED, + elements, + readSettledImport, + SETTLED_IMPORT, + UNREADABLE_SETTLED_IMPORT, +} from "./program-identity.ts"; +import type { + ProgramComponentRef, + ProgramSettlement, + ResolvedProgramComponent, } from "./program-identity.ts"; -import type { ProgramComponentRef, ResolvedProgramComponent } from "./program-identity.ts"; import { DeclaredMarkdownError } from "./components/declared-markdown.ts"; import type { PrivateImport } from "./components/declared-markdown.ts"; @@ -731,10 +739,11 @@ function authorityForBody( return undefined; } const privates = authority.declared?.closureFor(name, definition); - if (privates === authority.privates) { - return authority; - } - const { privates: _cleared, ...rest } = authority; + // A program's settlements belong to the occurrences its own parsed body + // writes. A component the program invokes expands its own bytes, at its own + // offsets, which nothing reconciled — carrying them in would make a + // settlement a name-wide override of somebody else's element. + const { privates: _cleared, settled: _scoped, ...rest } = authority; return privates === undefined ? rest : { ...rest, privates }; } @@ -2385,22 +2394,40 @@ function* expandComponent( // again would be a second lookup a provider could answer differently. What // still happens is the ordinary durable import the authored element makes, // restored from the answer that was already authorized. - const settled = authority?.settled?.get(name); - if (settled !== undefined) { - yield createDurableOperation( + const settled = position === undefined ? undefined : authority?.settled?.get(position.offset); + if (settled !== undefined && settled.name === name) { + if (settled.kind === "unresolved") { + // Settled as unresolved, so it does not fall through to the open chain: + // reconciliation already asked, and nothing may answer it now. + return [ + yield* raise({ + type: "error", + message: `Cannot resolve component: ${name}`, + source: name, + }), + ]; + } + const answer = settled.definition; + // The authored element still makes its ordinary durable import, and a + // continuation reads that record as the hostile data it is before anything + // is invoked. A record this protocol did not write authorizes nothing. + const record = yield createDurableOperation( { type: "import_component", name, ...sourceDescription(position) }, // deno-lint-ignore require-yield function* (): Operation { - return { kind: "settled" }; + return { settled: SETTLED_IMPORT, name }; }, ); - imported = settled; - if (settled.kind === "function") { - authority?.identities?.select(name, settled); - authority?.forms?.select(name, settled); + if (!readSettledImport(record, name)) { + throw new Error(UNREADABLE_SETTLED_IMPORT); + } + imported = answer; + if (answer.kind === "function") { + authority?.identities?.select(name, answer); + authority?.forms?.select(name, answer); } selected = selection?.settle(); - dispatcher = authority?.forms?.dispatcherFor(name, settled); + dispatcher = authority?.forms?.dispatcherFor(name, answer); } else { try { // The public chain answers, and canonical execution decides whether the @@ -4231,7 +4258,17 @@ export function* expandProgramBody( // called: the resolver is canonical execution's and reaches this expansion on // the authority, so a handler that answered the admission's own resolution // dishonestly is caught by the answer it cannot reach. - const current = yield* resolveProgramComponents(program.named, site.authority); + // The occurrences the retained source writes, in the order the admission + // retained them. Pairing them by position is what keeps two `` + // elements two: they resolve separately and settle separately. + const occurrences = elements(program.bodySegments, []); + if (occurrences.length !== program.named.length) { + throw new ProgramEvaluationError(INCOMPATIBLE); + } + const current = yield* resolveProgramComponents( + program.named.map((entry, index) => ({ ...entry, offset: occurrences[index]!.offset })), + site.authority, + ); if (current.some((entry) => entry.unidentified)) { throw new ProgramEvaluationError(UNIDENTIFIED); } @@ -4239,15 +4276,21 @@ export function* expandProgramBody( throw new ProgramEvaluationError(INCOMPATIBLE); } // The answers that passed the comparison are the answers the program - // invokes, and they reach expansion as settled answers rather than as a - // second lookup. The site's own closed tiers are untouched: a bundled or - // declared name is still that tier's to answer, and this adds nothing to - // what the site closes. - const settled = new Map(); + // invokes, and they reach expansion bound to the occurrence each was settled + // for rather than to the name. An unresolved occurrence is settled too: + // leaving it out would let that element fall through to the ordinary open + // chain and be answered by a lookup reconciliation never made. + // + // The site's own closed tiers are untouched: a bundled or declared name is + // still that tier's to answer, and this adds nothing to what the site closes. + const settled = new Map(); for (const entry of current) { - if (entry.definition !== undefined && !settled.has(entry.name)) { - settled.set(entry.name, entry.definition); - } + settled.set( + entry.offset, + entry.definition === undefined + ? { kind: "unresolved", name: entry.name } + : { kind: "resolved", name: entry.name, definition: entry.definition }, + ); } const authority: ExpansionAuthority | undefined = site.authority === undefined ? { settled } : { ...site.authority, settled }; @@ -4334,14 +4377,18 @@ export function* resolveProgramComponents( resolved.push({ name: entry.name, form: entry.form, + offset: entry.offset, identity: UNRESOLVED, definition: undefined, unidentified: false, }); continue; } + // Resolved per occurrence, not per name: a program writing one name twice + // asks twice, and a provider answering the two differently is answering + // about two elements rather than changing its mind about one. const settled = yield* resolve(entry.name); - resolved.push({ ...settled, name: entry.name, form: entry.form }); + resolved.push({ ...settled, name: entry.name, form: entry.form, offset: entry.offset }); } return resolved; } diff --git a/packages/core/src/program-evaluation.ts b/packages/core/src/program-evaluation.ts index a66024f9b..0bdf0c356 100644 --- a/packages/core/src/program-evaluation.ts +++ b/packages/core/src/program-evaluation.ts @@ -83,6 +83,7 @@ import { isJsonObject, parseJson } from "./json.ts"; import { sourceDescription } from "./source-position.ts"; import { validateProps } from "./validate.ts"; import { + elements, isProgramComponentForm, ProgramEvaluationError, readIdentity, @@ -249,20 +250,6 @@ export function programDigest(source: string): string { return sourceDigest(source); } -/** Every component the program names, with the form each element is written in. */ -function elements( - segments: readonly Segment[], - found: ProgramComponentRef[], -): ProgramComponentRef[] { - for (const segment of segments) { - if (segment.type === "component") { - found.push({ name: segment.name, form: segment.selfClosing ? "self-closing" : "paired" }); - elements(segment.children, found); - } - } - return found; -} - /** * The elements a candidate program writes, or none when it will not parse. * diff --git a/packages/core/src/program-identity.ts b/packages/core/src/program-identity.ts index 605a8bba3..a6b3a0f7d 100644 --- a/packages/core/src/program-identity.ts +++ b/packages/core/src/program-identity.ts @@ -45,7 +45,7 @@ import type { Operation } from "effection"; import type { ImportedDefinition } from "./components/import-authority.ts"; -import type { ComponentSelection, Json, JsonObject } from "./types.ts"; +import type { ComponentSelection, Json, JsonObject, Segment } from "./types.ts"; /** The authored forms an element is written in. */ export type ProgramComponentForm = "self-closing" | "paired"; @@ -103,12 +103,60 @@ export interface ProgramComponent { readonly identity: ProgramIdentity; } -/** One element a program names, before its identity has been resolved. */ +/** + * One element a program names, before its identity has been resolved. + * + * The offset is where the element was written in the program's own source. It + * is what makes an occurrence an occurrence: a program writing one name twice + * resolves it twice, and the two answers are two settlements rather than one + * that the second overwrites. It is derived from the retained source every time + * it is needed and never retained itself. + */ export interface ProgramComponentRef { readonly name: string; readonly form: ProgramComponentForm; + readonly offset: number; } +/** + * What canonical execution settled for one authored occurrence. + * + * An unresolved occurrence is a settled outcome like any other. Leaving it out + * would let the element fall through to the ordinary open chain and be answered + * by a lookup reconciliation never made. + */ +export type ProgramSettlement = + | { readonly kind: "resolved"; readonly name: string; readonly definition: ImportedDefinition } + | { readonly kind: "unresolved"; readonly name: string }; + +/** The durable result an already-authorized program occurrence records. */ +export const SETTLED_IMPORT = "program-occurrence"; + +/** + * The record one settled program import writes, read back as hostile data. + * + * A closed shape of exactly two members: the tag saying which protocol wrote + * it, and the name it was written for. A journal is data, so a restored value + * that is missing a member, carries one nobody wrote, spells one differently, + * or names another component is not this record — and nothing is invoked on the + * strength of it. + */ +export function readSettledImport(value: unknown, name: string): boolean { + if (typeof value !== "object" || value === null || Array.isArray(value)) { + return false; + } + const record = value as JsonObject; + const own = Object.keys(record); + if (own.length !== 2 || !Object.hasOwn(record, "settled") || !Object.hasOwn(record, "name")) { + return false; + } + return record.settled === SETTLED_IMPORT && record.name === name; +} + +/** What a settled program import that cannot be read as one says. */ +export const UNREADABLE_SETTLED_IMPORT = + "the retained record of this program component's import cannot be read as one."; + /** * What canonical resolution settled for one name: its identity, and the copy of * the answer core will invoke. @@ -119,6 +167,8 @@ export interface ProgramComponentRef { * for the invocation. */ export interface ResolvedProgramComponent extends ProgramComponent { + /** Where this occurrence was written in the program's own source. */ + readonly offset: number; /** * Core's own copy of the answer, or `undefined` when nothing was resolved or * the answer could not be copied. @@ -180,6 +230,25 @@ export const ANSWER_CHANGED = " cannot evaluate a program whose component answer was changed after the provider " + "that supplied it claimed it."; +/** Every component the program names, with the form each element is written in. */ +export function elements( + segments: readonly Segment[], + found: ProgramComponentRef[], +): ProgramComponentRef[] { + for (const segment of segments) { + if (segment.type === "component") { + found.push({ + name: segment.name, + form: segment.selfClosing ? "self-closing" : "paired", + // Where this occurrence is, so two of one name stay two. + offset: segment.position?.offset ?? -1, + }); + elements(segment.children, found); + } + } + return found; +} + /** The canonical identity of what a selection chose. */ export function selectionIdentity(selection: ComponentSelection): ProgramIdentity { switch (selection.kind) { diff --git a/packages/core/tests/program-evaluation.test.ts b/packages/core/tests/program-evaluation.test.ts index 77b0e708f..149ab4405 100644 --- a/packages/core/tests/program-evaluation.test.ts +++ b/packages/core/tests/program-evaluation.test.ts @@ -779,12 +779,17 @@ function openProvider(options: { then?: string; /** Which lookup starts answering `then`. */ switchAfter?: number; + /** Answer each successive lookup from this list, under one key each. */ + sequence?: readonly string[]; + /** Delegate to nothing for this many lookups, then answer `then`. */ + absentFor?: number; }): () => Operation { return function* install(): Operation { const claim = yield* useImportProvider( options.identity ?? { origin: "test/open", revision: options.mark }, ); let answered = 0; + const sequenced = new Map(); const first = openImplementation(options.mark); const later = options.then === undefined ? first : openImplementation(options.then); yield* Component.around( @@ -795,6 +800,24 @@ function openProvider(options: { return yield* next(name, position); } answered += 1; + if (options.absentFor !== undefined) { + // Delegates while the site is resolved and reconciled, and would + // answer only on a lookup that must never happen. + if (answered <= options.absentFor) { + return yield* next(name, position); + } + const late = openImplementation(options.then ?? "late"); + return claim(name, "Open", late); + } + if (options.sequence !== undefined) { + // One implementation per occurrence, each under its own stable key, + // answered in the order the program writes them. + const index = Math.min(answered, options.sequence.length) - 1; + const mark = options.sequence[index] ?? options.mark; + const supplied = sequenced.get(mark) ?? openImplementation(mark); + sequenced.set(mark, supplied); + return claim(name, `Open:${mark}`, supplied); + } const definition = answered <= (options.switchAfter ?? 1) ? first : later; return options.identity === null ? definition : claim(name, "Open", definition); }, @@ -1316,3 +1339,110 @@ describe("Tier PE — a provider's stated identity is captured once", () => { approved = APPROVED; }); }); + +describe("Tier PE — a settlement belongs to an occurrence", () => { + it("PE40 invokes A then B for two occurrences of one name", function* () { + opened.length = 0; + consulted.length = 0; + approved = "\n\n\n"; + + // Two occurrences, resolved twice in each pass. The provider answers A for + // the first and B for the second, under distinct stable keys. + const attempt = yield* run(PROBING_DOCUMENT, { + install: openProvider({ + mark: "A", + sequence: ["A", "B", "A", "B"], + identity: { origin: "test/open", revision: "1" }, + }), + }); + + expect(attempt.failure).toBeUndefined(); + // A then B, in the order the program writes them — not A twice, which is + // what a name-keyed settlement produces. + expect(opened).toEqual(["A", "B"]); + const named = decision(admissions(attempt.events)[0]!).named as Record[]; + expect(named.map((entry) => (entry.identity as Record).key)).toEqual([ + "Open:A", + "Open:B", + ]); + approved = APPROVED; + }); + + it("PE41 keeps an unresolved occurrence from reaching a later lookup", function* () { + opened.length = 0; + consulted.length = 0; + approved = OPEN_PROGRAM; + + // The provider delegates for the admission's resolution and for + // reconciliation, and would answer on a third lookup. + const attempt = yield* run(PROBING_DOCUMENT, { + install: openProvider({ + mark: "A", + absentFor: 2, + then: "late", + identity: { origin: "test/open", revision: "1" }, + }), + }); + + // Settled as unresolved, so the element says so rather than falling + // through to the open chain. + expect(attempt.failure ?? attempt.output ?? "").toContain("Cannot resolve component: Open"); + // The third lookup never happened, so the late implementation never ran. + expect(consulted.filter((name) => name === "Open")).toHaveLength(2); + expect(opened).toEqual([]); + approved = APPROVED; + }); + + it("PE42 refuses a corrupted settled-import record before invoking anything", function* () { + opened.length = 0; + approved = OPEN_PROGRAM; + + const first = yield* run(PROBING_DOCUMENT, { + install: openProvider({ mark: "A", identity: { origin: "test/open", revision: "A" } }), + }); + expect(first.failure).toBeUndefined(); + expect(opened).toEqual(["A"]); + + /** + * The history an interruption right after the settled import leaves, with + * that record rewritten. + * + * Truncated, because a completed journal replays as a terminal result and + * the element is never re-entered — which would prove nothing about what a + * continuation reads. + */ + const rewrite = (change: (record: Record) => Json): DurableEvent[] => { + const at = importsOf(first.events, "Open")[0]!; + return first.events.slice(0, at + 1).map((event, index) => { + if (index !== at || event.type !== "yield" || event.result.status !== "ok") { + return event; + } + const record = event.result.value as Record; + return { ...event, result: { ...event.result, value: change({ ...record }) } }; + }); + }; + + const corruptions: Record) => Json> = { + "a missing member": ({ name: _name, ...rest }) => rest, + "an additional member": (record) => ({ ...record, extra: 1 }), + "a mistyped member": (record) => ({ ...record, name: 7 }), + "an unknown tag": (record) => ({ ...record, settled: "something-else" }), + "another component's name": (record) => ({ ...record, name: "Probe" }), + "no record at all": () => "settled", + }; + + for (const [what, change] of Object.entries(corruptions)) { + opened.length = 0; + const replayed = yield* run(PROBING_DOCUMENT, { + stream: new InMemoryStream(rewrite(change)), + install: openProvider({ mark: "A", identity: { origin: "test/open", revision: "A" } }), + }); + + expect(`${what}: ${replayed.failure ?? replayed.output ?? ""}`).toContain( + "cannot be read as one", + ); + expect([what, opened]).toEqual([what, []]); + } + approved = APPROVED; + }); +}); diff --git a/specs/executable-mdx-spec.md b/specs/executable-mdx-spec.md index 8b2db1fca..6646f4b8a 100644 --- a/specs/executable-mdx-spec.md +++ b/specs/executable-mdx-spec.md @@ -4892,18 +4892,36 @@ admission's own resolution through `Component.resolveProgramSite()`, and answering it dishonestly refuses the evaluation rather than widening it, because the canonical comparison is what decides. -**The comparison and the invocation are one decision.** The answers that passed -the comparison are the answers the program invokes. Canonical execution keeps -its own copy of each, taken when the answer was witnessed, and hands those -copies to the program's expansion: an element whose name is among them does not -reach `Component.importComponent` at all. The chain is entered twice for one -evaluation — once for the admission's resolution, once for reconciliation — and -never again, so there is no third lookup for a provider to answer differently. +**The comparison and the invocation are one decision, and it is made per +occurrence.** Each element the program writes is resolved on its own, in the +order the admission retained, and the answer that passed the comparison is bound +to *that* element rather than to its name. A program writing one name twice +resolved it twice: if a provider supplies two implementations under two stable +keys, the two elements invoke the two implementations, in order. An element +settled as **unresolved** is settled too — it reports the ordinary unresolved +failure and never falls through to the open chain, where a later lookup could +answer what reconciliation did not. + +Canonical execution keeps its own copy of each answer, taken when it was +witnessed, and hands those copies to the program's expansion: a settled element +does not reach `Component.importComponent` at all. The chain is entered twice +for one evaluation — once for the admission's resolution, once for +reconciliation — and never again, so there is no third lookup for a provider to +answer differently. + +Settlements belong to the program's own parsed body. A component the program +invokes expands its own bytes, which nothing reconciled, so it imports exactly +as it always did. The authored element still makes its ordinary durable import: one -`import_component` record per occurrence, written after the admission and -restored from the answer already authorized, with identity-domain and form -selection recorded from that same answer. It rediscovers no authority. +`import_component` record per settled occurrence, written after the admission. +Its result is a closed shape of exactly two members — `settled`, holding +`"program-occurrence"`, and `name` — and a continuation parses it as the hostile +replay data it is before anything is invoked. A record missing a member, +carrying one nobody wrote, spelling one differently, holding an unknown tag, or +naming another component is refused, and the component does not run. Identity +domain and form selection are recorded from the already-authorized answer; the +record rediscovers no authority. **The retained record is hostile data.** A journal is a file, and replay hands back whatever it holds. The admitted and refused decisions and every nested @@ -12437,6 +12455,9 @@ forms sit beside the restricted one. | PE37 | The closed positive control | With no replacement, the declared answer is what the program invokes and what the admission retains | | PE38 | Identity captured once | Editing the stated identity after the claimant exists does not change what its claims are marked with | | PE39 | An alternating origin | A getter cannot answer the duplicate check and the claim differently | +| PE40 | Two occurrences of one name | A program writing `` against a provider supplying A then B under distinct keys invokes A then B, not A twice | +| PE41 | An unresolved occurrence | A provider that delegates through both resolution passes and would answer a third leaves the element unresolved: the third lookup never happens and the late implementation never runs | +| PE42 | A corrupted settled import | A missing, additional, mistyped or unknown member in the settled `import_component` result, and one naming another component, each refuse before the component is invoked | | EP1–EP8 | Run profile | The ordinary run profile declares ``; the forms, props, refusals and catalog entry hold through the real binary, and a program cannot reach ``'s private components | | WGAC17–WGAC20 | Workflow profile | A complete program records `evaluate_program` and no `generated_xmd`; the three forms cannot be combined; complete-program support is not reachable through `source` | From 91e266d156bf686aee2b5323ef4656cc734f9847 Mon Sep 17 00:00:00 2001 From: Taras Mankovski Date: Fri, 4 Sep 2026 12:06:39 -0400 Subject: [PATCH 6/7] =?UTF-8?q?=F0=9F=9B=A1=EF=B8=8F=20Detach=20a=20settle?= =?UTF-8?q?d=20import=20before=20reading=20it=20(#713)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `readSettledImport()` cast the restored value and inspected it directly, so a value that answers differently on a second read decided what a continuation invoked. It now detaches the value into plain JSON under a failure boundary and inspects only that copy: a member read once cannot answer twice, and a value that will not detach is simply not the record. The admission's own restored result is detached under the same boundary for the same reason. The prose said one settled `import_component` per settled occurrence. It is one per *resolved* occurrence: an unresolved one loads no component and records no such event, which PE41 now asserts. A retained value that refuses to be read at all — a Proxy trapping `ownKeys` or a descriptor, a throwing accessor, a cycle — never reaches this parser. The run's own retention check reads every retained result before the document body starts and refuses there, in the journal's own words. That is a persistence boundary rather than this one, so the evidence proves what this boundary can promise: the component is never invoked. --- architecture.md | 2 +- packages/core/src/expand.ts | 31 +++- packages/core/src/program-evaluation.ts | 11 +- packages/core/src/program-identity.ts | 23 ++- .../core/tests/program-evaluation.test.ts | 144 +++++++++++++++++- specs/executable-mdx-spec.md | 22 ++- 6 files changed, 211 insertions(+), 22 deletions(-) diff --git a/architecture.md b/architecture.md index 49acf519d..28591be94 100644 --- a/architecture.md +++ b/architecture.md @@ -3899,7 +3899,7 @@ Status is measured against main. | workflow Agent session | a workflow document's `` runs under a profile the host attaches only for a live or partial run: an empty host-owned working directory instead of any Workspace, checkout or caller path, no MCP servers, an empty requested native tool set, and `deny-all` with a permission path that denies every native request and fails the turn that asked without reaching the public permission chain. Within a run a session is identified by the Agent/Session expansion identity the engine derived — the authored name is descriptive, so two sibling `` elements are two sessions — routed inside a placement bound to its element and good for one use, so a kept placement cannot be substituted for the next. The conversation is retained as a row in the run's own database with the provider, resolved agent command and policy fingerprint beside it as compatibility attributes. The order is placement, the backend's acceptance of the session's first turn, the provider's canonical tagged assertion, then the mapping commit — and only then is anything that turn produced exposed. A placement is inert: it creates no provider session and writes no row. Occupancy of a provider key is not an assertion, and a record held for a first turn nobody accepted asserts nothing at all; the pre-commit window reconciles only from exactly one canonical assertion, and a missing, conflicting, replaced or ambiguous assertion is one explicit refusal that starts no replacement. Deleting a run removes the row with the run and the provider-session directory beside it, and reports the categories. The profile selects ACP-only capability explicitly — no native-launch advertisement and no client-native attachment advertisement — rather than inheriting the provider package's ordinary-run sets by omission, and it supplies no machine session coordinator, construction-route store or executable observer: a workflow session belongs to a run, and the machine-wide account describes a different thing entirely | built on the #302 stack, with the explicit ACP-only selection from #561; the portable proof that an adapter honours an empty tool set is tracked by #496 and does not widen the ceiling | | generated-XMD admission | admits one Agent-generated fragment through the trusted-host seam: host policy is a `read` table and a `write` table of exact pinned identities, each carrying the authored forms it is admitted for, and an authored `allow` selects a canonical non-empty subset of the closed classes — omitted means `read`. The complete source is preflighted inside one `generated_xmd` durable effect before its first generated effect; only the pinned identity the selected classes hold for that name **and** that form executes; and the admitted source, class selection, selected root, every selected entry with its forms, the identity and form of each element named, and the normalized request policy are retained in that effect's own result — so a continuation restores the decision without reading the current candidate and expands only the retained source. The roots are an as-of-admission retained basis checked by membership — the run's own later root publications and an advanced retained current root pass, while a lost admission root or lost selected root refuses — and every non-root term is checked exactly, refusing a run whose classes, identities, forms or requests have moved. The admission and every nested generated effect are offered inline by the owning expansion in authored order, so a partial continuation restores each completed one without another live execution. Each admitted effect is retained by its own ordinary record, and a read's value is collected while a mutation's is not | built on the #369 stack, continuation basis amended by #589; core owns the mechanics and the workflow policy wrapper is internal | | `` | the restricted generated-fragment form of ``, which the workflow host declares beside the complete-program forms below. The form an authored document writes where an observation should happen. The host does not register it: it **declares** it to the execution through `ExecutionInstallation.components`, captured before any installation runs, and canonical execution calls its factory once for that attachment with the claimant it minted and registers what comes back. Registration provides availability only — a name a trusted document may write — and carries none of the authority. It takes a `source` string and one optional `allow` array selecting a non-empty duplicate-free subset of the closed effect classes `read` and `write` — omitted means `read` — refuses paired content, and cannot be combined with `program`, with paired content, or with `props`; each of those refuses before the durable name is claimed and before the candidate is read. It declares no `returns` and answers with a detached value — `{ observations: [{ name, value }], output }`, each admitted read's own returned value under the name the fragment invoked it by, in invocation order, with whatever the fragment rendered under `output` rather than instead of them, and the pinned identity that produced one left in the retained admission rather than copied here — so an admitted ``, which renders nothing at all, still reaches the document. An admitted mutation contributes no entry and no receipt, so a write-only fragment binds `{ observations: [], output: "" }`; `as` is valid for every selection and binds that same shape. An ordinary `as` captures that value by reference, and an authored `` renders it into the next ``: deciding how a value becomes text is the document's. Every ceiling comes from values the host captured at installation — the run's retained roots and its authoritative current root read from the run's own storage at invocation, as-of-admission provenance a continuation holds by membership so the run's own later publications and an advanced retained current root invalidate nothing, core's pinned self-closing `` read, the write table of core's paired ``, this package's lexical `` built from the definition the ordinary registration owns, and core's self-closing ``, and `` only when the captured request ceiling is non-empty — and no prop, binding, context or middleware return value supplies or widens one. `allow` selects among those tables and adds nothing to them; approval, when a workflow needs one, is authored control flow before the element. Its durable operation is named through that claimant, on the exact invocation the engine handed it and in that invocation's own frame — not from a context a document could rebind, a contextual Api answer, a definition, or a registry answer. Generated source never resolves through the registration: the evaluator consults only its own closed table of pinned identities. It is deliberately not wrapped in `printErrors`, so a refused fragment stops the authored loop rather than becoming text the next turn could read as a read that happened | built on the #302 stack, extended by #369 | -| complete-program evaluation | admits one complete XMD root and evaluates it at the authored `` site, in the current execution: its lifecycle, journal, cancellation scope, contextual providers, working directory or Workspace, and the authority in force where the element was written. There is no child process, host profile, root lifecycle or journal of its own — program source requests behavior and grants none. The root's own frontmatter, imports, metadata, props schema, `returns` and `` selection decide what it does; a text root renders its selected output or binds it as a string under `as`, a value root requires `as` and binds its schema-validated result, and root props are the explicit `props` object defaulting to `{}` rather than the caller's own. Ordinary non-props caller bindings are visible as a copy, so nothing the program binds escapes, and relative paths resolve from the evaluation site's source origin. The producer's temporary authority is not inherited: an enclosing declaration's private closure does not cross, so a private name written in a program resolves to nothing, while imports and capabilities genuinely present at the site remain. Canonical execution answers `Component.expandProgram()` from the frame it already holds, which is what makes site authority a fact about the frame rather than something a prop, binding or middleware return value could supply. Which definition each name the program writes resolves to is part of what the site granted, so the admission retains it and canonical execution settles it again before the first program effect — through a resolver the execution builds and carries on the expansion authority, reachable from no document, component, contextual Api answer or middleware return value. That resolver goes through the ordinary import chain under a resolution-only terminal — it selects and loads what an authored import would, invokes nothing and journals nothing — because the answer the chain returns is the answer that runs, while an independent second resolution would describe a definition a provider replaced. `evaluate_program` therefore commits before the program's own ordinary import record. The site's existing closed tiers still decide: a bundled or declared name is answered by the tier that closed it, and a provider-supplied answer is witnessed rather than issued, so it authorizes nothing for such a name. Structural syntax resolves directly to its own identity and never reaches component import. The answers that passed the comparison are handed to the program's expansion bound to the authored occurrence each was settled for, not to the name, so a program writing one name twice invokes the two implementations its two elements resolved to rather than the first one twice. An unresolved occurrence is settled too, and reports the ordinary unresolved failure instead of falling through to the open chain. Settlements belong to the program's own parsed body and are not inherited by a component it invokes. The chain is therefore entered twice for one evaluation and never a third time; the authored element still records one ordinary import after the admission, whose result is a closed two-member shape parsed as hostile replay data before anything is invoked, with identity-domain and form selection taken from the already-authorized answer | built by #713 on the #722 `` contract; core owns the mechanics and every host that offers `` declares them | +| complete-program evaluation | admits one complete XMD root and evaluates it at the authored `` site, in the current execution: its lifecycle, journal, cancellation scope, contextual providers, working directory or Workspace, and the authority in force where the element was written. There is no child process, host profile, root lifecycle or journal of its own — program source requests behavior and grants none. The root's own frontmatter, imports, metadata, props schema, `returns` and `` selection decide what it does; a text root renders its selected output or binds it as a string under `as`, a value root requires `as` and binds its schema-validated result, and root props are the explicit `props` object defaulting to `{}` rather than the caller's own. Ordinary non-props caller bindings are visible as a copy, so nothing the program binds escapes, and relative paths resolve from the evaluation site's source origin. The producer's temporary authority is not inherited: an enclosing declaration's private closure does not cross, so a private name written in a program resolves to nothing, while imports and capabilities genuinely present at the site remain. Canonical execution answers `Component.expandProgram()` from the frame it already holds, which is what makes site authority a fact about the frame rather than something a prop, binding or middleware return value could supply. Which definition each name the program writes resolves to is part of what the site granted, so the admission retains it and canonical execution settles it again before the first program effect — through a resolver the execution builds and carries on the expansion authority, reachable from no document, component, contextual Api answer or middleware return value. That resolver goes through the ordinary import chain under a resolution-only terminal — it selects and loads what an authored import would, invokes nothing and journals nothing — because the answer the chain returns is the answer that runs, while an independent second resolution would describe a definition a provider replaced. `evaluate_program` therefore commits before the program's own ordinary import record. The site's existing closed tiers still decide: a bundled or declared name is answered by the tier that closed it, and a provider-supplied answer is witnessed rather than issued, so it authorizes nothing for such a name. Structural syntax resolves directly to its own identity and never reaches component import. The answers that passed the comparison are handed to the program's expansion bound to the authored occurrence each was settled for, not to the name, so a program writing one name twice invokes the two implementations its two elements resolved to rather than the first one twice. An unresolved occurrence is settled too, and reports the ordinary unresolved failure instead of falling through to the open chain. Settlements belong to the program's own parsed body and are not inherited by a component it invokes. The chain is therefore entered twice for one evaluation and never a third time; a resolved occurrence still records one ordinary import after the admission — an unresolved one loads nothing and records nothing — whose result is a closed two-member shape detached into plain JSON under a failure boundary, with only that copy inspected — so a member answering differently on a second read decides nothing, and a value that will not detach is not the record. A retained value that refuses to be read at all is refused earlier, by the run's own retention check, before the document body starts, with identity-domain and form selection taken from the already-authorized answer | built by #713 on the #722 `` contract; core owns the mechanics and every host that offers `` declares them | | `` complete-program forms | the two ways an author names a program at a composition site: paired content, which renders a source-producing component into a private program buffer, and `program={value}`, which supplies complete-program source directly. Optional `props={object}` is the root's props input and is valid only for these forms. The two are mutually exclusive and one is required; every ambiguous or misplaced combination refuses before program content is produced and before any program effect runs, so an ambiguous element does not even render its producer. Paired content is a private buffer rather than a quotation — an executable component written there executes under ordinary XMD semantics while producing the text sent to admission — and the element removes only the framing its own wrapper contributed: the line break after the opening tag with the indentation following it, the line break before the closing tag with the indentation preceding it, and the indentation those lines share. Interior bytes are untouched, so a sole `` producer's trailing newline survives and paired evaluation and `program={plan}` admit the same bytes and the same content digest. `program={value}` is exact supplied source and is neither trimmed, dedented nor newline-adjusted | built by #713 | | `evaluate_program` durable effect | the complete-program admission, recorded once before the first program effect and never as a widening of `generated_xmd` — the two are decisions about different kinds of source. `description.input` carries the source digest, the explicit props, the evaluation-site source origin and whether the result is captured; the result carries the exact admitted source, the root mode, the components the program names with their authored forms **and the identity canonical resolution settled for each name at that site**, as a closed tagged record whose tag decides its members, those same terms and the validated root props — or, for a refusal, the refused class alone and nothing of the source. Both results are closed shapes, and so is every record nested in them: a missing, additional or misspelled member at any depth is a record this evaluation did not write. Restored, it must also agree with itself before anything expands — the source hashes to the digest, reparsing produces the retained mode and a valid body structure, the retained elements are the ones that source writes in that order and those forms, and the supplied props validate to the retained validated props — and a record failing any of those is refused as unreadable with neither program performing an effect. A continuation whose site now answers one of those names with a different identity or admits it in a different form is refused before either implementation runs, which is a different answer from an unreadable record: the journal is intact and the environment moved. A partial continuation expands the retained source and restores the nested effects that already committed, repeating no planning; a changed program at the same evaluation occurrence is stale input and refuses before either the current or the retained source runs, so neither silently wins. The source digest identifies the program artifact and the authored element and loop iteration identify one execution of it, so two occurrences of one artifact have separate nested effect identities and the digest deduplicates neither | built by #713 | | generated mutation proposals | lets an Agent propose constrained executable changes that a separate admission then performs against the run's own Workspace | built on the #369 and #567 stacks, with directory creation added by #643: the standard Deno profile's write table is core's paired `File:write`, the paired `@executablemd/workflow/composition/dir-v2#Dir` and core's self-closing `File.Delete`, in that retained order and followed by any host extension. `allow={["write"]}` intentionally authorizes Dir's persistent recursive directory creation; its versioned identity makes every continuation retained under the former non-mutating Dir identity refuse before generated execution. Admitted mutations run as the ordinary components they are through the run's effect transactions, and the evaluator adds no mutation API or receipt. Approval is authored control flow before the write-enabled element. Local Git, Git-host, issue, process, execution, credential and external-write effects are outside the class | diff --git a/packages/core/src/expand.ts b/packages/core/src/expand.ts index 803a6e7dd..fc8924ca9 100644 --- a/packages/core/src/expand.ts +++ b/packages/core/src/expand.ts @@ -2411,13 +2411,30 @@ function* expandComponent( // The authored element still makes its ordinary durable import, and a // continuation reads that record as the hostile data it is before anything // is invoked. A record this protocol did not write authorizes nothing. - const record = yield createDurableOperation( - { type: "import_component", name, ...sourceDescription(position) }, - // deno-lint-ignore require-yield - function* (): Operation { - return { settled: SETTLED_IMPORT, name }; - }, - ); + // Restoring the record is itself a read of journal data. A value that + // refuses to be read — a proxy that throws from `ownKeys`, an accessor that + // throws, something no JSON holds — can refuse before this frame ever sees + // it, and what it throws is its own text. So the restore and the parse are + // one boundary: either this run is holding the record it wrote, or it is + // not, and the one thing said about it is the same either way. + // + // A durability failure is not that. The journal no longer describing this + // run is a fact about the run, not about this record, and it travels. + let record: unknown; + try { + record = yield createDurableOperation( + { type: "import_component", name, ...sourceDescription(position) }, + // deno-lint-ignore require-yield + function* (): Operation { + return { settled: SETTLED_IMPORT, name }; + }, + ); + } catch (error) { + if (durabilityFailure(error) !== undefined) { + throw error; + } + throw new Error(UNREADABLE_SETTLED_IMPORT); + } if (!readSettledImport(record, name)) { throw new Error(UNREADABLE_SETTLED_IMPORT); } diff --git a/packages/core/src/program-evaluation.ts b/packages/core/src/program-evaluation.ts index 0bdf0c356..9bd438b44 100644 --- a/packages/core/src/program-evaluation.ts +++ b/packages/core/src/program-evaluation.ts @@ -501,7 +501,16 @@ export function* evaluateProgram(request: ProgramEvaluationRequest): Operation

admitProgram(request, terms, resolved), ); - const decided = readAdmission(parseJson(stored)); + // Detached under a boundary for the reason every other read of this record + // is: a restored value can refuse to be read at all, and a value that will + // not become plain JSON is not an admission. + let restoredRecord: Json; + try { + restoredRecord = parseJson(stored); + } catch { + throw new ProgramEvaluationError(UNREADABLE); + } + const decided = readAdmission(restoredRecord); if (decided === undefined) { throw new ProgramEvaluationError(UNREADABLE); } diff --git a/packages/core/src/program-identity.ts b/packages/core/src/program-identity.ts index a6b3a0f7d..888ac7233 100644 --- a/packages/core/src/program-identity.ts +++ b/packages/core/src/program-identity.ts @@ -45,6 +45,7 @@ import type { Operation } from "effection"; import type { ImportedDefinition } from "./components/import-authority.ts"; +import { isJsonObject, parseJson } from "./json.ts"; import type { ComponentSelection, Json, JsonObject, Segment } from "./types.ts"; /** The authored forms an element is written in. */ @@ -142,15 +143,27 @@ export const SETTLED_IMPORT = "program-occurrence"; * strength of it. */ export function readSettledImport(value: unknown, name: string): boolean { - if (typeof value !== "object" || value === null || Array.isArray(value)) { + // Detached first, under a boundary, and only then inspected. The restored + // value is whatever the journal holds: a Proxy can refuse `ownKeys`, answer a + // property differently each time it is read, or throw from a descriptor, and + // a value can be circular or hold something no JSON has. Reading such a value + // to decide anything is reading a value that decides for itself, so the only + // thing asked of it is whether it will become plain JSON — and every way of + // saying no is one answer, which is that this is not the record. + let detached: Json; + try { + detached = parseJson(value); + } catch { return false; } - const record = value as JsonObject; - const own = Object.keys(record); - if (own.length !== 2 || !Object.hasOwn(record, "settled") || !Object.hasOwn(record, "name")) { + if (!isJsonObject(detached)) { + return false; + } + const own = Object.keys(detached); + if (own.length !== 2 || !Object.hasOwn(detached, "settled") || !Object.hasOwn(detached, "name")) { return false; } - return record.settled === SETTLED_IMPORT && record.name === name; + return detached.settled === SETTLED_IMPORT && detached.name === name; } /** What a settled program import that cannot be read as one says. */ diff --git a/packages/core/tests/program-evaluation.test.ts b/packages/core/tests/program-evaluation.test.ts index 149ab4405..9a8c6c8b4 100644 --- a/packages/core/tests/program-evaluation.test.ts +++ b/packages/core/tests/program-evaluation.test.ts @@ -30,7 +30,7 @@ import { scoped } from "effection"; import type { Operation } from "effection"; import { createDurableOperation, InMemoryStream } from "@executablemd/durable-streams"; -import type { DurableEvent } from "@executablemd/durable-streams"; +import type { DurableEvent, DurableStream } from "@executablemd/durable-streams"; import { Component } from "../src/component-api.ts"; import { executeInstalled, programEvaluationComponents, sourceDigest } from "../host.ts"; @@ -132,7 +132,7 @@ function probeComponents(origin = "test/probe"): readonly IdentityComponent[] { function run( source: string, options: { - stream?: InMemoryStream; + stream?: DurableStream; /** The origin `` is registered under, so a site can be moved. */ probeOrigin?: string; declarations?: readonly DeclaredMarkdownComponent[]; @@ -142,7 +142,7 @@ function run( } = {}, ): Operation { return scoped(function* () { - const stream = options.stream ?? new InMemoryStream(); + const stream: DurableStream = options.stream ?? new InMemoryStream(); // An installation that refuses throws out of `executeInstalled` rather than // settling a document result, so both endings are reported the same way. let execution; @@ -1390,6 +1390,9 @@ describe("Tier PE — a settlement belongs to an occurrence", () => { // The third lookup never happened, so the late implementation never ran. expect(consulted.filter((name) => name === "Open")).toHaveLength(2); expect(opened).toEqual([]); + // And nothing was loaded: an unresolved occurrence imports no component, so + // it records no import of one. + expect(importsOf(attempt.events, "Open")).toHaveLength(0); approved = APPROVED; }); @@ -1446,3 +1449,138 @@ describe("Tier PE — a settlement belongs to an occurrence", () => { approved = APPROVED; }); }); + +/** + * A journal that hands its events back exactly as they were put in. + * + * `InMemoryStream` structured-clones what it stores, which is the right thing + * for a journal and the wrong thing for this case: a value that refuses to be + * read would be rejected by the clone, long before the parser under test sees + * it. This keeps the reference, so what a continuation reads is the hostile + * value itself. + */ +function unclonedStream(events: readonly DurableEvent[]): DurableStream { + const held = [...events]; + return { + // deno-lint-ignore require-yield + *readAll(): Operation { + return held; + }, + // deno-lint-ignore require-yield + *append(event: DurableEvent): Operation { + held.push(event); + }, + }; +} + +/** What a hostile retained value tried to smuggle into a diagnostic. */ +const PLANTED = "planted-by-the-journal-a3f9"; + +/** The tag a settled program occurrence records. */ +const SETTLED_IMPORT_TAG = "program-occurrence"; + +describe("Tier PE — a settled import that will not be read", () => { + /** The interrupted history, with the settled import's result replaced. */ + function* holding(value: unknown): Operation<{ events: DurableEvent[]; at: number }> { + opened.length = 0; + approved = OPEN_PROGRAM; + const first = yield* run(PROBING_DOCUMENT, { + install: openProvider({ mark: "A", identity: { origin: "test/open", revision: "A" } }), + }); + expect(first.failure).toBeUndefined(); + expect(opened).toEqual(["A"]); + const at = importsOf(first.events, "Open")[0]!; + return { + at, + events: first.events.slice(0, at + 1).map((event, index) => { + if (index !== at || event.type !== "yield" || event.result.status !== "ok") { + return event; + } + return { ...event, result: { ...event.result, value: value as Json } }; + }), + }; + } + + function resume(events: DurableEvent[]): Operation { + opened.length = 0; + return run(PROBING_DOCUMENT, { + stream: unclonedStream(events), + install: openProvider({ mark: "A", identity: { origin: "test/open", revision: "A" } }), + }); + } + + it("PE43 refuses a value the parser can reach but not read, planting nothing", function* () { + // A member no JSON holds. It survives the journal's own retention checks + // and reaches this boundary, which answers with its one fixed sentence. + const planted = { settled: SETTLED_IMPORT_TAG, name: () => PLANTED }; + const replayed = yield* resume((yield* holding(planted)).events); + + expect(replayed.failure ?? replayed.output ?? "").toContain("cannot be read as one"); + expect(replayed.failure ?? replayed.output ?? "").not.toContain(PLANTED); + expect(opened).toEqual([]); + approved = APPROVED; + }); + + it("PE44 reads a detached copy, so an alternating answer decides nothing", function* () { + // Read once into detached JSON, so a member that answers differently on a + // second read has no second read to answer. The first answer is the record, + // and it is the right one. + let reads = 0; + const alternating = { + settled: SETTLED_IMPORT_TAG, + get name(): string { + reads += 1; + return reads === 1 ? "Open" : PLANTED; + }, + }; + const replayed = yield* resume((yield* holding(alternating)).events); + + expect(replayed.failure).toBeUndefined(); + expect(replayed.output).toContain("open A"); + expect(opened).toEqual(["A"]); + approved = APPROVED; + }); + + it("PE45 refuses a value the journal itself will not retain, invoking nothing", function* () { + // These never reach this boundary: the run's own retention check reads + // every retained result before the document body starts, and a value that + // refuses there is refused there. What this proves is the part that is + // this boundary's to promise — the component is never invoked. + const hostile: Record = { + "a proxy refusing ownKeys": new Proxy( + { settled: SETTLED_IMPORT_TAG, name: "Open" }, + { + ownKeys() { + throw new Error(PLANTED); + }, + }, + ), + "a proxy refusing a descriptor": new Proxy( + { settled: SETTLED_IMPORT_TAG, name: "Open" }, + { + getOwnPropertyDescriptor() { + throw new Error(PLANTED); + }, + }, + ), + "a throwing accessor": { + settled: SETTLED_IMPORT_TAG, + get name(): string { + throw new Error(PLANTED); + }, + }, + "a circular value": (() => { + const circular: Record = { settled: SETTLED_IMPORT_TAG, name: "Open" }; + circular.self = circular; + return circular; + })(), + }; + + for (const [what, value] of Object.entries(hostile)) { + const replayed = yield* resume((yield* holding(value)).events); + expect([what, replayed.output ?? ""]).toEqual([what, ""]); + expect([what, opened]).toEqual([what, []]); + } + approved = APPROVED; + }); +}); diff --git a/specs/executable-mdx-spec.md b/specs/executable-mdx-spec.md index 6646f4b8a..605a3dc57 100644 --- a/specs/executable-mdx-spec.md +++ b/specs/executable-mdx-spec.md @@ -4913,13 +4913,22 @@ Settlements belong to the program's own parsed body. A component the program invokes expands its own bytes, which nothing reconciled, so it imports exactly as it always did. -The authored element still makes its ordinary durable import: one -`import_component` record per settled occurrence, written after the admission. +A **resolved** occurrence still makes its ordinary durable import: one +`import_component` record per resolved occurrence, written after the admission. +An unresolved occurrence loads no component and records no such event — there is +nothing to import. Its result is a closed shape of exactly two members — `settled`, holding `"program-occurrence"`, and `name` — and a continuation parses it as the hostile -replay data it is before anything is invoked. A record missing a member, +replay data it is before anything is invoked: the value is detached into plain +JSON under a failure boundary and only that copy is inspected, so a member that +answers differently on a second read decides nothing and a value that will not +detach is simply not the record. A record missing a member, carrying one nobody wrote, spelling one differently, holding an unknown tag, or -naming another component is refused, and the component does not run. Identity +naming another component is refused, and the component does not run. A retained +value that refuses to be read at all — a Proxy trapping `ownKeys` or a +descriptor, a throwing accessor, a cycle — is refused earlier still, by the +run's own retention check, before the document body starts; that refusal is the +journal's and carries the journal's wording. Identity domain and form selection are recorded from the already-authorized answer; the record rediscovers no authority. @@ -12456,7 +12465,10 @@ forms sit beside the restricted one. | PE38 | Identity captured once | Editing the stated identity after the claimant exists does not change what its claims are marked with | | PE39 | An alternating origin | A getter cannot answer the duplicate check and the claim differently | | PE40 | Two occurrences of one name | A program writing `` against a provider supplying A then B under distinct keys invokes A then B, not A twice | -| PE41 | An unresolved occurrence | A provider that delegates through both resolution passes and would answer a third leaves the element unresolved: the third lookup never happens and the late implementation never runs | +| PE41 | An unresolved occurrence | A provider that delegates through both resolution passes and would answer a third leaves the element unresolved: the third lookup never happens, the late implementation never runs, and no `import_component` is recorded for it | +| PE43 | A settled import that will not read | A retained result holding a member no JSON has reaches this boundary and produces its fixed unreadable diagnostic, planting nothing in it, with the component never invoked | +| PE44 | A detached read | The result is read once into detached JSON, so a member answering differently on a second read decides nothing | +| PE45 | A value the journal will not retain | A Proxy refusing `ownKeys` or a descriptor, a throwing accessor, and a circular value are refused by the run's own retention check before the document body starts; the component is never invoked | | PE42 | A corrupted settled import | A missing, additional, mistyped or unknown member in the settled `import_component` result, and one naming another component, each refuse before the component is invoked | | EP1–EP8 | Run profile | The ordinary run profile declares ``; the forms, props, refusals and catalog entry hold through the real binary, and a program cannot reach ``'s private components | | WGAC17–WGAC20 | Workflow profile | A complete program records `evaluate_program` and no `generated_xmd`; the three forms cannot be combined; complete-program support is not reachable through `source` | From 9a522a3d8a196ec501088d98cc61882e05426ebb Mon Sep 17 00:00:00 2001 From: Taras Mankovski Date: Fri, 4 Sep 2026 15:17:39 -0400 Subject: [PATCH 7/7] =?UTF-8?q?=F0=9F=A9=B9=20Keep=20the=20workflow=20``=20case=20on=20the=20runtime=20that=20has=20the=20comma?= =?UTF-8?q?nd=20(#713)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit EP9 proves a workflow run is given one `` and not two, and it proves it by starting a real run. `xmd workflow` exists on the Deno entrypoints alone, so under Node and Bun the command refused before a run existed and the case asserted nothing — which is why `test-node (2/7)` and `test-bun (3/4)` failed while the same file passed locally under Deno. The case moves to its own file and joins the Deno-only exclusions with its reason and issue, the way every other test of a workflow command does. Tier EP's portable half stays where it was and still runs on all three runtimes. --- .../tests/evaluate-program-component.test.ts | 88 -------------- .../tests/evaluate-program-workflow.test.ts | 109 ++++++++++++++++++ scripts/runtime-test-exclusions.ts | 6 + 3 files changed, 115 insertions(+), 88 deletions(-) create mode 100644 packages/cli/tests/evaluate-program-workflow.test.ts diff --git a/packages/cli/tests/evaluate-program-component.test.ts b/packages/cli/tests/evaluate-program-component.test.ts index 61b104faa..4614e303e 100644 --- a/packages/cli/tests/evaluate-program-component.test.ts +++ b/packages/cli/tests/evaluate-program-component.test.ts @@ -178,91 +178,3 @@ describe( }); }, ); - -/** - * A workflow run declares an `` of its own — the complete-program - * forms plus the restricted generated-fragment form only that profile has. The - * run profile must not declare a second component of that name beside it: one - * name names durable work in one domain, and canonical execution refuses two. - * - * This is the seam where that goes wrong, because it is the CLI's own - * installation array rather than anything either package assembles alone. - */ -const WORKFLOW_PROGRAM = [ - "# Compose", - "", - '', - "", - "", - "", -].join("\n"); - -interface Fixture { - readonly repository: string; - readonly runs: string; - readonly home: string; -} - -function* git(repository: string, args: string[]): Operation { - const result = yield* exec("git", { arguments: args, cwd: repository }).expect(); - if (result.code !== 0) { - throw new Error(`git ${args.join(" ")} failed: ${result.stderr}`); - } -} - -function useWorkflowFixture( - files: Record, - body: (fixture: Fixture) => Operation, -): Operation { - return scoped(function* () { - const root = join(tmpdir(), `xmd-ep-${randomUUID()}`); - const fixture: Fixture = { - repository: join(root, "repository"), - runs: join(root, "runs"), - home: join(root, "home"), - }; - yield* ensure(() => rm(root, { recursive: true, force: true })); - yield* ensureDir(fixture.repository); - yield* ensureDir(fixture.home); - for (const [name, content] of Object.entries(files)) { - const path = join(fixture.repository, name); - yield* ensureDir(join(path, "..")); - yield* writeTextFile(path, content); - } - yield* git(fixture.repository, ["init", "-q", "--initial-branch=main", "."]); - yield* git(fixture.repository, ["config", "user.email", "tier-ep@example.test"]); - yield* git(fixture.repository, ["config", "user.name", "Tier EP"]); - yield* git(fixture.repository, ["add", "-A"]); - yield* git(fixture.repository, [ - "-c", - "commit.gpgsign=false", - "commit", - "-q", - "-m", - "definition", - ]); - return yield* body(fixture); - }); -} - -describe( - "Tier EP — one `` per execution", - { sanitizeOps: false, sanitizeResources: false }, - () => { - it("EP9: a workflow run keeps its own `` and gains no second one", function* () { - yield* useWorkflowFixture({ "flows/compose.md": WORKFLOW_PROGRAM }, function* (fixture) { - const started = yield* runCli(["workflow", "start", "--id=compose-1", "flows/compose.md"], { - cwd: fixture.repository, - env: { HOME: fixture.home, XMD_WORKFLOW_RUNS: fixture.runs }, - }).join(); - - const reported = `${started.stdout}${started.stderr}`; - // The exact sentence canonical execution refuses a duplicate with. A - // run profile that declared its own beside the workflow's would fail - // every workflow run, not only one that writes ``. - expect(reported).not.toContain('two identity components called "Evaluate"'); - expect(reported).toContain("The workflow program ran."); - }); - }); - }, -); diff --git a/packages/cli/tests/evaluate-program-workflow.test.ts b/packages/cli/tests/evaluate-program-workflow.test.ts new file mode 100644 index 000000000..8042c9a92 --- /dev/null +++ b/packages/cli/tests/evaluate-program-workflow.test.ts @@ -0,0 +1,109 @@ +/** + * Tier EP — one `` per execution (issue #713). + * + * A workflow run declares an `` of its own, and the run profile must + * not declare a second component of that name beside it. This is its own file + * because proving it needs `xmd workflow start`, which exists on the Deno + * entrypoints alone: under Node and Bun the command refuses before a run is + * created, so the case would assert nothing there. Tier EP's portable half is + * `evaluate-program-component.test.ts`. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { runCli } from "@executablemd/test-support/launch"; +import { ensure, scoped } from "effection"; +import type { Operation } from "effection"; +import { ensureDir, rm, writeTextFile } from "@effectionx/fs"; +import { exec } from "@effectionx/process"; +import { randomUUID } from "node:crypto"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; + +/** + * A workflow run declares an `` of its own — the complete-program + * forms plus the restricted generated-fragment form only that profile has. The + * run profile must not declare a second component of that name beside it: one + * name names durable work in one domain, and canonical execution refuses two. + * + * This is the seam where that goes wrong, because it is the CLI's own + * installation array rather than anything either package assembles alone. + */ +const WORKFLOW_PROGRAM = [ + "# Compose", + "", + '', + "", + "", + "", +].join("\n"); + +interface Fixture { + readonly repository: string; + readonly runs: string; + readonly home: string; +} + +function* git(repository: string, args: string[]): Operation { + const result = yield* exec("git", { arguments: args, cwd: repository }).expect(); + if (result.code !== 0) { + throw new Error(`git ${args.join(" ")} failed: ${result.stderr}`); + } +} + +function useWorkflowFixture( + files: Record, + body: (fixture: Fixture) => Operation, +): Operation { + return scoped(function* () { + const root = join(tmpdir(), `xmd-ep-${randomUUID()}`); + const fixture: Fixture = { + repository: join(root, "repository"), + runs: join(root, "runs"), + home: join(root, "home"), + }; + yield* ensure(() => rm(root, { recursive: true, force: true })); + yield* ensureDir(fixture.repository); + yield* ensureDir(fixture.home); + for (const [name, content] of Object.entries(files)) { + const path = join(fixture.repository, name); + yield* ensureDir(join(path, "..")); + yield* writeTextFile(path, content); + } + yield* git(fixture.repository, ["init", "-q", "--initial-branch=main", "."]); + yield* git(fixture.repository, ["config", "user.email", "tier-ep@example.test"]); + yield* git(fixture.repository, ["config", "user.name", "Tier EP"]); + yield* git(fixture.repository, ["add", "-A"]); + yield* git(fixture.repository, [ + "-c", + "commit.gpgsign=false", + "commit", + "-q", + "-m", + "definition", + ]); + return yield* body(fixture); + }); +} + +describe( + "Tier EP — one `` per execution", + { sanitizeOps: false, sanitizeResources: false }, + () => { + it("EP9: a workflow run keeps its own `` and gains no second one", function* () { + yield* useWorkflowFixture({ "flows/compose.md": WORKFLOW_PROGRAM }, function* (fixture) { + const started = yield* runCli(["workflow", "start", "--id=compose-1", "flows/compose.md"], { + cwd: fixture.repository, + env: { HOME: fixture.home, XMD_WORKFLOW_RUNS: fixture.runs }, + }).join(); + + const reported = `${started.stdout}${started.stderr}`; + // The exact sentence canonical execution refuses a duplicate with. A + // run profile that declared its own beside the workflow's would fail + // every workflow run, not only one that writes ``. + expect(reported).not.toContain('two identity components called "Evaluate"'); + expect(reported).toContain("The workflow program ran."); + }); + }); + }, +); diff --git a/scripts/runtime-test-exclusions.ts b/scripts/runtime-test-exclusions.ts index 227651421..61b301499 100644 --- a/scripts/runtime-test-exclusions.ts +++ b/scripts/runtime-test-exclusions.ts @@ -371,6 +371,12 @@ const DENO_ONLY_TOOLING: RuntimeExclusion[] = [ "every case takes the run's executor lock, which packages/workflow/src/deno/advisory-lock.ts reaches through the `Deno` global; no other runtime has one, so the acquisition refuses before an export begins", issue: DERIVED_SCOPE, }, + { + path: "packages/cli/tests/evaluate-program-workflow.test.ts", + reason: + "starts a real workflow run with `xmd workflow start` to prove a workflow's own `` is the only one that execution is given; the workflow commands exist on the Deno entrypoints alone, so under Node and Bun the command refuses before a run exists and the case would assert nothing. Tier EP's portable half is packages/cli/tests/evaluate-program-component.test.ts", + issue: "https://github.com/taras/executable.md/issues/713", + }, { path: "packages/cli/tests/workflow-retention.test.ts", reason: