diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index ecc2c8a40..2aea93d7d 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -640,6 +640,34 @@ jobs: # on a `ci-main-red-fix` pull request, and may be skipped on an ordinary one. # Every other job must succeed outright — an unexpected skip is an unproven # job, which is exactly what this check exists to catch. + # The workerd suite. It runs where the runtime is real: acquisition lifetime, + # owner eviction and transaction atomicity are properties of a Durable Object + # rather than of any model of one, so none of them is provable in the Deno, + # Node or Bun corpora — which is also why these files carry a `.vitest.ts` + # suffix those corpora never discover. + # + # `pnpm install` comes last on purpose: `deno install` prunes the links pnpm + # placed (scripts/deps.ts says so in its own header), and the plugin only + # takes over the pool when it and the CLI hold the same vitest. + test-cloudflare: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4 + with: + node-version: "22" + + - uses: pnpm/action-setup@b906affcce14559ad1aafd4ab0e942779e9f58b1 # v4 + + - run: pnpm install + + - name: Typecheck the Cloudflare owner + run: pnpm check:cloudflare + + - name: Cloudflare Workers suite + run: pnpm test:cloudflare + green: needs: [ @@ -653,6 +681,7 @@ jobs: site, test-node, test-bun, + test-cloudflare, ] if: always() runs-on: ubuntu-latest diff --git a/architecture.md b/architecture.md index 879b0aec6..5bf9d7dbb 100644 --- a/architecture.md +++ b/architecture.md @@ -104,6 +104,17 @@ Existing documents and code get aligned to this section retroactively. | recovery tombstone | an ownership record left active because its owner never proved it stopped. A crash releases the kernel lock and not this, and no pid, elapsed time, released lock or empty transcript clears it | | provider partition | one complete, independently owned agent-provider state — runtime, store, managed sessions, queues, coordinator, teardown — selected by the one installed factory at each dispatch. Production is the single-partition case of the same path; holding a partition grants work, never permission | | `JournalProvenance` | a non-operational, equality-only witness that a live publication stream descends from the exact journal backend a provider selected for one workflow run; it grants no append, read, execution, publication or reconciliation capability, and is meaningful only because the provider retains the witness it established and later requires exact equality | +| factory run identity | the run ID a software-factory run is addressed by: the lowercase unpadded RFC 4648 Base32 encoding of the full SHA-256 digest of the UTF-8 bytes `github-issue-v1`, NUL, the canonical GitHub authority, NUL and the exact GitHub issue GraphQL node ID. The canonical GitHub authority and the node ID are the ones `specs/github-actions-software-factory-spec.md` §1.1 defines, byte for byte; there is no more general Issue-provider spelling of this hash. It is one host-selected public run ID, derived once from immutable provider identity, and it is distinct from the workflow definition SHA, the implementation revision, the Workspace root, the expansion identity and every delivery identity | +| authenticated intake | one bounded record a trusted host retains for an externally delivered request — a verified webhook, or an authenticated human form submission — keyed by the provider's own delivery or submission identity and holding only typed bounded fields. It is what a later execution reads; it is never a stage, an outcome, a transition or a credential, and receiving one authorizes nothing beyond finding the run it names | +| Project provider | an external service that owns project boards and the status of the items on them. GitHub Projects V2 is one adapter. It is a separate boundary from a Git host and from an Issue provider, because a project board need own neither a repository nor an issue collection | +| Project projection | the human-facing status a Project provider holds for one item, published from the journaled lifecycle rather than read as it. A projection ahead of the journal is drift to reconcile; it is never evidence that a lifecycle transition happened | +| executor connection | a remote host's form of executor acquisition: one authenticated connection whose lifetime *is* the acquisition. Like the local executor lock it is not a time lease — no duration, expiry, renewal, heartbeat, PID or liveness poll — and closing it releases executor ownership without rolling back what already committed | +| delivery-plane transaction | one authenticated transaction that retains an externally supplied value for an exact retained subject without executing the run: intake retention, typed answer delivery and terminal-decision delivery are the three. It generalizes delivery to subjects other than a suspension request, on the same terms: no executor acquisition, no document execution, no journal append, no run-status change | +| implementation revision | the evolving pair `{ headSha, baseSha }` one factory run is currently producing or reviewing: the exact head commit of the implementation branch and the exact target-branch commit it is evaluated against. It changes many times within one run and never takes part in run identity | +| exact-review subject | the implementation revision a review conclusion names. A conclusion authorizes only the pair it names, so a later revision inherits nothing and a moved half of the pair invalidates it | +| machine wait | a durable wait that asks nobody anything: it ends because a later execution observed a provider again, not because a value was delivered. It shares the atomic suspension boundary — its retained event and the `suspended` status commit together, and the executor acquisition is released only after that commit — but it is a distinct event kind identified by a `waitId`, and it has no response schema, no answer route, no form and no bound value. It is a second kind of wait inside the lifecycle, never a second lifecycle controller | +| wake notification | a bounded record correlated to one exact machine wait, retained by an authenticated intake as an ordinary delivery-plane transaction. It carries no answer, verdict, stage, transition or observation result; a later executor consumes one inside the run's transaction and appends the wake event that permits exactly one further observation | +| terminal settlement | the last transition of a run whose outcome required external projections: the retained terminal decision is published as run state only after every required projection has completed, so a completed replay never needs a provider to repair one | ## Three axes @@ -397,6 +408,37 @@ describe what failed without repeating retained props or journal payloads — including their member *names*, which can carry a credential as readily as a member value can. +### One remote owner for one run + +A remote host owns the same run the local host owns, through the same +provider-neutral surfaces. The Cloudflare topology is one SQLite-backed Durable +Object per run, selected from the public run ID by the same arithmetic local +discovery uses, so a remote run has exactly one durable owner and no second +registry can disagree with it. That object holds the WorkflowRun record and its +filtered journal, the immutable Workspace roots and their content-addressed +bytes, the Agent-session mappings and checkpoints, the retained delivery state, +the authenticated intake records, and executor ownership. + +The runtime-named Cloudflare entrypoint is the only place that topology appears. +Shared modules reach it through the contextual storage and lifecycle APIs they +already use, detect no runtime, and import nothing Cloudflare-specific — the +same boundary the Deno entrypoint sits behind. + +Native execution stays off that object. Native Git, evidence processes and Agent +clients run on an ephemeral runner against bounded materialized state; the +Durable Object runs none of them. The runner materializes one selected retained +root, works in it, and submits content-addressed changes; the owner validates +the executor acquisition, the expected root and the submitted content, then +atomically publishes the new root together with the filtered journal result. +That is the same effect transaction local Workspace mutation uses, with the +mutation performed where the tools are and the publication performed where the +authority is. + +A runner that dies mid-flight therefore exposes only a prior or a new complete +transaction, never a partial one. The next acquisition performs the ordinary +stale-execution recovery and resumes from the exact committed WorkflowRun and +Workspace frontier. A completed run replays as it does anywhere, and reading its own history is what it does: it may reach the durable owner holding its retained result — an ephemeral client has nothing else to replay from — while attaching no Workspace, Agent, process, Git, Git-host, Issue, Project, credential or other external-effect provider, performing no effect again and starting no native operation. Lifecycle storage access is not external-effect replay, and only the second is what a completed run must never do. + ## The workflow lifecycle `xmd workflow start [--id=] [--props-*=…] ` and @@ -700,6 +742,29 @@ while the deleting workflow executor still holds its lock. An empty lock file is not retained run state and is not reported as history or provider-session data. +### Remote executor connection + +A remote host acquires the same authority over one authenticated connection. +The acquisition is that connection's lifetime: the run's owner registers the +exact acquisition when the connection is admitted and invalidates it when the +connection closes, which is the staleness proof a remote host has in place of a +released kernel lock. It is not a time lease either — no duration, expiry, +renewal, heartbeat, generation record or liveness poll — and closing it releases +executor ownership without rolling back anything already committed. A second +healthy executor follows the active one or is refused; it cannot advance the +run. + +What the acquisition gates is the same list the lock gates locally, plus what a +split host adds: start and resume, stale-execution recovery, document execution, +Workspace mutation, Agent attachment, native Git and evidence execution, +lifecycle transition, accepted-outcome publication and terminal settlement. Each +of them validates the exact live acquisition *and* the expected Workspace root +inside its own mutating transaction, so a stale connection and a stale frontier +are refused at the same boundary rather than at two. + +What it does not gate is delivery and inspection. Those are described below and +below that, and neither becomes transition authority by being remote. + ### Read-only lifecycle inspection Inspection has its own provider-neutral immutable snapshot surface. It returns @@ -1027,6 +1092,24 @@ transaction that does not commit publishes nothing. Replay after that transaction commits restores the recorded answer event without reaching the live controller and without consuming or publishing again. +That separation is a property of delivery rather than of the local CLI, so it +survives a remote host and generalizes past a suspension request. A **delivery- +plane transaction** is any authenticated transaction that retains an externally +supplied value for an exact retained subject: intake retention, typed answer +delivery, and the terminal-decision delivery the software-factory specification +describes. Each validates its own delivery identity and +its own retained subject, retains only the typed bounded value that subject +describes, and does exactly what answer delivery does otherwise — takes no +executor acquisition, begins no document execution, attaches no provider, +appends no lifecycle outcome and changes no run status. A duplicate delivery of +one identity finds the retained record and writes nothing. + +Consumption stays where it already is. A later executor reads the retained value +inside the run's own transaction, appends the accepted durable event or outcome +exactly once, and only then may authored control flow decide what follows. A +delivery that could advance a lifecycle would be a second state machine beside +the journal, which is the thing this split exists to prevent. + Scheduling — automatic resume, watchers, unattended iteration and remote host selection — is #300's and is not part of this behavior. Nothing here waits on it: a suspended run continues through `xmd workflow answer` followed by an @@ -1406,6 +1489,75 @@ afterwards, and nothing catches its refusal to try somebody else, because a search is how a document that named one service quietly reaches another. A destination every provider delegated reaches the operation's own base error. +### A project board is a third external boundary + +A **Project provider** owns project boards and the status of the items on them, +and it is a boundary of its own for the reason the Issue boundary is one: a +project board need own neither a Git repository nor an issue collection, so a +Project status cannot truthfully execute or persist as a Git-host effect or an +Issue effect. `Project.Status` reaches its own contextual operation and journals +its own durable effect. Its natural key is the exact Project item plus the exact +field; its compatible pre-state is the option that item currently holds; and the +configured Project, item, field and option identities are a host ceiling rather +than something an authored prop can widen. + +The status a board shows is a **Project projection** of the journaled lifecycle, +never a reading of it. A board ahead of the journal is drift the next execution +reconciles, and it is not evidence that a stage was passed. + +Which option means which stage is configuration, not inference. A host that projects lifecycle onto a board holds a total bijection between its stages and the board's exact status option IDs, validated against a complete reread before it is used and refusing when it is missing, partial, duplicated, names an option the board does not offer or one under another field, or names an option whose display name is not the settled string for its stage. Admission maps a reread option ID to a stage through that table and projection maps a stage back through its inverse; neither direction parses a display string, because a name is what a person reads and an option ID is what the host compares. + +### Comments, readiness and closure + +Four more reconciled effects join the same boundary, each keyed by its own subject. `Issue.Comment` is an Issue-provider effect keyed by the canonical issue URL plus the engine-derived effect identity; `PullRequest.Comment` is a Git-host effect keyed by the canonical pull-request URL plus that identity. The body is presentation in both: keying a comment by its text would make an edited sentence a different comment. Creating an object is not by itself what makes an effect attempt-stateful: an Issue upsert and a pull-request upsert each reconcile on a key or an identity the provider gives them, and keep their existing complete-observation contracts. A comment has neither. A Git host issues no client-supplied idempotency key for one, so a comment provider has to support one stable opaque correlation marker it can write, preserve and completely query, and a provider that cannot refuses before its first mutation. The marker is provider transport metadata rather than authored prose: the authored logical body stays byte for byte what the document rendered, the correlation representation rides outside it, and the binding and every replay expose the body and the provider's comment identity rather than the encoding. What a complete observation means depends on the attempt state the effect retains — before an attempt, no marker is proven absence and permits one creation; after an attempt with no committed completion, no marker is permanent ambiguity, because it equally describes somebody having removed one. That is what stops an interrupted creation from becoming a duplicate without pretending a person cannot edit a comment. `PullRequest.Ready` and `PullRequest.Close` are Git-host effects keyed by their exact pull-request subject, and `Issue.Close` is an Issue-provider effect keyed by its exact issue subject and carrying a closed `reason` enum that has to match the retained terminal intent. Each observes before it mutates, adopts a compatible completion, performs once from proven absence or an exact compatible pre-state, and refuses conflict, permanent ambiguity, incomplete observation and temporary unavailability. + +`PullRequest.Merged` is the fourth, and it is the one that never mutates. A Git host records a pull request as merged when it notices its own ref move, which is not the same event as a target publication succeeding, so the fact has to be observed as its own retained step rather than inferred from the step before it. Adoption is its only completion: the host reporting the pull request merged at the exact published commit is the fact, and a merge at another commit is a conflict. A pull request still open is temporary unavailability — the host has not caught up, which is a different thing from refusing — while one closed unmerged is a conflict, because a person having intervened is not a state that resolves itself by waiting. A run that keeps observing the open case does so through a bounded host-configured retry and then a machine wait whose subject is the pull request and the expected commit. That wait is not a typed-answer suspension: it publishes no response schema, accepts no delivered value, and ends because an execution looked again rather than because somebody answered. An authenticated intake retains a bounded wake notification correlated to that exact wait and nothing more; a later executor consumes it and appends the wake event in one transaction, and only a compatible observation advances anything. + +### Ordered merge, and publishing a target + +`Git.Merge` is Workspace-local. It observes and fixes its first parent, second +parent and merge base before it mutates, and its two closed results are +distinct: a clean merge publishes the new commit, the new Workspace root and the +filtered result in one effect transaction, while a conflicted merge restores the +pre-merge root and publishes normalized conflict evidence against it. The parent +order is the caller's and is part of the request, because the two merges a +factory performs mean opposite things — synchronizing a target into an +implementation, and publishing an implementation onto a target. + +`Git.PublishTarget` is a Git-host effect and is not a spelling of `Git.Push`. +Push advances a branch this run published, from an ancestry relation proved +inside the authenticated object source. Target publication is a compare-and-swap +against a protected ref: it updates only after observing the target equal to the +expected commit, adopts a target already equal to the exact source commit with +nothing performed, and refuses everything else without mutating. The remote, the +ref, the credential and the non-force policy are host-owned; the reviewed head +and expected commit travel in the request so the record says what the +publication was authorized against. + +### Trusted evidence execution + +`Evidence.Run` executes an authored structured argv list natively, on the trusted runner, against one exact retained Workspace root, under host-owned executable, environment, working-root, per-command duration, whole-run duration, output and process-tree ceilings. It is not Worker Shell and not a document's own process capability: it runs where the tools are, and it is absent from the workflow Agent's capabilities and from every generated-XMD write table. + +It is a fail-fast pipeline. Commands run in authored order and the first one that does not exit with status `0` is the last one that runs, because a plan's evidence list is usually a pipeline and rows produced after a failed build are evaluated against prerequisites that are missing or stale. What the effect binds is therefore the executed prefix, stated as such: how it completed, how many commands were authored, and one row per command that ran, each naming its argv, how it ended, and its stdout and stderr as separate bounded channels that state their own truncation. Breadth belongs inside a command whose own contract runs a corpus to the end, or in separate elements the plan says are independent. + +Two host-owned ceilings bound it — one per command, one for the whole list — and a row that timed out says which fired. A timeout is an ordinary unsuccessful outcome: it records that the host enforced its ceiling, reaped the tree and captured its channels. A non-zero status is likewise evidence rather than an infrastructure failure, since it is the answer the effect exists to obtain. + +Being unable to say what happened is the failure. A launch the ceiling refused, a channel the host could not read, and a child it could not reap each fail the effect and bind no result, so no prefix is ever mistaken for an answer. Cancellation wins over everything and commits neither completion nor failure; otherwise the first infrastructure failure is authoritative and a teardown failure after it is retained as secondary evidence, while a teardown failure with nothing before it is authoritative on its own — a host that cannot prove its process ownership settled cannot publish a success, on the same terms lifecycle settlement applies. A failed effect still retains bounded diagnostic evidence on its error: the safely collected prefix, the bounded channels, and the primary and secondary failure categories. No successful binding is not the same as no retained evidence, and replaying a failed effect starts no process. + +### Terminal settlement follows its projections + +An outcome whose completion requires external projections retains the decision first and settles last: the accepted decision is journaled before any effect is attempted, the required projections are separate reconciled steps, and only after all of them complete is the terminal run state published. + +There are two terminal paths, and they do not share a step list. Reading one general sequence for both would require merge effects during an abandonment, or pull-request closure during a merge. + +**The merged path** retains the authenticated exact-revision merge decision, then constructs the trusted merge commit, publishes the target, retains the merged pull-request observation, closes the issue as completed, projects the Project item to its closed option, and publishes terminal kind `merged` carrying the actor, the exact revision, the merge commit, the resulting provider identities and the retained history. + +**The abandoned path** retains the authenticated exact-revision abandonment decision together with its required reason, then closes the pull request unmerged, closes the issue as not planned, projects the Project item to its closed option, and publishes terminal kind `abandoned` carrying the actor, the exact revision, the reason, the resulting provider identities and the retained history. It constructs no merge and publishes no target — there is nothing it reviewed that it is publishing. + +A third decision is not terminal at all. A change decision names the earliest stage it invalidates and a reason, and returns the run to that stage; it settles nothing and projects no closure. + +Every step on either path is a separate reconciled external effect or a separate retained transition, and no distributed transaction is claimed across the run's storage, native Git and processes, and the external services. An interruption resumes at the first uncommitted or unreconciled step. Terminal settlement is last on both paths for one reason: a completed run replays without contacting a provider, so a terminal state published before its projections would leave the repair to exactly the replay that is forbidden to reach a provider. + Every committed journal event references the current logical Workspace root. Only committed event boundaries are checkpoints. A history fork copies the selected root and the roots the inherited prefix names into the new run, replays @@ -3116,6 +3268,51 @@ or partial continuation they run and record through the ordinary durable protocol — an effect an earlier preparation already completed is restored from its retained record rather than performed again. +### A split trusted host + +A factory run has one trusted host in two pieces, and which piece owns what is +the whole of its security boundary. + +The **provider host** is the Cloudflare runtime-named entrypoint. It owns +persistence and transactions, the authenticated intake receiver, the +authorization gates, token minting, and executor admission. On GitHub that +receiver is one dedicated GitHub App: it verifies a webhook signature before it +parses the payload as anything but bytes, +rereads the complete provider objects through the API rather than trusting the +payload's copy of them, authenticates the installation and the human actor, and +only then retains one bounded intake keyed by the provider's delivery or +submission identity, and mints the short-lived installation token every external +effect is performed with. It admits a runner session — the OIDC client the +Actions job authenticates as — only after validating that session's claims: +issuer, configured audience, repository ID, repository-owner ID, event name, +workflow ref and SHA, and the configured immutable workflow identity. Names are +mutable and IDs are not, which is why the check is on IDs. + +The **runner host** is the ephemeral Actions job. It owns the native clients: +Git, the plan-evidence processes, and the Agent. It holds no durable authority +at all; what it holds is one authenticated executor connection and one +materialized Workspace root, and every mutation it proposes is validated and +published by the provider host. + +Credentials stay with the piece that mints them. The application private key, +the webhook secret, the OIDC verification configuration, every issued +installation token, the provider endpoints, the raw payloads, the pagination +cursors and the host paths are provider-host secrets and closure state. None of +them reaches props, context composition data, a durable request or result, a +comment, document output, or a diagnostic. A short-lived installation token +performs the external effects; the journal retains the human actor separately +from the token that acted, so the record says who decided as well as what was +done. + +Ceilings narrow in one direction. The installation is limited to configured +repositories, and the host narrows further per operation to the exact +repository, branch, target ref, project, field, option, subject, reviewed +revision, parent pair and non-force operation. A path the granted permission +could reach but the contract excludes — a workflow definition under +`.github/workflows/**` is the one that matters, since rewriting it would rewrite +the run's own authorization — is refused by the host rather than left to the +permission model. + ### The weak journal-provenance association Journal provenance is the one further exception, and it is deliberately narrow. @@ -3303,6 +3500,19 @@ Status is measured against main. | `xmd workflow answer ` | retains one schema-validated value for one retained wait, taking no executor lock and changing no run state | built by #300 | | `suspension_answer` durable effect | ends a wait from retained delivery state, publishing the answer and consuming that state in one transaction | built by #300 | | `` · `` · `` | read the reviews, comments and checks a Git host already holds for one numbered pull request, completely or not at all | built by #576 | +| `` | adds one comment to the issue a canonical URL names, from the paired content it renders, reconciled as an Issue-provider effect whose natural key is that URL plus the engine-derived effect identity — the body is presentation, so an edited sentence is the same comment | specified by #710; implementation unbuilt | +| `` | adds one comment to the pull request a canonical URL names, from the paired content it renders, reconciled as a Git-host effect keyed by that URL plus the engine-derived effect identity | specified by #710; implementation unbuilt | +| `` | publishes one item's status to a Project provider — a boundary of its own, because a board owns neither a repository nor an issue collection — keyed by the exact item and field, against the option the item currently holds, inside the host's configured project, field and option ceiling | specified by #710; implementation unbuilt | +| `` · `` | take one pull request out of draft, and close one unmerged, as Git-host effects keyed by their exact subject; readiness is authorized by an accepted review outcome rather than by observing the pull request | specified by #710; implementation unbuilt | +| `` | closes one issue as `completed` or `not_planned`, as an Issue-provider effect keyed by its exact subject, with the reason a closed enum that must match the retained terminal intent | specified by #710; implementation unbuilt | +| `` | merges two exact commits inside the retained Workspace, observing and fixing both parents and the merge base first; a clean result publishes commit, root and filtered result in one effect transaction, and a conflicted one restores the pre-merge root and publishes normalized conflict evidence. The parent order is the caller's — synchronizing a target into an implementation and publishing an implementation onto a target are opposite operations — and `purpose` authorizes it rather than merely recording it: the provider-authenticated merge ceiling supplies the pair each purpose may carry, and a swapped, stale or cross-purpose parent refuses before any Git mutation | specified by #710; implementation unbuilt | +| `` | updates a protected target ref by compare-and-swap: one non-force update after observing the target equal to the expected commit, adoption of a target already equal to the exact source commit, and refusal of everything else without mutation; remote, ref, credential and non-force policy are host-owned. Not a spelling of `Git.Push`, which advances a branch this run published from a proved ancestry relation | specified by #710; implementation unbuilt | +| `` | runs an authored structured argv list natively on the trusted runner against one exact retained Workspace root, under host-owned executable, environment, working-root, per-command duration, whole-run duration, output and process-tree ceilings; a fail-fast pipeline that stops at the first command not exiting `0` and binds the executed prefix — how it completed, how many commands were authored, and one row per command that ran carrying argv, how it ended, which ceiling fired on a timeout, and separately bounded stdout and stderr that state their own truncation; a launch, output-pump or teardown failure binds no result while retaining bounded error evidence, and cancellation commits nothing; absent from the workflow Agent's capabilities and from every generated-XMD write table, and a completed replay runs nothing | specified by #710; implementation unbuilt | +| `` | observes that a Git host now records one pull request as merged, at the exact commit a target publication published; a reconciled Git-host observation that mutates nothing, whose only completion is adoption, keyed by the canonical pull-request URL — a merge at another commit conflicts, a pull request still open is temporary unavailability the run waits out under a bounded retry and then a durable machine wait, one closed unmerged conflicts, and publication does not imply any of it | specified by #710; implementation unbuilt | +| remote `WorkflowHost` implementation | keeps the existing four-method host boundary — `useRunHost()`, `useLifecycle()`, `useDelivery()`, `attach()` — and adds a Cloudflare runtime-named implementation of it beside the Deno one; start, lookup, execute, deliver and inspect stay lifecycle operations reached through those four rather than becoming method names, and a remote host receives no transitions type of its own. One SQLite-backed Durable Object per run is selected from the public run ID, and executor acquisition is an authenticated connection lifetime | specified by #710; implementation unbuilt | +| provider-neutral lifecycle transition types | `WorkflowExecutionTransitions`, `WorkflowBeginRequest`, `WorkflowExecutionBegun`, `WorkflowForkRequest`, `WorkflowForkSelection` and `WorkflowRunCreation` describe what any host's lifecycle does rather than what one adapter retains, and become package-root public types; the Deno entrypoint may re-export them for source compatibility without owning their meaning, while runtime-specific implementations and retained encodings stay behind their runtime-named entrypoints | neutrality settled by #710; the export move is implementation work | +| same-release runner transport | the messages between an ephemeral runner client and its durable owner are private to one software-factory release: not journaled, exported, authored or supported across independently versioned builds. Connection admission validates an exact immutable build or protocol fingerprint from trusted deployment configuration and refuses a mismatch closed, before parsing, acquisition or state access; there is no cross-version adaptation or downgrade. Privacy of the transport is not privacy of the authority — the acquisition, expected-root validation, owner-side parsing and transactions, content validation, separate no-acquisition delivery and inspection paths, and provider-free completed replay stay public and exact | specified by #710; implementation unbuilt | +| factory protocol records | the closed versioned schemas `specs/github-actions-software-factory-spec.md` §11.2 defines — and normatively owns, every other document linking to it rather than restating it — for the subject, stage, implementation revision, handoff, actor, role outcome, invalidation, evidence reference, Planner and Architect verdicts, conflict suspension, Stage 7 decision, merged-observation wait, stage-to-option table, active frontier and the two terminal settlements one issue-driven run retains. Each carries a schema discriminant and a version, and an unknown schema, version, member or enum value refuses rather than being ignored — provider-neutral durable protocol, neither an XMD component nor a TypeScript lifecycle controller | specified by #710; implementation unbuilt | | workflow scheduling (watchers, unattended iteration, remote host selection) | — | #300 | | `` | binds `{ok: true, value}` or `{ok: false, error}`; a failure becomes a bound value, not a raise | defined, unbuilt | | error middleware (JS api) | retry · suspend · decline | defined, unbuilt | diff --git a/deno.json b/deno.json index 97508df8b..70563ebed 100644 --- a/deno.json +++ b/deno.json @@ -1,5 +1,8 @@ { - "workspace": ["packages/*", "site"], + "workspace": [ + "packages/*", + "site" + ], "exclude": [ "scripts/tests/fixtures", ".xmd-eval", @@ -7,7 +10,12 @@ "packages/workflow/vendor/cloudflare-computer-dofs/upstream", "packages/workflow/vendor/cloudflare-computer-dofs/generated/**/*.d.ts", "packages/acp/vendor/acpx/upstream", - "packages/acp/vendor/acpx/generated/**/*.d.ts" + "packages/acp/vendor/acpx/generated/**/*.d.ts", + "packages/workflow/src/cloudflare", + "packages/workflow/tests/cloudflare", + "vitest.config.ts", + "packages/workflow/tsconfig.cloudflare.json", + "packages/workflow/cloudflare.ts" ], "nodeModulesDir": "auto", "lock": { @@ -71,6 +79,8 @@ "review:local": "deno run --allow-all packages/cli/src/deno.ts run .reviews/ReviewPR.local.md --component-dir .reviews/components --component-dir .reviews/policies --component-dir packages/core/components -j .reviews/journal.local.jsonl", "analyze": "deno run --allow-all packages/cli/src/deno.ts run .reviews/AnalyzeRepo.md --component-dir .reviews/components --component-dir .reviews/policies --component-dir packages/core/components -j .reviews/journal.analyze.jsonl", "analyze:ci": "deno run --allow-all packages/cli/src/deno.ts run .reviews/AnalyzeRepoCI.md --component-dir .reviews/components --component-dir .reviews/policies --component-dir packages/core/components -j .reviews/journal.analyze.ci.jsonl", - "analyze:dispatch": "deno run --allow-all packages/cli/src/deno.ts run .reviews/DispatchRepoAnalysis.md --component-dir .reviews/components --component-dir .reviews/policies --component-dir packages/core/components -j .reviews/journal.dispatch.jsonl" + "analyze:dispatch": "deno run --allow-all packages/cli/src/deno.ts run .reviews/DispatchRepoAnalysis.md --component-dir .reviews/components --component-dir .reviews/policies --component-dir packages/core/components -j .reviews/journal.dispatch.jsonl", + "test:cloudflare": "pnpm test:cloudflare", + "check:cloudflare": "pnpm check:cloudflare" } } diff --git a/deno.lock b/deno.lock index b255615bd..c89620201 100644 --- a/deno.lock +++ b/deno.lock @@ -41,6 +41,8 @@ "npm:@agentclientprotocol/sdk@1.3.0": "1.3.0_zod@4.4.3", "npm:@babel/core@^7.28.0": "7.29.7", "npm:@babel/preset-react@^7.27.1": "7.29.7_@babel+core@7.29.7", + "npm:@cloudflare/vitest-plugin@1.1.3": "1.1.3_@vitest+runner@4.1.11_@vitest+snapshot@4.1.11_vitest@4.1.11__@opentelemetry+api@1.9.1__@types+node@24.13.3__vite@7.3.6___@types+node@24.13.3___tsx@4.23.1_@cloudflare+workers-types@5.20260901.1", + "npm:@cloudflare/workers-types@^5.20260831.1": "5.20260901.1", "npm:@durable-streams/client@~0.2.2": "0.2.6", "npm:@durable-streams/server@~0.3.8": "0.3.8", "npm:@effectionx/context-api@0.6.0": "0.6.0_effection@4.1.0", @@ -74,6 +76,8 @@ "npm:@types/babel__core@^7.20.5": "7.20.5", "npm:@types/node@22": "22.19.15", "npm:@types/node@^24.5.2": "24.13.3", + "npm:@vitest/runner@4.1.11": "4.1.11", + "npm:@vitest/snapshot@4.1.11": "4.1.11", "npm:acorn@^8.16.0": "8.16.0", "npm:acpx@0.12.0": "0.12.0", "npm:ajv@8.20.0": "8.20.0", @@ -111,6 +115,7 @@ "npm:unist-util-select@5": "5.1.0", "npm:vite@^7.1.3": "7.3.6_@types+node@24.13.3_tsx@4.23.1", "npm:vite@^7.1.4": "7.3.6_@types+node@24.13.3_tsx@4.23.1", + "npm:vitest@4.1.11": "4.1.11_@opentelemetry+api@1.9.1_@types+node@24.13.3_vite@7.3.6__@types+node@24.13.3__tsx@4.23.1", "npm:zod@4": "4.4.3", "npm:zod@^4.3.6": "4.4.3" }, @@ -312,7 +317,7 @@ "debug", "gensync", "json5", - "semver" + "semver@6.3.1" ] }, "@babel/generator@7.29.7": { @@ -321,7 +326,7 @@ "@babel/parser", "@babel/types", "@jridgewell/gen-mapping", - "@jridgewell/trace-mapping", + "@jridgewell/trace-mapping@0.3.31", "jsesc" ] }, @@ -338,7 +343,7 @@ "@babel/helper-validator-option", "browserslist", "lru-cache", - "semver" + "semver@6.3.1" ] }, "@babel/helper-globals@7.29.7": { @@ -481,9 +486,69 @@ "sisteransi" ] }, + "@cloudflare/kv-asset-handler@0.5.0": { + "integrity": "sha512-jxQYkj8dSIzc0cD6cMMNdOc1UVjqSqu8BZdor5s8cGjW2I8BjODt/kWPVdY+u9zj3ms75Q5qaZgnxUad83+eAg==" + }, + "@cloudflare/unenv-preset@2.16.1_unenv@2.0.0-rc.24_workerd@1.20260831.1": { + "integrity": "sha512-ECxObrMfyTl5bhQf/lZCXwo5G6xX9IAUo+nDMKK4SZ8m4Jvvxp52vilxyySSWh2YTZz8+HQ07qGH/2rEom1vDw==", + "dependencies": [ + "unenv", + "workerd" + ], + "optionalPeers": [ + "workerd" + ] + }, + "@cloudflare/vitest-plugin@1.1.3_@vitest+runner@4.1.11_@vitest+snapshot@4.1.11_vitest@4.1.11__@opentelemetry+api@1.9.1__@types+node@24.13.3__vite@7.3.6___@types+node@24.13.3___tsx@4.23.1_@cloudflare+workers-types@5.20260901.1": { + "integrity": "sha512-ED1Rkaq5Wr5rCeHXpLoDyV4WGJzD0Ju0clM8jS7Hj+wjj/CwaMHeb8DXzUfUQPiHW9rTgwcttPPQnzau2kp6Jg==", + "dependencies": [ + "@vitest/runner", + "@vitest/snapshot", + "cjs-module-lexer", + "esbuild@0.28.1", + "miniflare", + "vitest", + "wrangler", + "zod" + ] + }, + "@cloudflare/workerd-darwin-64@1.20260831.1": { + "integrity": "sha512-oyZ8xhu+gYTvoxV/sn6NRmTHK95RhEO1Dk54/6oPb0Uu70w7ZeRoCjkJ5aNmfS8Vrkdu6+oL0HNg6EcC61uQ2Q==", + "os": ["darwin"], + "cpu": ["x64"] + }, + "@cloudflare/workerd-darwin-arm64@1.20260831.1": { + "integrity": "sha512-s6Go53KPnoXZ1sTGBZ3en3otfHDuMPJhiwXMYWU21JkJQkpoeRt6HFUwM0GPhK3YhXWm+8baGMvCGZYS/KA9eA==", + "os": ["darwin"], + "cpu": ["arm64"] + }, + "@cloudflare/workerd-linux-64@1.20260831.1": { + "integrity": "sha512-WxNKBgjKgeYTolW3yl1Lt3Lu67UlxdeyzWYi9MIqrKBdyQcz+UNG36RevSBf8rv1sTWapRW234VX2keZ+wXapA==", + "os": ["linux"], + "cpu": ["x64"] + }, + "@cloudflare/workerd-linux-arm64@1.20260831.1": { + "integrity": "sha512-JTF9+9clUT3gaCq7Xnmd+Q/wEMaitpngSTOec/Ffb/r3xexA9XwNJVFSOKfk6q61flHGjAYJ4H9B7Mu5Qur49w==", + "os": ["linux"], + "cpu": ["arm64"] + }, + "@cloudflare/workerd-windows-64@1.20260831.1": { + "integrity": "sha512-do+KDYw0PABwsrKUQIccWBZB70kqKcADoSnvzJ8pvMaWUVB4qaCspEZYfm97WNdtY1wt8mlKYqIJyYUNOkTvQg==", + "os": ["win32"], + "cpu": ["x64"] + }, + "@cloudflare/workers-types@5.20260901.1": { + "integrity": "sha512-m1rNbR3UYC1pgaEyXkSlwLFLDB11QziYUlY0z/nqjwuYtg5dw9zBrgDudoSfYM6ebbPDKU81ogBQAzLp6h1y4w==" + }, "@colors/colors@1.5.0": { "integrity": "sha512-ooWCrlZP11i8GImSjTHYHLkvFDP48nS4+204nGb1RiX/WXYHmJA2III9/e2DWVabCESdW7hBAEzHRqUn9OUVvQ==" }, + "@cspotcode/source-map-support@0.8.1": { + "integrity": "sha512-IchNf6dN4tHoMFIn/7OE8LWZ19Y6q/67Bmf6vnGREv8RSbBVb9LPJxEcnwrcwX6ixSvaiGoomAUvu4YSxXrVgw==", + "dependencies": [ + "@jridgewell/trace-mapping@0.3.9" + ] + }, "@durable-streams/client@0.2.6": { "integrity": "sha512-uHKKbWpsKLhFMeGjG0PgM6LXE3oEIi7FHKlJZkmYGxcqd4Yjjd/QEvnQnDzteRP4Av1uJVM8qjTL7kfKsgeS/w==", "dependencies": [ @@ -593,6 +658,12 @@ "effection" ] }, + "@emnapi/runtime@1.11.3": { + "integrity": "sha512-Xz4Tpyki7XyrpbUK1jR1AhdAdaXyhhY4lZ3neLodmhpuWfy2PAQN5B46sAiU4liOXGLkHypn/qU+jvfWSCYYLA==", + "dependencies": [ + "tslib" + ] + }, "@esbuild/aix-ppc64@0.25.12": { "integrity": "sha512-Hhmwd6CInZ3dwpuGTF8fJG6yoWmsToE+vYgD4nytZVxcu1ulHpUQRAB1UJ8+N1Am3Mz4+xOByoQoSZf4D+CpkA==", "os": ["aix"], @@ -1016,6 +1087,174 @@ "@harperfast/extended-iterable@1.0.3": { "integrity": "sha512-sSAYhQca3rDWtQUHSAPeO7axFIUJOI6hn1gjRC5APVE1a90tuyT8f5WIgRsFhhWA7htNkju2veB9eWL6YHi/Lw==" }, + "@img/colour@1.1.0": { + "integrity": "sha512-Td76q7j57o/tLVdgS746cYARfSyxk8iEfRxewL9h4OMzYhbW4TAcppl0mT4eyqXddh6L/jwoM75mo7ixa/pCeQ==" + }, + "@img/sharp-darwin-arm64@0.35.2": { + "integrity": "sha512-eEieHsMksAW4IiO5NzauESRl2D2qz3J/kwUxUrSfV06A93eEaRfMpHXyUb1mAqrR7i8U9A0GRqE9pjn6u1Jjpg==", + "optionalDependencies": [ + "@img/sharp-libvips-darwin-arm64" + ], + "os": ["darwin"], + "cpu": ["arm64"] + }, + "@img/sharp-darwin-x64@0.35.2": { + "integrity": "sha512-BaktuGPCeHJMARpodR8jK4uKiZrPAy9WrfQW0sdI37clracq8Bp01AYS3SZgi5FS/y5twa9t4+LIuuxQjqRrWw==", + "optionalDependencies": [ + "@img/sharp-libvips-darwin-x64" + ], + "os": ["darwin"], + "cpu": ["x64"] + }, + "@img/sharp-freebsd-wasm32@0.35.2": { + "integrity": "sha512-YoAxdnd8hPUkvLHd3bWY+YA8nw3xM/RyRopYucNsWHVSan8NLVM3X2volsfoRDcXdUJPg6tXahSd7HXPK7lRnw==", + "dependencies": [ + "@img/sharp-wasm32" + ], + "os": ["freebsd"] + }, + "@img/sharp-libvips-darwin-arm64@1.3.1": { + "integrity": "sha512-4V/M3roRMTYjiwZY9IOVQOE8OyeCxFAkYmyZDrZl51uOKjibm3oeEJ4WAmLxutAfzFbC9jqUiPs2gbnGflH+7g==", + "os": ["darwin"], + "cpu": ["arm64"] + }, + "@img/sharp-libvips-darwin-x64@1.3.1": { + "integrity": "sha512-c0/DxItpJv2+dGhgycJBBgotdqruGYDvA79drdh0MD1dFpy7JzJ/PlXwi1H4rFf0eTy8tgbI91aHDnZIceY3jQ==", + "os": ["darwin"], + "cpu": ["x64"] + }, + "@img/sharp-libvips-linux-arm64@1.3.1": { + "integrity": "sha512-JznefmcK9j1JKPz8AkQDh89kjojubyfOasWBPKfzMIhPwsgDy9evpE/naJTXXXmghS1iFwR8u/kTwh/I2/+GCw==", + "os": ["linux"], + "cpu": ["arm64"] + }, + "@img/sharp-libvips-linux-arm@1.3.1": { + "integrity": "sha512-aGGy9aWzXgHBG7HNyQPWorZthlp7+x6fDRoPAQbGO3ThcttuTyKIx3NuSHb6zb4gBNq6/yNn9f1cy9nFKS/Vmg==", + "os": ["linux"], + "cpu": ["arm"] + }, + "@img/sharp-libvips-linux-ppc64@1.3.1": { + "integrity": "sha512-1EkwGNCZk6iWNCMWqrvdJ+r1j0PT1zIz60CNPhYnJlK/zyeWqlsPZIe+ocBVqPF8k/Ssee/NCk+tE9Ryrko6ng==", + "os": ["linux"], + "cpu": ["ppc64"] + }, + "@img/sharp-libvips-linux-riscv64@1.3.1": { + "integrity": "sha512-Ilays+w2bXdnxzxtQdmXR62u8o8GYa3eL4+Gr+1KiE4xperMZUslRaVPJwwPkzlHEjGfXAfRVAa/7CYCtSqsBw==", + "os": ["linux"], + "cpu": ["riscv64"] + }, + "@img/sharp-libvips-linux-s390x@1.3.1": { + "integrity": "sha512-VfBwVHQTbRoj4XlpA/KLZ7ltgMpz+4WSejFzQ+GnoImjo1PtEJ59QB2qR1xQEeRPYIkNrPIm2L4cICMvz4C2ew==", + "os": ["linux"], + "cpu": ["s390x"] + }, + "@img/sharp-libvips-linux-x64@1.3.1": { + "integrity": "sha512-+c8ukgwU62DS54nCAjw7keOfHUkmr0B5QHEdcOqRnodF/MNXJbVI8Eopoj4B/0H8Asr65I+A4Amrn7a85/md6A==", + "os": ["linux"], + "cpu": ["x64"] + }, + "@img/sharp-libvips-linuxmusl-arm64@1.3.1": { + "integrity": "sha512-qlKb/pwbkAi1WMsJrYHk7CuDrd12s27U2QnRhFYUoJNrRCmkosMTttuRFat/DDB3IlDm5qE1TJgZ4JDnHX8Ldw==", + "os": ["linux"], + "cpu": ["arm64"] + }, + "@img/sharp-libvips-linuxmusl-x64@1.3.1": { + "integrity": "sha512-yO21HwoUVLN8Qa+/SBjQLMYwBWAVJjeGPNe+hc0OUeMeifEtJqu5a1c4HayE1nNpDih9y3/KkoltfkDodmKAlg==", + "os": ["linux"], + "cpu": ["x64"] + }, + "@img/sharp-linux-arm64@0.35.2": { + "integrity": "sha512-af12Pnd0ZGu2HfP8NayB0kk6eC/lrfbQE6HlR4jD+34wdJ1Vw9TF6TMn6ZvffT+WgqVsl0hRbmNvz2u/23VmwA==", + "optionalDependencies": [ + "@img/sharp-libvips-linux-arm64" + ], + "os": ["linux"], + "cpu": ["arm64"] + }, + "@img/sharp-linux-arm@0.35.2": { + "integrity": "sha512-SE4kzF2mepn6z+6E7L6lsV8FzuLL6IPQdyX8ZiwROAG/G8td+hP/m7FsFPwidtrF19gvajuC9l6TxAVcsA4S7A==", + "optionalDependencies": [ + "@img/sharp-libvips-linux-arm" + ], + "os": ["linux"], + "cpu": ["arm"] + }, + "@img/sharp-linux-ppc64@0.35.2": { + "integrity": "sha512-hYSBm7zcNtDCozCxQHYZJiu63b/bXsgRZuOxCIBZsStMM9Vap47iFHdbX4kCvQsblPB/k+clhELpdQJHQLSHvg==", + "optionalDependencies": [ + "@img/sharp-libvips-linux-ppc64" + ], + "os": ["linux"], + "cpu": ["ppc64"] + }, + "@img/sharp-linux-riscv64@0.35.2": { + "integrity": "sha512-qQt0Kc13+Hoan/Awq/qMSQw3L+RI1NCRPgD5cUJ/1WSSmIoysLOc72jlRM3E0OHN9Yr313jgeQ2T+zW+F03QFA==", + "optionalDependencies": [ + "@img/sharp-libvips-linux-riscv64" + ], + "os": ["linux"], + "cpu": ["riscv64"] + }, + "@img/sharp-linux-s390x@0.35.2": { + "integrity": "sha512-E4fLLfRPzDLlEeDaTzI98OFLcv++WL5ChLLMwPoVd0CIoZQqupBSNbOisPL5am9XsbQ9T84+iiMpUvbFtkunbA==", + "optionalDependencies": [ + "@img/sharp-libvips-linux-s390x" + ], + "os": ["linux"], + "cpu": ["s390x"] + }, + "@img/sharp-linux-x64@0.35.2": { + "integrity": "sha512-gi0zFJJRLswfCZmHtJdikXPOc5u7qamSOS3NHedLqLd4W8Q0NqjdBr6TTRIgsfFjqfTsHFgdfvJ9LwqSgcHiAA==", + "optionalDependencies": [ + "@img/sharp-libvips-linux-x64" + ], + "os": ["linux"], + "cpu": ["x64"] + }, + "@img/sharp-linuxmusl-arm64@0.35.2": { + "integrity": "sha512-siWbOW1u6HFnFLrp0waKyW7VEf7jYvcDWdrXEFa8AkdAQgEvuu5Fz8/Y70w9EeqAdwDtfU012BhEHHaDqvQNzg==", + "optionalDependencies": [ + "@img/sharp-libvips-linuxmusl-arm64" + ], + "os": ["linux"], + "cpu": ["arm64"] + }, + "@img/sharp-linuxmusl-x64@0.35.2": { + "integrity": "sha512-YBqMMcjDi4QGYiSn4vNOYBhmlC4z5AXqkOUUqI2e0AFA4urNv4ESgOgwNl3K+4etQhha0twXlzeF20bbULm9Yg==", + "optionalDependencies": [ + "@img/sharp-libvips-linuxmusl-x64" + ], + "os": ["linux"], + "cpu": ["x64"] + }, + "@img/sharp-wasm32@0.35.2": { + "integrity": "sha512-Mrv4JQNYVQ94xH+jzZ9r+gowleN8mv2FTgKT+PI6bx5C0G8TdNYndu161pg2i7uoBwxy2ImPMHrJOM2LZef7Bw==", + "dependencies": [ + "@emnapi/runtime" + ] + }, + "@img/sharp-webcontainers-wasm32@0.35.2": { + "integrity": "sha512-QNV27pxs9wpApEiCfvHM1RDoP1w1+2KrUWWDPEhEwg+latvOrfuhWrHWZKwdSFwU6jh3myjw/yOCRsUIuOft3g==", + "dependencies": [ + "@img/sharp-wasm32" + ], + "cpu": ["wasm32"] + }, + "@img/sharp-win32-arm64@0.35.2": { + "integrity": "sha512-BiVRYc/t6/Vl3e1hBx0hugG4oN9Pydf4fgMSpxTQJmwGUg/YoXTWHiFeRymHfCZzifxu4F4rpk/I67D0LQ20wQ==", + "os": ["win32"], + "cpu": ["arm64"] + }, + "@img/sharp-win32-ia32@0.35.2": { + "integrity": "sha512-YYEhx9PImCC7T0tI8JDMi4DB9LwLCXCU5OWNYEXAxh5Q1ShKkyC6byxzoBJ3gEFDnH2lQckWuDe70G7mB2XJog==", + "os": ["win32"], + "cpu": ["ia32"] + }, + "@img/sharp-win32-x64@0.35.2": { + "integrity": "sha512-imoOyBcoM/iiUr4J6VPpCNjPnjvP/Gks95898yB8YqoGGYmHYbOyCuNv9FMhFgtaiHFGbHW8bxKqRV6VjtXThQ==", + "os": ["win32"], + "cpu": ["x64"] + }, "@jest/diff-sequences@30.3.0": { "integrity": "sha512-cG51MVnLq1ecVUaQ3fr6YuuAOitHK1S4WUJHnsPFE/quQr33ADUx1FfrTCpMCRxvy0Yr9BThKpDjSlcTi91tMA==" }, @@ -1057,14 +1296,14 @@ "integrity": "sha512-2kkt/7niJ6MgEPxF0bYdQ6etZaA+fQvDcLKckhy1yIQOzaoKjBBjSj63/aLVjYE3qhRt5dvM+uUyfCg6UKCBbA==", "dependencies": [ "@jridgewell/sourcemap-codec", - "@jridgewell/trace-mapping" + "@jridgewell/trace-mapping@0.3.31" ] }, "@jridgewell/remapping@2.3.5": { "integrity": "sha512-LI9u/+laYG4Ds1TDKSJW2YPrIlcVYOwi2fUC6xB43lueCjgxV4lffOCZCtYFiH6TNOX+tQKXx97T4IKHbhyHEQ==", "dependencies": [ "@jridgewell/gen-mapping", - "@jridgewell/trace-mapping" + "@jridgewell/trace-mapping@0.3.31" ] }, "@jridgewell/resolve-uri@3.1.2": { @@ -1080,6 +1319,13 @@ "@jridgewell/sourcemap-codec" ] }, + "@jridgewell/trace-mapping@0.3.9": { + "integrity": "sha512-3Belt6tdc8bPgAtbcmdtNJlirVoTmEb5e2gC94PnkwEW9jI6CAHUeoG85tjWP5WquqfavoMtMwiG4P926ZKKuQ==", + "dependencies": [ + "@jridgewell/resolve-uri", + "@jridgewell/sourcemap-codec" + ] + }, "@lmdb/lmdb-darwin-arm64@3.5.6": { "integrity": "sha512-mY5FG4TjPAkY4P0w+OhHaUka5mDh2TX2WKYIwuKzJ1zeW3VvRgxdam/lGJTquI+bthTx5CSHDW+BAQCnNAzkEA==", "os": ["darwin"], @@ -1374,6 +1620,23 @@ "os": ["win32"], "cpu": ["x64"] }, + "@poppinss/colors@4.1.6": { + "integrity": "sha512-H9xkIdFswbS8n1d6vmRd8+c10t2Qe+rZITbbDHHkQixH5+2x1FDGmi/0K+WgWiqQFKPSlIYB7jlH6Kpfn6Fleg==", + "dependencies": [ + "kleur" + ] + }, + "@poppinss/dumper@0.6.5": { + "integrity": "sha512-NBdYIb90J7LfOI32dOewKI1r7wnkiH6m920puQ3qHUeZkxNkQiFnXVWoE6YtFSv6QOiPPf7ys6i+HWWecDz7sw==", + "dependencies": [ + "@poppinss/colors", + "@sindresorhus/is@7.2.0", + "supports-color@10.2.2" + ] + }, + "@poppinss/exception@1.2.3": { + "integrity": "sha512-dCED+QRChTVatE9ibtoaxc+WkdzOSjYTKi/+uacHWIsfodVfpsueo3+DKpgU5Px8qXjgmXkSvhXvSCz3fnP9lw==" + }, "@preact/signals-core@1.14.4": { "integrity": "sha512-HNB6HYeYKhQbJ1aKl+YRjrS4+QWHLKX6qKoUsfS/m0vqzsVaEBiZiaKbG/e+NKk2ch5ALQr/ihWaMHxiCuuWHA==" }, @@ -1835,7 +2098,7 @@ "@rollup/pluginutils@4.2.1": { "integrity": "sha512-iKnFXr7NkdZAIHiIWE+BX5ULi/ucVFYWD6TbAV+rZctiRTY2PL6tsIKhoIOaoskiWAkgu+VsbXgUVDNLHf+InQ==", "dependencies": [ - "estree-walker", + "estree-walker@2.0.2", "picomatch@2.3.2" ] }, @@ -1988,6 +2251,12 @@ "@sindresorhus/is@4.6.0": { "integrity": "sha512-t09vSN3MdfsyCHoFcTRCH/iUtG7OJ0CsjzB8cjAmKc/va/kIgeDI/TxsigdncE/4be734m0cvIYwNaV4i2XqAw==" }, + "@sindresorhus/is@7.2.0": { + "integrity": "sha512-P1Cz1dWaFfR4IR+U13mqqiGsLFf1KbayybWwdd2vfctdV6hDpUkgCY0nKOLLTMSoRd/jJNjtbqzf13K8DCCXQw==" + }, + "@speed-highlight/core@1.2.24": { + "integrity": "sha512-qeW2e1l78afw8VhRPfPQ1Gjj+KU5XFQ/OFV5ti6eTa9bruO7mJyZtA4vw0ofqmA3tKCkROE9xLk3VZoeRc98nw==" + }, "@standard-schema/spec@1.1.0": { "integrity": "sha512-l2aFy5jALhniG5HgqrD6jXLi/rUWrKvqN/qJx6yoJsgKhblVd+iqqU4RCXavm/jPityDo5TCvKMnpjKnOriy0w==" }, @@ -2117,12 +2386,22 @@ "@babel/types" ] }, + "@types/chai@5.2.3": { + "integrity": "sha512-Mw558oeA9fFbv65/y4mHtXDs9bPnFMZAL/jxdPFUpOHHIXX91mcgEHbS5Lahr+pwZFR8A7GQleRWeI6cGFC2UA==", + "dependencies": [ + "@types/deep-eql", + "assertion-error" + ] + }, "@types/debug@4.1.12": { "integrity": "sha512-vIChWdVG3LG1SMxEvI/AK+FWJthlrqlTu7fbrlywTkkaONwk/UAGaULXRlf8vkzFBLVm0zkMdCquhL5aOjhXPQ==", "dependencies": [ "@types/ms" ] }, + "@types/deep-eql@4.0.2": { + "integrity": "sha512-c9h9dVVMigMPc4bwTvC5dxqtqJZwQPePsWjPlpSOnojbor6pGqdk541lfA7AqFQr5pB1BRdq0juY9db81BwyFw==" + }, "@types/estree@1.0.9": { "integrity": "sha512-GhdPgy1el4/ImP05X05Uw4cw2/M93BCUmnEvWZNStlCzEKME4Fkk+YpoA5OiHNQmoS7Cafb8Xa3Pya8m1Qrzeg==" }, @@ -2189,6 +2468,62 @@ "@ungap/structured-clone@1.3.2": { "integrity": "sha512-5jsZFwgR5rTdKwidH9Qmat75RKwqfpKlWWB1frDkljN127mwqBu8K0PYo7/hFpF03IEJpfVPpCQDY/eDx3iHvA==" }, + "@vitest/expect@4.1.11": { + "integrity": "sha512-VX2x5vNJXET47KAFzwERI+KRMtTTCSWTfSMKsW7JsUsXV4psq++e3DvZpuTDOpHcxytiDs6p2nhVb2tVDiiUYw==", + "dependencies": [ + "@standard-schema/spec", + "@types/chai", + "@vitest/spy", + "@vitest/utils", + "chai", + "tinyrainbow" + ] + }, + "@vitest/mocker@4.1.11_vite@7.3.6__@types+node@24.13.3__tsx@4.23.1": { + "integrity": "sha512-2XJVD55d1o5AZous5CCGKS74g/riOj9odEt2bQpCVZeblHyHdnMeFl4jl0XjU21stf4mbjUkew2eXQZt65g5CQ==", + "dependencies": [ + "@vitest/spy", + "estree-walker@3.0.3", + "magic-string", + "vite" + ], + "optionalPeers": [ + "vite" + ] + }, + "@vitest/pretty-format@4.1.11": { + "integrity": "sha512-yiZzPbGTS9Sr/JpFl8zHrcIkAofNbFV6k21vIgQN/cY/oxZeXhJv5sc/MBJ5jFKWmWs+oJHw0UXLZjmf931+Vw==", + "dependencies": [ + "tinyrainbow" + ] + }, + "@vitest/runner@4.1.11": { + "integrity": "sha512-LztvUgdwMNJMIkj3hQnnxiC2Xy1zNxq928W/xhjCLaNCzqTZOudjwbQf6v9IntZGPw132i2Lq2rgTRZHD3JHNw==", + "dependencies": [ + "@vitest/utils", + "pathe" + ] + }, + "@vitest/snapshot@4.1.11": { + "integrity": "sha512-pN7ikn1ON7h8ee4gIAp4AzyK+zBtJPzVbqOgu5LCEh4VaJVbPQcgYQYJIMGQPXVeJJq1fnfazis7a5pFNPahog==", + "dependencies": [ + "@vitest/pretty-format", + "@vitest/utils", + "magic-string", + "pathe" + ] + }, + "@vitest/spy@4.1.11": { + "integrity": "sha512-apNa/prQy2qCeywhnixOHPRCgGNhvg7T4Dapfl1GahLp/R+uhBm5cPyFoNVyqsNd2h1nJxL6BqqdIjiABL60YA==" + }, + "@vitest/utils@4.1.11": { + "integrity": "sha512-zTCVGpyFsGWBhllOyKlTw/vnr6D9qxsfSDyfbyZmTyjHw5N/VuvzHpHoQjm2ZJzn4RJgx5w4r7V0er69CmLgPQ==", + "dependencies": [ + "@vitest/pretty-format", + "convert-source-map", + "tinyrainbow" + ] + }, "@x0k/json-schema-merge@1.0.4": { "integrity": "sha512-KvmMgAftbVzATq4IRnkno/SKSu+gjaR2ZUPJG5JUlY4W3twRJo03sk2914u8scmosibBZ0m7s6euZlJuqpv8Ww==", "dependencies": [ @@ -2264,6 +2599,9 @@ "tslib" ] }, + "assertion-error@2.0.1": { + "integrity": "sha512-Izi8RQcffqCeNVgFigKli1ssklIbpHnCYc6AknXGYoB6grJqyeby7jv12JUQgmTAnIDnbck1uxksT4dzN3PWBA==" + }, "b4a@1.8.1": { "integrity": "sha512-aiqre1Nr0B/6DgE2N5vwTc+2/oQZ4Wh1t4NznYY4E00y8LCt6NqdRv81so00oo27D8MVKTpUa/MwUUtBLXCoDw==" }, @@ -2308,6 +2646,9 @@ "integrity": "sha512-HYXq73DDpCtNzOmrFsm9eSwCvWCql0RzqjpDzXN9EadiLJ4DNat0nsZ/Bzmy+Ud12mb4/zKDY0cQ805ZzN+i0A==", "bin": true }, + "blake3-wasm@2.1.5": { + "integrity": "sha512-F1+K8EbfOZE49dtoPtmxUQrpXaBIl3ICvasLh+nJta0xkz+9kF/7uet9fLnwKqhDrmj6g+6K3Tw9yQPUg2ka5g==" + }, "boolbase@1.0.0": { "integrity": "sha512-JZOSA7Mo9sNGB8+UjSgzdLtokWAky1zbztM3WRLCbZ70/3cTANmQmOdR7y2g+J0e2WXywy1yS468tY+IruqEww==" }, @@ -2331,11 +2672,14 @@ "ccount@2.0.1": { "integrity": "sha512-eyrF0jiFpY+3drT6383f1qhkbGsLSifNAjA61IUjZjmLCWjItY6LB9ft9YhoDgwfmclB2zhu51Lc7+95b8NRAg==" }, + "chai@6.2.2": { + "integrity": "sha512-NUPRluOfOiTKBKvWPtSD4PhFvWCqOi0BGStNWs57X9js7XGTprSmFoz5F0tWhR4WPjNeR9jXqdC7/UpSJTnlRg==" + }, "chalk@4.1.2": { "integrity": "sha512-oKnbhFyRIXpUuez8iBMmyEa4nbj4IOQyuhc/wy9kY7/WVPcwIO9VA668Pu8RkO7+0G76SLROeyw9CpQ061i4mA==", "dependencies": [ "ansi-styles@4.3.0", - "supports-color" + "supports-color@7.2.0" ] }, "chalk@5.6.2": { @@ -2356,6 +2700,9 @@ "ci-info@4.4.0": { "integrity": "sha512-77PSwercCZU2Fc4sX94eF8k8Pxte6JAwL4/ICZLFjJLqegs7kCuAsqqj/70NQF6TvDpgFjkubQB2FW2ZZddvQg==" }, + "cjs-module-lexer@1.2.3": { + "integrity": "sha512-0TNiGstbQmCFwt4akjjBg5pLRTSyj/PkWQ1ZoO2zntmg9yLqSRxwEa4iCfQLGjqhiqBfOJa7W/E8wfGrTDmlZQ==" + }, "class-variance-authority@0.7.1": { "integrity": "sha512-Ka+9Trutv7G8M6WT6SeiRWz792K5qEqIGEGzXKhAE6xOWAY6pPH8U+9IY3oCMv6kqTmLsv7Xh/2w2RigkePMsg==", "dependencies": [ @@ -2429,6 +2776,9 @@ "convert-source-map@2.0.0": { "integrity": "sha512-Kvp459HrV2FEJ1CAsi1Ku+MY3kasH19TFykTz2xWmMeq6bk2NU3XXvfJ+Q61m0xktWwt+1HSYf3JZsTms3aRJg==" }, + "cookie@1.1.1": { + "integrity": "sha512-ei8Aos7ja0weRpFzJnEA9UHJ/7XQmqglbRwnf2ATjcB9Wq874VKH9kfjjirM6UhU2/E5fFYadylyhFldcqSidQ==" + }, "cross-spawn@7.0.6": { "integrity": "sha512-uV2QOWP2nWzsy2aMp8aRibhi9dlzF5Hgh5SHaB9OiTGEyDTiJJyx0uy51QXdyWbtAHNua4XJzUKca3OzKUd3vA==", "dependencies": [ @@ -2492,6 +2842,12 @@ "environment@1.1.0": { "integrity": "sha512-xUtoPkMggbz0MPyPiIWr1Kp4aeWJjDZ6SMvURhimjdZgsRuDplF5/s9hcgGhyXMhs+6vpnuoiZ2kFiu3FMnS8Q==" }, + "error-stack-parser-es@1.0.5": { + "integrity": "sha512-5qucVt2XcuGMcEGgWI7i+yZpmpByQ8J1lHhcL7PwqCwu9FPP3VUXzT4ltHe5i2z9dePwEHcDVOAfSnHsOlCXRA==" + }, + "es-module-lexer@2.3.2": { + "integrity": "sha512-poHGpORABojJJucnV9KbOavETW8lBVnphkW77ER5/BQ5Fz7oXSoCNek7IH3vR5nRjdsEz926ibFYX8KtLQmdyw==" + }, "esbuild-wasm@0.25.12": { "integrity": "sha512-rZqkjL3Y6FwLpSHzLnaEy8Ps6veCNo1kZa9EOfJvmWtBq5dJH4iVjfmOO6Mlkv9B0tt9WFPFmb/VxlgJOnueNg==", "bin": true @@ -2608,12 +2964,21 @@ "estree-walker@2.0.2": { "integrity": "sha512-Rfkk/Mp/DL7JVje3u18FxFujQlTNR2q6QfMSMB7AvCBx91NGj/ba3kCfza0f6dVDbw7YlRf/nDrn7pQrCCyQ/w==" }, + "estree-walker@3.0.3": { + "integrity": "sha512-7RUKfXgSMMkzt6ZuXmqapOurLGPPfgj6l9uRZ7lRGolvk0y2yocc35LdcxKC5PQZdn2DMqioAQ2NoWcrTKmm6g==", + "dependencies": [ + "@types/estree" + ] + }, "events-universal@1.0.1": { "integrity": "sha512-LUd5euvbMLpwOF8m6ivPCbhQeSiYVNb8Vs0fQ8QjXo0JTkEHpz8pxdQf0gStltaPpw0Cca8b39KxvK9cfKRiAw==", "dependencies": [ "bare-events" ] }, + "expect-type@1.4.0": { + "integrity": "sha512-KfYbmpRm0VbLjEvVa9yGwCi9GI34xvi7A/HXYWQO65CSD2u3MczUJSuwXKFIxlGsgBQizV9q5J9NHj4VG0n+pA==" + }, "expect@30.3.0": { "integrity": "sha512-1zQrciTiQfRdo7qJM1uG4navm8DayFa2TgCSRlzUyNkhcJ6XUZF3hjnpkyr3VhAqPH7i/9GkG7Tv5abz6fqz0Q==", "dependencies": [ @@ -2845,6 +3210,9 @@ "kind-of@6.0.3": { "integrity": "sha512-dcS1ul+9tmeD95T+x28/ehLgd9mENa3LsvDTtzm3vyBEO7RPptvAD+t44WVXaUjTBRcrpFeFlC8WCruUR456hw==" }, + "kleur@4.1.5": { + "integrity": "sha512-o+NO+8WrRiQEE4/7nwRJhN1HWpVmJm511pBHUxPLtp0BUISzlBplORYSmTclCnJvQq2tKu/sgl3xVpkc7ZWuQQ==" + }, "lightningcss-android-arm64@1.32.0": { "integrity": "sha512-YK7/ClTt4kAK0vo6w3X+Pnm0D2cf2vPHbhOXdoNti1Ga0al1P4TBZhwjATvjNwLEBCnKvjJc2jQgHXH0NEwlAg==", "os": ["android"], @@ -3228,6 +3596,17 @@ "micromark-util-types" ] }, + "miniflare@5.20260831.0-alpha": { + "integrity": "sha512-Hwgh1VDUiPCPGQKODQfUmy7hRAje1D55icB+9png3ueiM64rlSM87nSrtqpxAD+DlLWI4ehnYBuECaXV43zGmQ==", + "dependencies": [ + "@cspotcode/source-map-support", + "sharp", + "undici", + "workerd", + "ws", + "youch" + ] + }, "ms@2.1.3": { "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==" }, @@ -3271,7 +3650,7 @@ "node-emoji@2.2.0": { "integrity": "sha512-Z3lTE9pLaJF47NyMhd4ww1yFTAP8YhYI8SleJiHzM46Fgpm5cnNzSl9XfzFNqbaz+VlJrIj3fXQ4DeN1Rjm6cw==", "dependencies": [ - "@sindresorhus/is", + "@sindresorhus/is@4.6.0", "char-regex", "emojilib", "skin-tone" @@ -3296,6 +3675,9 @@ "object-assign@4.1.1": { "integrity": "sha512-rJgTQnkUnH1sFw8yT6VSU3zD3sWmu6sZhIseY8VX+GRu3P6F7Fu+JNDoXfklElbLJSnc3FUQHVe4cU5hj+BcUg==" }, + "obug@2.1.4": { + "integrity": "sha512-4a+OsYv9UktOJKE+l1A4OufDgdRF9PifWj+tJnHURo/P+WOxpG4GzUFL9qCalmWauao6ogiG+QvnCovwPoyAWA==" + }, "ordered-binary@1.6.1": { "integrity": "sha512-QkCdPooczexPLiXIrbVOPYkR3VO3T6v2OyKRkR1Xbhpy7/LAVXwahnRCgRp78Oe/Ehf0C/HATAxfSr6eA1oX+w==" }, @@ -3385,6 +3767,12 @@ "path-key@3.1.1": { "integrity": "sha512-ojmeN0qd+y0jszEtoY48r0Peq5dwMEkIlCOu6Q5f41lfkswXuKtYrhgoTpLnyIcHm24Uhqx+5Tqm2InSwLhE6Q==" }, + "path-to-regexp@6.3.0": { + "integrity": "sha512-Yhpw4T9C6hPpgPeA28us07OJeqZ5EzQTkbfwuhsUg0c237RomFoETJgmp2sa3F/41gfLE6G5cqcYwznmeEeOlQ==" + }, + "pathe@2.0.3": { + "integrity": "sha512-WUjGcAqP1gQacoQe+OBJsFA7Ld4DyXuUIjZ5cc75cLHvJ7dtNsTugphxIADwspS+AraAUePCKrSVtPLFj/F88w==" + }, "picocolors@1.1.1": { "integrity": "sha512-xceH2snhtb5M9liqDsmEw56le376mTZkEX/jEb/RxNFyegNul7eNslCXP9FDj/Lcu0X8KEyMceP2ntpaHrDEVA==" }, @@ -3594,6 +3982,45 @@ "integrity": "sha512-BR7VvDCVHO+q2xBEWskxS6DJE1qRnb7DxzUrogb71CWoSficBxYsiAGd+Kl0mmq/MprG9yArRkyrQxTO6XjMzA==", "bin": true }, + "semver@7.8.5": { + "integrity": "sha512-Y7/KDsb8LjooZpwaqGyulO6DQlksgCncchHGk+sZIY4SBvUocMBEFH5Ur1fI4dV+Jvl0w6cjvucaIi40puRioA==", + "bin": true + }, + "sharp@0.35.2": { + "integrity": "sha512-FVtFjtBCMiJS6yb5CX7Sop45WFMpeGw6oRKuJnXYgf/f1ms/D7LE/ZUSNxnW7rZ/dbslQWYkoqFHGPaDBtaK4w==", + "dependencies": [ + "@img/colour", + "detect-libc", + "semver@7.8.5" + ], + "optionalDependencies": [ + "@img/sharp-darwin-arm64", + "@img/sharp-darwin-x64", + "@img/sharp-freebsd-wasm32", + "@img/sharp-libvips-darwin-arm64", + "@img/sharp-libvips-darwin-x64", + "@img/sharp-libvips-linux-arm", + "@img/sharp-libvips-linux-arm64", + "@img/sharp-libvips-linux-ppc64", + "@img/sharp-libvips-linux-riscv64", + "@img/sharp-libvips-linux-s390x", + "@img/sharp-libvips-linux-x64", + "@img/sharp-libvips-linuxmusl-arm64", + "@img/sharp-libvips-linuxmusl-x64", + "@img/sharp-linux-arm", + "@img/sharp-linux-arm64", + "@img/sharp-linux-ppc64", + "@img/sharp-linux-riscv64", + "@img/sharp-linux-s390x", + "@img/sharp-linux-x64", + "@img/sharp-linuxmusl-arm64", + "@img/sharp-linuxmusl-x64", + "@img/sharp-webcontainers-wasm32", + "@img/sharp-win32-arm64", + "@img/sharp-win32-ia32", + "@img/sharp-win32-x64" + ] + }, "shebang-command@2.0.0": { "integrity": "sha512-kHxr2zZpYtdmrN1qDjrrX/Z1rR1kG8Dx+gkpK1G4eXmvXswmcE1hTWBWYUzlraYw1/yZp6YuDY77YtvbN0dmDA==", "dependencies": [ @@ -3606,6 +4033,9 @@ "shellwords-ts@3.0.1": { "integrity": "sha512-GabK4ApLMqHFRGlpgNqg8dmtHTnYHt0WUUJkIeMd3QaDrUUBEDXHSSNi3I0PzMimg8W+I0EN4TshQxsnHv1cwg==" }, + "siginfo@2.0.0": { + "integrity": "sha512-ybx0WO1/8bSBLEWXZvEd7gMW3Sn3JFlW3TvX1nREbDLRNQNaeNN8WK0meBwPdAaOI7TtRRRJn/Es1zhrrCHu7g==" + }, "sisteransi@1.0.5": { "integrity": "sha512-bLGGlR1QxBcynn2d5YmDX4MGjlZvy2MRBDRNHLJ8VI6l6+9FUiyTFNJ0IveOSP0bcXgVDPRcfGqA0pjaqUpfVg==" }, @@ -3641,6 +4071,12 @@ "escape-string-regexp" ] }, + "stackback@0.0.2": { + "integrity": "sha512-1XMJE5fQo1jGH6Y/7ebnwPOBEkIEnT4QF32d5R1+VXdXveM0IBMJt8zfaxX1P3QhVwrYe+576+jkANtSS2mBbw==" + }, + "std-env@4.2.0": { + "integrity": "sha512-oCUKSupKTHX53EyjDtuZQ64pjLJ6yYCtpmEw0goYxtjG9KpbRe8KAsl2tBUGU9DyMcJ0RwJ8GqJAFzMXcXW1Rw==" + }, "streamx@2.28.0": { "integrity": "sha512-1Yowhzjf0ivGMrTIkY9hav5TxobO9qIVqUE41fiCGMGgc3CLlf4MY+9AHmZqBWgDTue0fY9zWjYFVyf6Diuobw==", "dependencies": [ @@ -3679,6 +4115,9 @@ "boundary" ] }, + "supports-color@10.2.2": { + "integrity": "sha512-SS+jx45GF1QjgEXQx4NJZV9ImqmO2NPz5FNsIHrsDjh2YsHnawpan7SNQ1o8NuhrbHZy9AZhIoCUiCeaW/C80g==" + }, "supports-color@7.2.0": { "integrity": "sha512-qpCAvRl9stuOHveKsn7HncJRvv501qIacKzQlO/+Lwxc9+0q2wLyv4Dfvt80/DPn2pqOBsJdDiogXGR9+OvwRw==", "dependencies": [ @@ -3689,7 +4128,7 @@ "integrity": "sha512-zFObLMyZeEwzAoKCyu1B91U79K2t7ApXuQfo8OuxwXLDgcKxuwM+YvcbIhm6QWqz7mHUH1TVytR1PwVVjEuMig==", "dependencies": [ "has-flag", - "supports-color" + "supports-color@7.2.0" ] }, "tailwind-merge@3.6.0": { @@ -3740,6 +4179,12 @@ "any-promise" ] }, + "tinybench@2.9.0": { + "integrity": "sha512-0+DUvqWMValLmha6lr4kD8iAMK1HzV0/aKnCtWb9v9641TnP/MFb7Pc2bxoxQjTXAErryXVgUOfv2YqNllqGeg==" + }, + "tinyexec@1.3.0": { + "integrity": "sha512-QKAl9m8gWWGHV8jZcPeym6j+XULi6tOf1mT83WYJ4Lk2ytW/uwAWkrP0uFsdoYMdueVJ0qs26wZ+23xeB4ibNQ==" + }, "tinyglobby@0.2.17": { "integrity": "sha512-wXR/dYpcqKmfWpEdZjiKJOwCNFndD0DMnrW/cYjVGttEkBfVgcLFHoNrlj47mjOVic9yyNu65alsgF4NQyTa2g==", "dependencies": [ @@ -3750,6 +4195,9 @@ "tinypool@2.1.0": { "integrity": "sha512-Pugqs6M0m7Lv1I7FtxN4aoyToKg1C4tu+/381vH35y8oENM/Ai7f7C4StcoK4/+BSw9ebcS8jRiVrORFKCALLw==" }, + "tinyrainbow@3.1.1": { + "integrity": "sha512-yau8yJdTt989Mm0Bd/236QnzEiPf2xLLTqUZRUJOo/3CB078LSwzei343DgtJVmfJKJE3TMINY1u42SQsP6mXw==" + }, "trim-lines@3.0.1": { "integrity": "sha512-kRj8B+YHZCc9kQYdWfJB2/oUl9rA99qbowYYBtr4ui4mZyAQ2JpvVBd/6U2YloATfqBhBTSMhTpgBHtU0Mf3Rg==" }, @@ -3779,6 +4227,15 @@ "undici-types@7.18.2": { "integrity": "sha512-AsuCzffGHJybSaRrmr5eHr81mwJU3kjw6M+uprWvCXiNeN9SOGwQ3Jn8jb8m3Z6izVgknn1R0FTCEAP2QrLY/w==" }, + "undici@7.29.0": { + "integrity": "sha512-IDxfleLmmbSskfWSUATiN1nfn2rDuvnMOqb5CWR92iIfojA0Ud+ulOAAEQ57LPr9rWmsreUyf5lwyao+7GNNVw==" + }, + "unenv@2.0.0-rc.24": { + "integrity": "sha512-i7qRCmY42zmCwnYlh9H2SvLEypEFGye5iRmEMKjcGi7zk9UquigRjFtTLz0TYqr0ZGLZhaMHl/foy1bZR+Cwlw==", + "dependencies": [ + "pathe" + ] + }, "unicode-emoji-modifier-base@1.0.0": { "integrity": "sha512-yLSH4py7oFH3oG/9K+XWrz1pSi3dfUrWEnInbxMfArOfc1+33BlGPQtLsOYwvdMy11AwUBetYuaRxSPqgkq+8g==" }, @@ -3900,6 +4357,38 @@ ], "bin": true }, + "vitest@4.1.11_@opentelemetry+api@1.9.1_@types+node@24.13.3_vite@7.3.6__@types+node@24.13.3__tsx@4.23.1": { + "integrity": "sha512-fhACrNXUidIbGSBr5FlbuBkO7VWC1ZyLl0DO4CU2DrQoAPxX84Ysxs+HeGQpii5lZWV1Q4gBZTTu49mF+A6Edw==", + "dependencies": [ + "@opentelemetry/api", + "@types/node@24.13.3", + "@vitest/expect", + "@vitest/mocker", + "@vitest/pretty-format", + "@vitest/runner", + "@vitest/snapshot", + "@vitest/spy", + "@vitest/utils", + "es-module-lexer", + "expect-type", + "magic-string", + "obug", + "pathe", + "picomatch@4.0.5", + "std-env", + "tinybench", + "tinyexec", + "tinyglobby", + "tinyrainbow", + "vite", + "why-is-node-running" + ], + "optionalPeers": [ + "@opentelemetry/api", + "@types/node@24.13.3" + ], + "bin": true + }, "weak-lru-cache@1.2.2": { "integrity": "sha512-DEAoo25RfSYMuTGc9vPJzZcZullwIqRDSI9LOy+fkCJPi6hykCnfKaXTuPBDuXAUcqHXyOgFtHNp/kB2FjYHbw==" }, @@ -3910,6 +4399,47 @@ ], "bin": true }, + "why-is-node-running@2.3.0": { + "integrity": "sha512-hUrmaWBdVDcxvYqnyh09zunKzROWjbZTiNy8dBEjkS7ehEDQibXJ7XvlmtbwuTclUiIyN+CyXQD4Vmko8fNm8w==", + "dependencies": [ + "siginfo", + "stackback" + ], + "bin": true + }, + "workerd@1.20260831.1": { + "integrity": "sha512-A2LwrkBel/FnKABPfeBAMiL6v70+rugnunqQRfWsWZjlhsTZoBScWUVunMy/xLCGLjWCQL2zp39AVR6aO0jurQ==", + "optionalDependencies": [ + "@cloudflare/workerd-darwin-64", + "@cloudflare/workerd-darwin-arm64", + "@cloudflare/workerd-linux-64", + "@cloudflare/workerd-linux-arm64", + "@cloudflare/workerd-windows-64" + ], + "scripts": true, + "bin": true + }, + "wrangler@4.128.0_@cloudflare+workers-types@5.20260901.1": { + "integrity": "sha512-jNXy9e8/pbx8iqTzXPiuflnitKJZoAfEUSUUDLW87bwyeMvJ7kb3yQMSbxEcfNdfHqJW38KRcKaLljOYV4N/4w==", + "dependencies": [ + "@cloudflare/kv-asset-handler", + "@cloudflare/unenv-preset", + "@cloudflare/workers-types", + "blake3-wasm", + "esbuild@0.28.1", + "miniflare", + "path-to-regexp", + "unenv", + "workerd" + ], + "optionalDependencies": [ + "fsevents" + ], + "optionalPeers": [ + "@cloudflare/workers-types" + ], + "bin": true + }, "wrap-ansi@7.0.0": { "integrity": "sha512-YVGIj2kamLSTxw6NsZjoBxfSwsn0ycdesmc4p+Q21c5zPuZ1pl+NfxVdxPtdHvmNVOQ6XSYG4AUtyt/Fi7D16Q==", "dependencies": [ @@ -3918,6 +4448,9 @@ "strip-ansi" ] }, + "ws@8.21.0": { + "integrity": "sha512-Vsp28b7DRcimFQvrqu2Wek3z1iYxDCWqHYB8Qsnk/S4RfaCQzPGPyBNuVjJV3cd6UiKtUtp6sNM77gWvzcCH+g==" + }, "y18n@5.0.8": { "integrity": "sha512-0pfFzegeDWJHJIAmTLRP2DwHjdF5s7jo9tuztdQxAhINCdvS+3nGINqPd00AphqJR/0LhANUS6/+7SCb98YOfA==" }, @@ -3939,6 +4472,23 @@ "yargs-parser" ] }, + "youch-core@0.3.3": { + "integrity": "sha512-ho7XuGjLaJ2hWHoK8yFnsUGy2Y5uDpqSTq1FkHLK4/oqKtyUU1AFbOOxY4IpC9f0fTLjwYbslUz0Po5BpD1wrA==", + "dependencies": [ + "@poppinss/exception", + "error-stack-parser-es" + ] + }, + "youch@4.1.0-beta.10": { + "integrity": "sha512-rLfVLB4FgQneDr0dv1oddCVZmKjcJ6yX6mS4pU82Mq/Dt9a3cLZQ62pDBL4AUO+uVrCvtWz3ZFUL2HFAFJ/BXQ==", + "dependencies": [ + "@poppinss/colors", + "@poppinss/dumper", + "@speed-highlight/core", + "cookie", + "youch-core" + ] + }, "zod@4.4.3": { "integrity": "sha512-ytENFjIJFl2UwYglde2jchW2Hwm4GJFLDiSXWdTrJQBIN9Fcyp7n4DhxJEiWNAJMV1/BqWfW/kkg71UDcHJyTQ==" }, @@ -3980,6 +4530,8 @@ ], "packageJson": { "dependencies": [ + "npm:@cloudflare/vitest-plugin@1.1.3", + "npm:@cloudflare/workers-types@^5.20260831.1", "npm:@durable-streams/client@~0.2.2", "npm:@durable-streams/server@~0.3.8", "npm:@effectionx/context-api@0.6.0", @@ -3994,6 +4546,8 @@ "npm:@effectionx/test-adapter@0.7.4", "npm:@effectionx/timebox@0.4.3", "npm:@types/node@22", + "npm:@vitest/runner@4.1.11", + "npm:@vitest/snapshot@4.1.11", "npm:acorn@^8.16.0", "npm:ajv@^8.17.1", "npm:effection@4.1.0", @@ -4010,6 +4564,7 @@ "npm:tsx@^4.19.0", "npm:typescript@5", "npm:unist-util-select@5", + "npm:vitest@4.1.11", "npm:zod@^4.3.6" ] }, diff --git a/package.json b/package.json index 4e507ca7f..4a867b228 100644 --- a/package.json +++ b/package.json @@ -1,7 +1,7 @@ { "private": true, "type": "module", - "description": "executable.md — treat markdown documents as executable workflows.", + "description": "executable.md \u2014 treat markdown documents as executable workflows.", "homepage": "https://executable.md", "repository": { "type": "git", @@ -24,7 +24,6 @@ "packageManager": "pnpm@9.15.0", "dependencies": { "@durable-streams/client": "^0.2.2", - "effection": "4.1.0", "@effectionx/context-api": "0.6.0", "@effectionx/converge": "0.1.4", "@effectionx/fetch": "0.2.1", @@ -38,17 +37,20 @@ "@effectionx/timebox": "0.4.3", "acorn": "^8.16.0", "ajv": "^8.17.1", + "effection": "4.1.0", "gray-matter": "^4.0.3", "magic-string": "^0.30.21", "marked": "^17.0.4", "marked-terminal": "^7.3.0", + "mdast-util-to-string": "^4", "remark": "15", "remend": "^1.2.2", - "zod": "^4.3.6", "unist-util-select": "^5", - "mdast-util-to-string": "^4" + "zod": "^4.3.6" }, "devDependencies": { + "@cloudflare/vitest-plugin": "1.1.3", + "@cloudflare/workers-types": "^5.20260831.1", "@durable-streams/server": "^0.3.8", "@executablemd/acp": "workspace:*", "@executablemd/cli": "workspace:*", @@ -61,20 +63,31 @@ "@executablemd/testing": "workspace:*", "@executablemd/workflow": "workspace:*", "@types/node": "^22.0.0", + "@vitest/runner": "4.1.11", + "@vitest/snapshot": "4.1.11", "expect": "^30.0.0", "oxfmt": "^0.41.0", "oxlint": "1.74.0", "tsx": "^4.19.0", - "typescript": "^5.0.0" + "typescript": "^5.0.0", + "vitest": "4.1.11" }, "scripts": { "test:node": "tsx scripts/runtime-tests.ts node", "test:bun": "bun scripts/runtime-tests.ts bun", "test:deno": "deno task test", "lint": "oxlint -c .oxlintrc.json --ignore-pattern 'scripts/tests/fixtures/**' --ignore-pattern '**/npm/**' --ignore-pattern 'packages/workflow/vendor/cloudflare-computer-dofs/**' --ignore-pattern 'packages/acp/vendor/acpx/**' packages scripts .reviews/components && oxfmt --check packages scripts .reviews/components/*.ts", - "fmt": "oxfmt --write packages scripts .reviews/components/*.ts" + "fmt": "oxfmt --write packages scripts .reviews/components/*.ts", + "test:cloudflare": "vitest run --config vitest.config.ts", + "check:cloudflare": "tsc -p packages/workflow/tsconfig.cloudflare.json" }, "workspaces": [ "packages/*" - ] + ], + "pnpm": { + "overrides": { + "tsx": "4.23.1", + "@cloudflare/workers-types": "5.20260831.1" + } + } } diff --git a/packages/cli/src/deno-workflow.ts b/packages/cli/src/deno-workflow.ts index 4b234dd49..4e6d089d1 100644 --- a/packages/cli/src/deno-workflow.ts +++ b/packages/cli/src/deno-workflow.ts @@ -29,8 +29,7 @@ import { useWorkflowRunHost, withWorkflowWorkspace, } from "@executablemd/workflow/deno"; -import type { WorkflowExecutionTransitions } from "@executablemd/workflow/deno"; -import type { WorkflowRunDatabase } from "@executablemd/workflow"; +import type { WorkflowExecutionTransitions, WorkflowRunDatabase } from "@executablemd/workflow"; import type { HelperAssembly } from "@executablemd/workflow/credential-helper"; import { readDefinitionSource } from "./workflow-source.ts"; import type { WorkflowHost } from "./workflow.ts"; diff --git a/packages/cli/src/workflow-fork.ts b/packages/cli/src/workflow-fork.ts index 2d8865bf2..2c3545bd6 100644 --- a/packages/cli/src/workflow-fork.ts +++ b/packages/cli/src/workflow-fork.ts @@ -64,10 +64,7 @@ import { } from "@executablemd/workflow"; import type { ForkSelection, WorkflowRun } from "@executablemd/workflow"; import type { WorkflowRunDatabase } from "@executablemd/workflow"; -import type { - WorkflowExecutionTransitions, - WorkflowRunCreation, -} from "@executablemd/workflow/deno"; +import type { WorkflowExecutionTransitions, WorkflowRunCreation } from "@executablemd/workflow"; import type { EstablishedDefinition } from "./workflow-definition.ts"; import type { WorkflowExecution } from "./workflow.ts"; diff --git a/packages/cli/src/workflow.ts b/packages/cli/src/workflow.ts index d68fb4752..ebed19b44 100644 --- a/packages/cli/src/workflow.ts +++ b/packages/cli/src/workflow.ts @@ -90,7 +90,7 @@ import type { WorkflowExecutionBegun, WorkflowExecutionTransitions, WorkflowRunCreation, -} from "@executablemd/workflow/deno"; +} from "@executablemd/workflow"; import type { SuspensionControllerOptions, SuspensionNotice } from "@executablemd/workflow/deno"; import { SUSPENSION_REQUEST } from "@executablemd/workflow"; import { describeError } from "./props.ts"; diff --git a/packages/cli/tests/workflow-host-boundary.test.ts b/packages/cli/tests/workflow-host-boundary.test.ts new file mode 100644 index 000000000..fcd3d752e --- /dev/null +++ b/packages/cli/tests/workflow-host-boundary.test.ts @@ -0,0 +1,95 @@ +/** + * Tier WRH — the host assembly boundary a second host has to satisfy. + * + * `WorkflowHost` is four methods, and a remote host is one more implementation + * of them rather than a wider surface. That is the settled contract, and the + * way it fails quietly is by growing: a fifth method, or a transitions type only + * one adapter can name, and the "same four questions" claim stops being true + * while every existing test still passes. + * + * So both halves are pinned here. The key set is compared exactly, and the + * provider-neutral lifecycle types are imported from the package root — which + * is where they mean what they mean — so this stops compiling if they retreat + * behind a runtime-named entrypoint. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import type { Operation } from "effection"; +import type { WorkflowRunDatabase } from "@executablemd/workflow"; +import type { + WorkflowBeginRequest, + WorkflowExecutionBegun, + WorkflowExecutionTransitions, + WorkflowForkRequest, + WorkflowForkSelection, + WorkflowRunCreation, +} from "@executablemd/workflow"; +import type { WorkflowHost } from "../src/workflow.ts"; + +/** + * Compile-time proofs. `Assert` is the only instantiation that checks, so + * each of these stops compiling the moment its claim becomes false. + */ +type Assert = T; + +/** The host boundary is exactly these four methods. */ +type FourMethods = Assert< + keyof WorkflowHost extends "useRunHost" | "useLifecycle" | "useDelivery" | "attach" ? true : false +>; +const FOUR_METHODS: FourMethods = true; + +/** Every provider-neutral lifecycle type resolves through the package root. */ +type NeutralTypes = Assert< + [ + WorkflowExecutionTransitions, + WorkflowBeginRequest, + WorkflowExecutionBegun, + WorkflowForkRequest, + WorkflowForkSelection, + WorkflowRunCreation, + ] extends [unknown, unknown, unknown, unknown, unknown, unknown] + ? true + : false +>; +const NEUTRAL_TYPES: NeutralTypes = true; + +/** + * A host built only from the four methods and only from root-exported types. + * + * It answers nothing — the point is that it type-checks, which is the claim a + * second adapter depends on. + */ +function neutralHost(): WorkflowHost { + return { + useRunHost(): Operation { + throw new Error("not this test's question"); + }, + useLifecycle(): Operation { + throw new Error("not this test's question"); + }, + useDelivery(): Operation { + throw new Error("not this test's question"); + }, + attach(_database: WorkflowRunDatabase, operation: Operation): Operation { + return operation; + }, + }; +} + +describe("the workflow host boundary", () => { + it("is exactly four methods", function* () { + expect(FOUR_METHODS).toEqual(true); + expect(Object.keys(neutralHost()).toSorted()).toEqual([ + "attach", + "useDelivery", + "useLifecycle", + "useRunHost", + ]); + }); + + it("is satisfiable from the package root alone", function* () { + expect(NEUTRAL_TYPES).toEqual(true); + expect(typeof neutralHost().attach).toEqual("function"); + }); +}); diff --git a/packages/cli/tests/workflow-installation.test.ts b/packages/cli/tests/workflow-installation.test.ts index 0263f6e7e..f2a71bf0d 100644 --- a/packages/cli/tests/workflow-installation.test.ts +++ b/packages/cli/tests/workflow-installation.test.ts @@ -26,7 +26,7 @@ import { useWorkflowLifecycle, useWorkflowRunHost, } from "@executablemd/workflow/deno"; -import type { WorkflowExecutionTransitions } from "@executablemd/workflow/deno"; +import type { WorkflowExecutionTransitions } from "@executablemd/workflow"; import { Git, WorkflowLifecycle, WorkflowRunStorage } from "@executablemd/workflow"; import type { WorkflowRunDatabase, WorkflowRunStatus } from "@executablemd/workflow"; import type { Json } from "@executablemd/core"; diff --git a/packages/cli/tests/workflow-lifecycle-control.test.ts b/packages/cli/tests/workflow-lifecycle-control.test.ts index 13d2a754d..eab347016 100644 --- a/packages/cli/tests/workflow-lifecycle-control.test.ts +++ b/packages/cli/tests/workflow-lifecycle-control.test.ts @@ -24,7 +24,7 @@ import { useWorkflowLifecycle, useWorkflowRunHost, } from "@executablemd/workflow/deno"; -import type { WorkflowExecutionTransitions } from "@executablemd/workflow/deno"; +import type { WorkflowExecutionTransitions } from "@executablemd/workflow"; import { Git, suspendFor, WorkflowLifecycle } from "@executablemd/workflow"; import type { WorkflowRunDatabase } from "@executablemd/workflow"; import { collect, inlineSource, registerComponents } from "@executablemd/core"; diff --git a/packages/cli/tests/workflow-suspension.test.ts b/packages/cli/tests/workflow-suspension.test.ts index 9a14429b5..a635d3a97 100644 --- a/packages/cli/tests/workflow-suspension.test.ts +++ b/packages/cli/tests/workflow-suspension.test.ts @@ -49,7 +49,7 @@ import { useWorkflowLifecycle, useWorkflowRunHost, } from "@executablemd/workflow/deno"; -import type { WorkflowExecutionTransitions } from "@executablemd/workflow/deno"; +import type { WorkflowExecutionTransitions } from "@executablemd/workflow"; import { Git, SUSPENSION_REQUEST, suspendFor, WorkflowLifecycle } from "@executablemd/workflow"; import type { WorkflowRunDatabase } from "@executablemd/workflow"; import { workflowRunPath } from "@executablemd/workflow/deno"; diff --git a/packages/core/canonicalize.ts b/packages/core/canonicalize.ts new file mode 100644 index 000000000..cad1f446a --- /dev/null +++ b/packages/core/canonicalize.ts @@ -0,0 +1,13 @@ +/** + * @module + * + * Canonical JSON ordering, for runtimes that cannot load a Node builtin. + * + * `canonicalize` is already public from the package root. This subpath exists + * so a consumer can select it without loading the root barrel, which reaches + * `node:crypto`, `node:process` and the rest of the host surface — a Cloudflare + * Worker resolving that graph fails to typecheck, and the operation it needs is + * pure. Same function, same behavior, narrower resolution path. + */ + +export { canonicalize } from "./src/canonicalize.ts"; diff --git a/packages/core/component-name.ts b/packages/core/component-name.ts new file mode 100644 index 000000000..7f3e49c03 --- /dev/null +++ b/packages/core/component-name.ts @@ -0,0 +1,13 @@ +/** + * @module + * + * How a document spells a component name, for runtimes that cannot load the + * engine. + * + * `isComponentName` is already public from the package root. This subpath + * selects it without the root barrel, which reaches `node:crypto`, + * `node:process` and the rest of the host surface. Same function, narrower + * resolution path. + */ + +export { isComponentName } from "./src/component-name.ts"; diff --git a/packages/core/deno.json b/packages/core/deno.json index ac93c2365..1b62eeae6 100644 --- a/packages/core/deno.json +++ b/packages/core/deno.json @@ -3,6 +3,9 @@ "version": "0.9.0", "exports": { ".": "./mod.ts", + "./canonicalize": "./canonicalize.ts", + "./component-name": "./component-name.ts", + "./document-target": "./document-target.ts", "./host": "./host.ts" }, "imports": { diff --git a/packages/core/document-target.ts b/packages/core/document-target.ts new file mode 100644 index 000000000..202533cb8 --- /dev/null +++ b/packages/core/document-target.ts @@ -0,0 +1,13 @@ +/** + * @module + * + * How an exact document target is spelled, for runtimes that cannot load a + * Markdown parser. + * + * `isCanonicalDocumentTarget` is already public from the package root under + * that fuller name. This subpath selects the spelling predicate without the + * catalog and selector machinery behind it, and without the root barrel's host + * surface. Same function, narrower resolution path. + */ + +export { isCanonicalTarget as isCanonicalDocumentTarget } from "./src/document-target-spelling.ts"; diff --git a/packages/core/package.json b/packages/core/package.json index 6f80ed609..f787a04cb 100644 --- a/packages/core/package.json +++ b/packages/core/package.json @@ -5,6 +5,9 @@ "type": "module", "exports": { ".": "./mod.ts", + "./canonicalize": "./canonicalize.ts", + "./component-name": "./component-name.ts", + "./document-target": "./document-target.ts", "./host": "./host.ts" }, "dependencies": { diff --git a/packages/core/src/canonical.ts b/packages/core/src/canonical.ts index 473065517..38b7c3f6e 100644 --- a/packages/core/src/canonical.ts +++ b/packages/core/src/canonical.ts @@ -7,6 +7,11 @@ * replay would stop matching. Sorting the keys before serializing is what makes * the name depend on what the value *is*. * + * The canonicalization itself lives in `./canonicalize.ts`, which names no + * host; this module adds the digest, which needs one. Both remain exported + * from the package root, and `@executablemd/core/canonicalize` publishes the + * pure half for consumers that cannot load a Node builtin. + * * Callers compose their own identity and hash it here, rather than handing over * a shape this module defines: what belongs in a fingerprint is a property of * the thing being identified, and two callers disagree about it. `` @@ -15,31 +20,10 @@ */ import { createHash } from "node:crypto"; -import type { Json, JsonObject } from "./types.ts"; +import { canonicalize } from "./canonicalize.ts"; +import type { Json } from "./types.ts"; -/** The same value with every object's keys in sorted order. */ -export function canonicalize(value: Json): Json { - if (Array.isArray(value)) { - return value.map(canonicalize); - } - if (value === null || typeof value !== "object") { - return value; - } - const sorted: JsonObject = {}; - for (const key of Object.keys(value).sort()) { - // Defined rather than assigned: `sorted[key] = …` reaches - // `Object.prototype`'s setter for `__proto__` and drops the key on Node and - // Bun, so a schema declaring that name would canonicalize differently - // depending on where it ran. - Object.defineProperty(sorted, key, { - value: canonicalize(value[key]), - enumerable: true, - writable: true, - configurable: true, - }); - } - return sorted; -} +export { canonicalize }; /** The SHA-256 of a canonicalized value, as hex. */ export function canonicalFingerprint(value: Json): string { diff --git a/packages/core/src/canonicalize.ts b/packages/core/src/canonicalize.ts new file mode 100644 index 000000000..930688e29 --- /dev/null +++ b/packages/core/src/canonicalize.ts @@ -0,0 +1,41 @@ +/** + * A stable name for a JSON value, with no host behind it. + * + * Two values that differ only in key order are the same value, and + * `JSON.stringify` would otherwise make them different names — so a document + * that reordered a schema's properties would look like a different question and + * replay would stop matching. Sorting the keys before serializing is what makes + * the name depend on what the value *is*. + * + * This is a leaf on purpose. The operation is pure arithmetic over a JSON + * value, and it sat beside `canonicalFingerprint()`, which reaches + * `node:crypto` — so a runtime that has no Node builtins could not import one + * without the other, and a Cloudflare Worker that needs to canonicalize a + * record could not do it at all. Nothing here imports anything but a type. + */ + +import type { Json, JsonObject } from "./types.ts"; + +/** The same value with every object's keys in sorted order. */ +export function canonicalize(value: Json): Json { + if (Array.isArray(value)) { + return value.map(canonicalize); + } + if (value === null || typeof value !== "object") { + return value; + } + const sorted: JsonObject = {}; + for (const key of Object.keys(value).sort()) { + // Defined rather than assigned: `sorted[key] = …` reaches + // `Object.prototype`'s setter for `__proto__` and drops the key on Node and + // Bun, so a schema declaring that name would canonicalize differently + // depending on where it ran. + Object.defineProperty(sorted, key, { + value: canonicalize(value[key]), + enumerable: true, + writable: true, + configurable: true, + }); + } + return sorted; +} diff --git a/packages/core/src/component-name.ts b/packages/core/src/component-name.ts new file mode 100644 index 000000000..634e7d181 --- /dev/null +++ b/packages/core/src/component-name.ts @@ -0,0 +1,18 @@ +/** + * How a document spells a component name, with nothing else behind it. + * + * The grammar registration is held to, offered as a predicate so a host + * deciding what a name may be does not restate it. It answers about spelling + * alone: a name that passes may still be structural syntax, a reserved + * registration, or a name nothing supplies. + * + * A leaf, so a consumer validating a retained name — a stored workflow + * definition checking its component bundle — does not load the registration + * machinery, or the engine behind it, to ask one question about a string. + */ + +const SEGMENT = /^[A-Z][A-Za-z0-9_]*$/; + +export function isComponentName(name: string): boolean { + return name.length > 0 && name.split(".").every((segment) => SEGMENT.test(segment)); +} diff --git a/packages/core/src/components/registration.ts b/packages/core/src/components/registration.ts index 10b34314d..4f3f77420 100644 --- a/packages/core/src/components/registration.ts +++ b/packages/core/src/components/registration.ts @@ -17,6 +17,9 @@ import type { Context, Operation } from "effection"; import { Component } from "../component-api.ts"; import { updateOwn } from "../scope-local.ts"; import { RESERVED_STRUCTURAL } from "../structural.ts"; +import { isComponentName } from "../component-name.ts"; + +export { isComponentName }; import { compilePropsSchema, compileReturnsSchema } from "../validate.ts"; import type { ComponentRegistry, @@ -79,20 +82,6 @@ const OwnContributions: Context = createContext( new Map(), ); -const SEGMENT = /^[A-Z][A-Za-z0-9_]*$/; - -/** - * Whether `name` is spelled the way a document writes a component name. - * - * The grammar registration is held to, offered as a predicate so a host - * deciding what a name may be does not restate it. It answers about spelling - * alone: a name that passes may still be structural syntax, a reserved - * registration, or a name nothing supplies. - */ -export function isComponentName(name: string): boolean { - return name.length > 0 && name.split(".").every((segment) => SEGMENT.test(segment)); -} - function kindOf(registration: ComponentRegistration): Kind { return registration.reserved === true ? "reserved" : "default"; } diff --git a/packages/core/src/document-target-spelling.ts b/packages/core/src/document-target-spelling.ts new file mode 100644 index 000000000..370892b76 --- /dev/null +++ b/packages/core/src/document-target-spelling.ts @@ -0,0 +1,123 @@ +/** + * How an exact document target is spelled, with no host and no parser behind + * it. + * + * Percent-encoding a label, decoding one, normalizing it, and asking whether a + * fragment is already canonical are string arithmetic. They live apart from the + * catalog and selector machinery that uses them because a consumer that only + * needs to validate a retained target — a stored workflow definition checking + * the one it kept — should not have to load a Markdown parser, or a runtime + * that has one, to do it. + */ + +const UNRESERVED = /^[A-Za-z0-9\-._~]$/; +const HEX = /^[0-9A-Fa-f]$/; + +const ENCODER = new TextEncoder(); + +function encodeCharacter(character: string): string { + let encoded = ""; + for (const byte of ENCODER.encode(character)) { + encoded += `%${byte.toString(16).toUpperCase().padStart(2, "0")}`; + } + return encoded; +} + +/** + * Percent-encode one canonical label. Everything outside RFC 3986's unreserved + * set is escaped, so `/`, `*`, `#`, and `%` inside a heading cannot be read as + * hierarchy or operator syntax. + */ +export function encodeTargetLabel(label: string): string { + let encoded = ""; + for (const character of label) { + encoded += UNRESERVED.test(character) ? character : encodeCharacter(character); + } + return encoded; +} + +/** + * Percent-encode a decoded filesystem path. Separators survive as raw `/`; a + * `/` that is part of a filename cannot be told apart from one afterwards, so + * this is a formatter for paths the caller already holds, not a round trip. + */ +export function encodeDocumentPath(path: string): string { + let encoded = ""; + for (const character of path) { + encoded += + character === "/" || UNRESERVED.test(character) ? character : encodeCharacter(character); + } + return encoded; +} + +/** + * Decode one percent-encoded chunk, or `undefined` when it is not decodable. + * + * Malformed escapes, byte sequences that are not UTF-8, and NUL are all + * refused rather than repaired: a selector that cannot be read exactly is not a + * selector this can match against. `+` is an ordinary character — this is URI + * path syntax, not a form encoding. + */ +export function decodePercentEncoded(text: string): string | undefined { + const characters = Array.from(text); + const bytes: number[] = []; + for (let index = 0; index < characters.length; index++) { + const character = characters[index]!; + if (character !== "%") { + for (const byte of ENCODER.encode(character)) { + bytes.push(byte); + } + continue; + } + const high = characters[index + 1]; + const low = characters[index + 2]; + if (high === undefined || low === undefined || !HEX.test(high) || !HEX.test(low)) { + return undefined; + } + bytes.push(Number.parseInt(`${high}${low}`, 16)); + index += 2; + } + try { + // `ignoreBOM` is stated rather than defaulted: it is already false + // everywhere this runs, and Cloudflare's own type declares both options + // required, so saying it keeps one spelling readable to every runtime. + const decoded = new TextDecoder("utf-8", { fatal: true, ignoreBOM: false }).decode( + new Uint8Array(bytes), + ); + return decoded.includes("\u0000") ? undefined : decoded; + } catch { + return undefined; + } +} + +/** + * The canonical form of rendered heading text: NFC, every run of Unicode + * whitespace collapsed to one ASCII space, trimmed, case preserved. + */ +export function normalizeLabel(text: string): string { + return text.normalize("NFC").replace(/\s+/gu, " ").trim(); +} + +/** + * Whether a fragment is already an exact canonical target. + * + * A level is canonical only when decoding it, normalizing the label, and + * re-encoding that label reproduce the level byte for byte. Requiring the whole + * round trip is what makes this total: it rejects a wildcard operator, an empty + * level, a lowercase escape, a raw `#`, an NFD spelling, a tab, and leading, + * trailing, or uncollapsed whitespace without naming any of them, because none + * of them is what this module would have written. + */ +export function isCanonicalTarget(target: string): boolean { + if (target.length === 0) { + return false; + } + return target.split("/").every((level) => { + const decoded = decodePercentEncoded(level); + if (decoded === undefined || decoded.length === 0) { + return false; + } + const label = normalizeLabel(decoded); + return label === decoded && encodeTargetLabel(label) === level; + }); +} diff --git a/packages/core/src/document-targets.ts b/packages/core/src/document-targets.ts index 6567a8519..6a5897c97 100644 --- a/packages/core/src/document-targets.ts +++ b/packages/core/src/document-targets.ts @@ -28,6 +28,21 @@ import { remark } from "remark"; import { toString as mdastToString } from "mdast-util-to-string"; import type { ComponentSpan } from "./scanner.ts"; +import { + decodePercentEncoded, + encodeDocumentPath, + encodeTargetLabel, + isCanonicalTarget, + normalizeLabel, +} from "./document-target-spelling.ts"; + +export { + decodePercentEncoded, + encodeDocumentPath, + encodeTargetLabel, + isCanonicalTarget, + normalizeLabel, +}; /** A half-open slice of the original document body. */ export interface SourceRange { @@ -518,113 +533,6 @@ function sameList(left: readonly string[], right: readonly string[]): boolean { return left.length === right.length && left.every((item, index) => item === right[index]); } -const UNRESERVED = /^[A-Za-z0-9\-._~]$/; -const HEX = /^[0-9A-Fa-f]$/; - -const ENCODER = new TextEncoder(); - -function encodeCharacter(character: string): string { - let encoded = ""; - for (const byte of ENCODER.encode(character)) { - encoded += `%${byte.toString(16).toUpperCase().padStart(2, "0")}`; - } - return encoded; -} - -/** - * Percent-encode one canonical label. Everything outside RFC 3986's unreserved - * set is escaped, so `/`, `*`, `#`, and `%` inside a heading cannot be read as - * hierarchy or operator syntax. - */ -export function encodeTargetLabel(label: string): string { - let encoded = ""; - for (const character of label) { - encoded += UNRESERVED.test(character) ? character : encodeCharacter(character); - } - return encoded; -} - -/** - * Percent-encode a decoded filesystem path. Separators survive as raw `/`; a - * `/` that is part of a filename cannot be told apart from one afterwards, so - * this is a formatter for paths the caller already holds, not a round trip. - */ -export function encodeDocumentPath(path: string): string { - let encoded = ""; - for (const character of path) { - encoded += - character === "/" || UNRESERVED.test(character) ? character : encodeCharacter(character); - } - return encoded; -} - -/** - * Decode one percent-encoded chunk, or `undefined` when it is not decodable. - * - * Malformed escapes, byte sequences that are not UTF-8, and NUL are all - * refused rather than repaired: a selector that cannot be read exactly is not a - * selector this can match against. `+` is an ordinary character — this is URI - * path syntax, not a form encoding. - */ -export function decodePercentEncoded(text: string): string | undefined { - const characters = Array.from(text); - const bytes: number[] = []; - for (let index = 0; index < characters.length; index++) { - const character = characters[index]!; - if (character !== "%") { - for (const byte of ENCODER.encode(character)) { - bytes.push(byte); - } - continue; - } - const high = characters[index + 1]; - const low = characters[index + 2]; - if (high === undefined || low === undefined || !HEX.test(high) || !HEX.test(low)) { - return undefined; - } - bytes.push(Number.parseInt(`${high}${low}`, 16)); - index += 2; - } - try { - const decoded = new TextDecoder("utf-8", { fatal: true }).decode(new Uint8Array(bytes)); - return decoded.includes("\u0000") ? undefined : decoded; - } catch { - return undefined; - } -} - -/** - * The canonical form of rendered heading text: NFC, every run of Unicode - * whitespace collapsed to one ASCII space, trimmed, case preserved. - */ -export function normalizeLabel(text: string): string { - return text.normalize("NFC").replace(/\s+/gu, " ").trim(); -} - -/** - * Whether a fragment is already an exact canonical target. - * - * A level is canonical only when decoding it, normalizing the label, and - * re-encoding that label reproduce the level byte for byte. Requiring the whole - * round trip is what makes this total: it rejects a wildcard operator, an empty - * level, a lowercase escape, a raw `#`, an NFD spelling, a tab, and leading, - * trailing, or uncollapsed whitespace without naming any of them, because none - * of them is what this module would have written. - */ -export function isCanonicalTarget(target: string): boolean { - if (target.length === 0) { - return false; - } - return target.split("/").every((level) => { - const decoded = decodePercentEncoded(level); - if (decoded === undefined || decoded.length === 0) { - return false; - } - const label = normalizeLabel(decoded); - return label === decoded && encodeTargetLabel(label) === level; - }); -} - type LevelPart = | { readonly kind: "literal"; readonly text: string } | { readonly kind: "wildcard" }; diff --git a/packages/core/tests/canonicalize.test.ts b/packages/core/tests/canonicalize.test.ts new file mode 100644 index 000000000..a2d8b2d06 --- /dev/null +++ b/packages/core/tests/canonicalize.test.ts @@ -0,0 +1,63 @@ +/** + * The pure half of canonicalization, and the host-capable half beside it. + * + * `canonicalize()` moved into a leaf so a runtime without Node builtins can + * reach it — a Cloudflare Worker validating a retained record needs the key + * ordering and not the digest. The risk in that move is two implementations + * that drift, so what is asserted here is that there is exactly one: the + * package root and the subpath answer identically, and the fingerprint that + * composes over it is unchanged. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { canonicalize as fromRoot, canonicalFingerprint } from "@executablemd/core"; +import { canonicalize as fromSubpath } from "@executablemd/core/canonicalize"; +import { isComponentName as componentNameFromRoot } from "@executablemd/core"; +import { isComponentName as componentNameFromSubpath } from "@executablemd/core/component-name"; +import { isCanonicalDocumentTarget as targetFromRoot } from "@executablemd/core"; +import { isCanonicalDocumentTarget as targetFromSubpath } from "@executablemd/core/document-target"; +import type { Json } from "@executablemd/core"; + +/** Values chosen for the properties canonicalization is about. */ +const VALUES: Json[] = [ + null, + 0, + "text", + [3, 1, 2], + { b: 1, a: 2 }, + { outer: { z: [{ y: 1, x: 2 }], a: null } }, + // The name whose ordinary assignment would reach `Object.prototype`. + { ["__proto__"]: { polluted: true }, after: 1 }, +]; + +describe("canonicalization through both paths", () => { + it("answers identically from the package root and the subpath", function* () { + for (const value of VALUES) { + expect(JSON.stringify(fromSubpath(value))).toEqual(JSON.stringify(fromRoot(value))); + } + }); + + it("still sorts keys and leaves arrays in order", function* () { + expect(JSON.stringify(fromSubpath({ b: 1, a: 2 }))).toEqual('{"a":2,"b":1}'); + expect(JSON.stringify(fromSubpath([3, 1, 2]))).toEqual("[3,1,2]"); + }); + + it("keeps the fingerprint composing over the same ordering", function* () { + // The digest is the half that needs a host; it is unchanged by the split. + expect(canonicalFingerprint({ b: 1, a: 2 })).toEqual(canonicalFingerprint({ a: 2, b: 1 })); + expect(canonicalFingerprint({ a: 1 })).not.toEqual(canonicalFingerprint({ a: 2 })); + expect(canonicalFingerprint({ a: 1 })).toMatch(/^[0-9a-f]{64}$/); + }); +}); + +describe("the other predicates a retained descriptor validates with", () => { + it("answers identically from the package root and the subpath", function* () { + for (const name of ["Repository", "Ns.Sub", "lower", "", "9Bad", "A_1"]) { + expect(componentNameFromSubpath(name)).toEqual(componentNameFromRoot(name)); + } + for (const target of ["Heading", "A/B", "", "a%2Fb", "Lower case", "%2f", "Tab\there"]) { + expect(targetFromSubpath(target)).toEqual(targetFromRoot(target)); + } + }); +}); diff --git a/packages/workflow/cloudflare.ts b/packages/workflow/cloudflare.ts new file mode 100644 index 000000000..3b2f079c0 --- /dev/null +++ b/packages/workflow/cloudflare.ts @@ -0,0 +1,44 @@ +/** + * @module + * + * The Cloudflare host's workflow-run owner. + * + * Keeping this behind its own entrypoint is what lets the shared package stay + * provider-neutral, exactly as `./deno` does for the local host. Durable + * Objects, the runtime's SQLite, WebSocket acquisition and OIDC admission live + * here and nowhere above; `@executablemd/workflow` names none of them, so the + * Deno host is unaffected by this module existing and neither host has to know + * the other does. + * + * What an operator assembles is the owner and its policy: + * + * ```ts + * import { WorkflowOwnerObject } from "@executablemd/workflow/cloudflare"; + * + * export class WorkflowOwner extends WorkflowOwnerObject { + * protected configuration() { + * return { policy: POLICY }; + * } + * } + * ``` + * + * Provider endpoints, OIDC tokens, credentials, private message shapes, + * storage handles and acquisition evidence are deliberately absent from what + * this publishes. They are host closure state, and a value a document or a + * runner could name would be authority a document or a runner could hold. + */ + +export { WorkflowOwnerObject, refusalOf } from "./src/cloudflare/owner.ts"; +export type { AdmissionRequest, OwnerConfiguration } from "./src/cloudflare/owner.ts"; + +export { AdmissionError } from "./src/cloudflare/admission.ts"; +export type { AdmissionPolicy, AdmissionRefusal } from "./src/cloudflare/admission.ts"; + +export { ReleaseIdentityError } from "./src/cloudflare/release.ts"; +export type { ReleaseRefusal } from "./src/cloudflare/release.ts"; + +export { admitRunId, ownerFor, RunIdError } from "./src/cloudflare/routing.ts"; +export type { OwnerNamespace, RunIdRefusal } from "./src/cloudflare/routing.ts"; + +export { WorkflowObjectStorageError } from "./src/cloudflare/recognition.ts"; +export type { RecognitionFailure } from "./src/cloudflare/recognition.ts"; diff --git a/packages/workflow/deno.json b/packages/workflow/deno.json index ed6dfab38..ab4ae02a3 100644 --- a/packages/workflow/deno.json +++ b/packages/workflow/deno.json @@ -5,6 +5,7 @@ "exports": { ".": "./mod.ts", "./deno": "./deno.ts", + "./software-factory": "./software-factory.ts", "./credential-helper": "./src/deno/composition/credential-helper.ts" }, "publish": { diff --git a/packages/workflow/deno.ts b/packages/workflow/deno.ts index 368e3ea1c..012dada52 100644 --- a/packages/workflow/deno.ts +++ b/packages/workflow/deno.ts @@ -28,6 +28,15 @@ export { useWorkflowRunStorage } from "./src/deno/provider.ts"; export type { WorkflowRunStorageOptions } from "./src/deno/provider.ts"; export { useWorkflowLifecycle } from "./src/deno/lifecycle.ts"; export { useWorkflowRunHost } from "./src/deno/run-host.ts"; +/** + * Re-exported for source compatibility only. + * + * These are provider-neutral: they describe what any host's lifecycle does, not + * what this adapter retains, and `@executablemd/workflow` owns their meaning. + * Import them from there. What belongs behind this entrypoint is the + * implementation and its retained encoding — SQLite, DOFS, run-id hashing, + * filesystem paths — not the shape of a request. + */ export type { WorkflowBeginRequest, WorkflowExecutionTransitions, diff --git a/packages/workflow/mod.ts b/packages/workflow/mod.ts index 85f09ea39..18a0a049c 100644 --- a/packages/workflow/mod.ts +++ b/packages/workflow/mod.ts @@ -314,6 +314,18 @@ export type { WorkflowLifecycleApi, WorkflowLifecycleSnapshot, } from "./src/lifecycle/api.ts"; +// What a trusted host needs to move a run's lifecycle. These describe what any +// host's lifecycle does rather than what one adapter retains, so this entrypoint +// owns their meaning; `./deno` re-exports them for source compatibility and a +// second host implements the same shapes without that module being loaded. +export type { + WorkflowBeginRequest, + WorkflowExecutionBegun, + WorkflowExecutionTransitions, + WorkflowForkRequest, + WorkflowForkSelection, + WorkflowRunCreation, +} from "./src/lifecycle/execution.ts"; // The export request, its result and the boundary it names. The retained record // shapes an artifact also carries are DOFS and SQLite rows, so they are the // Deno entrypoint's to publish rather than this one's. diff --git a/packages/workflow/package.json b/packages/workflow/package.json index bb7e1758e..e7ffd74a5 100644 --- a/packages/workflow/package.json +++ b/packages/workflow/package.json @@ -6,6 +6,8 @@ "exports": { ".": "./mod.ts", "./deno": "./deno.ts", + "./cloudflare": "./cloudflare.ts", + "./software-factory": "./software-factory.ts", "./credential-helper": "./src/deno/composition/credential-helper.ts" }, "dependencies": { diff --git a/packages/workflow/software-factory.ts b/packages/workflow/software-factory.ts new file mode 100644 index 000000000..e5d70442c --- /dev/null +++ b/packages/workflow/software-factory.ts @@ -0,0 +1,47 @@ +/** + * @module + * + * The GitHub Actions software factory's public identity rule. + * + * This subpath is deliberately not the package root. `@executablemd/workflow` + * names no provider — that is what lets a second host implement the same + * lifecycle — and the derivation here names GitHub in its scheme tag, its + * authority rule and its node id, because the software factory is a GitHub + * product by definition rather than one adapter of a neutral boundary. + * + * So the two surfaces are separate on purpose. Anything that needs the factory's + * own contract asks for it by name: + * + * ```ts + * import { deriveFactoryRunId } from "@executablemd/workflow/software-factory"; + * + * const runId = yield* deriveFactoryRunId({ + * authority: "github.com", + * issueNodeId: node, + * }); + * ``` + * + * One issue is one durable run, so this is the whole of "one issue, one run": + * every host that admits the same issue arrives at the same 52 characters + * without asking anybody. It is specified in + * `specs/github-actions-software-factory-spec.md` §1.1 and restated in + * `specs/workflow-spec.md` §9.1. + * + * The seam is deliberately small: admit a subject, or derive its id. The scheme + * tag, the Base32 alphabet, the authority rule, the preimage layout and the + * encoder are implementation, not promises — a caller that could reach them + * could also reimplement the hash, and two implementations of an identity that + * must agree byte for byte is the failure §1.1 exists to prevent. + * + * Nothing here is runtime-specific. It uses the cross-runtime Web primitives — + * `TextEncoder` and `crypto.subtle` — and names no host, so the provider host + * and a GitHub intake reach the same single implementation rather than each + * carrying a hash that has to agree byte for byte with the other's. + */ + +export { + admitFactoryRunSubject, + deriveFactoryRunId, + FactoryRunSubjectError, +} from "./src/software-factory/run-id.ts"; +export type { FactoryRunSubject, FactoryRunSubjectFailure } from "./src/software-factory/run-id.ts"; diff --git a/packages/workflow/src/cloudflare/acquisition.ts b/packages/workflow/src/cloudflare/acquisition.ts new file mode 100644 index 000000000..c1cd862ec --- /dev/null +++ b/packages/workflow/src/cloudflare/acquisition.ts @@ -0,0 +1,173 @@ +/** + * Executor ownership, as one authenticated WebSocket. + * + * The acquisition *is* the connection. There is no lease, expiry, renewal, + * heartbeat, alarm, PID or liveness poll: a healthy socket owns the run, and a + * socket that closes stops owning it because the runtime stops listing it. That + * is the same shape the local host has, where the operating system releases an + * advisory lock when the executor exits, and it is why nothing here has to + * decide whether an absent executor is slow or gone. + * + * Hibernation is why ownership cannot live in a field. An idle Durable Object + * is evicted while its sockets stay open, so the object that wakes up has no + * memory of what it admitted. The runtime hands back the live sockets and the + * bounded attachment each was accepted with, and that pair is the authority: + * `ctx.getWebSockets()` says which sockets are real, and the attachment says + * what one was admitted as. + * + * Attachment bytes alone are not authority. A copy of them proves nothing, + * because the check is not "does this value look right" but "is the socket this + * message arrived on the one live socket carrying an acquisition". A second + * connection cannot manufacture that by holding a copy. + */ + +import type { OwnerStorage } from "./storage.ts"; + +/** What one admitted connection carries, and all it carries. */ +export interface AcquisitionAttachment { + readonly kind: "executor"; + readonly runId: string; + readonly acquisitionId: string; +} + +/** Why an acquisition was refused. */ +export type AcquisitionRefusal = + | "already-running" + | "not-acquired" + | "foreign-connection" + | "wrong-run"; + +export class AcquisitionError extends Error { + override name = "AcquisitionError"; + + constructor(readonly refusal: AcquisitionRefusal) { + super(`this connection does not own this run's executor (${refusal})`); + } +} + +/** The bits of a Durable Object's context this module uses. */ +export interface AcquisitionContext { + getWebSockets(tag?: string): WebSocket[]; + acceptWebSocket(socket: WebSocket, tags?: string[]): void; + readonly storage: OwnerStorage; +} + +/** The tag every executor connection is accepted under. */ +export const EXECUTOR_TAG = "executor"; + +function attachmentOf(socket: WebSocket): AcquisitionAttachment | undefined { + const value = socket.deserializeAttachment(); + if (value === null || typeof value !== "object" || Array.isArray(value)) { + return undefined; + } + const members: Map = new Map(Object.entries(value)); + if (members.get("kind") !== "executor") { + return undefined; + } + const runId = members.get("runId"); + const acquisitionId = members.get("acquisitionId"); + if (typeof runId !== "string" || typeof acquisitionId !== "string") { + return undefined; + } + return { kind: "executor", runId, acquisitionId }; +} + +/** + * Every live connection currently holding an acquisition. + * + * A socket the runtime still lists but whose attachment was cleared is not one: + * closing releases ownership immediately, while the runtime may take its own + * time to stop listing the socket, and ownership must end at the earlier of the + * two. + */ +export function acquisitionHolders( + ctx: AcquisitionContext, +): { socket: WebSocket; held: AcquisitionAttachment }[] { + const found: { socket: WebSocket; held: AcquisitionAttachment }[] = []; + for (const socket of ctx.getWebSockets(EXECUTOR_TAG)) { + const held = attachmentOf(socket); + if (held !== undefined) { + found.push({ socket, held }); + } + } + return found; +} + +/** + * Admit one connection as this run's executor. + * + * A second healthy executor is refused rather than followed: it cannot advance + * the run, and the caller learns that from the refusal rather than from a + * mutation that quietly did nothing. + */ +export function acquireExecutor( + ctx: AcquisitionContext, + socket: WebSocket, + runId: string, + acquisitionId: string, + beforeAccept: () => void = () => undefined, +): AcquisitionAttachment { + if (acquisitionHolders(ctx).length > 0) { + throw new AcquisitionError("already-running"); + } + beforeAccept(); + const attachment: AcquisitionAttachment = { kind: "executor", runId, acquisitionId }; + ctx.acceptWebSocket(socket, [EXECUTOR_TAG]); + // Bounded, and only what admission needs to be reconstructed after an + // eviction. Nothing here is a credential and nothing here is durable run + // state. + socket.serializeAttachment(attachment); + return attachment; +} + +/** + * Prove that a message arrived on the one live acquisition. + * + * Called before the requested mutation is parsed, and again — by the caller — + * inside the transaction that writes, because a socket can close between the + * two and the transaction is where the run actually changes. + */ +export function requireAcquisition( + ctx: AcquisitionContext, + socket: WebSocket, + runId: string, +): AcquisitionAttachment { + const mine = requireExecutorSocket(ctx, socket); + if (mine.runId !== runId) { + throw new AcquisitionError("wrong-run"); + } + return mine; +} + +export function requireExecutorSocket( + ctx: AcquisitionContext, + socket: WebSocket, +): AcquisitionAttachment { + const live = acquisitionHolders(ctx); + if (live.length === 0) { + throw new AcquisitionError("not-acquired"); + } + const mine = live.find((holder) => holder.socket === socket); + if (mine === undefined) { + // Either this socket was never admitted, or it was superseded and closed. + throw new AcquisitionError("foreign-connection"); + } + if (live.length > 1) { + // Two live holders is a state this module refuses to choose between. + throw new AcquisitionError("already-running"); + } + return mine.held; +} + +/** + * Release ownership when a connection ends. + * + * The runtime has already stopped listing the socket by the time this runs, so + * there is nothing to revoke — this exists to make the absence of a rollback + * explicit. A closed connection invalidates the acquisition and changes no + * committed state, and it settles no lifecycle: an executor that disappeared + * did not decide anything. + */ +export function releaseExecutor(socket: WebSocket): void { + socket.serializeAttachment(null); +} diff --git a/packages/workflow/src/cloudflare/admission.ts b/packages/workflow/src/cloudflare/admission.ts new file mode 100644 index 000000000..53a7ee687 --- /dev/null +++ b/packages/workflow/src/cloudflare/admission.ts @@ -0,0 +1,142 @@ +/** + * Who is allowed to become this run's executor. + * + * A runner authenticates with a GitHub Actions OIDC token, and the owner + * validates it before the connection is accepted and before an acquisition + * exists. Everything checked here is an identity the deployment configured, and + * the checks are on IDs rather than names: a repository can be renamed and an + * owner can be renamed, so a check on `repository` would admit whoever holds + * the name today. + * + * The claims reaching this module have already been proved to come from the + * issuer — `token.ts` verifies the signature, the algorithm and the temporal + * validity first. That order is the whole security property: comparing claim + * values a caller could have written is arithmetic, not authentication. + * + * Nothing about the token survives the check. The raw JWT, the JWKS endpoint, + * the claims this contract does not name, and the reason a signature failed are + * all provider state: none of them reaches durable storage, a journal event, a + * public value or an error message. What a refusal says is which category it + * fell into, because that is what an operator can act on and what a test can + * assert without pinning provider wording. + */ + +import type { Operation } from "effection"; +import { type TokenVerification, verifyToken } from "./token.ts"; + +/** What a deployment must state before any runner can be admitted. */ +export interface AdmissionPolicy { + readonly issuer: string; + readonly audience: string; + readonly repositoryId: string; + readonly repositoryOwnerId: string; + readonly eventName: string; + readonly workflowRef: string; + readonly workflowSha: string; + /** The immutable identity of the workflow allowed to execute this run. */ + readonly jobWorkflowRef: string; + /** The exact build both sides must be. */ + readonly release: string; +} + +/** + * The claims this contract reads. + * + * Deliberately a closed set. A token carries far more than this, and reading a + * claim here is what makes it part of the contract — so anything not named is + * not consulted, cannot be depended on, and never leaves the verifier. + */ +export interface ActionsClaims { + readonly iss: unknown; + readonly aud: unknown; + readonly repository_id: unknown; + readonly repository_owner_id: unknown; + readonly event_name: unknown; + readonly workflow_ref: unknown; + readonly workflow_sha: unknown; + readonly job_workflow_ref: unknown; +} + +/** Which part of the admission a token failed. */ +export type AdmissionRefusal = + | "token-absent" + | "token-malformed" + | "issuer" + | "audience" + | "repository-id" + | "repository-owner-id" + | "event-name" + | "workflow-ref" + | "workflow-sha" + | "workflow-identity"; + +export class AdmissionError extends Error { + override name = "AdmissionError"; + + constructor(readonly refusal: AdmissionRefusal) { + super(`this runner is not admitted to execute this run (${refusal})`); + } +} + +/** Compare one claim, naming the check rather than the values. */ +function requireClaim(claim: unknown, expected: string, refusal: AdmissionRefusal): void { + if (typeof claim !== "string" || claim !== expected) { + throw new AdmissionError(refusal); + } +} + +/** + * Hold verified claims to the configured policy. + * + * Private to this module's own admission path. It is not exported, because an + * exported "check these claims" is exactly the surface that made the previous + * revision forgeable: a caller reaching it directly would be a caller choosing + * its own identity. Reaching it goes through `admitToken()`, which verifies + * first. + */ +function admitClaims(policy: AdmissionPolicy, claims: ActionsClaims): void { + requireClaim(claims.iss, policy.issuer, "issuer"); + // `aud` may be a string or an array of them; only the exact configured + // audience admits, and an array containing it is that audience. + const audience = claims.aud; + const audiences = Array.isArray(audience) ? audience : [audience]; + if (!audiences.some((value) => value === policy.audience)) { + throw new AdmissionError("audience"); + } + requireClaim(claims.repository_id, policy.repositoryId, "repository-id"); + requireClaim(claims.repository_owner_id, policy.repositoryOwnerId, "repository-owner-id"); + requireClaim(claims.event_name, policy.eventName, "event-name"); + requireClaim(claims.workflow_ref, policy.workflowRef, "workflow-ref"); + requireClaim(claims.workflow_sha, policy.workflowSha, "workflow-sha"); + requireClaim(claims.job_workflow_ref, policy.jobWorkflowRef, "workflow-identity"); +} + +/** Read a claim set out of a verified payload. */ +function parseClaims(payload: Map): ActionsClaims { + return { + iss: payload.get("iss"), + aud: payload.get("aud"), + repository_id: payload.get("repository_id"), + repository_owner_id: payload.get("repository_owner_id"), + event_name: payload.get("event_name"), + workflow_ref: payload.get("workflow_ref"), + workflow_sha: payload.get("workflow_sha"), + job_workflow_ref: payload.get("job_workflow_ref"), + }; +} + +/** + * Verify a token and hold what it proved to the configured policy. + * + * The only way into this module. It takes the bytes a runner presented and the + * verification material the deployment configured, and nothing a request can + * name reaches either. + */ +export function* admitToken( + policy: AdmissionPolicy, + verification: TokenVerification, + token: unknown, +): Operation { + const payload = yield* verifyToken(verification, token); + admitClaims(policy, parseClaims(payload)); +} diff --git a/packages/workflow/src/cloudflare/client.ts b/packages/workflow/src/cloudflare/client.ts new file mode 100644 index 000000000..0ae1b4092 --- /dev/null +++ b/packages/workflow/src/cloudflare/client.ts @@ -0,0 +1,936 @@ +/** + * The runner's side of the private protocol. + * + * This is the only place that knows both languages. Above it, `src/remote/**` + * speaks in workflow records and Workspace roots; below it, the connection + * carries private commands and a private refusal union. Translating between + * them here is what keeps the neutral code neutral, and what keeps the private + * shapes private. + * + * Nothing arrives as a semantic value because the owner said so. A performed + * answer is parsed into a record, a manifest or a verified content piece before + * anything above can see it, and a refusal is narrowed to the exact union this + * release declares. Both sides are the same build — admission proved that — so + * a category this build has never heard of is not a new failure to report + * upward, it is a channel that is not what it claims to be, and the connection + * fails closed. + * + * Content is verified again on arrival. The owner validated it before sending, + * and that says nothing about what happened in between; a digest is cheap and + * the alternative is materializing bytes that are not the bytes the root names. + * + * The journal is reassembled here from anchored pages, and the assembly is + * checked rather than assumed: each page must continue the previous one, name + * no event twice, and end exactly at the anchor. A page that skipped, repeated + * or reordered an event closes the connection before a single event reaches a + * caller — half a journal that looks whole is worse than no journal. + */ + +import { Err, Ok, type Operation, type Result } from "effection"; +import { serializeDurableEvent } from "@executablemd/durable-streams"; +import type { JournalEntry } from "../storage/api.ts"; +import { parseMembers, requireMemberNames } from "../storage/members.ts"; +import type { DefinitionRetrieval, WorkflowRunRecord } from "../storage/record.ts"; +import type { CommitIntent, OwnerLink, StartingFrontier } from "../remote/collector.ts"; +import type { CommitDecision } from "../remote/publication.ts"; +import { OwnerLinkError, type OwnerAnswer, type OwnerConnection } from "../remote/client.ts"; +import { + parseRemoteExecution, + parseRemoteInvocationSnapshot, + parseRemoteJournalEntry, + type RemoteInvocationSnapshot, + parseRemoteRetrieval, + parseRemoteRunRecord, + RemoteRecordError, +} from "../remote/records.ts"; +import { + type RemoteContent, + type RemoteContentRequest, + type RemoteFrontierSnapshot, + type RemoteReadLink, + startingFrontier, +} from "../remote/read.ts"; +import { + parseWorkspaceRootManifest, + SHA256, + WORKSPACE_ROOT_DOMAIN, + type WorkspaceRootManifest, +} from "../workspace/root-manifest.ts"; +import { decodeContentManifest } from "../workspace/content-manifest.ts"; +import { + EXECUTION_PAGE_BYTES, + EXECUTION_PAGE_ENTRIES, + executionPageBytes, + JOURNAL_PAGE_ENTRIES, + MAX_CONTENT_BYTES, +} from "./commands.ts"; +import type { RemoteRunLink, RemoteWorkspaceLink } from "../remote/database.ts"; +import { isSchemaVersion, SCHEMA_VERSION } from "../sqlite/workflow-schema.ts"; +import { canonicalJson } from "../storage/record.ts"; +import { + WorkflowDatabaseCorruptError, + WorkflowDatabaseFormatError, + WorkflowSchemaVersionError, + WorkflowRecordMalformedError, + WorkflowRequestError, + WorkflowStorageError, + WorkflowTransactionError, +} from "../storage/errors.ts"; +import type { DocumentExecutionRecord } from "../storage/record.ts"; +import { decodeBase64, encodeBase64, sha256Hex } from "./encoding.ts"; + +export type PrivateRefusal = + | "acquisition:already-running" + | "acquisition:not-acquired" + | "acquisition:foreign-connection" + | "acquisition:wrong-run" + | "command:not-an-object" + | "command:unknown-command" + | "command:unknown-member" + | "command:malformed-member" + | "command:too-large" + | "command:duplicate-conflict" + | "command:capacity" + | "command:unavailable" + | "command:stale-root" + | "command:stale-journal" + | "command:mapping-conflict" + | "storage:foreign" + | `storage:unsupported-version-v${number}` + | "storage:corrupt"; + +export class CloudflareOwnerRefusalError extends Error { + override name = "CloudflareOwnerRefusalError"; + + constructor(readonly refusal: PrivateRefusal) { + super(`the workflow owner refused the request (${refusal})`); + } +} + +interface FrontierHeader { + readonly record: WorkflowRunRecord; + readonly retrieval: DefinitionRetrieval | undefined; + readonly workspaceRootId: string; + readonly journalEventId: string | null; +} + +interface JournalPage { + readonly anchorEventId: string | null; + readonly afterEventId: string | null; + readonly entries: readonly { + readonly previousEventId: string | null; + readonly entry: JournalEntry; + }[]; + readonly done: boolean; +} + +function fail(reason: string): never { + throw new RemoteRecordError(`the owner returned a malformed private answer: ${reason}`); +} + +function members(value: unknown, names: readonly string[]): Map { + const found = parseMembers(value, "$", (reason) => new RemoteRecordError(reason)); + requireMemberNames(found, names, "$", (reason) => new RemoteRecordError(reason)); + if (found.size !== names.length || names.some((name) => !found.has(name))) { + return fail("it omitted a declared member"); + } + return found; +} + +function rootId(value: unknown): string { + if (typeof value !== "string" || !SHA256.test(value)) { + return fail("it did not name a canonical Workspace root"); + } + return value; +} + +function nullableIdentity(value: unknown): string | null { + if (value === null) { + return null; + } + if (typeof value !== "string" || value === "") { + return fail("it did not name an event identity"); + } + return value; +} + +function privateRefusal(value: string): PrivateRefusal { + switch (value) { + case "acquisition:already-running": + case "acquisition:not-acquired": + case "acquisition:foreign-connection": + case "acquisition:wrong-run": + case "command:not-an-object": + case "command:unknown-command": + case "command:unknown-member": + case "command:malformed-member": + case "command:too-large": + case "command:duplicate-conflict": + case "command:capacity": + case "command:unavailable": + case "command:stale-root": + case "command:stale-journal": + case "command:mapping-conflict": + case "storage:foreign": + case "storage:corrupt": + return value; + default: { + // The one category that carries a value: the schema version the owner + // actually read, bounded and parsed rather than guessed. + const unsupported = readUnsupportedVersion(value); + if (unsupported !== undefined) { + return `storage:unsupported-version-v${unsupported}`; + } + return fail("it named an unknown refusal category"); + } + } +} + +function answer(offered: OwnerAnswer): T { + if (offered.outcome === "refused") { + throw new CloudflareOwnerRefusalError(privateRefusal(offered.refusal)); + } + return offered.value; +} + +function parseFrontier(value: unknown): FrontierHeader { + const found = members(value, ["record", "retrieval", "workspaceRootId", "journalEventId"]); + return { + record: parseRemoteRunRecord(found.get("record")), + retrieval: parseRemoteRetrieval(found.get("retrieval")), + workspaceRootId: rootId(found.get("workspaceRootId")), + journalEventId: nullableIdentity(found.get("journalEventId")), + }; +} + +function parseJournalPage(value: unknown): JournalPage { + const found = members(value, ["anchorEventId", "afterEventId", "entries", "done"]); + const offered = found.get("entries"); + if (!Array.isArray(offered) || offered.length > JOURNAL_PAGE_ENTRIES) { + return fail("it did not contain one bounded journal page"); + } + if (typeof found.get("done") !== "boolean") { + return fail("it did not say whether the journal page was terminal"); + } + return { + anchorEventId: nullableIdentity(found.get("anchorEventId")), + afterEventId: nullableIdentity(found.get("afterEventId")), + entries: offered.map((entry) => { + const item = members(entry, ["eventId", "previousEventId", "record", "workspaceRootId"]); + return { + previousEventId: nullableIdentity(item.get("previousEventId")), + entry: parseRemoteJournalEntry({ + eventId: item.get("eventId"), + record: item.get("record"), + workspaceRootId: item.get("workspaceRootId"), + }), + }; + }), + done: found.get("done") === true, + }; +} + +function parseAnchoredJournalPage( + value: unknown, + anchorEventId: string, + afterEventId: string | null, + seen: ReadonlySet, +): JournalPage { + const page = parseJournalPage(value); + if ( + page.anchorEventId !== anchorEventId || + page.afterEventId !== afterEventId || + page.entries.length === 0 + ) { + return fail("a journal page did not continue its anchored snapshot"); + } + let previous = afterEventId; + const found = new Set(seen); + for (const item of page.entries) { + if (item.previousEventId !== previous) { + return fail("an anchored journal page skipped or reordered an event"); + } + if (found.has(item.entry.eventId)) { + return fail("an anchored journal repeated an event"); + } + found.add(item.entry.eventId); + previous = item.entry.eventId; + } + if ((page.done && previous !== anchorEventId) || (!page.done && previous === anchorEventId)) { + return fail("an anchored journal page disagreed with its terminal event"); + } + return page; +} + +function parseRoot(value: unknown): { workspaceRootId: string; manifest: WorkspaceRootManifest } { + const found = members(value, ["workspaceRootId", "manifest"]); + const identity = rootId(found.get("workspaceRootId")); + const manifest = found.get("manifest"); + if ( + typeof manifest !== "string" || + new TextEncoder().encode(manifest).length > MAX_CONTENT_BYTES + ) { + return fail("it did not contain one bounded root manifest"); + } + const parsed = parseWorkspaceRootManifest(manifest, fail); + if (sha256Hex(`${WORKSPACE_ROOT_DOMAIN}${manifest}`) !== identity) { + return fail("the root manifest disagreed with its identity"); + } + return { workspaceRootId: identity, manifest: parsed }; +} + +function parseContent(value: unknown): RemoteContent { + const found = members(value, ["kind", "digest", "size", "bytes"]); + const kind = found.get("kind"); + if (kind !== "manifest" && kind !== "blob") { + return fail("it did not name a content kind"); + } + const digest = rootId(found.get("digest")); + const size = found.get("size"); + const encoded = found.get("bytes"); + if ( + typeof size !== "number" || + !Number.isSafeInteger(size) || + size < 1 || + size > MAX_CONTENT_BYTES || + typeof encoded !== "string" + ) { + return fail("it did not contain one bounded content piece"); + } + const bytes = decodeBase64(encoded); + if (bytes.length !== size || sha256Hex(bytes) !== digest) { + return fail("the content disagreed with its identity or size"); + } + if (kind === "manifest") { + decodeContentManifest(bytes, fail); + } + return { kind, digest, bytes }; +} + +/** + * The schema version an unsupported-version refusal names, if it names one. + * + * The grammar covers exactly the versions the owner can recognize as + * unsupported, so a same-release owner and client never disagree about whether + * a refusal is readable. Anything else is not this category. + */ +function readUnsupportedVersion(refusal: string): number | undefined { + const found = /^storage:unsupported-version-v(\d{1,10})$/.exec(refusal); + if (found === null) { + return undefined; + } + const version = Number(found[1]); + return isSchemaVersion(version) ? version : undefined; +} + +export function cloudflareReadLink( + connection: OwnerConnection, + nextId: () => string, + expectedRunId: string, +): RemoteReadLink { + return { + *invocationSnapshot(): Operation { + return answer( + yield* connection.ask( + nextId(), + { command: "mappings" }, + (value) => parseRemoteInvocationSnapshot(value), + privateRefusal, + ), + ); + }, + + *frontier(): Operation { + const header = answer( + yield* connection.ask( + nextId(), + { command: "frontier" }, + (value) => { + const parsed = parseFrontier(value); + if (parsed.record.runId !== expectedRunId) { + return fail("a frontier answer named another run"); + } + return parsed; + }, + privateRefusal, + ), + ); + const entries: JournalEntry[] = []; + const seen = new Set(); + let afterEventId: string | null = null; + let done = header.journalEventId === null; + while (!done) { + const page: JournalPage = answer( + yield* connection.ask( + nextId(), + { command: "journal", anchorEventId: header.journalEventId, afterEventId }, + (value) => + parseAnchoredJournalPage(value, header.journalEventId ?? "", afterEventId, seen), + privateRefusal, + ), + ); + for (const item of page.entries) { + const entry = item.entry; + seen.add(entry.eventId); + entries.push(entry); + afterEventId = entry.eventId; + } + done = page.done; + } + return { ...header, entries }; + }, + *root(workspaceRootId: string): Operation { + const read = answer( + yield* connection.ask( + nextId(), + { command: "root", workspaceRootId }, + (value) => { + const parsed = parseRoot(value); + if (parsed.workspaceRootId !== workspaceRootId) { + return fail("a root answer named another root"); + } + return parsed; + }, + privateRefusal, + ), + ); + return read.manifest; + }, + *content(workspaceRootId, request: RemoteContentRequest): Operation { + const read = answer( + yield* connection.ask( + nextId(), + { + command: "content", + workspaceRootId, + kind: request.kind, + digest: request.digest, + sourceManifest: request.kind === "blob" ? request.manifestDigest : null, + }, + (value) => { + const parsed = parseContent(value); + if (parsed.kind !== request.kind || parsed.digest !== request.digest) { + return fail("a content answer named another piece"); + } + return parsed; + }, + privateRefusal, + ), + ); + return read; + }, + }; +} + +/** + * The runner's production link to its owner. + * + * `commit()` is the whole publication path: stage the pieces the owner does not + * have, encode one closed command, send it, and read the decision. The command + * identity is minted once per intent and reused verbatim on a retry, because + * the owner recognizes a retry by that identity and a regenerated one would be + * a second proposal rather than the same question asked again. + */ +export function cloudflareOwnerLink( + connection: OwnerConnection, + reads: RemoteReadLink, + nextId: () => string, +): OwnerLink { + return { + *frontier(): Operation { + return startingFrontier(yield* reads.frontier()); + }, + + *commit(intent: CommitIntent): Operation> { + // Derived from the request rather than counted. The owner recognizes a + // retry by this identity, so retrying one proposal has to produce the + // identity it already decided — a counter would make the second attempt a + // second question, and the owner would apply it again. + const request = commitRequest(intent); + const id = commandIdentity(request); + try { + yield* stageMissing(connection, nextId, intent); + const answered = yield* ask(connection, id, request, intent); + return answered.outcome === "refused" + ? Err(new CloudflareOwnerRefusalError(answered.refusal)) + : Ok(answered.decision); + } catch (error) { + if (error instanceof OwnerLinkError || error instanceof RemoteRecordError) { + // The connection went while the answer was in flight, or the owner + // answered in a way this build cannot read. Whether the owner + // committed is exactly what cannot be known from either, so the + // caller learns the outcome is undecided rather than being told it + // failed — retrying this same id is what settles it. + return Err(error); + } + throw error; + } + }, + }; +} + +/** + * The identity one closed command is known by. + * + * A digest of the exact bytes that will be sent, so two attempts at the same + * proposal share an identity and two different proposals cannot. It is bounded + * well inside the correlation limit and carries nothing about the run: it is a + * name for a request, not a fact about the Workspace. + */ +function commandIdentity(request: Record): string { + return `commit-${sha256Hex(JSON.stringify(request))}`; +} + +/** + * One command sent and one answer read, checked against what was asked. + * + * A performed answer is not taken on its word. It has to name the root this + * proposal selected — the proposed one when there is a publication, the + * unchanged expected one when there is not — and one event identity for each + * event that was sent. An owner agreeing to something else is not an owner this + * runner can go on talking to: it would promote a Workspace nobody proposed, so + * the channel fails closed instead. + */ +function* ask( + connection: OwnerConnection, + id: string, + request: Record, + intent: CommitIntent, +): Operation< + | { outcome: "performed"; decision: CommitDecision } + | { outcome: "refused"; refusal: PrivateRefusal } +> { + const selected = + intent.publication === null + ? intent.expectedWorkspaceRootId + : intent.publication.proposedWorkspaceRootId; + const offered = yield* connection.ask( + id, + request, + (value): CommitDecision => { + const found = members(value, ["workspaceRootId", "journalEventIds"]); + const workspaceRootId = rootId(found.get("workspaceRootId")); + const ids = found.get("journalEventIds"); + if (!Array.isArray(ids) || ids.some((entry) => typeof entry !== "string" || entry === "")) { + return fail("a commit answer did not name the events it retained"); + } + if (workspaceRootId !== selected) { + return fail("a commit answer named a Workspace root this proposal did not select"); + } + if (ids.length !== intent.events.length) { + return fail("a commit answer did not retain one identity for each proposed event"); + } + return Object.freeze({ workspaceRootId, journalEventIds: Object.freeze([...ids]) }); + }, + privateRefusal, + ); + return offered.outcome === "refused" + ? { outcome: "refused", refusal: privateRefusal(offered.refusal) } + : { outcome: "performed", decision: offered.value }; +} + +/** + * Send the pieces the owner does not already hold. + * + * Staging is idempotent by identity, so a retry after an ambiguous answer + * re-offers the same bytes and the owner recognizes them rather than storing + * them twice. Anything the owner already has is not sent at all: content is + * addressed by what it is, and re-uploading a Workspace it never lost would be + * bytes crossing for nothing. + */ +function* stageMissing( + connection: OwnerConnection, + nextId: () => string, + intent: CommitIntent, +): Operation { + if (intent.publication === null) { + return; + } + for (const piece of intent.publication.content) { + const bytes = intent.bytes.get(piece.digest); + if (bytes === undefined) { + // The owner is expected to hold this one already. If it does not, the + // commit refuses rather than this guessing at bytes it does not have. + continue; + } + // The sealed bytes have to be the piece they were sealed as. Staging + // something else would mean the command identity described one proposal and + // the content described another. + if (bytes.length !== piece.size || sha256Hex(bytes) !== piece.digest) { + return fail("a sealed content piece does not match the identity it was proposed under"); + } + yield* stageCloudflareContent(connection, nextId(), piece.kind, bytes); + } +} + +/** The one closed command a complete intent becomes. */ +function commitRequest(intent: CommitIntent): Record { + return { + command: "commit", + expectedWorkspaceRootId: intent.expectedWorkspaceRootId, + expectedJournalEventId: intent.expectedJournalEventId, + publication: + intent.publication === null + ? null + : { + proposedWorkspaceRootId: intent.publication.proposedWorkspaceRootId, + proposedManifest: intent.publication.proposedManifest, + content: intent.publication.content.map((piece) => ({ + kind: piece.kind, + digest: piece.digest, + size: piece.size, + })), + }, + mappings: intent.mappings.map((mapping) => + mapping.kind === "repository" + ? { kind: mapping.kind, record: { ...mapping.record }, locator: mapping.locator } + : { kind: mapping.kind, record: { ...mapping.record } }, + ), + // Exactly what the serializer produces, in the order the transaction + // appended them. The owner parses each one and requires these same bytes. + events: intent.events.map((event) => serializeDurableEvent(event)), + }; +} + +export function* stageCloudflareContent( + connection: OwnerConnection, + id: string, + kind: RemoteContent["kind"], + bytes: Uint8Array, +): Operation<{ kind: RemoteContent["kind"]; digest: string; size: number }> { + if (bytes.length === 0 || bytes.length > MAX_CONTENT_BYTES) { + return fail("the staged content is outside the private piece bound"); + } + const digest = sha256Hex(bytes); + return answer( + yield* connection.ask( + id, + { command: "stage", kind, digest, bytes: encodeBase64(bytes) }, + (value) => { + const found = members(value, ["kind", "digest", "size"]); + if ( + found.get("kind") !== kind || + found.get("digest") !== digest || + found.get("size") !== bytes.length + ) { + return fail("a staging answer named another content piece"); + } + return { kind, digest, size: bytes.length }; + }, + privateRefusal, + ), + ); +} + +/** + * The runner's production link to everything the database asks for. + * + * Wraps the publication link with the two reads and one mutation the database + * needs, so a handle receives one seam rather than assembling the protocol + * itself. Every answer is parsed and cross-checked against the request before + * it becomes a semantic value, and every failure crosses as a provider-neutral + * storage error rather than as a private refusal. + */ +/** + * One run's whole owner link, from one connection. + * + * The read link is made here rather than accepted, so the reads a Workspace + * invocation is admitted from and the commits it publishes cannot be two + * different owners. A caller holding this holds one authority. + */ +export function cloudflareRunLink( + connection: OwnerConnection, + nextId: () => string, + expectedRunId: string, +): RemoteWorkspaceLink { + const reads = cloudflareReadLink(connection, nextId, expectedRunId); + const publication = cloudflareOwnerLink(connection, reads, nextId); + return { + ...reads, + + /** + * Both halves of the publication link, translated. + * + * The database returns these failures through a provider-neutral interface, + * so a private refusal or a transport error must not travel as itself. This + * is the one place that translation happens. + */ + *frontier(): Operation { + try { + return yield* publication.frontier(); + } catch (error) { + throw translate(error); + } + }, + + *commit(intent: CommitIntent): Operation> { + try { + const committed = yield* publication.commit(intent); + return committed.ok ? committed : Err(translate(committed.error)); + } catch (error) { + return Err(translate(error)); + } + }, + + *frontierSnapshot(): Operation { + try { + return yield* reads.frontier(); + } catch (error) { + throw translate(error); + } + }, + + *replaceRetrieval( + expectedWorkspaceRootId: string, + metadata: string | null, + ): Operation> { + // One identity per invocation, minted here. Two calls carrying identical + // metadata are two replacements and must not collapse into one, so the + // identity is not derived from the request's content. + const id = nextId(); + try { + const answered = yield* connection.ask( + id, + { command: "retrieval", expectedWorkspaceRootId, metadata }, + (value) => { + const found = members(value, ["retrieval"]); + const held = found.get("retrieval"); + if (held === null) { + if (metadata !== null) { + return fail("a retrieval answer cleared a replacement that was not a clear"); + } + return undefined; + } + const parsed = parseRemoteRetrieval(held); + if (parsed === undefined || metadata === null) { + return fail("a retrieval answer disagreed with the replacement it answered"); + } + // Compared here, where the answer arrives. An owner that performed + // a different replacement than the one asked for is a channel the + // two sides disagree on, so it fails closed rather than handing + // back a value the caller would have to notice was wrong. + if (canonicalJson(parsed.metadata) !== metadata) { + return fail("a retrieval answer named metadata the request did not ask for"); + } + return parsed; + }, + privateRefusal, + ); + return answered.outcome === "refused" + ? Err(storageFailure(privateRefusal(answered.refusal))) + : Ok(answered.value); + } catch (error) { + return Err(translate(error)); + } + }, + + *readExecutions(): Operation> { + try { + const found: DocumentExecutionRecord[] = []; + let anchor: number | null | undefined; + let after: number | null = null; + let done = false; + while (!done) { + const page: ExecutionPage = yield* askPage( + connection, + nextId(), + expectedRunId, + anchor ?? null, + after, + ); + // The first page chooses the snapshot. Every later one is held to it, + // and to the cursor it was asked to continue from. + const expected = anchor === undefined ? page.anchor : anchor; + anchor = expected; + if (page.anchor !== expected || page.after !== after) { + return Err(pageFailure("a page did not continue its anchored snapshot")); + } + if (page.anchor === null) { + // An empty snapshot is terminal and carries nothing. + if (page.rows.length > 0 || !page.done || after !== null) { + return Err(pageFailure("an empty snapshot carried rows or did not terminate")); + } + break; + } + if (page.rows.length === 0) { + // A page with nothing in it can only be the empty snapshot, which + // was handled above. Otherwise the read would never advance. + return Err(pageFailure("a page of an anchored snapshot carried no rows")); + } + let previous: number = after ?? 0; + for (const row of page.rows) { + if (row.sequence !== previous + 1) { + // Exactly adjacent: a gap would be retained history omitted from + // a snapshot that claims to be complete. + return Err(pageFailure("a page skipped, repeated or reordered a row")); + } + if (row.sequence > page.anchor) { + return Err(pageFailure("a page carried a row outside its snapshot")); + } + previous = row.sequence; + found.push(row.record); + } + if (page.done !== (previous === page.anchor)) { + // Terminal exactly at the anchor, and only there. + return Err(pageFailure("a page disagreed with its terminal row")); + } + after = previous; + done = page.done; + } + return Ok(found); + } catch (error) { + return Err(translate(error)); + } + }, + }; +} + +/** What a page that does not describe the snapshot it claims becomes. */ +function pageFailure(reason: string): WorkflowStorageError { + return new WorkflowRecordMalformedError("document executions", reason); +} + +/** One execution page, with the private ordering the runner checks adjacency by. */ +interface ExecutionPage { + readonly anchor: number | null; + readonly after: number | null; + readonly rows: readonly { readonly sequence: number; readonly record: DocumentExecutionRecord }[]; + readonly done: boolean; +} + +function* askPage( + connection: OwnerConnection, + id: string, + expectedRunId: string, + anchor: number | null, + after: number | null, +): Operation { + const answered = yield* connection.ask( + id, + { command: "executions", anchor, after }, + (value): ExecutionPage => { + const found = members(value, ["runId", "anchor", "after", "rows", "done"]); + if (found.get("runId") !== expectedRunId) { + // Another run's retained history is not this run's, however well formed. + return fail("an execution page named another run"); + } + const offered = found.get("rows"); + if (!Array.isArray(offered) || offered.length > EXECUTION_PAGE_ENTRIES) { + return fail("an execution page was not one bounded page"); + } + if (executionPageBytes(offered) > EXECUTION_PAGE_BYTES) { + // The page bound, not the message envelope. A page that ignored it + // would make the number of requests depend on how large one row is. + return fail("an execution page carried more than one page of rows"); + } + if (typeof found.get("done") !== "boolean") { + return fail("an execution page did not say whether it was terminal"); + } + const rows = offered.map((entry) => { + const item = members(entry, ["sequence", "record"]); + const sequence = item.get("sequence"); + if (typeof sequence !== "number" || !Number.isSafeInteger(sequence) || sequence < 1) { + return fail("an execution row did not carry a position"); + } + return { sequence, record: parseRemoteExecution(item.get("record")) }; + }); + return { + anchor: nullableSequence(found.get("anchor")), + after: nullableSequence(found.get("after")), + rows, + done: found.get("done") === true, + }; + }, + privateRefusal, + ); + if (answered.outcome === "refused") { + throw new CloudflareOwnerRefusalError(privateRefusal(answered.refusal)); + } + return answered.value; +} + +function nullableSequence(value: unknown): number | null { + if (value === null) { + return null; + } + if (typeof value !== "number" || !Number.isSafeInteger(value) || value < 1) { + return fail("an execution page did not name a position"); + } + return value; +} + +/** + * The provider-neutral failure one private refusal becomes. + * + * A caller learns the category the local host would have reported for the same + * condition. Command names, refusal spellings, rows and cursors stay below this + * line: they describe a protocol nobody above here is party to. + */ +function storageFailure(refusal: PrivateRefusal): WorkflowStorageError { + // A host acts on these differently: storage belonging to something else may + // not be written, a version this build does not implement may not be + // migrated, and damage may not be repaired. Collapsing them would make all + // three look like the one that says "restore from a backup". + if (refusal === "storage:foreign") { + return new WorkflowDatabaseFormatError(REMOTE_STORE, "it belongs to something else"); + } + const unsupported = readUnsupportedVersion(refusal); + if (unsupported !== undefined) { + return new WorkflowSchemaVersionError(REMOTE_STORE, unsupported, SCHEMA_VERSION); + } + if (refusal === "storage:corrupt") { + return new WorkflowDatabaseCorruptError(REMOTE_STORE, "its retained records do not agree"); + } + if (refusal === "command:stale-root" || refusal === "command:stale-journal") { + return new WorkflowTransactionError( + "this run has moved since the operation read it, so the change was not applied.", + ); + } + if (refusal === "command:capacity") { + return new WorkflowRequestError("this run's owner cannot accept more work on this connection."); + } + return new WorkflowTransactionError("this run's owner refused the operation."); +} + +/** + * What a public error names instead of a path. + * + * A remote run has no file, and naming one would be an invitation to look for + * it. The store is named as what it is. + */ +const REMOTE_STORE = "this run's remote storage"; + +/** + * Any failure from the private protocol, as a provider-neutral one. + * + * Nothing private crosses: not a refusal class, not a refusal spelling, not the + * message a parser wrote about a value it refused. A record this build cannot + * read is a malformed record rather than an unreachable owner, because those + * are different facts and a caller acts on them differently. + */ +function translate(error: unknown): WorkflowStorageError { + if (error instanceof CloudflareOwnerRefusalError) { + return storageFailure(error.refusal); + } + if (error instanceof WorkflowStorageError) { + return error; + } + if (error instanceof RemoteRecordError) { + return new WorkflowRecordMalformedError( + "record this run's owner returned", + "it is not a record this build can read", + ); + } + if (error instanceof OwnerLinkError) { + if (error.refusal === "too-large") { + // The channel measured the whole request and never sent it. That is a + // request this caller cannot make, not an owner it could not reach, and + // the two lead a host to do different things. + return new WorkflowRequestError( + "this request is larger than one message may carry, so it was not sent.", + ); + } + return new WorkflowTransactionError("this run's owner could not be reached."); + } + return new WorkflowTransactionError("this run's owner could not answer the operation."); +} diff --git a/packages/workflow/src/cloudflare/commands.ts b/packages/workflow/src/cloudflare/commands.ts new file mode 100644 index 000000000..0cdd1f8d9 --- /dev/null +++ b/packages/workflow/src/cloudflare/commands.ts @@ -0,0 +1,599 @@ +import { + type DocumentExecutionCompletion, + parseDocumentExecutionCompletion, +} from "../storage/record.ts"; +import { parseDurableEvent, serializeDurableEvent } from "@executablemd/durable-streams"; +import { SHA256 } from "../workspace/root-manifest.ts"; +import { admitLocator, locatorFingerprintOf } from "../composition/locator.ts"; +import { + parseRepositoryRecord, + parseWorktreeRecord, + type RepositoryRecord, + type WorktreeRecord, +} from "../composition/records.ts"; +import { type AgentSessionRecord, parseAgentSessionRecord } from "../storage/agent-session.ts"; +import { MAX_MESSAGE_BYTES } from "../remote/client.ts"; + +export { MAX_MESSAGE_BYTES }; + +export const MAX_CONTENT_BYTES = 1024 * 1024; +export const MAX_STAGED_BYTES = 2 * 1024 * 1024; +export const MAX_COMMANDS = 256; +export const MAX_LEDGER_BYTES = 2 * 1024 * 1024; +export const JOURNAL_PAGE_ENTRIES = 128; +export const JOURNAL_PAGE_BYTES = 512 * 1024; +/** The most document-execution rows one private page carries. */ +export const EXECUTION_PAGE_ENTRIES = 128; +/** The most serialized bytes of retained execution rows one page carries. */ +export const EXECUTION_PAGE_BYTES = 512 * 1024; + +/** + * How both ends measure one execution page. + * + * One function rather than two similar sums: the owner decides what fits and + * the runner checks it, and if they measured different things an honest page + * near the bound would be sent by one and refused by the other. What is + * measured is the exact `rows` member as it crosses, wrappers and punctuation + * included, because that is what the bound is about. + */ +export function executionPageBytes(rows: readonly unknown[]): number { + return new TextEncoder().encode(JSON.stringify(rows)).length; +} +/** The most content identities one proposal may name. */ +export const MAX_PROPOSED_PIECES = 8192; +/** The most retained mapping changes one proposal may carry. */ +export const MAX_MAPPINGS = 256; +/** The longest canonical root manifest this owner reads. */ +export const MAX_ROOT_MANIFEST_BYTES = MAX_CONTENT_BYTES; + +export type CommandName = + | "frontier" + | "journal" + | "root" + | "content" + | "stage" + | "commit" + | "retrieval" + | "executions" + | "mappings" + | "settle"; + +export type CommandRefusal = + | "not-an-object" + | "unknown-command" + | "unknown-member" + | "malformed-member" + | "too-large" + | "duplicate-conflict" + | "capacity" + | "unavailable" + // The frontier moved under the proposal. Not malformed and not a conflict of + // identity: the request was true when it was built and is not true now. + | "stale-root" + | "stale-journal" + // A retained mapping already exists and describes something else. Creation + // identity is immutable, so this is refused rather than rewritten. + | "mapping-conflict"; + +export class CommandError extends Error { + override name = "CommandError"; + + constructor(readonly refusal: CommandRefusal) { + super(`this owner refused a runner command (${refusal})`); + } +} + +export interface CommandEnvelope { + readonly id: string; + readonly command: CommandName; +} + +export interface FrontierCommand extends CommandEnvelope { + readonly command: "frontier"; +} + +export interface JournalCommand extends CommandEnvelope { + readonly command: "journal"; + readonly anchorEventId: string | null; + readonly afterEventId: string | null; +} + +export interface RootCommand extends CommandEnvelope { + readonly command: "root"; + readonly workspaceRootId: string; +} + +export type ContentKind = "manifest" | "blob"; + +export interface ContentCommand extends CommandEnvelope { + readonly command: "content"; + readonly workspaceRootId: string; + readonly kind: ContentKind; + readonly digest: string; + readonly sourceManifest: string | null; +} + +export interface StageCommand extends CommandEnvelope { + readonly command: "stage"; + readonly kind: ContentKind; + readonly digest: string; + readonly bytes: string; +} + +/** + * One closed proposal, and everything the owner needs to decide it. + * + * The earlier shape carried a proposed root identity and nothing that could + * justify it — an identity with no manifest and no content closure is a name, + * not a proposal, and an owner adopting one would be taking the runner's word + * for what a root contains. This carries the whole thing: what the runner + * started from, what it proposes, the canonical manifest that identity is the + * digest of, the exact content that manifest closes over, the retained mappings + * the same operation produced, and the filtered events to append. + * + * `publication` is absent for a transaction that only appended to the journal. + * That is a real case rather than a degenerate one, and inventing a Workspace + * change to fill it would publish a root nothing asked for. + */ +export interface CommitCommand extends CommandEnvelope { + readonly command: "commit"; + readonly expectedWorkspaceRootId: string; + readonly expectedJournalEventId: string | null; + readonly publication: ProposedPublication | null; + readonly mappings: readonly ProposedMapping[]; + /** Exactly what `serializeDurableEvent` produced, terminating newline included. */ + readonly events: readonly string[]; +} + +/** The Workspace half of a proposal, when there is one. */ +export interface ProposedPublication { + readonly proposedWorkspaceRootId: string; + readonly proposedManifest: string; + readonly content: readonly ProposedPiece[]; +} + +/** One content identity the proposed root closes over. */ +export interface ProposedPiece { + readonly kind: ContentKind; + readonly digest: string; + readonly size: number; +} + +/** One retained mapping the proposal carries, already parsed. */ +export type ProposedMapping = + | { readonly kind: "repository"; readonly record: RepositoryRecord; readonly locator: string } + | { readonly kind: "worktree"; readonly record: WorktreeRecord } + | { readonly kind: "agent-session"; readonly record: AgentSessionRecord }; + +/** + * Replace or clear where the definition can be fetched from. + * + * Its own mutation rather than a degenerate commit: it appends no journal + * event, publishes no root, and its revision is authoritative rather than + * proposed. `metadata` is `null` to clear, which is a different act from + * writing an empty object — clearing removes the row and the next replacement + * starts counting again. + * + * The expected root travels with it so the owner can refuse a replacement + * proposed against a frontier that has moved, the same way a commit is refused. + */ +export interface RetrievalCommand extends CommandEnvelope { + readonly command: "retrieval"; + readonly expectedWorkspaceRootId: string; + /** Canonical JSON, already encoded by the runner, or `null` to clear. */ + readonly metadata: string | null; +} + +/** + * One page of the document executions this run has begun. + * + * Anchored like the journal: the first page fixes the last execution that + * existed when the read began, and every later page is constrained to it, so an + * execution started while the read is in flight cannot appear halfway through. + */ +export interface ExecutionsCommand extends CommandEnvelope { + readonly command: "executions"; + /** The terminal sequence this snapshot is anchored to, or `null` for empty. */ + readonly anchor: number | null; + /** The sequence the previous page ended at, or `null` for the first page. */ + readonly after: number | null; +} + +/** One coherent admitted state, asked for exactly once per invocation. */ +export interface MappingsCommand extends CommandEnvelope { + readonly command: "mappings"; +} + +export interface SettleCommand extends CommandEnvelope { + readonly command: "settle"; + readonly completion: DocumentExecutionCompletion; + readonly expectedWorkspaceRootId: string; +} + +export type RunnerCommand = + | FrontierCommand + | JournalCommand + | RootCommand + | ContentCommand + | StageCommand + | CommitCommand + | RetrievalCommand + | ExecutionsCommand + | MappingsCommand + | SettleCommand; + +export type CommandResult = + | { readonly id: string; readonly outcome: "performed"; readonly value: unknown } + | { readonly id: string; readonly outcome: "refused"; readonly refusal: string }; + +const MAX_ID = 128; +const MAX_EVENTS = 4096; +const ENVELOPE = ["id", "command"]; +const MEMBERS: Record = { + frontier: ENVELOPE, + journal: [...ENVELOPE, "anchorEventId", "afterEventId"], + root: [...ENVELOPE, "workspaceRootId"], + content: [...ENVELOPE, "workspaceRootId", "kind", "digest", "sourceManifest"], + stage: [...ENVELOPE, "kind", "digest", "bytes"], + commit: [ + ...ENVELOPE, + "expectedWorkspaceRootId", + "expectedJournalEventId", + "publication", + "mappings", + "events", + ], + retrieval: [...ENVELOPE, "expectedWorkspaceRootId", "metadata"], + executions: [...ENVELOPE, "anchor", "after"], + mappings: ENVELOPE, + settle: [...ENVELOPE, "completion", "expectedWorkspaceRootId"], +}; + +function object(value: unknown): Map { + if (value === null || typeof value !== "object" || Array.isArray(value)) { + throw new CommandError("not-an-object"); + } + return new Map(Object.entries(value)); +} + +function closed(members: Map, allowed: readonly string[]): void { + for (const key of members.keys()) { + if (!allowed.includes(key)) { + throw new CommandError("unknown-member"); + } + } + if (members.size !== allowed.length) { + throw new CommandError("malformed-member"); + } +} + +function text( + members: Map, + key: string, + maximum = Number.MAX_SAFE_INTEGER, +): string { + const value = members.get(key); + if (typeof value !== "string" || value === "" || value.length > maximum) { + throw new CommandError( + value !== "" && typeof value === "string" ? "too-large" : "malformed-member", + ); + } + return value; +} + +function nullableText(members: Map, key: string): string | null { + const value = members.get(key); + if (value === null) { + return null; + } + if (typeof value !== "string" || value === "") { + throw new CommandError("malformed-member"); + } + return value; +} + +function digest(members: Map, key: string): string { + const value = members.get(key); + if (typeof value !== "string" || !SHA256.test(value)) { + throw new CommandError("malformed-member"); + } + return value; +} + +function kind(members: Map): ContentKind { + const value = members.get("kind"); + if (value !== "manifest" && value !== "blob") { + throw new CommandError("malformed-member"); + } + return value; +} + +/** + * The exact serialized events a proposal appends. + * + * A record is not admitted because it is a non-empty string, and not because + * SQLite will accept it as JSON. It is parsed with the authoritative durable + * event parser and then serialized again, and the result must be the same bytes + * that arrived, terminating newline included. + * + * That round trip is the point. Retaining something that parses as JSON but not + * as an event would create history a later read cannot understand, and the run + * would become unreplayable at exactly the moment it was told it had committed. + * Re-encoding a nearly-right record would be worse: the owner would retain + * something the runner never proposed. + */ +/** A physical sequence, which is a positive whole number or nothing. */ +function sequence(members: Map, key: string): number | null { + const value = members.get(key); + if (value === null) { + return null; + } + if (typeof value !== "number" || !Number.isSafeInteger(value) || value < 1) { + throw new CommandError("malformed-member"); + } + return value; +} + +function eventRecords(value: unknown): string[] { + if (!Array.isArray(value)) { + throw new CommandError("malformed-member"); + } + if (value.length > MAX_EVENTS) { + throw new CommandError("too-large"); + } + return value.map((entry) => { + if (typeof entry !== "string" || entry === "") { + throw new CommandError("malformed-member"); + } + const parsed = parseDurableEvent(entry); + if (!parsed.ok || serializeDurableEvent(parsed.value) !== entry) { + throw new CommandError("malformed-member"); + } + return entry; + }); +} + +export function parseCommand(raw: string): RunnerCommand { + if (new TextEncoder().encode(raw).length > MAX_MESSAGE_BYTES) { + throw new CommandError("too-large"); + } + let decoded: unknown; + try { + decoded = JSON.parse(raw); + } catch { + throw new CommandError("not-an-object"); + } + const members = object(decoded); + const id = text(members, "id", MAX_ID); + const command = members.get("command"); + if ( + command !== "frontier" && + command !== "journal" && + command !== "root" && + command !== "content" && + command !== "stage" && + command !== "commit" && + command !== "retrieval" && + command !== "executions" && + command !== "mappings" && + command !== "settle" + ) { + throw new CommandError("unknown-command"); + } + closed(members, MEMBERS[command]); + + if (command === "frontier") { + return { id, command }; + } + if (command === "journal") { + return { + id, + command, + anchorEventId: nullableText(members, "anchorEventId"), + afterEventId: nullableText(members, "afterEventId"), + }; + } + if (command === "root") { + return { id, command, workspaceRootId: digest(members, "workspaceRootId") }; + } + if (command === "content") { + const contentKind = kind(members); + if (contentKind === "manifest" && members.get("sourceManifest") !== null) { + throw new CommandError("malformed-member"); + } + const sourceManifest = contentKind === "manifest" ? null : digest(members, "sourceManifest"); + return { + id, + command, + workspaceRootId: digest(members, "workspaceRootId"), + kind: contentKind, + digest: digest(members, "digest"), + sourceManifest, + }; + } + if (command === "stage") { + return { + id, + command, + kind: kind(members), + digest: digest(members, "digest"), + bytes: text(members, "bytes", Math.ceil((MAX_CONTENT_BYTES * 4) / 3) + 4), + }; + } + if (command === "retrieval") { + const metadata = members.get("metadata"); + if (metadata !== null && (typeof metadata !== "string" || metadata === "")) { + throw new CommandError("malformed-member"); + } + if (metadata !== null && new TextEncoder().encode(metadata).length > MAX_MESSAGE_BYTES) { + throw new CommandError("too-large"); + } + return { + id, + command, + expectedWorkspaceRootId: digest(members, "expectedWorkspaceRootId"), + metadata, + }; + } + if (command === "mappings") { + return { id, command }; + } + if (command === "executions") { + const anchor = sequence(members, "anchor"); + const after = sequence(members, "after"); + if (anchor === null && after !== null) { + // An empty snapshot has nothing to continue from. + throw new CommandError("malformed-member"); + } + if (anchor !== null && after !== null && after >= anchor) { + throw new CommandError("malformed-member"); + } + return { id, command, anchor, after }; + } + if (command === "settle") { + const completion = parseDocumentExecutionCompletion(members.get("completion")); + if (!completion.ok) { + throw new CommandError("malformed-member"); + } + return { + id, + command, + completion: completion.value, + expectedWorkspaceRootId: digest(members, "expectedWorkspaceRootId"), + }; + } + return { + id, + command, + expectedWorkspaceRootId: digest(members, "expectedWorkspaceRootId"), + expectedJournalEventId: nullableText(members, "expectedJournalEventId"), + publication: publication(members.get("publication")), + mappings: mappings(members.get("mappings")), + events: eventRecords(members.get("events")), + }; +} + +/** + * The Workspace half of a proposal, or its absence. + * + * `null` is a journal-only transaction and is admitted as such. Everything else + * must be a complete proposal: an identity, the canonical manifest that + * identity is supposed to be the digest of, and the exact inventory. Whether + * the identity really is that digest, and whether the inventory really is the + * closure, is the owner's to recompute — this only decides whether the request + * is shaped like a proposal at all. + */ +function publication(value: unknown): ProposedPublication | null { + if (value === null) { + return null; + } + const members = object(value); + closed(members, ["proposedWorkspaceRootId", "proposedManifest", "content"]); + const manifest = members.get("proposedManifest"); + if (typeof manifest !== "string" || manifest === "") { + throw new CommandError("malformed-member"); + } + if (new TextEncoder().encode(manifest).length > MAX_ROOT_MANIFEST_BYTES) { + throw new CommandError("too-large"); + } + return { + proposedWorkspaceRootId: digest(members, "proposedWorkspaceRootId"), + proposedManifest: manifest, + content: pieces(members.get("content")), + }; +} + +/** + * The inventory, in the order it must arrive. + * + * Canonical order and no repeats, checked here rather than sorted into shape: a + * proposal that named one piece twice, or named them in an order this build did + * not produce, is not the proposal the runner computed its identity over. + */ +function pieces(value: unknown): ProposedPiece[] { + if (!Array.isArray(value)) { + throw new CommandError("malformed-member"); + } + if (value.length > MAX_PROPOSED_PIECES) { + throw new CommandError("too-large"); + } + const found: ProposedPiece[] = []; + let previous: string | undefined; + for (const entry of value) { + const members = object(entry); + closed(members, ["kind", "digest", "size"]); + const size = members.get("size"); + if (typeof size !== "number" || !Number.isSafeInteger(size) || size < 0) { + throw new CommandError("malformed-member"); + } + if (size > MAX_CONTENT_BYTES) { + throw new CommandError("too-large"); + } + const piece: ProposedPiece = { + kind: kind(members), + digest: digest(members, "digest"), + size, + }; + const ordering = `${piece.kind}:${piece.digest}`; + if (previous !== undefined && ordering <= previous) { + throw new CommandError("malformed-member"); + } + previous = ordering; + found.push(piece); + } + return found; +} + +/** + * The retained mappings a proposal carries, read through the shared parsers. + * + * The parsers are the ones the local host holds its own rows to. A private + * approximation here would be the two hosts disagreeing about what a retained + * Repository is, and the owner would be the one that found out. + */ +function mappings(value: unknown): ProposedMapping[] { + if (!Array.isArray(value)) { + throw new CommandError("malformed-member"); + } + if (value.length > MAX_MAPPINGS) { + throw new CommandError("too-large"); + } + return value.map((entry) => { + const members = object(entry); + const which = members.get("kind"); + closed(members, which === "repository" ? ["kind", "record", "locator"] : ["kind", "record"]); + const offered = members.get("record"); + if (which === "repository") { + const record = parseRepositoryRecord(offered); + const offeredLocator = members.get("locator"); + if (record === undefined || typeof offeredLocator !== "string") { + throw new CommandError("malformed-member"); + } + // Admitted first, by the same closed allowlist the local host uses. A + // matching fingerprint says the two values agree with each other; it says + // nothing about whether the locator is one this system will ever hand to + // Git, and an authenticated proposal must not be able to retain a + // credential-bearing URL or an executable transport form. + const locator = admitLocator(offeredLocator); + if (locator === undefined || locatorFingerprintOf(locator) !== record.locatorFingerprint) { + throw new CommandError("malformed-member"); + } + return { kind: which, record, locator }; + } + if (which === "worktree") { + const record = parseWorktreeRecord(offered); + if (record === undefined) { + throw new CommandError("malformed-member"); + } + return { kind: which, record }; + } + if (which === "agent-session") { + const record = parseAgentSessionRecord(offered); + if (record === undefined) { + throw new CommandError("malformed-member"); + } + return { kind: which, record }; + } + throw new CommandError("malformed-member"); + }); +} diff --git a/packages/workflow/src/cloudflare/dispatcher.ts b/packages/workflow/src/cloudflare/dispatcher.ts new file mode 100644 index 000000000..bf3345ec7 --- /dev/null +++ b/packages/workflow/src/cloudflare/dispatcher.ts @@ -0,0 +1,392 @@ +/** + * Deciding one command, once. + * + * A runner that does not hear an answer cannot tell a lost question from a lost + * answer, so it asks again. That is only safe if asking twice is the same as + * asking once — which is what this arranges. Each command ID is decided once + * within one acquisition, and the decision is retained beside the acquisition + * that made it. + * + * Two requests are the same request when their *parsed* commands are equal. + * Member order and equivalent encodings are not differences; a different value + * is. Reusing an ID for a different request is not a retry, and it is refused + * rather than answered, because answering it would mean one identifier named + * two decisions. + * + * What is retained is the decision, not always the response. A read whose + * answer is fixed by immutable state and a snapshot anchor the request already + * carries is remembered as a decision to read again, and re-reading returns the + * same bytes because the request names what to read. The frontier is the + * exception and is kept whole: it is the one read whose answer would otherwise + * move, and a retry that returned a later frontier would hand a runner a + * snapshot it never asked for. + * + * The ledger is bounded and never evicts. Dropping an older ID would make a + * retry of it look like a new command, which for a mutation is the difference + * between doing something once and doing it twice — so a full ledger refuses + * the new command and fails the connection closed instead. + * + * Everything happens inside one short synchronous transaction, and the exact + * live acquisition is proved twice: before parsing, and again inside the + * transaction, because a socket can close between the two and the transaction + * is where the object actually changes. + */ + +import type { AcquisitionContext } from "./acquisition.ts"; +import { requireAcquisition } from "./acquisition.ts"; +import { + type CommandResult, + CommandError, + MAX_COMMANDS, + MAX_CONTENT_BYTES, + MAX_LEDGER_BYTES, + MAX_STAGED_BYTES, + type RunnerCommand, +} from "./commands.ts"; +import { bytesOf, decodeBase64, sha256Hex } from "./encoding.ts"; +import { + readContent, + readExecutions, + readInvocationSnapshot, + readFrontier, + readJournalPage, + readRoot, +} from "./owner-reads.ts"; +import type { OwnerTransactions } from "./owner-transaction.ts"; +import { COMMAND_TABLE, MUTATION_TABLE, STAGING_TABLE } from "./private-schema.ts"; +import { applyCommit, applyRetrieval } from "./publish.ts"; +import { recognizeObject } from "./recognition.ts"; + +function requestFingerprint(command: RunnerCommand): string { + // The command name is part of the fingerprint, so one textual id used for a + // commit and for a retrieval replacement is two different requests rather + // than one recognized retry. + return sha256Hex(JSON.stringify({ kind: command.command, command })); +} + +/** + * Whether this command changes the run, and therefore whether its decision has + * to outlive the connection that asked for it. + * + * A read can be asked again; a mutation cannot, so its answer is retained where + * the next connection can find it. + */ +function mutating(command: RunnerCommand): boolean { + return command.command === "commit" || command.command === "retrieval"; +} + +function integer(value: unknown): number { + if (typeof value !== "number" || !Number.isSafeInteger(value) || value < 0) { + throw new Error("private protocol storage holds a malformed count"); + } + return value; +} + +function storedDecision(value: unknown, id: string): CommandResult | "reconstruct" { + if (typeof value !== "string") { + throw new Error("private protocol storage holds a malformed result"); + } + let parsed: unknown; + try { + parsed = JSON.parse(value); + } catch { + throw new Error("private protocol storage holds a malformed result"); + } + if (parsed === null || typeof parsed !== "object" || Array.isArray(parsed)) { + throw new Error("private protocol storage holds a malformed result"); + } + const members = new Map(Object.entries(parsed)); + if (members.get("id") !== id) { + throw new Error("private protocol storage holds a result for another command"); + } + const outcome = members.get("outcome"); + if (outcome === "reconstruct" && members.size === 2) { + return "reconstruct"; + } + if (outcome === "performed" && members.size === 3 && members.has("value")) { + return { id, outcome, value: members.get("value") }; + } + const refusal = members.get("refusal"); + if (outcome === "refused" && members.size === 3 && typeof refusal === "string") { + return { id, outcome, refusal }; + } + throw new Error("private protocol storage holds a malformed result"); +} + +/** + * A fresh opaque identity for one retained event. + * + * Minted by the owner inside the transaction that writes the row. An id the + * runner chose would be a runner deciding what a retained event is called, and + * two runners could choose the same one. + */ +function mintEventId(): string { + return crypto.randomUUID(); +} + +/** + * The moment the owner records against a mutation it just made. + * + * The owner's clock, not the runner's. A time a runner supplied would be a + * caller deciding when the run's history happened. + */ +function ownerTime(): string { + return new Date().toISOString(); +} + +function retainedDecision(command: RunnerCommand, result: CommandResult): string { + if ( + result.outcome === "performed" && + (command.command === "journal" || + command.command === "root" || + command.command === "content" || + command.command === "executions" || + command.command === "mappings") + ) { + return JSON.stringify({ id: command.id, outcome: "reconstruct" }); + } + return JSON.stringify(result); +} + +function sameBytes(left: Uint8Array, right: Uint8Array): boolean { + if (left.length !== right.length) { + return false; + } + let difference = 0; + for (let index = 0; index < left.length; index += 1) { + difference |= (left[index] ?? 0) ^ (right[index] ?? 0); + } + return difference === 0; +} + +function stage( + ctx: AcquisitionContext, + acquisitionId: string, + command: Extract, +): { kind: "manifest" | "blob"; digest: string; size: number } { + const bytes = decodeBase64(command.bytes); + if (bytes.length === 0 || bytes.length > MAX_CONTENT_BYTES) { + throw new CommandError(bytes.length === 0 ? "malformed-member" : "too-large"); + } + if (sha256Hex(bytes) !== command.digest) { + throw new CommandError("malformed-member"); + } + const existing = ctx.storage.sql + .exec( + `SELECT size, bytes FROM ${STAGING_TABLE} + WHERE acquisition_id = ? AND kind = ? AND digest = ?`, + acquisitionId, + command.kind, + command.digest, + ) + .toArray()[0]; + if (existing !== undefined) { + const retained = bytesOf(existing["bytes"]); + if (!sameBytes(retained, bytes)) { + throw new Error("private staging disagrees with its content identity"); + } + return { kind: command.kind, digest: command.digest, size: bytes.length }; + } + const total = ctx.storage.sql + .exec( + `SELECT coalesce(sum(size), 0) AS total FROM ${STAGING_TABLE} WHERE acquisition_id = ?`, + acquisitionId, + ) + .toArray()[0]; + if (integer(total?.["total"]) + bytes.length > MAX_STAGED_BYTES) { + throw new CommandError("capacity"); + } + ctx.storage.sql.exec( + `INSERT INTO ${STAGING_TABLE} (acquisition_id, kind, digest, size, bytes) + VALUES (?, ?, ?, ?, ?)`, + acquisitionId, + command.kind, + command.digest, + bytes.length, + new Uint8Array(bytes), + ); + return { kind: command.kind, digest: command.digest, size: bytes.length }; +} + +function perform( + ctx: AcquisitionContext, + runId: string, + acquisitionId: string, + command: RunnerCommand, +): CommandResult { + if (command.command === "frontier") { + return { id: command.id, outcome: "performed", value: readFrontier(ctx.storage, runId) }; + } + if (command.command === "journal") { + return { + id: command.id, + outcome: "performed", + value: readJournalPage(ctx.storage, command.anchorEventId, command.afterEventId), + }; + } + if (command.command === "root") { + return { + id: command.id, + outcome: "performed", + value: readRoot(ctx.storage, command.workspaceRootId), + }; + } + if (command.command === "content") { + return { + id: command.id, + outcome: "performed", + value: readContent( + ctx.storage, + command.workspaceRootId, + command.kind, + command.digest, + command.sourceManifest, + ), + }; + } + if (command.command === "stage") { + return { id: command.id, outcome: "performed", value: stage(ctx, acquisitionId, command) }; + } + if (command.command === "commit") { + return { + id: command.id, + outcome: "performed", + value: applyCommit(ctx.storage, acquisitionId, command, mintEventId), + }; + } + if (command.command === "retrieval") { + return { + id: command.id, + outcome: "performed", + value: applyRetrieval(ctx.storage, command, ownerTime), + }; + } + if (command.command === "executions") { + return { + id: command.id, + outcome: "performed", + value: readExecutions(ctx.storage, runId, command.anchor, command.after), + }; + } + if (command.command === "mappings") { + return { + id: command.id, + outcome: "performed", + value: readInvocationSnapshot(ctx.storage, runId), + }; + } + // `settle` is a later checkpoint's. It parses strictly and is declined, + // because a placeholder that reported success is the one answer a runner + // cannot recover from. + return { id: command.id, outcome: "refused", refusal: "command:unavailable" }; +} + +export function dispatchCommand( + ctx: AcquisitionContext, + transactions: OwnerTransactions, + socket: WebSocket, + runId: string, + command: RunnerCommand, +): CommandResult { + const held = requireAcquisition(ctx, socket, runId); + const fingerprint = requestFingerprint(command); + return transactions.run(ctx.storage, () => { + const inside = requireAcquisition(ctx, socket, runId); + if (inside.acquisitionId !== held.acquisitionId) { + throw new CommandError("duplicate-conflict"); + } + recognizeObject(ctx.storage); + + // A mutation's decision is looked for by the run, not by the connection. + // The case this exists for is the one where the connection that asked is + // gone: the owner committed, the answer never arrived, and the runner + // reconnected to ask the same question again. + if (mutating(command)) { + const decided = ctx.storage.sql + .exec( + `SELECT request_fingerprint, response FROM ${MUTATION_TABLE} WHERE command_id = ?`, + command.id, + ) + .toArray()[0]; + if (decided !== undefined) { + if (decided.request_fingerprint !== fingerprint) { + throw new CommandError("duplicate-conflict"); + } + const decision = storedDecision(decided.response, command.id); + if (decision === "reconstruct") { + // A mutation's decision is always retained whole. Reconstructing one + // would mean applying it again. + throw new Error("private protocol storage holds a malformed result"); + } + return decision; + } + } + + const previous = ctx.storage.sql + .exec( + `SELECT request_fingerprint, response FROM ${COMMAND_TABLE} + WHERE acquisition_id = ? AND command_id = ?`, + held.acquisitionId, + command.id, + ) + .toArray()[0]; + if (previous !== undefined) { + if (previous.request_fingerprint !== fingerprint) { + throw new CommandError("duplicate-conflict"); + } + const decision = storedDecision(previous.response, command.id); + return decision === "reconstruct" + ? perform(ctx, runId, held.acquisitionId, command) + : decision; + } + const usage = ctx.storage.sql + .exec( + `SELECT count(*) AS commands, coalesce(sum(response_bytes), 0) AS bytes + FROM ${COMMAND_TABLE} WHERE acquisition_id = ?`, + held.acquisitionId, + ) + .toArray()[0]; + if ( + integer(usage?.["commands"]) >= MAX_COMMANDS || + integer(usage?.["bytes"]) >= MAX_LEDGER_BYTES + ) { + throw new CommandError("capacity"); + } + const result = perform(ctx, runId, held.acquisitionId, command); + const encoded = retainedDecision(command, result); + const responseBytes = new TextEncoder().encode(encoded).length; + if (integer(usage?.["bytes"]) + responseBytes > MAX_LEDGER_BYTES) { + throw new CommandError("capacity"); + } + ctx.storage.sql.exec( + `INSERT INTO ${COMMAND_TABLE} + (acquisition_id, command_id, request_fingerprint, response, response_bytes) + VALUES (?, ?, ?, ?, ?)`, + held.acquisitionId, + command.id, + fingerprint, + encoded, + responseBytes, + ); + if (mutating(command)) { + // Recorded in this same transaction as the mutation it describes, so a + // crash cannot leave one without the other. + const mutations = ctx.storage.sql + .exec(`SELECT count(*) AS decided FROM ${MUTATION_TABLE}`) + .toArray()[0]; + if (integer(mutations?.["decided"]) >= MAX_COMMANDS) { + throw new CommandError("capacity"); + } + ctx.storage.sql.exec( + `INSERT INTO ${MUTATION_TABLE} + (command_id, request_fingerprint, response, response_bytes) + VALUES (?, ?, ?, ?)`, + command.id, + fingerprint, + encoded, + responseBytes, + ); + } + return result; + }); +} diff --git a/packages/workflow/src/cloudflare/encoding.ts b/packages/workflow/src/cloudflare/encoding.ts new file mode 100644 index 000000000..b99308f3a --- /dev/null +++ b/packages/workflow/src/cloudflare/encoding.ts @@ -0,0 +1,57 @@ +/** + * The two encodings the private protocol carries bytes and identities in. + * + * A WebSocket text frame carries text, and content-addressed bytes are not + * text, so base64 is what the private protocol uses. It is canonical in both + * directions: a value that decodes and then re-encodes to something else is + * refused rather than accepted as though the difference did not matter, because + * a digest is taken over bytes and two spellings of one byte sequence would be + * two names for one piece of content. + * + * `bytesOf` is the storage side of the same question. SQLite hands back a blob + * as whatever the runtime models one as, and a column that is not bytes at all + * is damage rather than something to coerce. + */ + +import { CommandError } from "./commands.ts"; +export { sha256Hex } from "../workspace/sha256.ts"; + +export function encodeBase64(bytes: Uint8Array): string { + let binary = ""; + const stride = 32 * 1024; + for (let offset = 0; offset < bytes.length; offset += stride) { + binary += String.fromCharCode(...bytes.slice(offset, offset + stride)); + } + return btoa(binary); +} + +export function decodeBase64(value: string): Uint8Array { + if (value === "" || value.length % 4 !== 0 || !/^[A-Za-z0-9+/]+={0,2}$/.test(value)) { + throw new CommandError("malformed-member"); + } + let binary: string; + try { + binary = atob(value); + } catch { + throw new CommandError("malformed-member"); + } + const bytes = Uint8Array.from(binary, (character) => character.charCodeAt(0)); + if (encodeBase64(bytes) !== value) { + throw new CommandError("malformed-member"); + } + return bytes; +} + +export function bytesOf(value: unknown): Uint8Array { + if (value instanceof Uint8Array) { + return new Uint8Array(value); + } + if (value instanceof ArrayBuffer) { + return new Uint8Array(value.slice(0)); + } + throw new Error("stored bytes are not a byte sequence"); +} + +export function hex(bytes: Uint8Array): string { + return Array.from(bytes, (byte) => byte.toString(16).padStart(2, "0")).join(""); +} diff --git a/packages/workflow/src/cloudflare/marker.ts b/packages/workflow/src/cloudflare/marker.ts new file mode 100644 index 000000000..da15833c9 --- /dev/null +++ b/packages/workflow/src/cloudflare/marker.ts @@ -0,0 +1,99 @@ +/** + * How the Cloudflare owner says which schema its storage holds. + * + * The Deno host writes `PRAGMA application_id` and `PRAGMA user_version` into + * the SQLite header, and recognition reads them back to tell three conditions + * apart: a database belonging to something else, a version this build has not + * learned, and a database that claims version 1 and is not shaped like one. + * + * A Durable Object's SQLite refuses both pragmas — `not authorized: + * SQLITE_AUTH`, on read as well as write — so this adapter carries the same two + * values in a table of its own. The logical schema version is unchanged and + * shared with Deno; only the physical carrier differs, which is why this table + * is adapter-private recognition metadata rather than a WorkflowRun record. It + * is never a journal value, an exported field, an authored value, a public API, + * or a second schema. + * + * The constraints are what make the claim trustworthy. `id` is fixed at 1 by a + * CHECK and is the primary key, so a second identity row cannot exist; both + * values are non-null integers; and a row that disagrees with this build is + * refused rather than migrated. + */ + +import { APPLICATION_ID, isSchemaVersion, SCHEMA_VERSION } from "../sqlite/workflow-schema.ts"; + +/** The adapter-private table carrying this database's identity. */ +export const MARKER_TABLE = "_xmd_workflow_schema"; + +export const MARKER_SQL = `CREATE TABLE ${MARKER_TABLE} ( + id INTEGER PRIMARY KEY NOT NULL CHECK (id = 1), + application_id INTEGER NOT NULL, + schema_version INTEGER NOT NULL +) STRICT, WITHOUT ROWID`; + +/** What one marker row says. */ +export interface SchemaMarker { + readonly applicationId: number; + readonly schemaVersion: number; +} + +/** Why a marker could not be accepted. */ +export type MarkerFailure = + | { readonly kind: "absent" } + | { readonly kind: "duplicated"; readonly rows: number } + | { readonly kind: "malformed" } + | { readonly kind: "foreign-application"; readonly applicationId: number } + | { readonly kind: "incomplete-version" } + | { readonly kind: "unknown-version"; readonly schemaVersion: number }; + +/** + * Read a marker out of rows the caller already selected. + * + * Takes rows rather than a connection so the comparison is the same whoever + * consumed the cursor — Cloudflare requires a cursor to be drained + * synchronously, and that is the caller's concern rather than this one's. + */ +export function readMarker(rows: readonly Record[]): SchemaMarker | MarkerFailure { + if (rows.length === 0) { + return { kind: "absent" }; + } + if (rows.length > 1) { + return { kind: "duplicated", rows: rows.length }; + } + const row = rows[0]; + if (row === undefined) { + return { kind: "absent" }; + } + const applicationId = row["application_id"]; + const schemaVersion = row["schema_version"]; + if ( + typeof applicationId !== "number" || + !Number.isInteger(applicationId) || + typeof schemaVersion !== "number" || + !Number.isInteger(schemaVersion) + ) { + return { kind: "malformed" }; + } + if (applicationId !== APPLICATION_ID) { + return { kind: "foreign-application", applicationId }; + } + if (schemaVersion === 0) { + // The identity is this project's and the version says nothing was + // finished. That is a database left partly initialized, not an older one. + return { kind: "incomplete-version" }; + } + if (!isSchemaVersion(schemaVersion)) { + // Outside what the version carrier can hold, so no build wrote it. The row + // is damaged retained data rather than a version to report. + return { kind: "malformed" }; + } + if (schemaVersion !== SCHEMA_VERSION) { + return { kind: "unknown-version", schemaVersion }; + } + return { applicationId, schemaVersion }; +} + +/** Whether a read produced a marker rather than a reason it could not. */ +export function isSchemaMarker(value: SchemaMarker | MarkerFailure): value is SchemaMarker { + return "applicationId" in value && !("kind" in value); +} diff --git a/packages/workflow/src/cloudflare/owner-reads.ts b/packages/workflow/src/cloudflare/owner-reads.ts new file mode 100644 index 000000000..d1ec640f5 --- /dev/null +++ b/packages/workflow/src/cloudflare/owner-reads.ts @@ -0,0 +1,684 @@ +/** + * What the owner answers a read with, read out of its own storage. + * + * Every value here is rebuilt from checked columns. The rows are this object's + * own and were written by this build, which is a reason to expect them to be + * right and no reason at all to skip asking: a row that does not parse is + * storage damage, and storage damage answered as though it were a workflow + * value is how damage travels. + * + * Three properties hold the reads together. The frontier is *coherent*: the run + * record, the current root and the journal anchor are read as one, and the + * anchor is the last event that existed at that moment, so later appends cannot + * enter an earlier snapshot. Reads are *bounded*: a journal is returned in + * pages anchored to that event, and content comes back one piece at a time. + * Reads are *referenced*: a root is returned only once its complete content + * graph has been proved present and self-consistent, and a piece is admitted + * only if that root actually names it, so this is a read of one retained root + * rather than of a content-addressed store. Validating the graph up front is + * the point: a root is a starting frontier, and a frontier that turns out not + * to be materializable after the runner has it is a failure arriving too late + * to mean anything. + * + * A refusal says the category and nothing else. Column values, retained JSON + * and request data never appear in one: the caller learns that storage is + * damaged, which is the only thing it can act on. + */ + +import { parseDurableEvent } from "@executablemd/durable-streams"; +import { readDocumentExecution, readRetrieval, readRunRecord, type Row } from "../sqlite/rows.ts"; +import type { DocumentExecutionRecord } from "../storage/record.ts"; +import { + parseRepositoryRecord, + parseWorktreeRecord, + type RepositoryRecord, + type WorktreeRecord, +} from "../composition/records.ts"; +import { type AgentSessionRecord, parseAgentSessionRecord } from "../storage/agent-session.ts"; +import { WorkflowRecordMalformedError } from "../storage/errors.ts"; +import { + parseWorkspaceRootManifest, + SHA256, + WORKSPACE_ROOT_DOMAIN, + type WorkspaceRootManifest, +} from "../workspace/root-manifest.ts"; +import { type ContentManifest, decodeContentManifest } from "../workspace/content-manifest.ts"; +import { + CommandError, + EXECUTION_PAGE_BYTES, + EXECUTION_PAGE_ENTRIES, + executionPageBytes, + MAX_LEDGER_BYTES, + MAX_MAPPINGS, + JOURNAL_PAGE_BYTES, + JOURNAL_PAGE_ENTRIES, + MAX_CONTENT_BYTES, +} from "./commands.ts"; +import { bytesOf, encodeBase64, sha256Hex } from "./encoding.ts"; +import type { OwnerStorage } from "./storage.ts"; + +export interface FrontierValue { + readonly record: ReturnType; + readonly retrieval: ReturnType | null; + readonly workspaceRootId: string; + readonly journalEventId: string | null; +} + +export interface JournalPageValue { + readonly anchorEventId: string | null; + readonly afterEventId: string | null; + readonly entries: readonly { + readonly eventId: string; + readonly previousEventId: string | null; + readonly record: string; + readonly workspaceRootId: string; + }[]; + readonly done: boolean; +} + +export interface RootValue { + readonly workspaceRootId: string; + readonly manifest: string; +} + +export interface ContentValue { + readonly kind: "manifest" | "blob"; + readonly digest: string; + readonly size: number; + readonly bytes: string; +} + +/** + * One retained root, and the whole content graph it names, proved. + * + * `manifests` and `blobs` are not a description of what the root refers to — + * they are what was found and checked. A `StoredRoot` therefore cannot exist + * for a root whose graph is incomplete or disagrees with itself. + */ +interface StoredRoot { + readonly manifest: string; + readonly parsed: WorkspaceRootManifest; + readonly manifests: ReadonlyMap; + readonly blobs: ReadonlySet; +} + +function corrupt(reason: string): never { + throw new WorkflowRecordMalformedError("workflow owner storage", reason); +} + +function exactlyOne(rows: Row[], name: string): Row { + if (rows.length !== 1 || rows[0] === undefined) { + return corrupt(`expected exactly one ${name} row`); + } + return rows[0]; +} + +function safeText(row: Row, column: string): string { + const value = row[column]; + if (typeof value !== "string" || value === "") { + return corrupt(`expected ${column} to be non-empty text`); + } + return value; +} + +function safeInteger(value: unknown, name: string): number { + if (typeof value !== "number" || !Number.isSafeInteger(value) || value < 0) { + return corrupt(`expected ${name} to be a nonnegative whole number`); + } + return value; +} + +function rootIdentity(manifest: string): string { + return sha256Hex(`${WORKSPACE_ROOT_DOMAIN}${manifest}`); +} + +function byteRows(storage: OwnerStorage, sql: string, ...bindings: unknown[]): Row[] { + return storage.sql.exec(sql, ...bindings).toArray(); +} + +/** Every content identity one root's reference table holds, in order. */ +function referenceRows( + storage: OwnerStorage, + table: string, + column: string, + rootId: string, +): string[] { + return byteRows( + storage, + `SELECT lower(hex(${column})) AS digest FROM ${table} WHERE root_id = ? ORDER BY digest`, + rootId, + ).map((row) => safeText(row, "digest")); +} + +/** + * Whether a reference table holds exactly the identities the content names. + * + * Both directions matter and for different reasons. A missing row is content + * the root depends on that nothing is keeping alive, so retention may already + * have collected it. An extra row is the root claiming content it does not use, + * which keeps bytes reachable that no manifest accounts for. Neither is a root + * this owner will hand to a runner as a starting frontier. + */ +function requireReferenceSet( + found: readonly string[], + expected: ReadonlySet, + what: string, +): void { + if (found.length !== expected.size || found.some((digest) => !expected.has(digest))) { + corrupt(`a Workspace root's ${what} references disagree with its content`); + } +} + +/** One retained DOFS manifest, proved against its identity, size and entries. */ +function validatedManifest( + storage: OwnerStorage, + parsed: WorkspaceRootManifest, + digest: string, +): { bytes: Uint8Array; manifest: ContentManifest } { + const row = exactlyOne( + byteRows(storage, "SELECT size, encoded FROM vfs_manifests WHERE lower(hex(hash)) = ?", digest), + "DOFS manifest", + ); + const bytes = bytesOf(row["encoded"]); + if (bytes.length > MAX_CONTENT_BYTES || sha256Hex(bytes) !== digest) { + return corrupt("a retained DOFS manifest disagrees with its identity"); + } + const manifest = decodeContentManifest(bytes, corrupt); + if (safeInteger(row["size"], "manifest size") !== manifest.size) { + return corrupt("a retained DOFS manifest disagrees with its recorded size"); + } + for (const entry of parsed.entries) { + if (entry.kind === "file" && entry.manifest === digest && entry.size !== manifest.size) { + return corrupt("a Workspace file size disagrees with its retained manifest"); + } + } + return { bytes, manifest }; +} + +/** One retained blob, proved against its identity and every chunk naming it. */ +function validatedBlob( + storage: OwnerStorage, + rootId: string, + manifests: ReadonlyMap, + digest: string, +): Uint8Array { + const row = exactlyOne( + byteRows( + storage, + `SELECT b.size, x.bytes FROM workspace_root_blob_refs AS r + JOIN vfs_blobs AS b ON b.hash = r.blob_hash + JOIN vfs_blob_bytes AS x ON x.hash = r.blob_hash + WHERE r.root_id = ? AND lower(hex(r.blob_hash)) = ?`, + rootId, + digest, + ), + "DOFS blob", + ); + const bytes = bytesOf(row["bytes"]); + if (bytes.length > MAX_CONTENT_BYTES || sha256Hex(bytes) !== digest) { + return corrupt("a retained DOFS blob disagrees with its identity"); + } + if (safeInteger(row["size"], "blob size") !== bytes.length) { + return corrupt("a retained DOFS blob disagrees with its recorded size"); + } + for (const manifest of manifests.values()) { + for (const chunk of manifest.chunks) { + if (chunk.hash === digest && chunk.size !== bytes.length) { + return corrupt("a DOFS chunk size disagrees with the blob it names"); + } + } + } + return bytes; +} + +/** + * One retained root, with its complete content graph proved before it is a root + * at all. + * + * Accepting a root is accepting a starting frontier: the runner will + * materialize it, work in it, and propose against it. A root whose graph cannot + * be materialized is not a frontier, and discovering that one piece at a time — + * after the frontier has already crossed to the runner — would mean the failure + * arrives once the run has already been told where it stands. + * + * So the whole graph is walked here. The manifests the entries name must be + * exactly the manifests the root retains; each must exist, be bounded, decode + * canonically, hash to its identity, and agree with its recorded size and with + * every file that names it. The blobs those manifests name must be exactly the + * blobs the root retains; each must exist, be bounded, hash to its identity, + * and agree with its recorded size and with every chunk that names it. + * + * The bytes are read and dropped. What is kept is the proof, and a later + * content request re-reads the single piece it is sending — which is what keeps + * the transport piece-oriented rather than turning a validated root into one + * unbounded answer. + */ +/** + * Prove one retained root is complete, for a caller that is about to write. + * + * The same validator the reads use. Keeping the read boundary and the write + * boundary on one proof is what stops a root being publishable by one path and + * refused by the other. + */ +export function validateRetainedRoot(storage: OwnerStorage, rootId: string): void { + referencedRoot(storage, rootId); +} + +function referencedRoot(storage: OwnerStorage, rootId: string): StoredRoot { + if (!SHA256.test(rootId)) { + return corrupt("a Workspace root identity is malformed"); + } + const root = exactlyOne( + byteRows( + storage, + "SELECT root_id, format_version, manifest FROM workspace_roots WHERE root_id = ?", + rootId, + ), + "Workspace root", + ); + const manifest = safeText(root, "manifest"); + if (root["format_version"] !== 1) { + return corrupt("a Workspace root has an unsupported format"); + } + const parsed = parseWorkspaceRootManifest(manifest, corrupt); + if (rootIdentity(manifest) !== rootId || root["root_id"] !== rootId) { + return corrupt("a Workspace root disagrees with its identity"); + } + if (new TextEncoder().encode(manifest).length > MAX_CONTENT_BYTES) { + throw new CommandError("too-large"); + } + + const named = new Set( + parsed.entries.flatMap((entry) => (entry.kind === "file" ? [entry.manifest] : [])), + ); + requireReferenceSet( + referenceRows(storage, "workspace_root_manifest_refs", "manifest_hash", rootId), + named, + "manifest", + ); + + const manifests = new Map(); + for (const digest of named) { + manifests.set(digest, validatedManifest(storage, parsed, digest).manifest); + } + + const reachable = new Set(); + for (const decoded of manifests.values()) { + for (const chunk of decoded.chunks) { + reachable.add(chunk.hash); + } + } + requireReferenceSet( + referenceRows(storage, "workspace_root_blob_refs", "blob_hash", rootId), + reachable, + "blob", + ); + for (const digest of reachable) { + validatedBlob(storage, rootId, manifests, digest); + } + + return { manifest, parsed, manifests, blobs: reachable }; +} + +export function readFrontier(storage: OwnerStorage, runId: string): FrontierValue { + const record = readRunRecord( + exactlyOne( + byteRows( + storage, + `SELECT run_id, definition, base, props, status, + stop_reason_kind, stop_reason_code, stop_reason_event_id, + created_at, updated_at FROM workflow_run`, + ), + "workflow run", + ), + ); + if (record.runId !== runId) { + return corrupt("the retained run identity does not address this owner"); + } + const state = exactlyOne( + byteRows(storage, "SELECT current_root_id FROM workspace_state WHERE singleton_id = 1"), + "Workspace state", + ); + const workspaceRootId = safeText(state, "current_root_id"); + referencedRoot(storage, workspaceRootId); + const retrievalRows = byteRows( + storage, + "SELECT metadata, revision, updated_at FROM definition_retrieval WHERE id = 1", + ); + if (retrievalRows.length > 1) { + return corrupt("the definition retrieval is not a singleton"); + } + const last = byteRows( + storage, + "SELECT event_id FROM journal_events ORDER BY sequence DESC LIMIT 1", + )[0]; + return { + record, + retrieval: retrievalRows[0] === undefined ? null : readRetrieval(retrievalRows[0]), + workspaceRootId, + journalEventId: last === undefined ? null : safeText(last, "event_id"), + }; +} + +/** + * One coherent admission fact: the root, the journal anchor and every mapping. + * + * Read together, in one owner-side read, because they are one state. Taking the + * mappings from one request and the root from another would let an invocation + * begin against a Workspace whose retained Repository rows describe a different + * moment — and nothing later could notice, because each answer was true when it + * was given. + * + * Complete rather than paged. The mapping tables are insert-only, so a cursor + * over sorted names cannot be made safe by root and journal equality alone: a + * name inserted later can sort before the cursor and never be seen. A count and + * byte ceiling refuses instead, and refuses whole. + */ +export function readInvocationSnapshot( + storage: OwnerStorage, + runId: string, +): InvocationSnapshotValue { + const frontier = readFrontier(storage, runId); + const repositories = byteRows( + storage, + `SELECT name, locator, locator_fingerprint, requested_base, creation_commit, + primary_branch, object_format, checkout_path + FROM workspace_repositories ORDER BY name`, + ).map((row) => ({ + record: readRepositoryRecord(row), + locator: safeText(row, "locator"), + })); + const worktrees = byteRows( + storage, + `SELECT repository_name, name, requested_branch, requested_base, + creation_commit, checkout_path + FROM workspace_worktrees ORDER BY repository_name, name`, + ).map((row) => readWorktreeRecord(row)); + const agentSessions = byteRows( + storage, + `SELECT session_key, provider, agent_command, session_identity, policy, + assertion_kind, assertion_value, created_at + FROM agent_sessions ORDER BY session_key`, + ).map((row) => readAgentSessionRow(row)); + + const entries = repositories.length + worktrees.length + agentSessions.length; + if (entries > MAX_MAPPINGS) { + throw new CommandError("too-large"); + } + const snapshot = { + workspaceRootId: frontier.workspaceRootId, + journalEventId: frontier.journalEventId, + repositories, + worktrees, + agentSessions, + }; + // Measured over the complete semantic answer, before any of it is returned. A + // ceiling checked per record would let an aggregate no message can carry + // through one record at a time. + if (new TextEncoder().encode(JSON.stringify(snapshot)).length > MAX_LEDGER_BYTES) { + throw new CommandError("too-large"); + } + return snapshot; +} + +function readRepositoryRecord(row: Row): RepositoryRecord { + const parsed = parseRepositoryRecord({ + name: row["name"], + locatorFingerprint: row["locator_fingerprint"], + requestedBase: row["requested_base"] ?? null, + creationCommit: row["creation_commit"], + primaryBranch: row["primary_branch"], + objectFormat: row["object_format"], + checkoutPath: row["checkout_path"], + }); + if (parsed === undefined) { + return corrupt("a retained Repository row does not describe a Repository"); + } + return parsed; +} + +function readWorktreeRecord(row: Row): WorktreeRecord { + const parsed = parseWorktreeRecord({ + repositoryName: row["repository_name"], + name: row["name"], + requestedBranch: row["requested_branch"], + requestedBase: row["requested_base"] ?? null, + creationCommit: row["creation_commit"], + checkoutPath: row["checkout_path"], + }); + if (parsed === undefined) { + return corrupt("a retained Worktree row does not describe a Worktree"); + } + return parsed; +} + +function readAgentSessionRow(row: Row): AgentSessionRecord { + const parsed = parseAgentSessionRecord({ + sessionKey: row["session_key"], + provider: row["provider"], + agentCommand: row["agent_command"], + sessionIdentity: row["session_identity"], + policy: row["policy"], + assertion: { kind: row["assertion_kind"], value: row["assertion_value"] }, + createdAt: row["created_at"], + }); + if (parsed === undefined) { + return corrupt("a retained Agent session row does not describe a session"); + } + return parsed; +} + +export function readJournalPage( + storage: OwnerStorage, + anchorEventId: string | null, + afterEventId: string | null, +): JournalPageValue { + if (anchorEventId === null) { + if (afterEventId !== null) { + return corrupt("an empty journal snapshot names an earlier event"); + } + return { anchorEventId, afterEventId, entries: [], done: true }; + } + const anchor = exactlyOne( + byteRows(storage, "SELECT sequence FROM journal_events WHERE event_id = ?", anchorEventId), + "journal anchor", + ); + const anchorSequence = safeInteger(anchor["sequence"], "journal anchor sequence"); + let afterSequence = 0; + if (afterEventId !== null) { + const after = exactlyOne( + byteRows(storage, "SELECT sequence FROM journal_events WHERE event_id = ?", afterEventId), + "journal cursor", + ); + afterSequence = safeInteger(after["sequence"], "journal cursor sequence"); + if (afterSequence >= anchorSequence) { + return corrupt("a journal cursor is outside its anchored snapshot"); + } + } + const rows = byteRows( + storage, + `SELECT event_id, record, workspace_root_id, + (SELECT event_id FROM journal_events AS predecessor + WHERE predecessor.sequence < event.sequence + ORDER BY predecessor.sequence DESC LIMIT 1) AS previous_event_id + FROM journal_events AS event + WHERE sequence > ? AND sequence <= ? ORDER BY sequence ASC LIMIT ?`, + afterSequence, + anchorSequence, + JOURNAL_PAGE_ENTRIES + 1, + ); + const entries: JournalPageValue["entries"][number][] = []; + let encodedBytes = 0; + for (const row of rows.slice(0, JOURNAL_PAGE_ENTRIES)) { + const eventId = safeText(row, "event_id"); + const previous = row["previous_event_id"]; + if (previous !== null && typeof previous !== "string") { + return corrupt("a journal predecessor identity is malformed"); + } + const record = safeText(row, "record"); + const workspaceRootId = safeText(row, "workspace_root_id"); + if (!SHA256.test(workspaceRootId) || !parseDurableEvent(record).ok) { + return corrupt("a journal row is malformed"); + } + const entry = { eventId, previousEventId: previous, record, workspaceRootId }; + const nextBytes = new TextEncoder().encode(JSON.stringify(entry)).length; + if (entries.length > 0 && encodedBytes + nextBytes > JOURNAL_PAGE_BYTES) { + break; + } + if (nextBytes > MAX_CONTENT_BYTES) { + throw new CommandError("too-large"); + } + entries.push(entry); + encodedBytes += nextBytes; + } + const done = rows.length <= entries.length; + if (done && entries.at(-1)?.eventId !== anchorEventId) { + return corrupt("an anchored journal snapshot is incomplete"); + } + return { anchorEventId, afterEventId, entries, done }; +} + +export function readRoot(storage: OwnerStorage, workspaceRootId: string): RootValue { + const root = referencedRoot(storage, workspaceRootId); + return { workspaceRootId, manifest: root.manifest }; +} + +export function readContent( + storage: OwnerStorage, + workspaceRootId: string, + kind: "manifest" | "blob", + digest: string, + sourceManifest: string | null, +): ContentValue { + const root = referencedRoot(storage, workspaceRootId); + const bytes = piece(storage, workspaceRootId, root, kind, digest, sourceManifest); + if (bytes.length === 0) { + return corrupt("a retained content piece is empty"); + } + return { kind, digest, size: bytes.length, bytes: encodeBase64(bytes) }; +} + +/** + * The one piece a content request names, re-read from the proved graph. + * + * Membership is decided against what the root actually names rather than + * against the reference tables alone, and a blob is reached only through a + * manifest the request names. That is what keeps this a read of one retained + * root instead of a read of the content store: staged, orphaned or + * otherwise-unreferenced bytes are addressable by nobody through here. + */ +function piece( + storage: OwnerStorage, + rootId: string, + root: StoredRoot, + kind: "manifest" | "blob", + digest: string, + sourceManifest: string | null, +): Uint8Array { + if (kind === "manifest") { + if (!root.manifests.has(digest)) { + return corrupt("a DOFS manifest is not referenced by this Workspace root"); + } + return validatedManifest(storage, root.parsed, digest).bytes; + } + const source = sourceManifest === null ? undefined : root.manifests.get(sourceManifest); + if (source === undefined || !source.chunks.some((chunk) => chunk.hash === digest)) { + return corrupt("a blob is not referenced by the named DOFS manifest"); + } + return validatedBlob(storage, rootId, root.manifests, digest); +} + +/** One page of document executions, anchored to the snapshot that began it. */ +/** The one admitted state a remote Workspace invocation begins from. */ +export interface InvocationSnapshotValue { + readonly workspaceRootId: string; + readonly journalEventId: string | null; + readonly repositories: readonly { readonly record: RepositoryRecord; readonly locator: string }[]; + readonly worktrees: readonly WorktreeRecord[]; + readonly agentSessions: readonly AgentSessionRecord[]; +} + +export interface ExecutionsValue { + readonly runId: string; + readonly anchor: number | null; + readonly after: number | null; + readonly rows: readonly { readonly sequence: number; readonly record: DocumentExecutionRecord }[]; + readonly done: boolean; +} + +/** + * Read one bounded page of the executions this run has begun. + * + * Anchored the way the journal is, and for the same reason: a caller assembling + * a list across several requests must see one snapshot rather than whatever the + * table held at each moment. The first page fixes the terminal sequence; every + * later page is constrained to it, so an execution begun while the read is in + * flight cannot appear halfway through the answer. + * + * The run identity travels with the page so the runner can refuse an answer + * from another run, and the sequence travels so it can prove adjacency. Neither + * becomes part of the semantic record. + */ +export function readExecutions( + storage: OwnerStorage, + runId: string, + anchor: number | null, + after: number | null, +): ExecutionsValue { + // The first request carries no anchor because the runner has nothing to + // anchor to yet. The owner chooses it — the terminal row at this moment — and + // answers with it, so every later page is held to the snapshot this one + // began. An empty run answers with an explicit empty anchor. + const selected = anchor ?? (after === null ? executionAnchor(storage) : null); + if (selected === null) { + if (after !== null) { + return corrupt("an empty execution snapshot names an earlier row"); + } + return { runId, anchor: null, after, rows: [], done: true }; + } + + const found = byteRows( + storage, + `SELECT sequence, execution_id, started_at, stopped_at, stop_status, + stop_reason_kind, stop_reason_code, stop_reason_event_id + FROM document_executions + WHERE sequence > ? AND sequence <= ? ORDER BY sequence ASC LIMIT ?`, + after ?? 0, + selected, + EXECUTION_PAGE_ENTRIES + 1, + ); + + const page: { sequence: number; record: DocumentExecutionRecord }[] = []; + for (const row of found.slice(0, EXECUTION_PAGE_ENTRIES)) { + const at = safeInteger(row["sequence"], "execution sequence"); + // The semantic record is what crosses, not the physical row. A row this + // owner cannot read is storage damage; sending its columns would make the + // runner responsible for a shape it has no business knowing. + const entry = { sequence: at, record: readDocumentExecution(row) }; + const grown = [...page, entry]; + if (executionPageBytes(grown) > EXECUTION_PAGE_BYTES) { + if (page.length === 0) { + // One record larger than a whole page: this snapshot cannot be paged, + // and answering with it would send what the runner must refuse. + throw new CommandError("too-large"); + } + break; + } + page.push(entry); + } + + const done = found.length <= page.length; + if (done && page.at(-1)?.sequence !== selected) { + return corrupt("an anchored execution snapshot is incomplete"); + } + return { runId, anchor: selected, after, rows: page, done }; +} + +/** The terminal execution sequence right now, or `null` when there is none. */ +export function executionAnchor(storage: OwnerStorage): number | null { + const last = byteRows( + storage, + "SELECT sequence FROM document_executions ORDER BY sequence DESC LIMIT 1", + )[0]; + return last === undefined ? null : safeInteger(last["sequence"], "execution sequence"); +} diff --git a/packages/workflow/src/cloudflare/owner-transaction.ts b/packages/workflow/src/cloudflare/owner-transaction.ts new file mode 100644 index 000000000..b91a3ec7c --- /dev/null +++ b/packages/workflow/src/cloudflare/owner-transaction.ts @@ -0,0 +1,144 @@ +/** + * The one real transaction an owner commit runs inside. + * + * A Durable Object's SQLite accepts exactly one shape of transaction: the + * runtime's own `transactionSync()`, entered once. It refuses `BEGIN`, `COMMIT` + * and `SAVEPOINT` through `sql.exec()`, and it refuses a reentrant + * `transactionSync()`. The vendored DOFS `Database` does not know that — asked + * to transact while it believes a transaction is already open, it falls back to + * `SAVEPOINT`, and every DOFS filesystem primitive opens a transaction of its + * own on the way in. + * + * So the owner enters the real transaction itself and hands DOFS a wrapper + * whose `transactionSync` runs its callback directly. Inside the real + * callback that is not a weaker promise: the outer transaction is already + * open, so a body that returns has had its work applied to the same + * transaction, and a body that throws unwinds through the real callback and + * Cloudflare rolls the whole thing back. + * + * That substitution is only safe because the owner does not use a DOFS + * savepoint as a recovery boundary. The runner has already performed the live + * Workspace work against disposable materialization; what reaches the owner is + * a complete proposal. The owner commits all of it or, treating any validation + * or application failure as infrastructure failure, none of it. + * + * The wrapper is created for one callback and refuses use outside it, so + * nothing can retain it and reach the storage later. Its DOFS caches are built + * fresh for the same reason: a resolution or blob cache populated from + * uncommitted rows must not survive a rollback or be read by a later + * operation. + */ + +import { Database as DofsDatabase } from "../../vendor/cloudflare-computer-dofs/generated/storage.js"; +import { clearBlobCache } from "../../vendor/cloudflare-computer-dofs/generated/fs/blobCache.js"; +import { clearResolveCache } from "../../vendor/cloudflare-computer-dofs/generated/fs/resolveCache.js"; +import { dofsStorage, type OwnerStorage } from "./storage.ts"; + +/** Using an enlistment after its transaction returned. */ +export class OwnerTransactionClosedError extends Error { + override name = "OwnerTransactionClosedError"; + + constructor() { + super( + "this owner transaction has finished; a DOFS enlistment is valid only inside the callback that created it.", + ); + } +} + +/** Opening an owner transaction inside one. */ +export class OwnerTransactionNestedError extends Error { + override name = "OwnerTransactionNestedError"; + + constructor() { + super( + "an owner transaction is already open; Durable Object storage admits exactly one, and a second would reach SAVEPOINT.", + ); + } +} + +/** What the body of an owner transaction is given. */ +export interface OwnerTransaction { + /** The DOFS database, enlisted in this transaction and valid only inside it. */ + readonly dofs: DofsDatabase; +} + +/** + * One Durable Object's claim on its own storage. + * + * Owned by the object rather than by this module. A module-scoped flag would be + * shared by every object in an isolate, so one object's transaction would + * refuse another's; a module-scoped registry keyed by storage would fix that + * and still be a process-lifetime table this package's rules do not allow. An + * instance the object creates and holds says the same thing without either + * problem: the gate's lifetime is the object's, and no other object can see it. + */ +export class OwnerTransactions { + #open = false; + + /** + * Run `body` inside one real `ctx.storage.transactionSync()`. + * + * `body` must complete synchronously. Nothing may await, suspend, hold a + * cursor, wait on a WebSocket or reach the runner from inside it: the runtime + * requires the callback to finish before it can commit, and a value that + * arrived later would be applied to a transaction nobody is holding. + */ + run(storage: OwnerStorage, body: (transaction: OwnerTransaction) => T): T { + if (this.#open) { + throw new OwnerTransactionNestedError(); + } + this.#open = true; + try { + return enter(storage, body); + } finally { + this.#open = false; + } + } +} + +/** + * Run `body` inside one real `ctx.storage.transactionSync()`. + * + * `body` must complete synchronously. Nothing may await, suspend, hold a + * cursor, wait on a WebSocket or reach the runner from inside it: the runtime + * requires the callback to finish before it can commit, and a value that + * arrived later would be applied to a transaction nobody is holding. + */ +/** + * Enter the one real transaction and enlist DOFS inside it. + * + * Separate from the gate above so the claim and the runtime call are two + * things: the gate says whether this object may transact, and this says what a + * transaction is. + */ +function enter(storage: OwnerStorage, body: (transaction: OwnerTransaction) => T): T { + return storage.transactionSync(() => { + let live = true; + const dofs = new DofsDatabase(dofsStorage(storage)); + // Fresh caches for this transaction alone. They are keyed by database, so + // an entry populated from rows this transaction may roll back would + // otherwise outlive it and be read by a later operation. + clearResolveCache(dofs); + clearBlobCache(dofs); + // The substitution: DOFS believes it is opening a transaction, and runs in + // the one already open. Reentrancy inside DOFS becomes ordinary nesting of + // plain function calls, which is what the runtime allows. + Object.defineProperty(dofs, "transactionSync", { + value: (closure: () => R): R => { + if (!live) { + throw new OwnerTransactionClosedError(); + } + return closure(); + }, + configurable: false, + writable: false, + }); + try { + return body({ dofs }); + } finally { + live = false; + clearResolveCache(dofs); + clearBlobCache(dofs); + } + }); +} diff --git a/packages/workflow/src/cloudflare/owner.ts b/packages/workflow/src/cloudflare/owner.ts new file mode 100644 index 000000000..41d985f55 --- /dev/null +++ b/packages/workflow/src/cloudflare/owner.ts @@ -0,0 +1,281 @@ +/** + * The Durable Object that owns one workflow run. + * + * One run, one object, selected arithmetically from the public run ID. It holds + * the WorkflowRun record and its filtered journal, the immutable Workspace roots + * and their content, and executor ownership — and it holds them in one embedded + * SQLite database, because a second store would be a second thing to keep in + * agreement with the first. + * + * What it does *not* do is as much of the contract as what it does. It runs no + * native client: no Git, no evidence process, no Agent. Those live on the + * ephemeral runner against disposable materialization, and what crosses the + * connection is a proposal this object validates and publishes. The runner + * performs; the owner decides. + * + * Three planes reach it and only one of them can advance a run. The executor + * plane is one authenticated WebSocket whose lifetime is the acquisition. + * Delivery and inspection arrive over ordinary requests, take no acquisition, + * and cannot move the lifecycle — which is why they are separate methods here + * rather than commands on the socket. + */ + +import { DurableObject } from "cloudflare:workers"; +import type { Operation } from "effection"; +import { OwnerTransactions } from "./owner-transaction.ts"; +import { + acquireExecutor, + type AcquisitionAttachment, + AcquisitionError, + releaseExecutor, + requireAcquisition, + requireExecutorSocket, +} from "./acquisition.ts"; +import { admitToken, type AdmissionPolicy, AdmissionError } from "./admission.ts"; +import { TokenError, type TokenVerification } from "./token.ts"; +import { CommandError, type CommandResult, parseCommand, type RunnerCommand } from "./commands.ts"; +import { dispatchCommand } from "./dispatcher.ts"; +import { WorkflowRecordMalformedError } from "../storage/errors.ts"; +import { discardPriorAcquisitions, PRIVATE_OBJECT_NAMES } from "./private-schema.ts"; +import { + declaredObjects, + initializeObject, + isPristine, + recognizeObject, + WorkflowObjectStorageError, +} from "./recognition.ts"; +import { ReleaseIdentityError, requireSameRelease } from "./release.ts"; +import { admitRunId, RunIdError } from "./routing.ts"; +import type { OwnerStorage } from "./storage.ts"; + +/** + * What one admission presents. + * + * Bytes and identifiers, all of them untrusted. There is deliberately no member + * for a verified result, a claim set, an acquisition identity or verification + * material: a request that could name any of those would be a request choosing + * what it is allowed to be. + */ +export interface AdmissionRequest { + readonly runId: unknown; + readonly release: unknown; + /** The raw short-lived OIDC token, exactly as presented. */ + readonly token: unknown; +} + +/** Everything a deployment must state before this object admits anybody. */ +export interface OwnerConfiguration { + readonly policy: AdmissionPolicy; + /** The issuer's keys and clock. Trusted closure state, never request data. */ + readonly verification: TokenVerification; +} + +/** Name a refusal without repeating what caused it. */ +export function refusalOf(error: unknown): string { + if (error instanceof AcquisitionError) { + return `acquisition:${error.refusal}`; + } + if (error instanceof AdmissionError) { + return `admission:${error.refusal}`; + } + if (error instanceof TokenError) { + return `token:${error.refusal}`; + } + if (error instanceof ReleaseIdentityError) { + return `release:${error.refusal}`; + } + if (error instanceof RunIdError) { + return `run-id:${error.refusal}`; + } + if (error instanceof CommandError) { + return `command:${error.refusal}`; + } + if (error instanceof WorkflowObjectStorageError) { + if (error.failure.kind === "unsupported-version") { + // The version travels in the category rather than beside it, because the + // answer envelope carries a refusal and nothing else. It is the one fact + // a host needs to decide whether this build may open the store, and a + // public error that guessed it would state something untrue. + return `storage:unsupported-version-v${error.failure.schemaVersion}`; + } + return `storage:${error.failure.kind}`; + } + if (error instanceof WorkflowRecordMalformedError) { + return "storage:corrupt"; + } + if ( + error instanceof Error && + (error.message.startsWith("private protocol storage") || + error.message.startsWith("private staging") || + error.message.startsWith("stored bytes")) + ) { + return "storage:corrupt"; + } + return "internal"; +} + +/** + * The owner, minus the deployment's own configuration. + * + * Subclassed rather than configured through a binding because the policy is + * trusted host state: a value a request could supply would be a runner naming + * the identities it must satisfy. + */ +export abstract class WorkflowOwnerObject extends DurableObject { + /** + * This object's claim on its own storage. + * + * One per Durable Object, so a transaction here cannot refuse one in another + * object and no table outlives the object that owns it. + */ + protected readonly transactions: OwnerTransactions = new OwnerTransactions(); + + protected abstract configuration(): OwnerConfiguration; + + /** This object's storage, as the shared modules expect to see it. */ + protected get owned(): OwnerStorage { + return this.ctx.storage; + } + + /** + * Admit one executor connection. + * + * The order is the contract: the build is compared before any token work, the + * token is verified before the run is touched, and the acquisition is taken + * last. A refusal at any step leaves no acquisition and no object state. + * + * The correlation value is minted here, after both checks pass, and never + * taken from the request. A caller-selected one would let a later connection + * reuse an abandoned identifier and collide with the private staging that + * identifier partitions. + */ + *admit(request: AdmissionRequest, socket: WebSocket): Operation { + const { policy, verification } = this.configuration(); + requireSameRelease(policy.release, request.release); + yield* admitToken(policy, verification, request.token); + const runId = admitRunId(request.runId); + const acquisitionId = mintAcquisitionId(); + return acquireExecutor(this.ctx, socket, runId, acquisitionId, () => { + const names = new Set(declaredObjects(this.owned).map((object) => object.name)); + if (PRIVATE_OBJECT_NAMES.every((name) => names.has(name))) { + recognizeObject(this.owned); + this.transactions.run(this.owned, () => { + discardPriorAcquisitions(this.owned, acquisitionId); + }); + } + }); + } + + /** + * Handle one message from an admitted connection. + * + * Acquisition is proved before the message is parsed, so a superseded or + * foreign socket never reaches the command reader — and proved again by + * whatever writes, inside the transaction that writes. + */ + onRunnerMessage(socket: WebSocket, runId: string, raw: string): CommandResult { + let command: RunnerCommand | undefined; + try { + requireAcquisition(this.ctx, socket, runId); + command = parseCommand(raw); + return dispatchCommand(this.ctx, this.transactions, socket, runId, command); + } catch (error) { + return { id: command?.id ?? "", outcome: "refused", refusal: refusalOf(error) }; + } + } + + webSocketMessage(socket: WebSocket, message: string | ArrayBuffer): void { + let answer: CommandResult; + if (typeof message !== "string") { + answer = { id: "", outcome: "refused", refusal: "command:malformed-member" }; + } else { + try { + const held = requireExecutorSocket(this.ctx, socket); + answer = this.onRunnerMessage(socket, held.runId, message); + } catch (error) { + answer = { id: "", outcome: "refused", refusal: refusalOf(error) }; + } + } + try { + socket.send(JSON.stringify(answer)); + } catch { + releaseExecutor(socket); + socket.close(1011, "send failed"); + return; + } + if (fatal(answer)) { + releaseExecutor(socket); + socket.close(1002, "protocol refused"); + } + } + + /** A connection that ended owns nothing, and rolled nothing back. */ + webSocketClose(socket: WebSocket): void { + releaseExecutor(socket); + } + + webSocketError(socket: WebSocket): void { + releaseExecutor(socket); + } + + /** + * Create this run's storage, or recognize what is already there. + * + * Pristine is asked first rather than inferred from a refusal: storage that + * holds nothing is the only storage this build may write into, and every + * other state — foreign, damaged, a version this build does not implement — + * is recognition's to refuse rather than initialization's to overwrite. + */ + open(runId: string, initializeRun: () => void): void { + admitRunId(runId); + if (isPristine(declaredObjects(this.owned))) { + initializeObject(this.owned, this.transactions, initializeRun); + return; + } + recognizeObject(this.owned); + } +} + +/** + * A fresh correlation value for one acquisition. + * + * Bounded and unpredictable, and used only to partition acquisition-private + * staging and duplicate handling. It is not a bearer credential, a lease, a + * generation record or a durable identity: what proves a message may act is the + * exact live socket, and this value proves nothing on its own. + */ +function mintAcquisitionId(): string { + const bytes = new Uint8Array(16); + crypto.getRandomValues(bytes); + return Array.from(bytes, (byte) => byte.toString(16).padStart(2, "0")).join(""); +} + +/** + * Whether a refusal means the connection itself is finished. + * + * Two kinds of refusal reach here and they deserve opposite treatment. One says + * the channel or the store is not what it claims — a message that would not + * parse, an acquisition this socket does not hold, storage that is damaged — + * and carrying on would mean guessing what the other side meant. + * + * The other is an answer about the request. A duplicate id, a frontier that has + * moved, a mapping that disagrees with what is already retained, and a command + * this release does not implement are all decisions the runner can act on: read + * the frontier again, propose against it, or stop. Closing the connection on + * those would turn every ordinary disagreement into a lost acquisition and make + * the runner reconnect to be told the same thing. + */ +const ANSWERED: readonly string[] = [ + "command:duplicate-conflict", + "command:unavailable", + "command:stale-root", + "command:stale-journal", + "command:mapping-conflict", +]; + +function fatal(answer: CommandResult): boolean { + if (answer.outcome === "performed") { + return false; + } + return !ANSWERED.includes(answer.refusal); +} diff --git a/packages/workflow/src/cloudflare/private-schema.ts b/packages/workflow/src/cloudflare/private-schema.ts new file mode 100644 index 000000000..ee3b6ae39 --- /dev/null +++ b/packages/workflow/src/cloudflare/private-schema.ts @@ -0,0 +1,121 @@ +/** + * The scratch state one acquisition keeps, and nothing else keeps. + * + * Hibernation is why this is in SQLite rather than in a field or an attachment. + * An idle Durable Object is evicted while its sockets stay open, so anything + * held in memory is gone by the time the next message arrives; and the + * attachment is bounded at 16 KiB and is the compact acquisition identity, not + * somewhere to put a growing ledger or a content payload. + * + * Two tables, both keyed by the owner-minted acquisition ID. One remembers what + * each command ID already decided, so a retry returns the decision rather than + * acting twice. The other holds content a runner has offered but nothing has + * adopted. + * + * Neither is run state. Staged bytes are not published content: they are in no + * root, referenced by nothing, invisible to every retained read, and adopting + * them is a later checkpoint's transaction to perform. Both are declared here + * rather than in the shared logical schema for exactly that reason — they are + * this adapter's physical scratch, and a host that had no hibernation would + * need neither. + * + * Recognition checks their exact shapes like any other declared object. Storage + * carrying a table this build did not write is refused rather than tolerated + * because its name looked familiar. + */ + +import { normalize, type SchemaObject } from "../sqlite/workflow-schema.ts"; +import type { OwnerStorage } from "./storage.ts"; + +export const COMMAND_TABLE = "_xmd_executor_commands"; +/** + * Decisions about mutations, which outlive the connection that asked for them. + * + * The acquisition-scoped ledger answers a retry on the same socket. It cannot + * answer the case that matters most: the owner committed, the answer was lost, + * and the connection died. A replacement acquisition discards its predecessor's + * scratch — correctly, because staged bytes and read decisions belong to the + * connection that produced them — but the fact that a mutation was applied is + * not scratch. It is the only thing that lets the next connection tell "this + * already happened" from "this never happened", and without it the same request + * meets a moved frontier and is refused as stale while the runner has no way to + * know whether to promote or discard. + * + * So a mutation decision is keyed by the run rather than the acquisition, and + * cleanup never touches it. It is not a lease and does not expire because a + * socket did. + */ +export const MUTATION_TABLE = "_xmd_run_mutations"; +export const STAGING_TABLE = "_xmd_executor_staging"; + +const COMMAND_SQL = `CREATE TABLE ${COMMAND_TABLE} ( + acquisition_id TEXT NOT NULL, + command_id TEXT NOT NULL, + request_fingerprint TEXT NOT NULL CHECK ( + length(request_fingerprint) = 64 AND request_fingerprint NOT GLOB '*[^0-9a-f]*' + ), + response TEXT NOT NULL CHECK (json_valid(response)), + response_bytes INTEGER NOT NULL CHECK (response_bytes >= 0), + PRIMARY KEY (acquisition_id, command_id) +) STRICT, WITHOUT ROWID`; + +const STAGING_SQL = `CREATE TABLE ${STAGING_TABLE} ( + acquisition_id TEXT NOT NULL, + kind TEXT NOT NULL CHECK (kind IN ('manifest', 'blob')), + digest TEXT NOT NULL CHECK ( + length(digest) = 64 AND digest NOT GLOB '*[^0-9a-f]*' + ), + size INTEGER NOT NULL CHECK (size > 0), + bytes BLOB NOT NULL, + PRIMARY KEY (acquisition_id, kind, digest) +) STRICT, WITHOUT ROWID`; + +const MUTATION_SQL = `CREATE TABLE ${MUTATION_TABLE} ( + command_id TEXT PRIMARY KEY, + request_fingerprint TEXT NOT NULL CHECK ( + length(request_fingerprint) = 64 AND request_fingerprint NOT GLOB '*[^0-9a-f]*' + ), + response TEXT NOT NULL CHECK (json_valid(response)), + response_bytes INTEGER NOT NULL CHECK (response_bytes >= 0) +) STRICT, WITHOUT ROWID`; + +const PRIVATE_OBJECTS = new Map([ + [COMMAND_TABLE, { type: "table", sql: COMMAND_SQL }], + [STAGING_TABLE, { type: "table", sql: STAGING_SQL }], + [MUTATION_TABLE, { type: "table", sql: MUTATION_SQL }], +]); + +export const PRIVATE_OBJECT_NAMES: readonly string[] = Object.freeze([...PRIVATE_OBJECTS.keys()]); + +export function initializePrivateSchema(storage: OwnerStorage): void { + storage.sql.exec(`${COMMAND_SQL};\n\n${STAGING_SQL};\n\n${MUTATION_SQL};`); +} + +export function privateStructureFailure( + objects: readonly SchemaObject[], +): { kind: "missing" | "misshapen"; name: string } | undefined { + const byName = new Map(objects.map((object) => [object.name, object])); + for (const [name, expected] of PRIVATE_OBJECTS) { + const found = byName.get(name); + if (found === undefined) { + return { kind: "missing", name }; + } + if (found.type !== expected.type || normalize(found.sql) !== normalize(expected.sql)) { + return { kind: "misshapen", name }; + } + } + return undefined; +} + +/** + * Discard what belonged to a connection that is gone. + * + * Staged bytes and read decisions are that connection's scratch and go with it. + * Mutation decisions deliberately do not: they are how the next connection + * learns that a commit already happened, and deleting one would turn a retry + * into a second mutation or a refusal the runner cannot interpret. + */ +export function discardPriorAcquisitions(storage: OwnerStorage, acquisitionId: string): void { + storage.sql.exec(`DELETE FROM ${COMMAND_TABLE} WHERE acquisition_id <> ?`, acquisitionId); + storage.sql.exec(`DELETE FROM ${STAGING_TABLE} WHERE acquisition_id <> ?`, acquisitionId); +} diff --git a/packages/workflow/src/cloudflare/publish.ts b/packages/workflow/src/cloudflare/publish.ts new file mode 100644 index 000000000..c8704718d --- /dev/null +++ b/packages/workflow/src/cloudflare/publish.ts @@ -0,0 +1,762 @@ +/** + * Deciding one proposal, and applying all of it or none of it. + * + * This is where a remote run actually moves. Everything before it is reading + * and staging; everything after it is history. The runner has done the work, + * captured a root, and offered a description of what it wants published — and + * none of that is authority. The owner recomputes every identity, resolves + * every piece against content it already holds or bytes this exact acquisition + * staged, and only then writes. + * + * The order is deliberate and each step exists because skipping it is a way to + * publish something nobody proposed: + * + * 1. The frontier is re-read *here*, inside the transaction, and compared with + * what the runner said it started from — root and terminal event both, + * `null` included exactly. A frontier read before the transaction is a + * frontier that can move before the write. + * 2. The proposed identity is recomputed from the manifest rather than + * believed. An identity is a digest, and a digest a caller supplies is a + * claim about bytes rather than a property of them. + * 3. The inventory must be exactly the closure of that manifest — every + * manifest its file entries name, every blob those manifests name, once + * each, and nothing else. A missing piece is a root that cannot be + * materialized; an extra one is content the root does not account for. + * 4. Each piece resolves from authoritative content or from this acquisition's + * staging. Staging supplies bytes and grants nothing: a digest another + * acquisition staged is not reachable, and a digest already authoritative + * under different bytes is a disagreement rather than an overwrite. + * 5. Content, root, references, mappings, the current pointer and the journal + * rows are written together. The pointer moves by compare-and-set from the + * expected root, so two commits racing the same frontier cannot both win. + * + * Journal rows are associated with the root this commit selected: the proposed + * root when there is a publication, the unchanged expected root when there is + * not. That is the same rule the local host follows, and it is what makes + * history readable against the Workspace it happened in. + * + * Nothing here awaits, yields, sends a frame or contacts the runner. It runs + * inside one synchronous transaction and returns a value the caller serializes + * afterwards. + */ + +import { WorkflowRecordMalformedError } from "../storage/errors.ts"; +import { + compareUtf8, + parseWorkspaceRootManifest, + WORKSPACE_ROOT_DOMAIN, +} from "../workspace/root-manifest.ts"; +import { decodeContentManifest } from "../workspace/content-manifest.ts"; +import { MAX_CONTENT_BYTES } from "./commands.ts"; +import { sha256Hex } from "../workspace/sha256.ts"; +import { CommandError, type CommitCommand, type ProposedMapping } from "./commands.ts"; +import { validateRetainedRoot } from "./owner-reads.ts"; +import { readRetrieval } from "../sqlite/rows.ts"; +import { bytesOf } from "./encoding.ts"; +import { STAGING_TABLE } from "./private-schema.ts"; +import type { OwnerStorage } from "./storage.ts"; + +/** What the owner answers a performed commit with. */ +export interface CommitValue { + readonly workspaceRootId: string; + readonly journalEventIds: readonly string[]; +} + +function corrupt(reason: string): never { + throw new WorkflowRecordMalformedError("workflow owner storage", reason); +} + +function rows( + storage: OwnerStorage, + sql: string, + ...bindings: unknown[] +): Record[] { + return storage.sql.exec(sql, ...bindings).toArray(); +} + +/** Content identities in the canonical order references are written in. */ +function sortedDigests(digests: Iterable): string[] { + const found = [...digests]; + found.sort(compareUtf8); + return found; +} + +function hexBytes(digest: string): Uint8Array { + const bytes = new Uint8Array(digest.length / 2); + for (let index = 0; index < bytes.length; index += 1) { + bytes[index] = Number.parseInt(digest.slice(index * 2, index * 2 + 2), 16); + } + return bytes; +} + +/** + * The frontier as it is right now, read where the write will happen. + * + * The current root is proved complete by the same validator the read boundary + * uses, not merely read out of the pointer. A commit accepts its starting root + * as the run's frontier, and accepting one whose content graph cannot be + * materialized would append history against a Workspace nothing can restore — + * a proposal is not a licence to repair, so damage is refused here rather than + * worked around. + */ +function frontier(storage: OwnerStorage): { rootId: string; journalEventId: string | null } { + const state = rows(storage, "SELECT current_root_id FROM workspace_state WHERE singleton_id = 1"); + const current = state[0]?.["current_root_id"]; + if (state.length !== 1 || typeof current !== "string") { + return corrupt("the Workspace has no single current root"); + } + validateRetainedRoot(storage, current); + const last = rows( + storage, + "SELECT event_id FROM journal_events ORDER BY sequence DESC LIMIT 1", + )[0]; + const eventId = last?.["event_id"]; + if (last !== undefined && typeof eventId !== "string") { + return corrupt("a journal row has no identity"); + } + return { rootId: current, journalEventId: last === undefined ? null : String(eventId) }; +} + +/** Bytes for one proposed identity, from what is authoritative or what was staged. */ +function resolve( + storage: OwnerStorage, + acquisitionId: string, + kind: "manifest" | "blob", + digest: string, +): { bytes: Uint8Array; authoritative: boolean } { + const table = kind === "manifest" ? "vfs_manifests" : "vfs_blob_bytes"; + const column = kind === "manifest" ? "encoded" : "bytes"; + const authoritative = rows( + storage, + `SELECT ${column} AS content FROM ${table} WHERE lower(hex(hash)) = ?`, + digest, + ); + if (authoritative.length > 1) { + return corrupt("retained content is stored more than once under one identity"); + } + const held = authoritative[0]; + if (held !== undefined) { + const bytes = bytesOf(held["content"]); + if (bytes.length > MAX_CONTENT_BYTES || sha256Hex(bytes) !== digest) { + return corrupt("retained content disagrees with the identity it is stored under"); + } + // The companion row is part of the same fact. A size that disagrees with + // the bytes is damage, and adopting a proposal over it would publish a root + // whose content the read path refuses. + confirmCompanion(storage, kind, digest, bytes); + return { bytes, authoritative: true }; + } + if (kind === "blob") { + // A metadata row with no bytes is a half-written identity. Falling through + // to staging here would complete it as a side effect of a proposal, and + // which durable state won would depend on the write path rather than on + // what the store actually holds. + const partial = rows(storage, "SELECT size FROM vfs_blobs WHERE lower(hex(hash)) = ?", digest); + if (partial.length > 0) { + return corrupt("a retained blob has no bytes"); + } + } + const staged = rows( + storage, + `SELECT bytes FROM ${STAGING_TABLE} WHERE acquisition_id = ? AND kind = ? AND digest = ?`, + acquisitionId, + kind, + digest, + )[0]; + if (staged === undefined) { + // Either never offered, or offered by an acquisition that is not this one. + // Both are the same refusal: this proposal names content this connection + // has not supplied. + throw new CommandError("malformed-member"); + } + const bytes = bytesOf(staged["bytes"]); + if (sha256Hex(bytes) !== digest) { + return corrupt("staged content disagrees with the identity it was stored under"); + } + return { bytes, authoritative: false }; +} + +/** + * The metadata stored beside one content identity, confirmed rather than fixed. + * + * A manifest's recorded size must equal what its chunks add up to; a blob's + * recorded size must equal its bytes; and a blob's byte row and its `vfs_blobs` + * row must both exist. Any disagreement is existing damage, refused here rather + * than silently repaired by an `ON CONFLICT DO NOTHING` that leaves the wrong + * row in place. + */ +function confirmCompanion( + storage: OwnerStorage, + kind: "manifest" | "blob", + digest: string, + bytes: Uint8Array, +): void { + if (kind === "manifest") { + const row = rows( + storage, + "SELECT size FROM vfs_manifests WHERE lower(hex(hash)) = ?", + digest, + )[0]; + const decoded = decodeContentManifest(bytes, corrupt); + if (row === undefined || Number(row["size"]) !== decoded.size) { + return corrupt("a retained manifest disagrees with its recorded size"); + } + return; + } + const row = rows(storage, "SELECT size FROM vfs_blobs WHERE lower(hex(hash)) = ?", digest)[0]; + if (row === undefined || Number(row["size"]) !== bytes.length) { + return corrupt("a retained blob disagrees with its recorded size"); + } +} + +/** + * Apply one proposal, entirely, inside the caller's open transaction. + * + * The caller has already proved the acquisition twice and recognized the store. + * What is left is deciding whether this proposal is true and writing it. + */ +export function applyCommit( + storage: OwnerStorage, + acquisitionId: string, + command: CommitCommand, + mintEventId: () => string, +): CommitValue { + const now = frontier(storage); + if (now.rootId !== command.expectedWorkspaceRootId) { + throw new CommandError("stale-root"); + } + if (now.journalEventId !== command.expectedJournalEventId) { + throw new CommandError("stale-journal"); + } + + const selected = + command.publication === null + ? command.expectedWorkspaceRootId + : publish(storage, acquisitionId, command); + + const selectedEntries = directoriesOf( + command.publication === null ? undefined : command.publication.proposedManifest, + ); + // The whole collection is decided before any of it is written. A Worktree may + // name a Repository that arrives in the same proposal, and which of the two + // happens to come first in an array is not a difference between proposals — + // an owner that applied them in order would accept one spelling of a + // transaction and refuse an identical one. + validateMappings(storage, command.mappings, selectedEntries); + // Applied in dependency order rather than the order they arrived in. A + // Worktree row references its Repository, so the parent has to exist when the + // child is written — but which one a proposal happens to list first is not a + // difference between proposals, and the owner decides that rather than making + // the runner arrange an array to suit the schema. + for (const mapping of dependencyOrder(command.mappings)) { + applyMapping(storage, mapping); + } + + const journalEventIds: string[] = []; + for (const record of command.events) { + const eventId = mintEventId(); + storage.sql.exec( + "INSERT INTO journal_events (event_id, record, workspace_root_id) VALUES (?, ?, ?)", + eventId, + record, + selected, + ); + journalEventIds.push(eventId); + } + + return { workspaceRootId: selected, journalEventIds }; +} + +/** Adopt the content and the root, and move the pointer to it. */ +function publish(storage: OwnerStorage, acquisitionId: string, command: CommitCommand): string { + const proposal = command.publication; + if (proposal === null) { + return command.expectedWorkspaceRootId; + } + if ( + sha256Hex(`${WORKSPACE_ROOT_DOMAIN}${proposal.proposedManifest}`) !== + proposal.proposedWorkspaceRootId + ) { + throw new CommandError("malformed-member"); + } + const parsed = parseWorkspaceRootManifest(proposal.proposedManifest, () => { + throw new CommandError("malformed-member"); + }); + + // The closure the manifest actually names, derived here rather than taken + // from the inventory the request supplied. + const named = new Set( + parsed.entries.flatMap((entry) => (entry.kind === "file" ? [entry.manifest] : [])), + ); + const offered = new Map( + proposal.content.map((piece) => [`${piece.kind}:${piece.digest}`, piece]), + ); + + const manifests = new Map(); + for (const digest of named) { + const piece = offered.get(`manifest:${digest}`); + if (piece === undefined) { + throw new CommandError("malformed-member"); + } + const { bytes } = resolve(storage, acquisitionId, "manifest", digest); + if (bytes.length !== piece.size) { + throw new CommandError("malformed-member"); + } + manifests.set(digest, bytes); + } + + const blobs = new Map(); + for (const [digest, bytes] of manifests) { + const decoded = decodeContentManifest(bytes, () => { + throw new CommandError("malformed-member"); + }); + for (const entry of parsed.entries) { + if (entry.kind === "file" && entry.manifest === digest && entry.size !== decoded.size) { + throw new CommandError("malformed-member"); + } + } + for (const chunk of decoded.chunks) { + const seen = blobs.get(chunk.hash); + if (seen !== undefined && seen !== chunk.size) { + throw new CommandError("malformed-member"); + } + blobs.set(chunk.hash, chunk.size); + } + } + + // Exactly the closure: nothing missing, nothing extra. + if (offered.size !== named.size + blobs.size) { + throw new CommandError("malformed-member"); + } + + const blobBytes = new Map(); + for (const [digest, size] of blobs) { + const piece = offered.get(`blob:${digest}`); + if (piece === undefined || piece.size !== size) { + throw new CommandError("malformed-member"); + } + const { bytes } = resolve(storage, acquisitionId, "blob", digest); + if (bytes.length !== size) { + throw new CommandError("malformed-member"); + } + blobBytes.set(digest, bytes); + } + + for (const [digest, bytes] of blobBytes) { + const hash = hexBytes(digest); + storage.sql.exec( + "INSERT INTO vfs_blobs (hash, size, last_seen) VALUES (?, ?, 0) ON CONFLICT(hash) DO NOTHING", + hash, + bytes.length, + ); + storage.sql.exec( + "INSERT INTO vfs_blob_bytes (hash, bytes) VALUES (?, ?) ON CONFLICT(hash) DO NOTHING", + hash, + bytes, + ); + } + for (const [digest, bytes] of manifests) { + storage.sql.exec( + `INSERT INTO vfs_manifests (hash, size, encoded, last_seen) VALUES (?, ?, ?, 0) + ON CONFLICT(hash) DO NOTHING`, + hexBytes(digest), + decodeContentManifest(bytes, () => { + throw new CommandError("malformed-member"); + }).size, + bytes, + ); + } + + // Immutable: a root already retained is confirmed rather than rewritten. + const existing = rows( + storage, + "SELECT manifest FROM workspace_roots WHERE root_id = ?", + proposal.proposedWorkspaceRootId, + )[0]; + if (existing === undefined) { + storage.sql.exec( + "INSERT INTO workspace_roots (root_id, format_version, manifest) VALUES (?, 1, ?)", + proposal.proposedWorkspaceRootId, + proposal.proposedManifest, + ); + for (const digest of sortedDigests(named)) { + storage.sql.exec( + "INSERT INTO workspace_root_manifest_refs (root_id, manifest_hash) VALUES (?, ?)", + proposal.proposedWorkspaceRootId, + hexBytes(digest), + ); + } + for (const digest of sortedDigests(blobs.keys())) { + storage.sql.exec( + "INSERT INTO workspace_root_blob_refs (root_id, blob_hash) VALUES (?, ?)", + proposal.proposedWorkspaceRootId, + hexBytes(digest), + ); + } + } else if (existing["manifest"] !== proposal.proposedManifest) { + return corrupt("a retained Workspace root disagrees with the identity it is stored under"); + } + + // Whether it was just written or was already there, the root the pointer is + // about to name is proved to be a complete materializable root — the same + // proof the read boundary applies, so a root cannot be publishable by one + // path and refused by the other. + validateRetainedRoot(storage, proposal.proposedWorkspaceRootId); + + // Compare-and-set. Two commits racing one frontier cannot both move it. + storage.sql.exec( + "UPDATE workspace_state SET current_root_id = ? WHERE singleton_id = 1 AND current_root_id = ?", + proposal.proposedWorkspaceRootId, + command.expectedWorkspaceRootId, + ); + const moved = rows( + storage, + "SELECT current_root_id FROM workspace_state WHERE singleton_id = 1", + )[0]; + if (moved?.["current_root_id"] !== proposal.proposedWorkspaceRootId) { + throw new CommandError("stale-root"); + } + return proposal.proposedWorkspaceRootId; +} + +/** How one mapping is named within a proposal, whatever order it arrives in. */ +function mappingIdentity(mapping: ProposedMapping): string { + if (mapping.kind === "worktree") { + return `worktree:${mapping.record.repositoryName}/${mapping.record.name}`; + } + if (mapping.kind === "repository") { + return `repository:${mapping.record.name}`; + } + return `agent-session:${mapping.record.sessionKey}`; +} + +/** + * Decide the whole mapping collection, before any of it is written. + * + * Identity, duplication, parent relationships and checkout placement are all + * properties of the proposal rather than of one mapping, so they are settled + * here — against every mapping the proposal carries and the Workspace it + * selects. Deciding them one at a time during application would make acceptance + * depend on transport order. + */ +function validateMappings( + storage: OwnerStorage, + mappings: readonly ProposedMapping[], + selectedEntries: ReadonlySet | undefined, +): void { + const named = new Set(); + for (const mapping of mappings) { + const identity = mappingIdentity(mapping); + if (named.has(identity)) { + // One proposal naming one mapping twice cannot be applied once and is not + // two mappings either. + throw new CommandError("mapping-conflict"); + } + named.add(identity); + } + + for (const mapping of mappings) { + if (mapping.kind === "agent-session") { + continue; + } + if (retainedMapping(storage, mapping) !== undefined) { + // Already retained. Whether the proposal agrees with it is confirmed + // where the row is read; a mapping that exists needs no new checkout. + continue; + } + // A new checkout mapping is only true if this proposal publishes the + // Workspace that contains it. + if (selectedEntries === undefined || !selectedEntries.has(mapping.record.checkoutPath)) { + throw new CommandError("mapping-conflict"); + } + if ( + mapping.kind === "worktree" && + rows( + storage, + "SELECT name FROM workspace_repositories WHERE name = ?", + mapping.record.repositoryName, + )[0] === undefined && + !proposesRepository(mappings, mapping.record.repositoryName) + ) { + // A Worktree exists inside a Repository. One that named none — neither + // retained nor arriving in this same proposal — would be a checkout + // belonging to nothing. + throw new CommandError("mapping-conflict"); + } + } +} + +/** The row already retained for one mapping, if there is one. */ +function retainedMapping( + storage: OwnerStorage, + mapping: ProposedMapping, +): Record | undefined { + if (mapping.kind === "repository") { + return rows( + storage, + `SELECT locator, locator_fingerprint, requested_base, creation_commit, primary_branch, + object_format, checkout_path FROM workspace_repositories WHERE name = ?`, + mapping.record.name, + )[0]; + } + if (mapping.kind === "worktree") { + return rows( + storage, + `SELECT requested_branch, requested_base, creation_commit, checkout_path + FROM workspace_worktrees WHERE repository_name = ? AND name = ?`, + mapping.record.repositoryName, + mapping.record.name, + )[0]; + } + return rows( + storage, + `SELECT provider, agent_command, session_identity, policy, + assertion_kind, assertion_value, created_at + FROM agent_sessions WHERE session_key = ?`, + mapping.record.sessionKey, + )[0]; +} + +/** Parents before children, so a proposal's array order carries no meaning. */ +const APPLICATION_ORDER: readonly ProposedMapping["kind"][] = [ + "repository", + "worktree", + "agent-session", +]; + +function dependencyOrder(mappings: readonly ProposedMapping[]): ProposedMapping[] { + const ordered: ProposedMapping[] = []; + for (const kind of APPLICATION_ORDER) { + for (const mapping of mappings) { + if (mapping.kind === kind) { + ordered.push(mapping); + } + } + } + return ordered; +} + +/** Whether this proposal itself supplies the Repository a Worktree names. */ +function proposesRepository(mappings: readonly ProposedMapping[], name: string): boolean { + return mappings.some((mapping) => mapping.kind === "repository" && mapping.record.name === name); +} + +/** Every directory the selected root contains, for placement checks. */ +function directoriesOf(manifest: string | undefined): ReadonlySet | undefined { + if (manifest === undefined) { + return undefined; + } + const parsed = parseWorkspaceRootManifest(manifest, () => { + throw new CommandError("malformed-member"); + }); + return new Set( + parsed.entries.flatMap((entry) => (entry.kind === "directory" ? [entry.path] : [])), + ); +} + +function sameText(row: Record, column: string, expected: string | null): boolean { + const value = row[column]; + return expected === null ? value === null : value === expected; +} + +/** + * One retained mapping, inserted or confirmed in full. + * + * Creation identity is immutable, so an existing row is compared on every field + * that establishes it — not on a convenient subset. A partial comparison would + * report performed for a proposal that disagrees with what an earlier execution + * established, and the disagreement would only surface later, as a checkout + * that is not what its record says. + */ +function applyMapping(storage: OwnerStorage, mapping: ProposedMapping): void { + if (mapping.kind === "repository") { + const record = mapping.record; + const held = rows( + storage, + `SELECT locator, locator_fingerprint, requested_base, creation_commit, primary_branch, + object_format, checkout_path FROM workspace_repositories WHERE name = ?`, + record.name, + )[0]; + if (held === undefined) { + storage.sql.exec( + `INSERT INTO workspace_repositories + (name, locator, locator_fingerprint, requested_base, creation_commit, + primary_branch, object_format, checkout_path) + VALUES (?, ?, ?, ?, ?, ?, ?, ?)`, + record.name, + mapping.locator, + record.locatorFingerprint, + record.requestedBase, + record.creationCommit, + record.primaryBranch, + record.objectFormat, + record.checkoutPath, + ); + return; + } + if ( + held["locator"] !== mapping.locator || + held["locator_fingerprint"] !== record.locatorFingerprint || + !sameText(held, "requested_base", record.requestedBase) || + held["creation_commit"] !== record.creationCommit || + held["primary_branch"] !== record.primaryBranch || + held["object_format"] !== record.objectFormat || + held["checkout_path"] !== record.checkoutPath + ) { + throw new CommandError("mapping-conflict"); + } + return; + } + + if (mapping.kind === "worktree") { + const record = mapping.record; + const held = rows( + storage, + `SELECT requested_branch, requested_base, creation_commit, checkout_path + FROM workspace_worktrees WHERE repository_name = ? AND name = ?`, + record.repositoryName, + record.name, + )[0]; + if (held === undefined) { + storage.sql.exec( + `INSERT INTO workspace_worktrees + (repository_name, name, requested_branch, requested_base, creation_commit, checkout_path) + VALUES (?, ?, ?, ?, ?, ?)`, + record.repositoryName, + record.name, + record.requestedBranch, + record.requestedBase, + record.creationCommit, + record.checkoutPath, + ); + return; + } + if ( + held["requested_branch"] !== record.requestedBranch || + !sameText(held, "requested_base", record.requestedBase) || + held["creation_commit"] !== record.creationCommit || + held["checkout_path"] !== record.checkoutPath + ) { + throw new CommandError("mapping-conflict"); + } + return; + } + + const record = mapping.record; + const held = rows( + storage, + `SELECT provider, agent_command, session_identity, policy, + assertion_kind, assertion_value, created_at + FROM agent_sessions WHERE session_key = ?`, + record.sessionKey, + )[0]; + if (held === undefined) { + storage.sql.exec( + `INSERT INTO agent_sessions + (session_key, provider, agent_command, session_identity, policy, + assertion_kind, assertion_value, created_at) + VALUES (?, ?, ?, ?, ?, ?, ?, ?)`, + record.sessionKey, + record.provider, + record.agentCommand, + record.sessionIdentity, + record.policy, + record.assertion.kind, + record.assertion.value, + record.createdAt, + ); + return; + } + if ( + held["provider"] !== record.provider || + held["agent_command"] !== record.agentCommand || + held["session_identity"] !== record.sessionIdentity || + held["policy"] !== record.policy || + held["assertion_kind"] !== record.assertion.kind || + held["assertion_value"] !== record.assertion.value || + held["created_at"] !== record.createdAt + ) { + throw new CommandError("mapping-conflict"); + } +} + +/** What the owner answers a performed retrieval replacement with. */ +export interface RetrievalValue { + readonly retrieval: { + readonly metadata: unknown; + readonly revision: number; + readonly updatedAt: string; + } | null; +} + +/** + * Replace or clear where this run's definition can be fetched from. + * + * Its own mutation rather than a degenerate commit. Nothing is appended to the + * journal, no root moves, and the revision is the owner's arithmetic over what + * is stored rather than a number the runner proposed — two handles that both + * read revision one before either wrote would otherwise both write two, and the + * second would silently lose the first. + * + * The expected root is revalidated here, inside the transaction that writes, so + * a replacement proposed against a frontier that has moved is refused on the + * same terms a commit is. + */ +export function applyRetrieval( + storage: OwnerStorage, + command: { expectedWorkspaceRootId: string; metadata: string | null }, + now: () => string, +): RetrievalValue { + const state = rows(storage, "SELECT current_root_id FROM workspace_state WHERE singleton_id = 1"); + const current = state[0]?.["current_root_id"]; + if (state.length !== 1 || typeof current !== "string") { + return corrupt("the Workspace has no single current root"); + } + if (current !== command.expectedWorkspaceRootId) { + throw new CommandError("stale-root"); + } + validateRetainedRoot(storage, current); + + if (command.metadata === null) { + // Clearing removes the row. The next replacement starts counting again, + // because a revision counts replacements since the metadata last existed. + storage.sql.exec("DELETE FROM definition_retrieval WHERE id = 1"); + return { retrieval: null }; + } + + const held = rows(storage, "SELECT revision FROM definition_retrieval WHERE id = 1")[0]; + const revision = held === undefined ? 1 : safeRevision(held["revision"]) + 1; + const updatedAt = now(); + storage.sql.exec( + `INSERT INTO definition_retrieval (id, metadata, revision, updated_at) VALUES (1, ?, ?, ?) + ON CONFLICT(id) DO UPDATE SET metadata = excluded.metadata, + revision = excluded.revision, updated_at = excluded.updated_at`, + command.metadata, + revision, + updatedAt, + ); + + const written = rows( + storage, + "SELECT metadata, revision, updated_at FROM definition_retrieval WHERE id = 1", + )[0]; + if (written === undefined) { + return corrupt("a retrieval replacement wrote no row"); + } + // Read back and parsed, so the answer describes what is actually stored. + const parsed = readRetrieval(written); + return { + retrieval: { + metadata: parsed.metadata, + revision: parsed.revision, + updatedAt: parsed.updatedAt, + }, + }; +} + +function safeRevision(value: unknown): number { + if (typeof value !== "number" || !Number.isSafeInteger(value) || value < 1) { + return corrupt("a retained retrieval revision is not a positive whole number"); + } + return value; +} diff --git a/packages/workflow/src/cloudflare/recognition.ts b/packages/workflow/src/cloudflare/recognition.ts new file mode 100644 index 000000000..9368f9e31 --- /dev/null +++ b/packages/workflow/src/cloudflare/recognition.ts @@ -0,0 +1,214 @@ +/** + * Whether this Durable Object's storage is a version-1 workflow run, and how it + * becomes one. + * + * The conditions are the ones the Deno host distinguishes, because they are + * what a caller acts on differently: storage nobody has written yet may be + * initialized; storage belonging to something else, or claiming a version this + * build does not implement, must be left alone; and storage that claims version + * 1 and is not shaped like it is damaged. Collapsing them would leave a host + * guessing whether to create, refuse, or report damage. + * + * What differs from Deno is only where the claim is written. The pragmas that + * carry it in a file are refused here, so `_xmd_workflow_schema` carries it + * instead. Nothing initializes, migrates, repairs or replaces storage this + * module refuses. + */ + +import { initializeSchema as initializeDofsSchema } from "../../vendor/cloudflare-computer-dofs/generated/schema/index.js"; +import { + APPLICATION_ID, + declaredStructureFailure, + hasAnyDeclaredObject, + SCHEMA_SQL, + SCHEMA_VERSION, + type SchemaObject, +} from "../sqlite/workflow-schema.ts"; +import { isSchemaMarker, MARKER_SQL, MARKER_TABLE, readMarker } from "./marker.ts"; +import { + initializePrivateSchema, + PRIVATE_OBJECT_NAMES, + privateStructureFailure, +} from "./private-schema.ts"; +import type { OwnerTransactions } from "./owner-transaction.ts"; +import type { OwnerStorage } from "./storage.ts"; + +/** Why storage could not be read as a version-1 workflow run. */ +export type RecognitionFailure = + | { readonly kind: "foreign"; readonly detail: string } + | { readonly kind: "unsupported-version"; readonly schemaVersion: number } + | { readonly kind: "corrupt"; readonly detail: string }; + +export class WorkflowObjectStorageError extends Error { + override name = "WorkflowObjectStorageError"; + + constructor(readonly failure: RecognitionFailure) { + super(describeFailure(failure)); + } +} + +function describeFailure(failure: RecognitionFailure): string { + if (failure.kind === "foreign") { + return `this Durable Object's storage is not a workflow run: ${failure.detail}`; + } + if (failure.kind === "unsupported-version") { + return `this Durable Object's storage declares schema version ${failure.schemaVersion}, which this build does not implement`; + } + return `this Durable Object's storage is damaged: ${failure.detail}`; +} + +/** Every object the storage declares, drained where the cursor is created. */ +export function declaredObjects(storage: OwnerStorage): SchemaObject[] { + const rows = storage.sql + .exec("SELECT type, name, sql FROM sqlite_schema WHERE name NOT LIKE 'sqlite_%' ORDER BY name") + .toArray(); + return rows.map((row) => ({ + type: String(row["type"]), + name: String(row["name"]), + sql: row["sql"] === null || row["sql"] === undefined ? "" : String(row["sql"]), + })); +} + +/** + * Whether this storage holds nothing at all. + * + * Pristine means no object anybody created — not XMD's, not DOFS's, not the + * marker's, and nothing unrelated. Storage carrying any object but no marker is + * foreign or half-initialized, and is refused rather than written into. + */ +export function isPristine(objects: readonly SchemaObject[]): boolean { + return objects.length === 0; +} + +function markerRows(storage: OwnerStorage): Record[] { + return storage.sql.exec(`SELECT application_id, schema_version FROM ${MARKER_TABLE}`).toArray(); +} + +/** + * Make pristine storage into a version-1 workflow run, in one transaction. + * + * The marker is written last. Atomicity means no observer could see the + * ordering, so this is the code saying what the marker means: an identity claim + * over a schema that is already complete. + */ +export function initializeObject( + storage: OwnerStorage, + transactions: OwnerTransactions, + initializeRun: () => void, +): void { + const objects = declaredObjects(storage); + if (!isPristine(objects)) { + throw new WorkflowObjectStorageError({ + kind: "foreign", + detail: "it already holds objects and carries no workflow schema marker", + }); + } + transactions.run(storage, ({ dofs }) => { + storage.sql.exec(SCHEMA_SQL); + initializeDofsSchema(dofs, () => 0); + initializePrivateSchema(storage); + initializeRun(); + storage.sql.exec(MARKER_SQL); + storage.sql.exec( + `INSERT INTO ${MARKER_TABLE} (id, application_id, schema_version) VALUES (1, ?, ?)`, + APPLICATION_ID, + SCHEMA_VERSION, + ); + }); +} + +/** + * Refuse anything that is not a version-1 workflow run. + * + * Structure only. Whether the rows describe the run that was asked for is a + * separate question, asked after this one succeeds. + */ +export function recognizeObject(storage: OwnerStorage): void { + const objects = declaredObjects(storage); + if (isPristine(objects)) { + throw new WorkflowObjectStorageError({ + kind: "foreign", + detail: "it holds nothing at all", + }); + } + + const carriesMarker = objects.some((object) => object.name === MARKER_TABLE); + if (!carriesMarker) { + throw new WorkflowObjectStorageError({ + kind: "foreign", + detail: hasAnyDeclaredObject(objects) + ? "it declares workflow tables without the schema marker that identifies them" + : "it belongs to something else", + }); + } + + const marker = readMarker(markerRows(storage)); + if (!isSchemaMarker(marker)) { + if (marker.kind === "unknown-version") { + throw new WorkflowObjectStorageError({ + kind: "unsupported-version", + schemaVersion: marker.schemaVersion, + }); + } + if (marker.kind === "foreign-application") { + throw new WorkflowObjectStorageError({ + kind: "foreign", + detail: "its schema marker carries another application's identity", + }); + } + throw new WorkflowObjectStorageError({ + kind: "corrupt", + detail: + marker.kind === "absent" + ? "its schema marker table holds no identity row" + : marker.kind === "duplicated" + ? "its schema marker table holds more than one identity row" + : marker.kind === "incomplete-version" + ? "it carries the XMD application identity without a complete version-1 schema" + : "its schema marker row does not describe an identity", + }); + } + + const privateObjects = objects.filter((object) => PRIVATE_OBJECT_NAMES.includes(object.name)); + const privateFailure = privateStructureFailure(privateObjects); + if (privateFailure !== undefined) { + throw new WorkflowObjectStorageError({ + kind: "corrupt", + detail: + privateFailure.kind === "missing" + ? `it is missing the table ${privateFailure.name}` + : `its ${privateFailure.name} object is not shaped the way version ${SCHEMA_VERSION} declares it`, + }); + } + + const privateNames = new Set(PRIVATE_OBJECT_NAMES); + const declared = objects.filter( + (object) => object.name !== MARKER_TABLE && !privateNames.has(object.name), + ); + const failure = declaredStructureFailure(declared); + if (failure === undefined) { + return; + } + if (failure.kind === "incomplete-pre-release") { + throw new WorkflowObjectStorageError({ + kind: "corrupt", + detail: "it holds an incomplete pre-release of version 1", + }); + } + if (failure.kind === "undeclared-object") { + throw new WorkflowObjectStorageError({ + kind: "corrupt", + detail: `it declares an object that version ${SCHEMA_VERSION} does not`, + }); + } + if (failure.kind === "misshapen-object") { + throw new WorkflowObjectStorageError({ + kind: "corrupt", + detail: `its ${failure.name} object is not shaped the way version ${SCHEMA_VERSION} declares it`, + }); + } + throw new WorkflowObjectStorageError({ + kind: "corrupt", + detail: `it is missing the table ${failure.names.join(", ")}`, + }); +} diff --git a/packages/workflow/src/cloudflare/release.ts b/packages/workflow/src/cloudflare/release.ts new file mode 100644 index 000000000..7397eb3b6 --- /dev/null +++ b/packages/workflow/src/cloudflare/release.ts @@ -0,0 +1,62 @@ +/** + * Which build is allowed to talk to which owner. + * + * The runner client and the Durable Object owner ship as one software-factory + * release, so the messages between them are not a compatibility boundary and + * carry no version negotiation. What replaces one is this: admission compares + * an exact immutable fingerprint the deployment supplied on both sides, and a + * mismatch refuses closed — before any private message is parsed, before an + * acquisition exists, and before any run state is read. + * + * Two builds disagreeing about what was committed is the failure this exists to + * prevent rather than to survive, so there is no downgrade path and nothing + * adapts. + */ + +/** Why a build was not admitted. */ +export type ReleaseRefusal = "release-absent" | "release-malformed" | "release-mismatch"; + +export class ReleaseIdentityError extends Error { + override name = "ReleaseIdentityError"; + + constructor(readonly refusal: ReleaseRefusal) { + // The configured and presented fingerprints are deployment facts, and a + // refusal that printed them would put them in every log that saw one. + super(`this runner build is not admitted by this owner (${refusal})`); + } +} + +/** + * A fingerprint is opaque, non-empty and bounded. + * + * Bounded because it arrives from outside admission and is compared before + * anything else has looked at it; opaque because what a deployment derives it + * from — a commit, a container digest, a build id — is the deployment's + * business and never this module's. + */ +const FINGERPRINT = /^[A-Za-z0-9._:-]{1,200}$/; + +export function admitReleaseFingerprint(value: unknown): string { + if (typeof value !== "string" || value === "") { + throw new ReleaseIdentityError("release-absent"); + } + if (!FINGERPRINT.test(value)) { + throw new ReleaseIdentityError("release-malformed"); + } + return value; +} + +/** + * Compare a presented fingerprint with the configured one. + * + * Exactness rather than secrecy is the point: a fingerprint proves nothing by + * itself, and this is the one check that stops a build the owner never agreed + * to from parsing a private message. + */ +export function requireSameRelease(configured: string, presented: unknown): string { + const admitted = admitReleaseFingerprint(presented); + if (admitted !== configured) { + throw new ReleaseIdentityError("release-mismatch"); + } + return admitted; +} diff --git a/packages/workflow/src/cloudflare/routing.ts b/packages/workflow/src/cloudflare/routing.ts new file mode 100644 index 000000000..d9b59053e --- /dev/null +++ b/packages/workflow/src/cloudflare/routing.ts @@ -0,0 +1,68 @@ +/** + * Which Durable Object owns one run. + * + * The public run ID selects it arithmetically, through the namespace's own + * `idFromName`. There is no registry, no lookup table and nothing to keep in + * agreement with the objects themselves: a second authority that could disagree + * with the arithmetic is exactly what "one issue, one run, one owner" cannot + * have. + * + * The id is admitted before it is used. A malformed one must not reach + * `idFromName` at all — that call answers with an object for any string, so a + * mistyped id would silently address a fresh, empty owner rather than fail. + */ + +/** What a run ID has to be to address an owner. */ +export type RunIdRefusal = "run-id-absent" | "run-id-empty" | "run-id-has-nul" | "run-id-too-long"; + +export class RunIdError extends Error { + override name = "RunIdError"; + + constructor(readonly refusal: RunIdRefusal) { + super(`this run id cannot address a workflow owner (${refusal})`); + } +} + +/** + * The longest run ID this host routes. + * + * Public run IDs are opaque and caller-selectable, so a bound belongs here + * rather than in the derivation: the factory's own is 52 characters, and this + * leaves room for an authorized caller's without letting an unbounded string + * reach the runtime. + */ +const MAX_RUN_ID = 512; + +/** Hold a run ID to what storage requires of one, changing nothing about it. */ +export function admitRunId(value: unknown): string { + if (typeof value !== "string") { + throw new RunIdError("run-id-absent"); + } + if (value === "") { + throw new RunIdError("run-id-empty"); + } + if (value.includes("\0")) { + throw new RunIdError("run-id-has-nul"); + } + if (value.length > MAX_RUN_ID) { + throw new RunIdError("run-id-too-long"); + } + return value; +} + +/** The one namespace operation this host routes through. */ +export interface OwnerNamespace { + idFromName(name: string): { toString(): string }; + get(id: { toString(): string }): Stub; +} + +/** + * The owner for one run. + * + * Deterministic in the run ID and in nothing else: the same id reaches the same + * object from any worker, on any request, without either side having recorded + * where it went. + */ +export function ownerFor(namespace: OwnerNamespace, runId: unknown): Stub { + return namespace.get(namespace.idFromName(admitRunId(runId))); +} diff --git a/packages/workflow/src/cloudflare/storage.ts b/packages/workflow/src/cloudflare/storage.ts new file mode 100644 index 000000000..aa68916a9 --- /dev/null +++ b/packages/workflow/src/cloudflare/storage.ts @@ -0,0 +1,53 @@ +/** + * A Durable Object's storage, as the vendored DOFS layer expects to see it. + * + * The vendor describes storage structurally — `sql.exec()` answering a cursor + * whose rows are a caller-chosen `object` subtype — while the runtime types the + * same call concretely as `Record`. The two are + * compatible in fact and not in the type system, so this is the one place the + * shapes are reconciled, rather than every call site asserting it. + * + * Nothing is converted: the cursor is drained with `toArray()` exactly where + * the caller asks for it, because Cloudflare's SQL cursor does not survive an + * `await` and draining it late would read a different result than the query + * asked for. + */ + +import type { + DurableObjectStorageLike, + SQLCursorLike, + SQLStorageLike, +} from "../../vendor/cloudflare-computer-dofs/generated/types.d.ts"; + +/** The subset of the runtime's storage this adapter uses. */ +export interface OwnerStorage { + readonly sql: { + exec(query: string, ...bindings: unknown[]): { toArray(): Record[] }; + }; + transactionSync(closure: () => T): T; +} + +/** + * Present one Durable Object's storage as the vendored DOFS storage shape. + * + * `transactionSync` is deliberately *not* forwarded here. The owner opens + * exactly one real transaction of its own and enlists DOFS inside it; a wrapper + * that forwarded this method would let a nested call reach the runtime, which + * refuses transaction statements from `sql.exec()`. + */ +export function dofsStorage(storage: OwnerStorage): DurableObjectStorageLike { + const sql: SQLStorageLike = { + exec>( + query: string, + ...bindings: unknown[] + ): SQLCursorLike { + const rows = storage.sql.exec(query, ...bindings).toArray(); + return { + toArray(): Row[] { + return rows as Row[]; + }, + }; + }, + }; + return { sql }; +} diff --git a/packages/workflow/src/cloudflare/token.ts b/packages/workflow/src/cloudflare/token.ts new file mode 100644 index 000000000..1139ebf66 --- /dev/null +++ b/packages/workflow/src/cloudflare/token.ts @@ -0,0 +1,247 @@ +/** + * Verifying the token a runner presents. + * + * This is the authority boundary, so it takes bytes rather than a claim set. A + * caller that could hand over decoded claims would be a caller that could + * assert whatever the policy asks for, and no amount of equality checking after + * that point would mean anything — which is exactly the hole this module + * closes. + * + * What it does is ordinary compact-JWS verification, narrowed hard: one + * algorithm family, keys the deployment configured, and temporal validity + * checked before any payload member is read as a claim. Everything about the + * token stops here. The raw JWT, the key material, the header, the claims the + * policy does not name and the reason a signature failed are all provider + * state: none of it is retained, attached, journaled, logged, or returned. + */ + +import { type Operation, until } from "effection"; + +/** Why a token was not accepted. */ +export type TokenRefusal = + | "token-absent" + | "token-malformed" + | "token-too-large" + | "unsupported-algorithm" + | "unsupported-type" + | "unknown-key" + | "bad-signature" + | "malformed-claims" + | "expired" + | "not-yet-valid" + | "misconfigured-clock"; + +export class TokenError extends Error { + override name = "TokenError"; + + constructor(readonly refusal: TokenRefusal) { + super(`this runner's token was not accepted (${refusal})`); + } +} + +/** + * The one signature family this accepts. + * + * GitHub Actions signs with RS256. An allowlist rather than a lookup, because + * reading the algorithm out of the header and trusting it is how a token comes + * to be "verified" with `none` or with a symmetric key an attacker chose. + */ +const SUPPORTED = "RS256"; + +/** + * The longest token this reads at all, and the longest segment inside one. + * + * Bounded before anything is decoded, because decoding is the first work an + * unauthenticated caller can make this owner do. + */ +const MAX_TOKEN = 16 * 1024; +const MAX_SEGMENT = 8 * 1024; + +/** The most skew a deployment may configure. */ +const MAX_SKEW_SECONDS = 300; + +/** A NumericDate: a finite integer count of seconds. */ +function numericDate(value: unknown): number { + if (typeof value !== "number" || !Number.isFinite(value) || !Number.isInteger(value)) { + throw new TokenError("malformed-claims"); + } + return value; +} + +/** What a deployment configures before any token can be verified. */ +export interface TokenVerification { + /** + * The issuer's public keys. Fetched and rotated by the host. + * + * `kid` is carried beside the key rather than read off it: the runtime's + * `JsonWebKey` does not declare one, and a key set that narrows by id is what + * a JWKS is for. + */ + readonly keys: readonly VerificationKey[]; + /** + * How much clock skew to tolerate, in seconds. + * + * Adapter policy, not a user setting and never a request field. Bounded above + * because a large tolerance is indistinguishable from not checking, and below + * because a negative one would reject tokens for being on time. + */ + readonly skewSeconds: number; + /** Now, in seconds since the epoch. Injected so a test can be exact. */ + readonly now: () => number; +} + +/** One configured public key, and the id a token may name it by. */ +export interface VerificationKey { + readonly kid?: string; + readonly jwk: JsonWebKey; +} + +function decodeSegment(segment: string): unknown { + // base64url, without the padding a compact JWS omits. + const padded = segment.replaceAll("-", "+").replaceAll("_", "/"); + const filled = padded + "=".repeat((4 - (padded.length % 4)) % 4); + let text: string; + try { + const bytes = Uint8Array.from(atob(filled), (character) => character.charCodeAt(0)); + text = new TextDecoder().decode(bytes); + } catch { + throw new TokenError("token-malformed"); + } + try { + return JSON.parse(text); + } catch { + throw new TokenError("token-malformed"); + } +} + +function object(value: unknown): Map { + if (value === null || typeof value !== "object" || Array.isArray(value)) { + throw new TokenError("token-malformed"); + } + const members: Map = new Map(Object.entries(value)); + return members; +} + +function signatureBytes(segment: string): Uint8Array { + const padded = segment.replaceAll("-", "+").replaceAll("_", "/"); + const filled = padded + "=".repeat((4 - (padded.length % 4)) % 4); + try { + return Uint8Array.from(atob(filled), (character) => character.charCodeAt(0)); + } catch { + throw new TokenError("token-malformed"); + } +} + +/** + * Verify one compact JWS and answer with its payload. + * + * The order is the contract: shape, then algorithm, then signature, then time. + * A payload member is not a claim until every one of those has passed, which is + * why nothing here returns early with something a caller could mistake for one. + */ +export function* verifyToken( + configured: TokenVerification, + token: unknown, +): Operation> { + const skew = configured.skewSeconds; + if (!Number.isFinite(skew) || skew < 0 || skew > MAX_SKEW_SECONDS) { + throw new TokenError("misconfigured-clock"); + } + if (typeof token !== "string" || token === "") { + throw new TokenError("token-absent"); + } + if (token.length > MAX_TOKEN) { + throw new TokenError("token-too-large"); + } + const parts = token.split("."); + if (parts.length !== 3) { + throw new TokenError("token-malformed"); + } + if (parts.some((part) => part.length === 0 || part.length > MAX_SEGMENT)) { + throw new TokenError("token-malformed"); + } + const [encodedHeader, encodedPayload, encodedSignature] = parts; + if ( + encodedHeader === undefined || + encodedPayload === undefined || + encodedSignature === undefined + ) { + throw new TokenError("token-malformed"); + } + + const header = object(decodeSegment(encodedHeader)); + if (header.get("alg") !== SUPPORTED) { + throw new TokenError("unsupported-algorithm"); + } + // GitHub's Actions tokens carry `typ: "JWT"`. Requiring it is cheap and stops + // a token minted for another purpose from being read as one of these. + const type = header.get("typ"); + if (typeof type !== "string" || type.toUpperCase() !== "JWT") { + throw new TokenError("unsupported-type"); + } + + const signed = new TextEncoder().encode(`${encodedHeader}.${encodedPayload}`); + const signature = signatureBytes(encodedSignature); + // The token names exactly one configured key. Falling back to an unkeyed + // candidate when the id matched nothing would mean an unrecognized key id + // still got a signature check against whatever else was configured. + const keyId = header.get("kid"); + if (typeof keyId !== "string" || keyId === "") { + throw new TokenError("unknown-key"); + } + const candidates = configured.keys.filter((key) => key.kid === keyId); + if (candidates.length !== 1) { + // None means the id is unrecognized; more than one means the configuration + // cannot say which key that id is. + throw new TokenError("unknown-key"); + } + + let verified = false; + for (const candidate of candidates) { + const key = yield* until( + crypto.subtle.importKey( + "jwk", + candidate.jwk, + { name: "RSASSA-PKCS1-v1_5", hash: "SHA-256" }, + false, + ["verify"], + ), + ); + const matched = yield* until(crypto.subtle.verify("RSASSA-PKCS1-v1_5", key, signature, signed)); + if (matched) { + verified = true; + break; + } + } + if (!verified) { + throw new TokenError("bad-signature"); + } + + const payload = object(decodeSegment(encodedPayload)); + const now = configured.now(); + if (!Number.isFinite(now)) { + throw new TokenError("misconfigured-clock"); + } + + // All three are required. Checking a temporal claim only when it happens to + // be a number means a token that omits it is treated as one that satisfies + // it, which is the opposite of what the claim is for. + const expiry = numericDate(payload.get("exp")); + const issued = numericDate(payload.get("iat")); + const notBefore = numericDate(payload.get("nbf")); + + // RFC 7519 §4.1.4: the current time must be *before* the expiration, so the + // boundary itself is expired rather than the last valid instant. + if (now >= expiry + skew) { + throw new TokenError("expired"); + } + if (now + skew < notBefore) { + throw new TokenError("not-yet-valid"); + } + if (now + skew < issued) { + // Issued in the future by more than the tolerance: the token and this clock + // disagree about when now is, and nothing here can tell which is wrong. + throw new TokenError("not-yet-valid"); + } + return payload; +} diff --git a/packages/workflow/src/composition/locator.ts b/packages/workflow/src/composition/locator.ts new file mode 100644 index 000000000..af989e6c1 --- /dev/null +++ b/packages/workflow/src/composition/locator.ts @@ -0,0 +1,111 @@ +/** + * Admitting a Git locator, and naming one without publishing it. + * + * Two different questions. **Admission** decides whether a locator may be handed + * to Git at all. **Fingerprinting** produces the stable name the journal, the + * record and every compatibility comparison use, so a changed locator diverges + * without the bytes of either one being retained outside the single column that + * holds them. + * + * Admission is a closed allowlist rather than a search for bad shapes. Git's + * locator grammar reaches well past URLs — `ext::sh -c …` runs a command, a + * leading `-` is read as an option, and a transport helper is whatever is on + * `PATH` — so anything not recognized as one of the admitted forms is refused. + * Credentials in the string are refused rather than stripped: a locator that + * carries one is a secret a caller put in a durable input, and quietly editing + * it would retain a run nobody asked for. + * + * Both rules are shared because both hosts need them and neither may be more + * permissive than the other. The local host refuses a locator before Git sees + * it; the remote owner must refuse the same one before it becomes durable + * state, or an authenticated proposal could retain something the local host + * would never have produced. A second copy of an allowlist is the copy that + * ends up longer. + * + * Nothing here reaches a runtime. `URL` is the platform's, and the digest is + * the shared one. + */ + +import { sha256Hex } from "../workspace/sha256.ts"; + +/** Schemes this provider hands to Git. Everything else is refused. */ +const SCHEMES = new Set(["https", "http", "ssh", "git", "file"]); + +/** `user@host:path`, Git's scp-like form. A colon in the userinfo is a password. */ +const SCP_LIKE = /^([^/@:]+)@([^/@:]+):(.+)$/; + +function hasControlCharacters(value: string): boolean { + for (const character of value) { + const code = character.codePointAt(0) ?? 0; + if (code < 0x20 || code === 0x7f) { + return true; + } + } + return false; +} + +function admitUrl(locator: string): string | undefined { + let url: URL; + try { + url = new URL(locator); + } catch { + return undefined; + } + const scheme = url.protocol.replace(/:$/, ""); + if (!SCHEMES.has(scheme)) { + return undefined; + } + if (url.username !== "" || url.password !== "") { + return undefined; + } + // A query or a fragment is refused whole rather than searched for credentials. + // `?access_token=…` is the ordinary way a token is written into a URL, and a + // rule that named the parameters worth refusing would be a list of the ones + // somebody thought of — the same open-ended guessing this module rejects + // everywhere else. Git is given a repository's location, and neither part + // carries any of that location for the transports admitted here. + if (url.search !== "" || url.hash !== "") { + return undefined; + } + return locator; +} + +/** + * The locator this string is, or `undefined` when this provider will not use it. + * + * The answer is the original bytes, never a rewritten form: what is admitted is + * what Git is given and what the fingerprint names, so the three cannot drift. + */ +export function admitLocator(locator: string): string | undefined { + if (locator === "" || hasControlCharacters(locator) || /\s/.test(locator)) { + return undefined; + } + if (locator.startsWith("-")) { + return undefined; + } + if (/^[A-Za-z][A-Za-z0-9+.-]*:\/\//.test(locator)) { + return admitUrl(locator); + } + const scpLike = SCP_LIKE.exec(locator); + if (scpLike !== null) { + return locator; + } + // A local path. Absolute only: a relative one would name a different + // repository depending on which directory the host happened to run in, and a + // workflow's retained identity must not depend on that. + if (locator.startsWith("/")) { + return locator; + } + return undefined; +} + +/** + * The stable name an admitted locator is known by everywhere but its own column. + * + * Shared because both hosts retain it and both must derive it identically: a + * fingerprint is what a journal event carries in place of the locator, and two + * derivations would be two names for one repository. + */ +export function locatorFingerprintOf(locator: string): string { + return sha256Hex(locator); +} diff --git a/packages/workflow/src/deno/artifact-frontier.ts b/packages/workflow/src/deno/artifact-frontier.ts index c8957363d..e0f3aa5d8 100644 --- a/packages/workflow/src/deno/artifact-frontier.ts +++ b/packages/workflow/src/deno/artifact-frontier.ts @@ -44,7 +44,7 @@ import type { RetainedBlob, RetainedManifest } from "./fork-source.ts"; import { readForkLineage } from "./fork-write.ts"; import { readRepositories, readRetainedRows, readWorktrees } from "./fork-source.ts"; import { reading } from "./reading.ts"; -import { readDocumentExecution, readRetrieval } from "./rows.ts"; +import { readDocumentExecution, readRetrieval } from "../sqlite/rows.ts"; import { readAllAgentSessions } from "./workspace/agent-sessions.ts"; import { bytes, integer } from "./workspace/manifest.ts"; import { diff --git a/packages/workflow/src/deno/artifact/records.ts b/packages/workflow/src/deno/artifact/records.ts index 08cf5b18c..4752e1620 100644 --- a/packages/workflow/src/deno/artifact/records.ts +++ b/packages/workflow/src/deno/artifact/records.ts @@ -86,8 +86,8 @@ import { workspaceRoot, type WorkspaceRootManifest, } from "../workspace/manifest.ts"; -import { decodeDofsManifest } from "../workspace/root.ts"; -import type { DofsManifest } from "../workspace/root.ts"; +import { decodeContentManifest } from "../workspace/root.ts"; +import type { ContentManifest } from "../workspace/root.ts"; import { gitBlobIdentity } from "./source.ts"; import { canonicalJsonBytes, canonicalJsonText, entryKey } from "./manifest.ts"; import type { @@ -1335,7 +1335,7 @@ function verifyLifecycle( function verifyContentStore( contents: XmdArtifactContents, reject: Reject, -): ReadonlyMap { +): ReadonlyMap { const blobs = new Map(); for (const blob of contents.blobs) { const hash = toHex(blob.hash); @@ -1348,7 +1348,7 @@ function verifyContentStore( blobs.set(hash, blob.size); } - const manifests = new Map(); + const manifests = new Map(); for (const manifest of contents.manifests) { const hash = toHex(manifest.hash); if (manifests.has(hash)) { @@ -1357,7 +1357,7 @@ function verifyContentStore( if (toHex(sha256(manifest.encoded)) !== hash) { reject("a DOFS manifest's identity does not match its bytes"); } - const decoded = decodeDofsManifest(manifest.encoded, reject); + const decoded = decodeContentManifest(manifest.encoded, reject); if (decoded.size !== manifest.size) { reject("a DOFS manifest's declared size does not equal its chunks"); } @@ -1382,7 +1382,7 @@ function verifyContentStore( */ function verifyRoots( contents: XmdArtifactContents, - manifests: ReadonlyMap, + manifests: ReadonlyMap, path: string, reject: Reject, ): void { @@ -1395,7 +1395,7 @@ function verifyRoots( reject("a Workspace root identity does not match its manifest bytes"); } - const declared = new Map(); + const declared = new Map(); for (const entry of parsed.entries) { if (entry.kind !== "file") { continue; diff --git a/packages/workflow/src/deno/composition/locator.ts b/packages/workflow/src/deno/composition/locator.ts index 7b9198d60..5d0f1dbe9 100644 --- a/packages/workflow/src/deno/composition/locator.ts +++ b/packages/workflow/src/deno/composition/locator.ts @@ -18,80 +18,7 @@ * quietly editing it would retain a run nobody asked for. */ -import { createHash } from "node:crypto"; - -/** Schemes this provider hands to Git. Everything else is refused. */ -const SCHEMES = new Set(["https", "http", "ssh", "git", "file"]); - -/** `user@host:path`, Git's scp-like form. A colon in the userinfo is a password. */ -const SCP_LIKE = /^([^/@:]+)@([^/@:]+):(.+)$/; - -function hasControlCharacters(value: string): boolean { - for (const character of value) { - const code = character.codePointAt(0) ?? 0; - if (code < 0x20 || code === 0x7f) { - return true; - } - } - return false; -} - -function admitUrl(locator: string): string | undefined { - let url: URL; - try { - url = new URL(locator); - } catch { - return undefined; - } - const scheme = url.protocol.replace(/:$/, ""); - if (!SCHEMES.has(scheme)) { - return undefined; - } - if (url.username !== "" || url.password !== "") { - return undefined; - } - // A query or a fragment is refused whole rather than searched for credentials. - // `?access_token=…` is the ordinary way a token is written into a URL, and a - // rule that named the parameters worth refusing would be a list of the ones - // somebody thought of — the same open-ended guessing this module rejects - // everywhere else. Git is given a repository's location, and neither part - // carries any of that location for the transports admitted here. - if (url.search !== "" || url.hash !== "") { - return undefined; - } - return locator; -} - -/** - * The locator this string is, or `undefined` when this provider will not use it. - * - * The answer is the original bytes, never a rewritten form: what is admitted is - * what Git is given and what the fingerprint names, so the three cannot drift. - */ -export function admitLocator(locator: string): string | undefined { - if (locator === "" || hasControlCharacters(locator) || /\s/.test(locator)) { - return undefined; - } - if (locator.startsWith("-")) { - return undefined; - } - if (/^[A-Za-z][A-Za-z0-9+.-]*:\/\//.test(locator)) { - return admitUrl(locator); - } - const scpLike = SCP_LIKE.exec(locator); - if (scpLike !== null) { - return locator; - } - // A local path. Absolute only: a relative one would name a different - // repository depending on which directory the host happened to run in, and a - // workflow's retained identity must not depend on that. - if (locator.startsWith("/")) { - return locator; - } - return undefined; -} - -/** The stable name an admitted locator is known by everywhere but its own column. */ -export function locatorFingerprint(locator: string): string { - return createHash("sha256").update(locator, "utf8").digest("hex"); -} +export { + admitLocator, + locatorFingerprintOf as locatorFingerprint, +} from "../../composition/locator.ts"; diff --git a/packages/workflow/src/deno/database.ts b/packages/workflow/src/deno/database.ts index 2b439d65f..00e262223 100644 --- a/packages/workflow/src/deno/database.ts +++ b/packages/workflow/src/deno/database.ts @@ -61,7 +61,7 @@ import { holdsTransactionOn, useTransactionSavepoints, } from "./transaction.ts"; -import { readDocumentExecution, readRetrieval, readRunRecord } from "./rows.ts"; +import { readDocumentExecution, readRetrieval, readRunRecord } from "../sqlite/rows.ts"; import { reading } from "./reading.ts"; import { translateSqliteError } from "./schema.ts"; diff --git a/packages/workflow/src/deno/lifecycle.ts b/packages/workflow/src/deno/lifecycle.ts index 22eced242..a43c7d98f 100644 --- a/packages/workflow/src/deno/lifecycle.ts +++ b/packages/workflow/src/deno/lifecycle.ts @@ -104,7 +104,7 @@ import { readRetrievalMetadata, } from "./artifact-frontier.ts"; import type { WorkflowExportRequest, WorkflowExportResult } from "../lifecycle/export.ts"; -import { readDocumentExecution, readRetrieval, readRunRecord } from "./rows.ts"; +import { readDocumentExecution, readRetrieval, readRunRecord } from "../sqlite/rows.ts"; import { translateSqliteError, verifySchema, WorkflowReadonlyRollbackError } from "./schema.ts"; import { holdRecoveryCoordination } from "./recovery-coordination.ts"; diff --git a/packages/workflow/src/deno/remote-files.ts b/packages/workflow/src/deno/remote-files.ts new file mode 100644 index 000000000..1e807706b --- /dev/null +++ b/packages/workflow/src/deno/remote-files.ts @@ -0,0 +1,209 @@ +/** + * The runner's own filesystem, as materialization needs to see it. + * + * `@effectionx/fs` covers the ordinary work but not the whole Workspace + * contract: a retained root carries symbolic links, hardlink groups, modes and + * modification times, and preserving those is what makes an untouched + * materialization capture back to the root it came from. The operations it + * lacks are adapted here from the runtime's own asynchronous primitives with + * `until`, which is the sanctioned way to reach one — not by making production + * code asynchronous and not by reaching for a synchronous call. + * + * `node:fs/promises` rather than a runtime global, because the same adapter has + * to work wherever the runner runs. Nothing above this module names a runtime, + * and nothing in this module decides anything about a Workspace: it moves bytes + * and metadata where it is told, and the rules live in shared code. + */ + +import { + chmod, + link, + lstat, + lchmod, + lutimes, + mkdir, + readdir, + readFile, + mkdtemp, + readlink, + rm, + symlink, + utimes, + writeFile, +} from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { ensure, type Operation, resource, until } from "effection"; +import type { RunnerFiles, RunnerNode } from "../remote/materialize.ts"; +import type { TemporaryTrees } from "../remote/invocation.ts"; + +/** + * Whole milliseconds, which is the unit a retained entry records. + * + * Not seconds. The Workspace format carries whatever the retaining host's clock + * produced, and that clock is `Date.now`, so an adapter that reported seconds + * would describe every retained tree as a different one — and setting a + * millisecond value as though it were seconds would put the file tens of + * thousands of years from now, where the filesystem cannot keep it. + */ +function milliseconds(value: number): number { + return Math.round(value); +} + +function describeStats( + name: string, + stats: { + isDirectory(): boolean; + isSymbolicLink(): boolean; + mode: number; + mtimeMs: number; + size: number; + ino: number | bigint; + nlink: number | bigint; + }, + target: string | undefined, +): RunnerNode { + const kind = stats.isSymbolicLink() ? "symlink" : stats.isDirectory() ? "directory" : "file"; + return { + name, + kind, + // The permission bits only. The type bits are what `kind` already said, and + // a retained mode that carried them would not round-trip through the + // format's own bound. + mode: stats.mode & 0o7777, + mtime: milliseconds(stats.mtimeMs), + size: kind === "file" ? stats.size : 0, + // Only a file reached by more than one name can be part of a group, so + // anything else reports no identity and is captured on its own. + identity: kind === "file" && Number(stats.nlink) > 1 ? String(stats.ino) : undefined, + target, + }; +} + +/** + * `lchmod`, when the platform actually has it. + * + * BSD-derived systems do; Linux does not, and Node exposes the export + * regardless on some releases. Probing the export is the only honest test + * available before a real call. + */ +function lchmodOf(): ((path: string, mode: number) => Operation) | undefined { + if (typeof lchmod !== "function") { + return undefined; + } + return function* (path: string, mode: number): Operation { + yield* until(lchmod(path, mode)); + }; +} + +/** The runner's filesystem operations, for one materialized tree. */ +export function runnerFiles(): RunnerFiles { + return { + *makeDirectory(path: string, mode: number): Operation { + yield* until(mkdir(path, { recursive: false, mode })); + }, + + *writeFile(path: string, bytes: Uint8Array, mode: number): Operation { + yield* until(writeFile(path, bytes, { mode })); + }, + + *makeSymlink(target: string, path: string): Operation { + yield* until(symlink(target, path)); + }, + + *makeHardlink(existing: string, path: string): Operation { + yield* until(link(existing, path)); + }, + + *setMode(path: string, mode: number): Operation { + // Explicit rather than relying on the creation mode, which the process + // umask narrows. A retained mode is durable identity. + yield* until(chmod(path, mode)); + }, + + *setModifiedAt(path: string, mtime: number): Operation { + // `utimes` speaks seconds; the format speaks milliseconds. + yield* until(utimes(path, mtime / 1000, mtime / 1000)); + }, + + /** + * A link's own time, set without following it. + * + * `lutimes` is what makes this possible at all: `utimes` would follow the + * link and rewrite whatever it points at, which may be outside the tree + * entirely. + */ + *setLinkModifiedAt(path: string, mtime: number): Operation { + yield* until(lutimes(path, mtime / 1000, mtime / 1000)); + }, + + /** + * A link's own permissions, where the platform has them. + * + * Linux ignores symbolic-link permission bits and offers no `lchmod`, so + * this is deliberately absent there rather than faked. Materialization + * checks what it actually got and refuses a root this host cannot + * represent, which is the honest outcome; quietly writing a different mode + * would change durable identity. + */ + setLinkMode: lchmodOf(), + + *readFile(path: string): Operation { + return new Uint8Array(yield* until(readFile(path))); + }, + + *list(path: string): Operation { + const names = yield* until(readdir(path)); + const found: RunnerNode[] = []; + for (const name of names) { + const entry = join(path, name); + const stats = yield* until(lstat(entry)); + // Read, never resolved: what a retained link points at is part of the + // Workspace's description of itself, not somewhere to go looking. + const target: string | undefined = stats.isSymbolicLink() + ? yield* until(readlink(entry)) + : undefined; + found.push(describeStats(name, stats, target)); + } + return found; + }, + + *describe(path: string): Operation { + const stats = yield* until(lstat(path)); + return describeStats("", stats, undefined); + }, + }; +} + +/** + * Temporary trees for one invocation, owned by the scope that asked for them. + * + * Every tree this hands out is removed when that scope ends, however it ends. + * A run that left one behind would leave a materialized Workspace on a machine + * that has stopped being responsible for it. + */ +export function useRunnerTrees(): Operation { + return resource(function* (provide) { + const roots: string[] = []; + yield* ensure(function* () { + // In reverse, so a nested tree goes before whatever contains it. + for (const root of roots.toReversed()) { + yield* until(rm(root, { recursive: true, force: true })); + } + }); + yield* provide({ + *create(purpose: string): Operation { + const root = yield* until(mkdtemp(join(tmpdir(), `xmd-workflow-${purpose}-`))); + roots.push(root); + return root; + }, + *remove(path: string): Operation { + yield* until(rm(path, { recursive: true, force: true })); + const found = roots.indexOf(path); + if (found >= 0) { + roots.splice(found, 1); + } + }, + }); + }); +} diff --git a/packages/workflow/src/deno/remote-workspace-files.ts b/packages/workflow/src/deno/remote-workspace-files.ts new file mode 100644 index 000000000..af9a727b5 --- /dev/null +++ b/packages/workflow/src/deno/remote-workspace-files.ts @@ -0,0 +1,341 @@ +/** + * The Workspace filesystem, over the attempt directory this invocation owns. + * + * The runner's Workspace is a real directory it materialized from the owner, so + * the operations are the runtime's own asynchronous primitives adapted with + * `until`. Nothing above this module names a runtime, and nothing in it decides + * anything about a Workspace: it moves bytes where it is told, and refuses to + * be told anywhere outside the attempt. + * + * ## Why lexical admission is not containment here + * + * The Deno host's Workspace is rows in a database, so a path there has no + * outside to reach and admission is arithmetic. This one is a real directory on + * a host that has an outside, and a symbolic link is a path the kernel follows + * on its own. Comparing the *spelling* of a path with the attempt root admits + * `/link` while the syscall that follows reads whatever `/link` points at. + * + * So every operation resolves before it acts, on the same terms + * `packages/runtime/host-files.ts` states for the host provider: a complete + * `..` segment leaves and `..notes.md` does not; the existing prefix is walked + * so a path that does not exist yet can still be judged by its deepest + * existing ancestor; an operation that acts on a link does not follow it, and + * one whose contract follows a link follows only where that link lands inside + * the attempt. + * + * ## A link's target is a Workspace path, not a host path + * + * A retained symbolic link carries its target as text, and that text is + * interpreted in the Workspace it belongs to. An absolute target names the + * logical Workspace root — the root of the tree this invocation owns — not the + * runner host's root. Letting the kernel interpret `/etc/passwd` would turn a + * retained Workspace entry into authority over the machine, so resolution is + * done here, one segment at a time, and the host is asked only about paths that + * are already known to be inside. + * + * The stable-host-namespace limitation the host provider documents applies here + * too: another process can replace a directory between the moment this resolves + * a path and the moment it uses one. That window is not what this closes. + */ + +import { + chmod, + link, + lstat, + mkdir, + readdir, + readFile, + readlink, + rename, + rm, + stat, + symlink, + writeFile, +} from "node:fs/promises"; +import type { Stats } from "node:fs"; +import { type Operation, until } from "effection"; +import type { + WorkspaceEntry, + WorkspaceFilesystem, + WorkspaceStat, +} from "../workspace/filesystem.ts"; +import { throwWorkspaceFilesystemFailure } from "./workspace/errors.ts"; +import type { HostPath } from "../remote/materialize.ts"; + +/** A path no Workspace operation may reach, whatever it names. */ +export class WorkspacePathError extends Error { + override name = "WorkspacePathError"; + + constructor() { + // No path, no target and no host directory: what a document may learn is + // that it asked for somewhere it does not own. + super("this Workspace path is outside the tree this invocation owns."); + } +} + +/** How many links one resolution will follow before calling it a loop. */ +const MAX_LINKS = 32; + +/** + * The logical segments this path names, or `undefined` if it leaves the root. + * + * Pure arithmetic on POSIX segments, decided before anything touches the host. + * `.` and an empty segment are nothing; a complete `..` pops, and popping past + * the root is the escape. A segment that merely begins with two dots is an + * ordinary name and stays. + */ +function segmentsOf(base: readonly string[], path: string): string[] | undefined { + const segments = path.startsWith("/") ? [] : [...base]; + for (const segment of path.split("/")) { + if (segment === "" || segment === ".") { + continue; + } + if (segment === "..") { + if (segments.length === 0) { + return undefined; + } + segments.pop(); + continue; + } + segments.push(segment); + } + return segments; +} + +/** What one operation may act on: where it is, and whether a link was left alone. */ +interface Resolved { + readonly segments: readonly string[]; +} + +/** + * Walk the path, following the links inside it, and refuse the ones that leave. + * + * `followFinal` is the difference between an operation about a file and an + * operation about a link. A read follows the last link to the file it names, + * because replacing or reporting the link would surprise a caller that asked + * for the file; `lstat`, `readlink`, a removal and a rename act on the entry + * the caller named, so the last segment is left exactly as written. + * + * A path that does not exist is not an error: the walk stops at the deepest + * existing ancestor and keeps the rest, which is what lets a write name a file + * it is about to create and still be judged. + */ +function* resolve(root: string, path: string, followFinal: boolean): Operation { + if (path === "" || path.includes("\u0000")) { + throw new WorkspacePathError(); + } + const admitted = segmentsOf([], path); + if (admitted === undefined) { + throw new WorkspacePathError(); + } + let segments: string[] = admitted; + + for (let followed = 0; ; followed += 1) { + if (followed > MAX_LINKS) { + throw new WorkspacePathError(); + } + const crossing = yield* firstLink(root, segments, followFinal); + if (crossing === undefined) { + return { segments }; + } + // The target is read in the Workspace this link belongs to: absolute means + // the Workspace root, and relative means beside the link. + const next = segmentsOf(segments.slice(0, crossing.depth - 1), crossing.target); + if (next === undefined) { + throw new WorkspacePathError(); + } + segments = [...next, ...segments.slice(crossing.depth)]; + } +} + +/** + * The shallowest segment of this path that is a symbolic link, if any. + * + * Shallowest rather than any, because substituting a link's target changes + * every segment beneath it — resolving a deeper one first would resolve it + * against a prefix that is about to be replaced. + */ +function* firstLink( + root: string, + segments: readonly string[], + followFinal: boolean, +): Operation<{ depth: number; target: string } | undefined> { + const last = followFinal ? segments.length : segments.length - 1; + for (let depth = 1; depth <= last; depth += 1) { + const host = hostPath(root, segments.slice(0, depth)); + const entry: Stats | undefined = yield* describing(host); + if (entry === undefined) { + // Nothing here, so nothing below it exists either. What the caller named + // is judged by the ancestor that does exist, which this walk has passed. + return undefined; + } + if (entry.isSymbolicLink()) { + return { depth, target: yield* until(readlink(host)) }; + } + } + return undefined; +} + +/** What this entry is, or nothing when there is no entry here. */ +function* describing(host: string): Operation { + try { + return yield* until(lstat(host)); + } catch { + return undefined; + } +} + +function hostPath(root: string, segments: readonly string[]): string { + return segments.length === 0 ? root : `${root}/${segments.join("/")}`; +} + +function described(value: { + mode: number; + mtimeMs: number; + size: number; + isFile(): boolean; + isDirectory(): boolean; +}): WorkspaceStat { + const kind = value.isFile() ? "file" : value.isDirectory() ? "directory" : "symlink"; + // The retained mode is the permission bits; the type bits belong to the + // node's kind, which is reported beside it. + return { kind, mode: value.mode & 0o7777, mtime: Math.trunc(value.mtimeMs), size: value.size }; +} + +/** + * A runtime failure, named the way the shared classifier reads one. + * + * The classifier asks for a `WorkspaceFsError` carrying a documented code, + * because that is what the other host raises. Renaming here rather than + * widening the classifier keeps one list of documented conditions. The + * message and the host path inside it are dropped: what reaches a document is + * the condition, never where this invocation happened to put its tree. + */ +function named(error: unknown): unknown { + const code = error instanceof Error ? Reflect.get(error, "code") : undefined; + if (error instanceof Error && typeof code === "string") { + const renamed = new Error(`the Workspace operation failed (${code})`); + renamed.name = "WorkspaceFsError"; + Reflect.set(renamed, "code", code); + return renamed; + } + return error; +} + +export function createRemoteWorkspaceFilesystem( + at: HostPath, + authorize: () => void, +): WorkspaceFilesystem { + // The attempt's own root, taken from the same resolver every other path goes + // through. Every host path this module builds is this root plus segments it + // has already admitted, so no authored text reaches a syscall unexamined. + const root = at("/"); + + function* run( + path: string, + followFinal: boolean, + body: (host: string) => Promise, + ): Operation { + authorize(); + // Resolved immediately before the operation it authorizes, never cached: a + // path admitted once is not a capability to use later. + const resolved = yield* resolve(root, path, followFinal); + try { + return yield* until(body(hostPath(root, resolved.segments))); + } catch (error) { + // The same classification the Deno host applies: a documented filesystem + // condition is the effect's own outcome, and everything else is the run + // failing. + return throwWorkspaceFilesystemFailure(named(error)); + } + } + + /** Both ends of a two-path operation, each admitted at the time of use. */ + function* pair( + from: string, + to: string, + followFrom: boolean, + body: (source: string, destination: string) => Promise, + ): Operation { + authorize(); + const source = yield* resolve(root, from, followFrom); + const destination = yield* resolve(root, to, false); + try { + return yield* until( + body(hostPath(root, source.segments), hostPath(root, destination.segments)), + ); + } catch (error) { + return throwWorkspaceFilesystemFailure(named(error)); + } + } + + return { + *readFile(path): Operation { + return yield* run(path, true, (host) => readFile(host)); + }, + + *readTextFile(path): Operation { + const bytes = yield* run(path, true, (host) => readFile(host)); + return new TextDecoder().decode(bytes); + }, + + *stat(path): Operation { + return described(yield* run(path, true, (host) => stat(host))); + }, + + *lstat(path): Operation { + // About the entry, so the last segment stays what it is. + return described(yield* run(path, false, (host) => lstat(host))); + }, + + *readlink(path): Operation { + // The retained target, exactly as it was written. It is a Workspace path, + // and reading it back is not resolving it. + return yield* run(path, false, (host) => readlink(host)); + }, + + *readdir(path): Operation { + const entries = yield* run(path, true, (host) => readdir(host, { withFileTypes: true })); + return entries.map((entry) => ({ + name: entry.name, + kind: entry.isFile() ? "file" : entry.isDirectory() ? "directory" : "symlink", + })); + }, + + *writeFile(path, content, mode): Operation { + const bytes = typeof content === "string" ? new TextEncoder().encode(content) : content; + // Follows an internal link to the file it names: replacing the link would + // be the surprising outcome, and an outward one never got this far. + yield* run(path, true, (host) => writeFile(host, bytes, mode === undefined ? {} : { mode })); + }, + + *mkdir(path, options = {}): Operation { + yield* run(path, true, (host) => mkdir(host, options).then(() => undefined)); + }, + + *remove(path, options = {}): Operation { + // A removal takes the entry the caller named. Following a final link + // would remove something never mentioned. + yield* run(path, false, (host) => rm(host, options)); + }, + + *rename(from, to): Operation { + yield* pair(from, to, false, (source, destination) => rename(source, destination)); + }, + + *chmod(path, mode): Operation { + yield* run(path, true, (host) => chmod(host, mode)); + }, + + *symlink(target, path): Operation { + // The target is not resolved: a link's target is retained text, and it is + // interpreted when the link is walked. What is admitted here is where the + // link itself is created. + yield* run(path, false, (host) => symlink(target, host)); + }, + + *link(existingPath, newPath): Operation { + yield* pair(existingPath, newPath, true, (source, destination) => link(source, destination)); + }, + }; +} diff --git a/packages/workflow/src/deno/schema.ts b/packages/workflow/src/deno/schema.ts index 3b3fdc4c2..7073c5e7e 100644 --- a/packages/workflow/src/deno/schema.ts +++ b/packages/workflow/src/deno/schema.ts @@ -32,462 +32,20 @@ import { WorkflowIncompleteVersionOneError, WorkflowSchemaVersionError, } from "../storage/errors.ts"; +import { + APPLICATION_ID, + declaredStructureFailure, + EXPECTED_SCHEMA, + hasAnyDeclaredObject, + REQUIRED_OBJECTS, + REQUIRED_TABLES, + SCHEMA_SQL, + SCHEMA_VERSION, + type SchemaObject, +} from "../sqlite/workflow-schema.ts"; import { reading } from "./reading.ts"; import { initializeEmptyWorkspace, verifyWorkspace } from "./workspace/root.ts"; -/** - * The bytes `XMD1` as a 32-bit integer, written into the SQLite header. - * - * A database carries what wrote it, so a file that is perfectly valid SQLite - * and belongs to something else is refused on sight rather than through the - * confusing shape of its missing tables. - */ -export const APPLICATION_ID = 0x584d4431; - -/** The only schema version this build reads or writes. */ -export const SCHEMA_VERSION = 1; - -const STATUSES = "'running', 'suspended', 'interrupted', 'completed', 'failed', 'cancelled'"; - -/** - * A stop reason is three columns wide and has three legal shapes. - * - * Spreading the variant across columns is what lets SQLite hold the invariant - * rather than the code that writes rows: a host reason with an event id, or a - * journal reason with a code, is refused by the database itself. - */ -function coherentStopReason(): string { - return `CHECK ( - (stop_reason_kind IS NULL AND stop_reason_code IS NULL AND stop_reason_event_id IS NULL) - OR (stop_reason_kind = 'host' AND stop_reason_code IS NOT NULL AND stop_reason_event_id IS NULL) - OR (stop_reason_kind = 'journal' AND stop_reason_code IS NULL AND stop_reason_event_id IS NOT NULL) - )`; -} - -/** - * Version 1, one table at a time. - * - * Kept as separate definitions so verification can compare what a file holds - * with what this build writes, rather than settling for the table's name. - * - * The complete version-1 shape includes the pinned DOFS objects, retained - * Workspace roots, journal and metadata. Dependency order is explicit: DOFS - * content precedes root references, and roots precede the journal rows that - * name them. - */ -interface DeclaredObject { - readonly type: "table" | "index"; - readonly sql: string; -} - -const OBJECTS: ReadonlyMap = new Map([ - [ - "vfs_meta", - { - type: "table", - sql: `CREATE TABLE vfs_meta ( - k TEXT PRIMARY KEY, - v INTEGER NOT NULL - )`, - }, - ], - [ - "vfs_nodes", - { - type: "table", - sql: `CREATE TABLE vfs_nodes ( - inode INTEGER PRIMARY KEY AUTOINCREMENT, - type TEXT NOT NULL CHECK(type IN ('file','dir','symlink')), - mode INTEGER NOT NULL DEFAULT 493, - mtime INTEGER NOT NULL, - rev INTEGER NOT NULL DEFAULT 0, - mount_root TEXT, - stub_size INTEGER, - manifest_hash BLOB, - link_target TEXT, - size INTEGER NOT NULL DEFAULT 0 - )`, - }, - ], - [ - "vfs_dirents", - { - type: "table", - sql: `CREATE TABLE vfs_dirents ( - parent_inode INTEGER NOT NULL, - name TEXT NOT NULL, - child_inode INTEGER NOT NULL, - PRIMARY KEY (parent_inode, name) - ) WITHOUT ROWID`, - }, - ], - [ - "vfs_dirents_by_child", - { - type: "index", - sql: "CREATE INDEX vfs_dirents_by_child ON vfs_dirents(child_inode)", - }, - ], - [ - "vfs_nodes_by_rev", - { - type: "index", - sql: "CREATE INDEX vfs_nodes_by_rev ON vfs_nodes(rev)", - }, - ], - [ - "vfs_nodes_by_manifest_hash", - { - type: "index", - sql: `CREATE INDEX vfs_nodes_by_manifest_hash - ON vfs_nodes(manifest_hash) WHERE manifest_hash IS NOT NULL`, - }, - ], - [ - "vfs_blobs", - { - type: "table", - sql: `CREATE TABLE vfs_blobs ( - hash BLOB PRIMARY KEY, - size INTEGER NOT NULL, - last_seen INTEGER NOT NULL - )`, - }, - ], - [ - "vfs_blob_bytes", - { - type: "table", - sql: `CREATE TABLE vfs_blob_bytes ( - hash BLOB PRIMARY KEY REFERENCES vfs_blobs(hash) ON DELETE CASCADE, - bytes BLOB NOT NULL - )`, - }, - ], - [ - "vfs_chunks", - { - type: "table", - sql: `CREATE TABLE vfs_chunks ( - inode INTEGER NOT NULL, - idx INTEGER NOT NULL, - hash BLOB NOT NULL, - size INTEGER NOT NULL, - PRIMARY KEY (inode, idx) - ) WITHOUT ROWID`, - }, - ], - [ - "vfs_chunks_by_hash", - { - type: "index", - sql: "CREATE INDEX vfs_chunks_by_hash ON vfs_chunks(hash)", - }, - ], - [ - "vfs_manifests", - { - type: "table", - sql: `CREATE TABLE vfs_manifests ( - hash BLOB PRIMARY KEY, - size INTEGER NOT NULL, - encoded BLOB NOT NULL, - last_seen INTEGER NOT NULL DEFAULT 0 - )`, - }, - ], - [ - "vfs_changes", - { - type: "table", - sql: `CREATE TABLE vfs_changes ( - id INTEGER PRIMARY KEY AUTOINCREMENT, - rev INTEGER NOT NULL, - path TEXT NOT NULL, - op TEXT NOT NULL CHECK(op IN ('delete')) - )`, - }, - ], - [ - "vfs_changes_by_rev", - { - type: "index", - sql: "CREATE INDEX vfs_changes_by_rev ON vfs_changes(rev)", - }, - ], - [ - "vfs_changes_by_path", - { - type: "index", - sql: "CREATE INDEX vfs_changes_by_path ON vfs_changes(path, id DESC)", - }, - ], - [ - "_vfs_watermark", - { - type: "table", - sql: `CREATE TABLE _vfs_watermark ( - k TEXT NOT NULL, - backend TEXT NOT NULL DEFAULT 'default', - v INTEGER NOT NULL, - PRIMARY KEY (k, backend) - )`, - }, - ], - [ - "_vfs_fetch_cursor", - { - type: "table", - sql: `CREATE TABLE _vfs_fetch_cursor ( - k TEXT NOT NULL CHECK(k = 'fetch'), - backend TEXT NOT NULL DEFAULT 'default', - path TEXT, - PRIMARY KEY (k, backend) - )`, - }, - ], - [ - "_vfs_mounts", - { - type: "table", - sql: `CREATE TABLE _vfs_mounts ( - root TEXT PRIMARY KEY, - kind TEXT NOT NULL, - indexed INTEGER NOT NULL DEFAULT 0, - mode TEXT NOT NULL DEFAULT 'read-only' - CHECK(mode IN ('read-only', 'read-write')) - )`, - }, - ], - [ - "workspace_roots", - { - type: "table", - sql: `CREATE TABLE workspace_roots ( - root_id TEXT PRIMARY KEY CHECK ( - length(root_id) = 64 AND root_id NOT GLOB '*[^0-9a-f]*' - ), - format_version INTEGER NOT NULL CHECK (format_version = 1), - manifest TEXT NOT NULL CHECK (json_valid(manifest)) -) STRICT`, - }, - ], - [ - "workspace_root_manifest_refs", - { - type: "table", - sql: `CREATE TABLE workspace_root_manifest_refs ( - root_id TEXT NOT NULL REFERENCES workspace_roots(root_id) ON DELETE CASCADE, - manifest_hash BLOB NOT NULL REFERENCES vfs_manifests(hash) ON DELETE RESTRICT, - PRIMARY KEY (root_id, manifest_hash) -) STRICT, WITHOUT ROWID`, - }, - ], - [ - "workspace_root_blob_refs", - { - type: "table", - sql: `CREATE TABLE workspace_root_blob_refs ( - root_id TEXT NOT NULL REFERENCES workspace_roots(root_id) ON DELETE CASCADE, - blob_hash BLOB NOT NULL, - PRIMARY KEY (root_id, blob_hash), - FOREIGN KEY (blob_hash) REFERENCES vfs_blobs(hash) ON DELETE RESTRICT, - FOREIGN KEY (blob_hash) REFERENCES vfs_blob_bytes(hash) ON DELETE RESTRICT -) STRICT, WITHOUT ROWID`, - }, - ], - [ - "agent_sessions", - { - type: "table", - sql: `CREATE TABLE agent_sessions ( - session_key TEXT PRIMARY KEY, - provider TEXT NOT NULL, - agent_command TEXT NOT NULL, - session_identity TEXT NOT NULL, - policy TEXT NOT NULL, - assertion_kind TEXT NOT NULL, - assertion_value TEXT NOT NULL, - created_at TEXT NOT NULL -) STRICT`, - }, - ], - [ - "workspace_state", - { - type: "table", - sql: `CREATE TABLE workspace_state ( - singleton_id INTEGER PRIMARY KEY CHECK (singleton_id = 1), - current_root_id TEXT NOT NULL REFERENCES workspace_roots(root_id) ON DELETE RESTRICT -) STRICT`, - }, - ], - [ - "journal_events", - { - type: "table", - sql: `CREATE TABLE journal_events ( - sequence INTEGER PRIMARY KEY AUTOINCREMENT, - event_id TEXT NOT NULL UNIQUE, - record TEXT NOT NULL CHECK (json_valid(record)), - workspace_root_id TEXT NOT NULL REFERENCES workspace_roots(root_id) ON DELETE RESTRICT -) STRICT`, - }, - ], - [ - "workflow_run", - { - type: "table", - sql: `CREATE TABLE workflow_run ( - id INTEGER PRIMARY KEY CHECK (id = 1), - run_id TEXT NOT NULL, - definition TEXT NOT NULL CHECK (json_valid(definition)), - base TEXT NOT NULL, - props TEXT NOT NULL CHECK (json_valid(props) AND json_type(props) = 'object'), - status TEXT NOT NULL CHECK (status IN (${STATUSES})), - stop_reason_kind TEXT CHECK (stop_reason_kind IS NULL OR stop_reason_kind IN ('host', 'journal')), - stop_reason_code TEXT, - stop_reason_event_id TEXT REFERENCES journal_events (event_id), - created_at TEXT NOT NULL, - updated_at TEXT NOT NULL, - ${coherentStopReason()} -) STRICT`, - }, - ], - [ - "definition_retrieval", - { - type: "table", - sql: `CREATE TABLE definition_retrieval ( - id INTEGER PRIMARY KEY CHECK (id = 1), - metadata TEXT NOT NULL CHECK (json_valid(metadata)), - revision INTEGER NOT NULL CHECK (revision >= 1 AND revision <= 9007199254740991), - updated_at TEXT NOT NULL -) STRICT`, - }, - ], - [ - "document_executions", - { - type: "table", - sql: `CREATE TABLE document_executions ( - sequence INTEGER PRIMARY KEY AUTOINCREMENT, - execution_id TEXT NOT NULL UNIQUE, - started_at TEXT NOT NULL, - stopped_at TEXT, - stop_status TEXT CHECK (stop_status IS NULL OR stop_status IN (${STATUSES})), - stop_reason_kind TEXT CHECK (stop_reason_kind IS NULL OR stop_reason_kind IN ('host', 'journal')), - stop_reason_code TEXT, - stop_reason_event_id TEXT REFERENCES journal_events (event_id), - CHECK ((stopped_at IS NULL) = (stop_status IS NULL)), - CHECK (stop_status IS NOT NULL OR stop_reason_kind IS NULL), - ${coherentStopReason()} -) STRICT`, - }, - ], - [ - "workspace_repositories", - { - type: "table", - sql: `CREATE TABLE workspace_repositories ( - name TEXT PRIMARY KEY CHECK (length(name) > 0), - locator TEXT NOT NULL CHECK (length(locator) > 0), - locator_fingerprint TEXT NOT NULL CHECK ( - length(locator_fingerprint) = 64 AND locator_fingerprint NOT GLOB '*[^0-9a-f]*' - ), - requested_base TEXT CHECK (requested_base IS NULL OR length(requested_base) > 0), - creation_commit TEXT NOT NULL CHECK (length(creation_commit) > 0), - primary_branch TEXT NOT NULL CHECK (length(primary_branch) > 0), - object_format TEXT NOT NULL CHECK (object_format IN ('sha1', 'sha256')), - checkout_path TEXT NOT NULL UNIQUE CHECK ( - length(checkout_path) > 0 AND substr(checkout_path, 1, 1) = '/' - ) -) STRICT`, - }, - ], - [ - "workspace_worktrees", - { - type: "table", - sql: `CREATE TABLE workspace_worktrees ( - repository_name TEXT NOT NULL REFERENCES workspace_repositories(name) ON DELETE RESTRICT, - name TEXT NOT NULL CHECK (length(name) > 0), - requested_branch TEXT NOT NULL CHECK (length(requested_branch) > 0), - requested_base TEXT CHECK (requested_base IS NULL OR length(requested_base) > 0), - creation_commit TEXT NOT NULL CHECK (length(creation_commit) > 0), - checkout_path TEXT NOT NULL UNIQUE CHECK ( - length(checkout_path) > 0 AND substr(checkout_path, 1, 1) = '/' - ), - PRIMARY KEY (repository_name, name) -) STRICT, WITHOUT ROWID`, - }, - ], - [ - "workflow_suspension_answers", - { - type: "table", - sql: `CREATE TABLE workflow_suspension_answers ( - suspension_id TEXT PRIMARY KEY, - request_event_id TEXT NOT NULL REFERENCES journal_events(event_id) ON DELETE RESTRICT, - request_fingerprint TEXT NOT NULL CHECK ( - length(request_fingerprint) = 64 AND request_fingerprint NOT GLOB '*[^0-9a-f]*' - ), - answer TEXT NOT NULL CHECK (json_valid(answer)), - state TEXT NOT NULL CHECK (state IN ('pending', 'consumed')), - created_at TEXT NOT NULL, - consumed_at TEXT, - CHECK ((state = 'consumed') = (consumed_at IS NOT NULL)) -) STRICT`, - }, - ], - [ - "workflow_fork_lineage", - { - type: "table", - sql: `CREATE TABLE workflow_fork_lineage ( - id INTEGER PRIMARY KEY CHECK (id = 1), - source_run_id TEXT NOT NULL CHECK (length(source_run_id) > 0), - checkpoint_event_id TEXT NOT NULL CHECK (length(checkpoint_event_id) > 0), - checkpoint_workspace_root_id TEXT NOT NULL - REFERENCES workspace_roots(root_id) ON DELETE RESTRICT, - created_at TEXT NOT NULL -) STRICT`, - }, - ], - [ - "journal_event_provenance", - { - type: "table", - sql: `CREATE TABLE journal_event_provenance ( - event_id TEXT PRIMARY KEY REFERENCES journal_events(event_id) ON DELETE RESTRICT, - source_run_id TEXT NOT NULL CHECK (length(source_run_id) > 0), - source_event_id TEXT NOT NULL CHECK (length(source_event_id) > 0) -) STRICT, WITHOUT ROWID`, - }, - ], -]); - -export const EXPECTED_SCHEMA = Object.freeze( - [...OBJECTS.entries()].map(([name, object]) => - Object.freeze({ name, type: object.type, sql: normalize(object.sql) }), - ), -); - -/** Objects version 1 declares, including the pinned Cloudflare structure. */ -export const REQUIRED_OBJECTS: readonly string[] = Object.freeze([...OBJECTS.keys()]); - -/** Tables version 1 declares. */ -export const REQUIRED_TABLES: readonly string[] = Object.freeze( - [...OBJECTS.entries()].filter(([, object]) => object.type === "table").map(([name]) => name), -); - -/** Version 1 in full. */ -export const SCHEMA_SQL = [...OBJECTS.values()] - .filter((object) => object.type === "table" && !object.sql.startsWith("CREATE TABLE vfs_")) - .filter((object) => !object.sql.startsWith("CREATE TABLE _vfs_")) - .map((object) => `${object.sql};`) - .join("\n\n"); - /** * Write the version-1 schema into a database that holds nothing. * @@ -495,6 +53,15 @@ export const SCHEMA_SQL = [...OBJECTS.values()] * and the tables appear together or not at all — a half-initialized file would * be indistinguishable from one this build must refuse. */ +export { + APPLICATION_ID, + EXPECTED_SCHEMA, + REQUIRED_OBJECTS, + REQUIRED_TABLES, + SCHEMA_SQL, + SCHEMA_VERSION, +}; + export function initializeSchema( database: DatabaseSync, dofs: CloudflareDatabase, @@ -571,98 +138,33 @@ export function verifySchema(database: DatabaseSync, path: string, dofs: Cloudfl * has not learned yet. */ function verifyStructure(database: DatabaseSync, path: string): void { - const objects = schemaObjects(database, path); - if (isIncompletePreReleaseShape(objects)) { + const failure = declaredStructureFailure(schemaObjects(database, path)); + if (failure === undefined) { + return; + } + if (failure.kind === "incomplete-pre-release") { throw new WorkflowIncompleteVersionOneError(path); } - - for (const object of objects) { - const expected = OBJECTS.get(object.name); - if (expected === undefined) { - throw new WorkflowDatabaseCorruptError( - path, - `it declares an object that version ${SCHEMA_VERSION} does not`, - ); - } - if (object.type !== expected.type || normalize(object.sql) !== normalize(expected.sql)) { - throw new WorkflowDatabaseCorruptError( - path, - `its ${object.name} object is not shaped the way version ${SCHEMA_VERSION} declares it`, - ); - } + if (failure.kind === "undeclared-object") { + throw new WorkflowDatabaseCorruptError( + path, + `it declares an object that version ${SCHEMA_VERSION} does not`, + ); } - - const present = new Set(objects.map((object) => object.name)); - const missing = REQUIRED_OBJECTS.filter((name) => !present.has(name)); - if (missing.length > 0) { - throw new WorkflowDatabaseCorruptError(path, `it is missing the table ${missing.join(", ")}`); + if (failure.kind === "misshapen-object") { + throw new WorkflowDatabaseCorruptError( + path, + `its ${failure.name} object is not shaped the way version ${SCHEMA_VERSION} declares it`, + ); } + throw new WorkflowDatabaseCorruptError( + path, + `it is missing the table ${failure.names.join(", ")}`, + ); } function hasDeclaredVersionOneObjects(database: DatabaseSync, path: string): boolean { - return schemaObjects(database, path).some((object) => OBJECTS.has(object.name)); -} - -/** - * Every in-place amendment to version 1, newest first. - * - * Each entry names what that amendment added. Peeling them off in order is what - * reconstructs the shapes that once claimed to be a complete version 1, so a - * database an earlier build produced is refused as an incomplete pre-release - * rather than as arbitrary damage. - */ -const AMENDMENTS: readonly (readonly string[])[] = Object.freeze([ - Object.freeze(["workflow_fork_lineage", "journal_event_provenance"]), - Object.freeze(["workflow_suspension_answers"]), - Object.freeze(["workspace_repositories", "workspace_worktrees"]), -]); - -/** What the newest amendment added. Its presence marks a current-shape database. */ -const LATEST_AMENDMENT: readonly string[] = AMENDMENTS[0] ?? []; - -/** The very first pre-release shape, before Workspace root retention existed. */ -const EARLIEST_PRE_RELEASE_SHAPE: readonly string[] = [ - "definition_retrieval", - "document_executions", - "journal_events", - "workflow_run", -]; - -/** - * Every later shape that once claimed to be a complete version 1. - * - * Newest first: version 1 minus the newest amendment, then minus the one before - * it, and so on. - */ -const PRIOR_COMPLETE_SHAPES: readonly (readonly string[])[] = Object.freeze( - AMENDMENTS.map((_, index) => { - const removed = new Set(AMENDMENTS.slice(0, index + 1).flat()); - return Object.freeze(REQUIRED_OBJECTS.filter((name) => !removed.has(name))); - }), -); - -/** - * Whether these declarations describe an earlier shape that once claimed to be - * a complete version 1. - * - * The very first pre-release held only the run, journal and execution tables. - * Every shape after it is version 1 minus whichever amendments had not been - * made yet, and each is named here so the refusal reads as an incomplete - * pre-release rather than as corruption. - */ -function isIncompletePreReleaseShape(objects: readonly SchemaObject[]): boolean { - const present = new Set(objects.map((object) => object.name)); - if (LATEST_AMENDMENT.some((name) => present.has(name))) { - return false; - } - const earliest = new Set(EARLIEST_PRE_RELEASE_SHAPE); - if (present.size === earliest.size && [...present].every((name) => earliest.has(name))) { - return objects.every((object) => object.type === "table"); - } - return PRIOR_COMPLETE_SHAPES.some((shape) => { - const expected = new Set(shape); - return present.size === expected.size && [...present].every((name) => expected.has(name)); - }); + return hasAnyDeclaredObject(schemaObjects(database, path)); } /** @@ -693,12 +195,6 @@ function checkForeignKeys(database: DatabaseSync, path: string): void { } } -interface SchemaObject { - readonly type: string; - readonly name: string; - readonly sql: string; -} - /** * Everything somebody declared in this database. * @@ -725,11 +221,6 @@ function schemaObjects(database: DatabaseSync, path: string): SchemaObject[] { return objects; } -/** One statement's shape, independent of how it was laid out. */ -function normalize(sql: string): string { - return sql.replace(/\s+/g, " ").trim(); -} - function readPragmaNumber(database: DatabaseSync, pragma: string, path: string): number { const rows = query(database, `PRAGMA ${pragma}`, path); const value = rows[0]?.[pragma]; diff --git a/packages/workflow/src/deno/transitions.ts b/packages/workflow/src/deno/transitions.ts index 1c365834d..27808c7a7 100644 --- a/packages/workflow/src/deno/transitions.ts +++ b/packages/workflow/src/deno/transitions.ts @@ -60,7 +60,7 @@ import { reading } from "./reading.ts"; import { readJournalEntries } from "./journal.ts"; import type { ForkSourceSnapshot } from "./fork-source.ts"; import { readForkLineage, writeForkInheritance, type ForkHeadEvents } from "./fork-write.ts"; -import { readDocumentExecution, readRetrieval, stopReasonColumns } from "./rows.ts"; +import { readDocumentExecution, readRetrieval, stopReasonColumns } from "../sqlite/rows.ts"; import { initializeSchema, isSqliteForeignKeyConstraint, diff --git a/packages/workflow/src/deno/workspace/agent-sessions.ts b/packages/workflow/src/deno/workspace/agent-sessions.ts index ff55877b3..e3a418153 100644 --- a/packages/workflow/src/deno/workspace/agent-sessions.ts +++ b/packages/workflow/src/deno/workspace/agent-sessions.ts @@ -38,62 +38,33 @@ */ import type { DatabaseSync } from "node:sqlite"; -import { createHash } from "node:crypto"; - -/** A retained Agent session this host will not continue under. */ -export class WorkflowAgentSessionError extends Error { - override name = "WorkflowAgentSessionError"; -} - -/** - * One durable identity a provider asserted, and what kind of thing it is. - * - * Tagged, because "the adapter's own session id" and "an ACP session id" and "a - * record id in some store" are different claims that happen to be strings. A - * host comparing them without the tag would accept one for another. - */ -export interface ProviderAssertion { - readonly kind: string; - readonly value: string; -} - -/** What identifies one logical Agent session. */ -export interface AgentSessionIdentity { - /** Which provider holds the conversation, as that provider names itself. */ - readonly provider: string; - /** The resolved agent command, not the name a document wrote. */ - readonly agentCommand: string; - /** The engine-derived Agent/Session expansion identity. Never authored. */ - readonly sessionIdentity: string; -} - -/** One retained mapping, as the run's database holds it. */ -export interface AgentSessionRecord extends AgentSessionIdentity { - readonly sessionKey: string; - /** The session policy in force when the provider created this session. */ - readonly policy: string; - readonly assertion: ProviderAssertion; - readonly createdAt: string; -} - -function digest(value: string): string { - return createHash("sha256").update(value, "utf8").digest("hex").slice(0, 32); -} /** - * The key one logical session is retained under, within this run. + * The shape and the key derivation are the shared rule, not this adapter's. * - * The engine-derived Session expansion identity and nothing else. The provider - * and the resolved agent command are compatibility attributes stored beside it: - * changing either refuses reattachment rather than addressing a second mapping, - * because a `` element that changed agent is the same element asking - * for something this run cannot give it. - * - * Digested so it stays bounded, and namespaced so a row is recognizable. + * Both hosts retain these mappings, and two derivations would be two keys for + * one session — reattachment would quietly start a new conversation instead of + * finding the old one. What stays here is the storage: the columns, the + * statements, and the transaction they run in. */ -export function agentSessionKey(identity: AgentSessionIdentity): string { - return ["xmd", "workflow", "v1", digest(identity.sessionIdentity)].join(":"); -} +import { + type AgentSessionRecord, + type AgentSessions, + WorkflowAgentSessionError, +} from "../../storage/agent-session.ts"; + +export { + agentSessionKey, + parseAgentSessionRecord, + resolveAgentSession, + WorkflowAgentSessionError, +} from "../../storage/agent-session.ts"; +export type { AgentSessionResolution, AgentSessions } from "../../storage/agent-session.ts"; +export type { + AgentSessionIdentity, + AgentSessionRecord, + ProviderAssertion, +} from "../../storage/agent-session.ts"; const COLUMNS = `session_key, provider, agent_command, session_identity, policy, assertion_kind, assertion_value, created_at`; @@ -106,12 +77,6 @@ const INSERT = `INSERT INTO agent_sessions (session_key, provider, agent_command session_identity, policy, assertion_kind, assertion_value, created_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?)`; -/** Every retained mapping this run holds. Reading is not a transaction. */ -export interface AgentSessions { - read(sessionKey: string): AgentSessionRecord | undefined; - commit(record: AgentSessionRecord): void; -} - function text(value: unknown): string | undefined { return typeof value === "string" ? value : undefined; } @@ -191,82 +156,3 @@ export function createAgentSessions(database: DatabaseSync, authorize: () => voi }, }; } - -/** What a continuation may do with the session a key names. */ -export type AgentSessionResolution = - | { readonly kind: "create"; readonly sessionKey: string } - | { readonly kind: "reattach"; readonly record: AgentSessionRecord }; - -/** - * Decide what this attachment may do with the session this identity names. - * - * `asserted` is every canonical identity the provider currently asserts for that - * key — none, one, or more than one. It is deliberately not "does the provider - * hold this key": occupancy says something is there, not what conversation it - * is, and adopting one on that basis is how a run continues a session it cannot - * name. - */ -export function resolveAgentSession( - retained: AgentSessionRecord | undefined, - policy: string, - asserted: readonly ProviderAssertion[], - identity: AgentSessionIdentity, -): AgentSessionResolution { - const sessionKey = agentSessionKey(identity); - if (asserted.length > 1) { - throw new WorkflowAgentSessionError( - "the provider asserts more than one durable identity for this run's Agent session, so " + - "this host cannot tell which conversation it would be continuing. Start a new run " + - "rather than continuing this one.", - ); - } - const current = asserted[0]; - - if (retained === undefined) { - if (current === undefined) { - // Neither side holds anything: nothing was ever established here. - return { kind: "create", sessionKey }; - } - // The pre-commit window. An attempt was interrupted between the provider - // asserting an identity and this run recording it, and exactly one - // canonical assertion is what reconciles it — nothing else may. - return { - kind: "reattach", - record: { - sessionKey, - ...identity, - policy, - assertion: current, - createdAt: new Date().toISOString(), - }, - }; - } - - if ( - retained.provider !== identity.provider || - retained.agentCommand !== identity.agentCommand || - retained.sessionIdentity !== identity.sessionIdentity || - retained.policy !== policy - ) { - throw new WorkflowAgentSessionError( - "this run's Agent session was established under a different provider, agent or session " + - "policy than this host states, and a session created under one ceiling is not " + - "continued under another. Start a new run rather than continuing this one.", - ); - } - if (current === undefined) { - throw new WorkflowAgentSessionError( - "the provider asserts no durable identity for the Agent session this run retained, and " + - "this host does not reconstruct a conversation by replaying it into a new session. " + - "Start a new run rather than continuing this one.", - ); - } - if (current.kind !== retained.assertion.kind || current.value !== retained.assertion.value) { - throw new WorkflowAgentSessionError( - "the provider asserts a different durable identity than the Agent session this run " + - "retained, so it did not resume the conversation this run was having. This host does " + - "not continue under a replacement session.", - ); - } - return { kind: "reattach", record: retained }; -} diff --git a/packages/workflow/src/deno/workspace/errors.ts b/packages/workflow/src/deno/workspace/errors.ts index 571fd1fb0..7b65f09cb 100644 --- a/packages/workflow/src/deno/workspace/errors.ts +++ b/packages/workflow/src/deno/workspace/errors.ts @@ -13,31 +13,9 @@ const JOURNALABLE_CODES = new Set([ "ELOOP", ]); -/** - * A failure this effect publishes as its own durable outcome instead of raising. - * - * The distinction the effect layer needs is not "what went wrong" but "who - * this belongs to". A failure of this kind is part of what the effect *did*: it - * is written into the journal as the effect's result, the Workspace root stays - * where it was, and a replay reproduces it without performing anything. Every - * other failure is the run failing, and travels as an ordinary raise. - * - * It is a base class rather than a predicate over shapes so that being publishable - * is something a failure declares by construction. A module that wants its own - * refusal published extends this; nothing acquires the property by resembling - * something. - */ -export abstract class JournaledEffectFailure extends Error {} +export { isJournaledEffectFailure, JournaledEffectFailure } from "../../workspace/failure.ts"; -/** - * Whether this failure is the effect's outcome rather than the run's failure. - * - * Asked by the one place that has to choose between writing a result and - * letting a failure through. - */ -export function isJournaledEffectFailure(error: unknown): error is Error { - return error instanceof JournaledEffectFailure; -} +import { JournaledEffectFailure } from "../../workspace/failure.ts"; class JournalableWorkspaceFailure extends JournaledEffectFailure { override name = "WorkspaceFsError"; diff --git a/packages/workflow/src/deno/workspace/filesystem.ts b/packages/workflow/src/deno/workspace/filesystem.ts index 1ae4087a1..357adaf82 100644 --- a/packages/workflow/src/deno/workspace/filesystem.ts +++ b/packages/workflow/src/deno/workspace/filesystem.ts @@ -1,4 +1,9 @@ import { type Operation } from "effection"; +import type { + WorkspaceEntry, + WorkspaceFilesystem, + WorkspaceStat, +} from "../../workspace/filesystem.ts"; import { chmod as chmodPath } from "../../../vendor/cloudflare-computer-dofs/generated/fs/chmod.js"; import { link as linkFile } from "../../../vendor/cloudflare-computer-dofs/generated/fs/link.js"; import { mkdir as mkdirPath } from "../../../vendor/cloudflare-computer-dofs/generated/fs/mkdir.js"; @@ -17,33 +22,15 @@ import { writeFileSync } from "../../../vendor/cloudflare-computer-dofs/generate import type { RunConnection } from "../connections.ts"; import { throwWorkspaceFilesystemFailure } from "./errors.ts"; -export interface DenoWorkspaceEntry { - readonly name: string; - readonly kind: "file" | "directory" | "symlink"; -} - -export interface DenoWorkspaceStat { - readonly kind: "file" | "directory" | "symlink"; - readonly mode: number; - readonly mtime: number; - readonly size: number; -} - -export interface DenoWorkspaceFilesystem { - readFile(path: string): Operation; - readTextFile(path: string): Operation; - stat(path: string): Operation; - lstat(path: string): Operation; - readlink(path: string): Operation; - readdir(path: string): Operation; - writeFile(path: string, content: string | Uint8Array, mode?: number): Operation; - mkdir(path: string, options?: { recursive?: boolean; mode?: number }): Operation; - remove(path: string, options?: { recursive?: boolean; force?: boolean }): Operation; - rename(from: string, to: string): Operation; - chmod(path: string, mode: number): Operation; - symlink(target: string, path: string): Operation; - link(existingPath: string, newPath: string): Operation; -} +/** + * The names this host has always used, for the one shared contract. + * + * The interface moved rather than changed: this adapter is one implementation + * of it, and the runner's attempt-backed adapter is the other. + */ +export type DenoWorkspaceEntry = WorkspaceEntry; +export type DenoWorkspaceStat = WorkspaceStat; +export type DenoWorkspaceFilesystem = WorkspaceFilesystem; export function createDenoWorkspaceFilesystem( connection: RunConnection, diff --git a/packages/workflow/src/deno/workspace/manifest.ts b/packages/workflow/src/deno/workspace/manifest.ts index 47c0b3893..e57194887 100644 --- a/packages/workflow/src/deno/workspace/manifest.ts +++ b/packages/workflow/src/deno/workspace/manifest.ts @@ -1,58 +1,43 @@ import { createHash } from "node:crypto"; -import { z } from "zod"; import { WorkflowDatabaseCorruptError } from "../../storage/errors.ts"; +import { + hasUnpairedSurrogate, + parseWorkspaceRootManifest, + SHA256, + validateCanonicalWorkspacePath, + validateWorkspaceRootEntries, + WORKSPACE_ROOT_DOMAIN, + WORKSPACE_ROOT_FORMAT, + type WorkspaceRejection, + type WorkspaceRootEntry, + type WorkspaceRootManifest, +} from "../../workspace/root-manifest.ts"; + +export { + compareUtf8, + hasUnpairedSurrogate, + parentFirst, + parentPath, + WORKSPACE_ROOT_DOMAIN, + WORKSPACE_ROOT_FORMAT, +} from "../../workspace/root-manifest.ts"; +export type { + WorkspaceRejection, + WorkspaceRootEntry, + WorkspaceRootManifest, +} from "../../workspace/root-manifest.ts"; -export const WORKSPACE_ROOT_FORMAT = 1; -export const WORKSPACE_ROOT_DOMAIN = "xmd-workspace-root\0v1\0"; - -const SHA256 = /^[0-9a-f]{64}$/; -const encoder = new TextEncoder(); - -const directoryEntrySchema = z - .object({ - path: z.string(), - kind: z.literal("directory"), - mode: z.number().int().min(0).max(0o7777), - mtime: z.number().int().safe(), - }) - .strict(); - -const fileEntrySchema = z - .object({ - path: z.string(), - kind: z.literal("file"), - mode: z.number().int().min(0).max(0o7777), - mtime: z.number().int().safe(), - size: z.number().int().safe().nonnegative(), - manifest: z.string().regex(SHA256), - hardlink: z - .string() - .regex(/^h[0-9]+$/) - .nullable(), - }) - .strict(); - -const symlinkEntrySchema = z - .object({ - path: z.string(), - kind: z.literal("symlink"), - mode: z.number().int().min(0).max(0o7777), - mtime: z.number().int().safe(), - target: z.string(), - }) - .strict(); - -const rootManifestSchema = z - .object({ - format: z.literal(WORKSPACE_ROOT_FORMAT), - entries: z.array( - z.discriminatedUnion("kind", [directoryEntrySchema, fileEntrySchema, symlinkEntrySchema]), - ), - }) - .strict(); - -export type WorkspaceRootEntry = z.infer["entries"][number]; -export type WorkspaceRootManifest = z.infer; +/** + * How a caller other than a live run reports a Workspace root it cannot accept. + * + * The default names the run database the root was read from, which is what + * every live caller is holding. A sealed XMD artifact is not a run database and + * says so in its own words, so it supplies one of these rather than borrowing a + * sentence that would tell an operator to restore a run from a backup. + */ +function rejecting(databasePath: string): WorkspaceRejection { + return (reason: string) => corrupt(databasePath, reason); +} export interface StoredWorkspaceRoot { readonly rootId: string; @@ -96,40 +81,12 @@ export function encodeWorkspaceManifest( return JSON.stringify(manifest); } -/** - * How a caller other than a live run reports a Workspace root it cannot accept. - * - * The default names the run database the root was read from, which is what - * every live caller is holding. A sealed XMD artifact is not a run database and - * says so in its own words, so it supplies one of these rather than borrowing a - * sentence that would tell an operator to restore a run from a backup. - */ -export type WorkspaceRejection = (reason: string) => never; - -function rejecting(databasePath: string): WorkspaceRejection { - return (reason: string) => corrupt(databasePath, reason); -} - export function parseWorkspaceManifest( manifest: string, databasePath: string, reject: WorkspaceRejection = rejecting(databasePath), ): WorkspaceRootManifest { - let offered: unknown; - try { - offered = JSON.parse(manifest); - } catch { - reject("one of its retained Workspace roots is not JSON"); - } - const parsed = rootManifestSchema.safeParse(offered); - if (!parsed.success) { - reject("one of its retained Workspace roots has an invalid manifest"); - } - validateWorkspaceEntries(parsed.data.entries, databasePath, reject); - if (JSON.stringify(parsed.data) !== manifest) { - reject("one of its retained Workspace roots is not canonically encoded"); - } - return parsed.data; + return parseWorkspaceRootManifest(manifest, reject); } export function validateWorkspaceEntries( @@ -137,60 +94,7 @@ export function validateWorkspaceEntries( databasePath: string, reject: WorkspaceRejection = rejecting(databasePath), ): void { - if (entries.length === 0 || entries[0]?.path !== "/" || entries[0]?.kind !== "directory") { - reject("a Workspace root does not begin with its root directory"); - } - - let previous: string | undefined; - let nextHardlink = 0; - const directories = new Set(); - const hardlinkMembers = new Map(); - const hardlinkFirst = new Map(); - - for (const entry of entries) { - validateCanonicalPath(entry.path, databasePath, reject); - if (previous !== undefined && compareUtf8(previous, entry.path) >= 0) { - reject("a Workspace root's paths are duplicated or out of canonical order"); - } - previous = entry.path; - - if (entry.path !== "/" && !directories.has(parentPath(entry.path))) { - reject("a Workspace root contains an entry without a parent directory"); - } - if (entry.kind === "directory") { - directories.add(entry.path); - } - if ( - entry.kind === "symlink" && - (entry.target.includes("\0") || hasUnpairedSurrogate(entry.target)) - ) { - reject("a Workspace root contains an invalid symbolic-link target"); - } - if (entry.kind === "file" && entry.hardlink !== null) { - const first = hardlinkFirst.get(entry.hardlink); - if (first === undefined) { - if (entry.hardlink !== `h${nextHardlink}`) { - reject("a Workspace root's hardlinks are not canonically numbered"); - } - nextHardlink += 1; - hardlinkFirst.set(entry.hardlink, entry); - } else if ( - first.mode !== entry.mode || - first.mtime !== entry.mtime || - first.size !== entry.size || - first.manifest !== entry.manifest - ) { - reject("a Workspace root's hardlink group has inconsistent metadata"); - } - hardlinkMembers.set(entry.hardlink, (hardlinkMembers.get(entry.hardlink) ?? 0) + 1); - } - } - - for (const count of hardlinkMembers.values()) { - if (count < 2) { - reject("a Workspace root contains a one-member hardlink group"); - } - } + validateWorkspaceRootEntries(entries, reject); } export function validateCanonicalPath( @@ -198,22 +102,7 @@ export function validateCanonicalPath( databasePath: string, reject: WorkspaceRejection = rejecting(databasePath), ): void { - if (value === "/") { - return; - } - if ( - !value.startsWith("/") || - value.endsWith("/") || - value.includes("\0") || - hasUnpairedSurrogate(value) - ) { - reject("a Workspace root contains a noncanonical path"); - } - for (const part of value.slice(1).split("/")) { - if (part === "" || part === "." || part === "..") { - reject("a Workspace root contains a noncanonical path component"); - } - } + validateCanonicalWorkspacePath(value, reject); } export function validatePathName(name: string, databasePath: string): void { @@ -229,20 +118,6 @@ export function validatePathName(name: string, databasePath: string): void { } } -export function compareUtf8(left: string, right: string): number { - return Buffer.compare(encoder.encode(left), encoder.encode(right)); -} - -export function parentFirst(left: WorkspaceRootEntry, right: WorkspaceRootEntry): number { - const depth = left.path.split("/").length - right.path.split("/").length; - return depth === 0 ? compareUtf8(left.path, right.path) : depth; -} - -export function parentPath(path: string): string { - const boundary = path.lastIndexOf("/"); - return boundary === 0 ? "/" : path.slice(0, boundary); -} - export function sha256(value: Uint8Array): Uint8Array { return new Uint8Array(createHash("sha256").update(value).digest()); } @@ -292,19 +167,3 @@ export function mode(value: unknown, databasePath: string): number { export function corrupt(databasePath: string, reason: string): never { throw new WorkflowDatabaseCorruptError(databasePath, reason); } - -function hasUnpairedSurrogate(value: string): boolean { - for (let index = 0; index < value.length; index += 1) { - const code = value.charCodeAt(index); - if (code >= 0xd800 && code <= 0xdbff) { - const next = value.charCodeAt(index + 1); - if (next < 0xdc00 || next > 0xdfff) { - return true; - } - index += 1; - } else if (code >= 0xdc00 && code <= 0xdfff) { - return true; - } - } - return false; -} diff --git a/packages/workflow/src/deno/workspace/repositories.ts b/packages/workflow/src/deno/workspace/repositories.ts index 82befb9a7..cb21e7326 100644 --- a/packages/workflow/src/deno/workspace/repositories.ts +++ b/packages/workflow/src/deno/workspace/repositories.ts @@ -29,12 +29,9 @@ import { type WorktreeRecord, } from "../../composition/records.ts"; import { reading } from "../reading.ts"; +import type { StoredRepository, WorkspaceMetadata } from "../../workspace/metadata.ts"; -/** A Repository row: its journal-safe record, and the locator only storage sees. */ -export interface StoredRepository { - readonly record: RepositoryRecord; - readonly locator: string; -} +export type { StoredRepository, WorkspaceMetadata } from "../../workspace/metadata.ts"; const REPOSITORY_COLUMNS = `name, locator, locator_fingerprint, requested_base, creation_commit, primary_branch, object_format, checkout_path`; @@ -195,23 +192,6 @@ export function insertWorktree(database: DatabaseSync, record: WorktreeRecord): ); } -/** - * The metadata one Workspace transaction may read and write. - * - * Handed to a mutation beside the filesystem, so retained Git identity and - * retained Git bytes move together inside one transaction. It is the provider's - * surface and not a document's: a component reaches it only by asking the - * composition provider to perform an effect. - */ -export interface WorkspaceMetadata { - readRepository(name: string): StoredRepository | undefined; - readRepositories(): StoredRepository[]; - insertRepository(stored: StoredRepository): void; - readWorktree(repositoryName: string, name: string): WorktreeRecord | undefined; - readWorktreesForRepository(repositoryName: string): WorktreeRecord[]; - insertWorktree(record: WorktreeRecord): void; -} - export function createWorkspaceMetadata( database: DatabaseSync, authorize: () => void, diff --git a/packages/workflow/src/deno/workspace/restore.ts b/packages/workflow/src/deno/workspace/restore.ts index 375994ef9..524f16592 100644 --- a/packages/workflow/src/deno/workspace/restore.ts +++ b/packages/workflow/src/deno/workspace/restore.ts @@ -13,7 +13,7 @@ import { } from "./manifest.ts"; import { loadWorkspaceRoot, - readDofsManifest, + readContentManifest, setCurrentWorkspaceRoot, snapshotWorkspace, verifyWorkspace, @@ -131,7 +131,7 @@ function materializeNode( .run(entry.mode, entry.mtime, revision, entry.target); inode = Number(result.lastInsertRowid); } else { - const manifest = readDofsManifest(database, entry.manifest, databasePath); + const manifest = readContentManifest(database, entry.manifest, databasePath); if (manifest.size !== entry.size) { corrupt(databasePath, "a retained file size differs from its DOFS manifest"); } diff --git a/packages/workflow/src/deno/workspace/root.ts b/packages/workflow/src/deno/workspace/root.ts index 55db2b482..de87da60c 100644 --- a/packages/workflow/src/deno/workspace/root.ts +++ b/packages/workflow/src/deno/workspace/root.ts @@ -1,5 +1,4 @@ import type { DatabaseSync } from "node:sqlite"; -import { z } from "zod"; import type { Database as CloudflareDatabase } from "../../../vendor/cloudflare-computer-dofs/generated/storage.js"; import { buildManifest } from "../../../vendor/cloudflare-computer-dofs/generated/sync/manifests.js"; import type { RunConnection, RunTransaction } from "../connections.ts"; @@ -25,33 +24,18 @@ import { workspaceRoot, WORKSPACE_ROOT_FORMAT, } from "./manifest.ts"; - -const decoder = new TextDecoder("utf-8", { fatal: true }); -const SHA256 = /^[0-9a-f]{64}$/; - -const dofsManifestSchema = z - .object({ - version: z.literal(1), - chunks: z.array( - z - .object({ - hash: z.string().regex(SHA256), - size: z.number().int().safe().positive(), - }) - .strict(), - ), - }) - .strict(); +import { SHA256 } from "../../workspace/root-manifest.ts"; +import { + type ContentManifest, + decodeContentManifest as decodeSharedContentManifest, +} from "../../workspace/content-manifest.ts"; export interface DofsChunk { readonly hash: Uint8Array; readonly size: number; } -export interface DofsManifest { - readonly size: number; - readonly chunks: readonly { readonly hash: string; readonly size: number }[]; -} +export type { ContentManifest } from "../../workspace/content-manifest.ts"; interface NodeRow { readonly inode: number; @@ -216,9 +200,12 @@ export function snapshotWorkspace( for (const [index, paths] of groups.entries()) { const group = `h${index}`; const members = new Set(paths); - for (const item of entries) { + for (const [position, item] of entries.entries()) { if (item.entry.kind === "file" && members.has(item.entry.path)) { - item.entry.hardlink = group; + // Rebuilt rather than mutated: a manifest entry is what a root is + // hashed over, and a value nobody can edit in place is one nobody can + // edit after it has been counted. + entries[position] = { ...item, entry: { ...item.entry, hardlink: group } }; } } } @@ -425,11 +412,11 @@ export function verifyWorkspace( } } -export function readDofsManifest( +export function readContentManifest( database: DatabaseSync, hash: string, databasePath: string, -): DofsManifest { +): ContentManifest { const hashBytes = fromHex(hash, databasePath, "DOFS manifest identity"); const row = reading( database, @@ -447,7 +434,7 @@ export function readDofsManifest( if (toHex(sha256(encoded)) !== hash) { corrupt(databasePath, "a DOFS manifest hash does not match its bytes"); } - const decoded = decodeDofsManifest(encoded, (reason) => corrupt(databasePath, reason)); + const decoded = decodeContentManifest(encoded, (reason) => corrupt(databasePath, reason)); if (decoded.size !== size) { corrupt(databasePath, "a DOFS manifest size does not equal its chunks"); } @@ -467,24 +454,11 @@ export function readDofsManifest( * whether these bytes are a canonically encoded DOFS manifest at all, and what * size the chunks it lists add up to. */ -export function decodeDofsManifest(encoded: Uint8Array, reject: WorkspaceRejection): DofsManifest { - let text: string; - let offered: unknown; - try { - text = decoder.decode(encoded); - offered = JSON.parse(text); - } catch { - reject("a DOFS manifest is not canonical UTF-8 JSON"); - } - const parsed = dofsManifestSchema.safeParse(offered); - if (!parsed.success || JSON.stringify(parsed.data) !== text) { - reject("a DOFS manifest is not canonically encoded"); - } - const total = parsed.data.chunks.reduce((sum, chunk) => sum + chunk.size, 0); - if (!Number.isSafeInteger(total)) { - reject("a DOFS manifest names more bytes than a size can hold"); - } - return Object.freeze({ size: total, chunks: Object.freeze(parsed.data.chunks) }); +export function decodeContentManifest( + encoded: Uint8Array, + reject: WorkspaceRejection, +): ContentManifest { + return decodeSharedContentManifest(encoded, reject); } function parseStoredRoot( @@ -517,12 +491,12 @@ function rootFromManifest( parsed: ReturnType, databasePath: string, ): StoredWorkspaceRoot { - const manifests = new Map(); + const manifests = new Map(); for (const entry of parsed.entries) { if (entry.kind === "file") { let manifest = manifests.get(entry.manifest); if (manifest === undefined) { - manifest = readDofsManifest(database, entry.manifest, databasePath); + manifest = readContentManifest(database, entry.manifest, databasePath); manifests.set(entry.manifest, manifest); } if (entry.size !== manifest.size) { @@ -576,7 +550,7 @@ function validateFile( corrupt(databasePath, "a Workspace file has an invalid DOFS manifest identity"); } const manifest = toHex(manifestHash); - const encoded = readDofsManifest(database, manifest, databasePath); + const encoded = readContentManifest(database, manifest, databasePath); if ( encoded.size !== node.size || !equalChunks( @@ -616,7 +590,7 @@ function validateDofsContentStore(database: DatabaseSync, databasePath: string): if (hash.byteLength !== 32) { corrupt(databasePath, "a DOFS manifest has an invalid hash length"); } - readDofsManifest(database, toHex(hash), databasePath); + readContentManifest(database, toHex(hash), databasePath); } } diff --git a/packages/workflow/src/remote/client.ts b/packages/workflow/src/remote/client.ts new file mode 100644 index 000000000..d03d2e152 --- /dev/null +++ b/packages/workflow/src/remote/client.ts @@ -0,0 +1,385 @@ +/** + * The runner's side of the connection to its owner. + * + * One connection, one acquisition, and one request in flight at a time per + * command id. Requests and answers are correlated explicitly rather than by + * arrival order, because a socket delivers what the owner sent whenever it + * sent it, and a client that assumed order would attribute one command's + * refusal to another. + * + * Nothing here decides anything about the run. It carries a question to the + * owner and hands back what the owner said — including a refusal, which is an + * answer rather than a transport failure. What the owner does with a command is + * the owner's, and a client that interpreted a refusal would be a second place + * deciding what a run may do. + * + * What it will not do is carry on after the two sides disagree about which + * command completed. An answer it cannot read, an answer naming a request + * nobody made, and a second answer to a request already settled are each + * evidence that correlation has broken — and a commit may have landed on the + * owner while the caller waits for a reply that will never be attributed. So + * the channel fails closed: it stops, and every waiter learns, rather than + * dropping the answer and leaving somebody blocked forever. + */ + +import { ensure, type Operation, resource, withResolvers } from "effection"; + +/** + * The most bytes one message on this connection may carry. + * + * The bound is the whole message as it crosses, in both directions. Measuring + * one member of a request instead would let a request that clears the check + * still be too large once its correlation and framing are added. + */ +export const MAX_MESSAGE_BYTES = 8 * 1024 * 1024; + +/** Why the connection itself could not carry a request. */ +export type LinkRefusal = + | "closed" + | "malformed-answer" + | "unknown-answer" + | "duplicate-answer" + | "too-large" + | "malformed-request" + | "send-failed" + | "socket-error"; + +/** + * A parser's own failure, as something that can be settled and reported. + * + * A parser may throw anything. What travels back to the caller has to be an + * `Error`, and it has to stay the parser's failure rather than becoming the + * channel's, so the boundary that knows what the value meant can classify it. + */ +function unreadable(error: unknown): Error { + return error instanceof Error ? error : new OwnerLinkError("malformed-answer"); +} + +export class OwnerLinkError extends Error { + override name = "OwnerLinkError"; + + constructor(readonly refusal: LinkRefusal) { + super(`the connection to this run's owner cannot carry the request (${refusal})`); + } +} + +/** + * What the owner answered, once the caller's own parser has read the value. + * + * `T` is what the request asked for. A performed answer carries a parsed value + * and never an `unknown`: the JSON boundary is inside this module, and letting + * it out would make every consumer responsible for remembering to parse — which + * is the kind of thing that is remembered until it is not. + */ +export type OwnerAnswer = + | { readonly outcome: "performed"; readonly value: T } + | { readonly outcome: "refused"; readonly refusal: string }; + +/** + * How a request reads its own success value. + * + * Supplied with the request, because what a performed answer means is the + * command's business rather than the connection's. Raising is how it says the + * owner sent something this build cannot read. + */ +export type AnswerParser = (value: unknown) => T; + +/** One listener, kept so teardown can remove the exact callback it installed. */ +export type SocketListener = (event: { data?: unknown }) => void; + +/** The socket shape this client needs, so a test can supply one. */ +export interface OwnerSocket { + send(data: string): void; + close(): void; + addEventListener(type: "message" | "close" | "error", listener: SocketListener): void; + removeEventListener(type: "message" | "close" | "error", listener: SocketListener): void; +} + +/** One live connection to a run's owner. */ +export interface OwnerConnection { + /** + * Send one command and wait for the answer that names it. + * + * `parse` reads the success value. If it raises, the channel fails closed + * like any other disagreement about what completed — a value neither side + * agrees on is not something to hand a caller and carry on from. + */ + ask( + id: string, + command: Record, + parse: AnswerParser, + parseRefusal?: (refusal: string) => string, + ): Operation>; +} + +/** The most bytes one answer may carry. */ +const MAX_ANSWER = 8 * 1024 * 1024; + +/** The longest correlation id, in either direction. */ +const MAX_ID = 128; + +/** + * The longest refusal this reads. + * + * Small and its own bound: a refusal is a category, and the eight-megabyte + * envelope bound is for a command's payload rather than for a word. + */ +const MAX_REFUSAL = 200; + +/** Whether a correlation id is one this client will send or accept. */ +function usableId(value: unknown): value is string { + return typeof value === "string" && value !== "" && value.length <= MAX_ID; +} + +/** + * The shape a refusal category has. + * + * The owner answers with a category and an optional detail. This proves + * *spelling* and nothing more — a syntactically valid category this build has + * never heard of still passes here. Narrowing a refusal to the exact declared + * union is the Cloudflare adapter's job, where the union is known; what this + * bound is for is stopping an arbitrary remote sentence from travelling as + * though it were a category at all. + */ +const REFUSAL = /^[a-z][a-z0-9-]*(:[a-z][a-z0-9-]*)?$/; + +/** The envelope, before the caller's parser reads the value inside it. */ +type RawAnswer = + | { readonly outcome: "performed"; readonly value: unknown } + | { readonly outcome: "refused"; readonly refusal: string }; + +function readAnswer(raw: unknown): { id: string; answer: RawAnswer } { + if (typeof raw !== "string") { + throw new OwnerLinkError("malformed-answer"); + } + if (raw.length > MAX_ANSWER) { + throw new OwnerLinkError("too-large"); + } + let decoded: unknown; + try { + decoded = JSON.parse(raw); + } catch { + throw new OwnerLinkError("malformed-answer"); + } + if (decoded === null || typeof decoded !== "object" || Array.isArray(decoded)) { + throw new OwnerLinkError("malformed-answer"); + } + const members: Map = new Map(Object.entries(decoded)); + const id = members.get("id"); + const outcome = members.get("outcome"); + if (!usableId(id)) { + throw new OwnerLinkError("malformed-answer"); + } + + // Each branch declares its whole key set. A performed answer carrying a + // `refusal`, or a refused one carrying a `value`, is an answer the two sides + // disagree about the shape of — which is the thing this channel refuses to + // carry on past. + const declared = + outcome === "performed" ? ["id", "outcome", "value"] : ["id", "outcome", "refusal"]; + if (members.size !== declared.length) { + throw new OwnerLinkError("malformed-answer"); + } + for (const key of members.keys()) { + if (!declared.includes(key)) { + throw new OwnerLinkError("malformed-answer"); + } + } + + if (outcome === "performed") { + return { id, answer: { outcome, value: members.get("value") } }; + } + if (outcome === "refused") { + const refusal = members.get("refusal"); + if (typeof refusal !== "string" || refusal.length > MAX_REFUSAL || !REFUSAL.test(refusal)) { + throw new OwnerLinkError("malformed-answer"); + } + return { id, answer: { outcome, refusal } }; + } + throw new OwnerLinkError("malformed-answer"); +} + +/** + * Hold one connection open for the calling scope. + * + * The connection *is* the executor acquisition, so the scope that owns it owns + * ending it: there is no lease to expire and no heartbeat to miss, and an owner + * that still sees a healthy socket still considers this runner the executor. A + * scope that walked away without closing would leave the run unadvanceable by + * anybody, forever. + * + * So teardown is one operation with one owner. Scope exit, cancellation, a + * remote close, a socket error, a protocol failure and a failed send all reach + * it, it runs once, and it removes the exact listeners it installed and closes + * the socket. The failure that caused it is what the waiters are told — a close + * arriving afterwards must not rewrite `malformed-answer` into `closed`. + */ +export function useOwnerConnection(socket: OwnerSocket): Operation { + return resource(function* (provide) { + /** + * One waiting request, as the reader sees it. + * + * The command's own type stays inside the closure `ask()` built, so the + * reader settles an answer without naming it and nothing here has to assert + * what a value is. `deliver` settles the request either way and returns the + * failure that made an answer unreadable, so the caller that asked learns + * what was wrong with its own answer rather than only that the channel + * ended. + */ + interface Waiter { + deliver(answer: RawAnswer): Error | undefined; + fail(error: OwnerLinkError): void; + } + const waiting = new Map(); + /** Requests already answered, so a second answer is recognized as one. */ + const settled = new Set(); + let closed = false; + let torn = false; + + /** + * Read one incoming answer and settle the request it names. + * + * Synchronous, and deliberately so. If this queued the message and read it + * later, a close arriving in the same turn would reach teardown first and + * the caller would be told `closed` for an answer that was actually + * unreadable. What went wrong is decided where it is observed. + */ + const onMessage: SocketListener = (event) => { + if (torn) { + return; + } + let read: { id: string; answer: RawAnswer }; + try { + read = readAnswer(event.data); + } catch (error) { + // The owner said something this build cannot read. Whether it was meant + // for a waiter is exactly what cannot be established. + teardown(error instanceof OwnerLinkError ? error.refusal : "malformed-answer"); + return; + } + const pending = waiting.get(read.id); + if (pending === undefined) { + // Either a request nobody made, or a second answer to one already + // settled. Both mean the two sides disagree about what completed. + teardown(settled.has(read.id) ? "duplicate-answer" : "unknown-answer"); + return; + } + waiting.delete(read.id); + settled.add(read.id); + if (pending.deliver(read.answer) !== undefined) { + // The owner performed the command and described the result in a way + // this build cannot read. Handing the caller an unparsed value is the + // one outcome that must not happen — but the caller that asked has + // already been told why, so teardown here is about everyone else. + teardown("malformed-answer"); + } + }; + const onClose: SocketListener = () => teardown("closed"); + const onError: SocketListener = () => teardown("socket-error"); + + /** + * End the connection, once. + * + * `refusal` is what the waiters are told. The first caller decides it: a + * remote close after a malformed answer is the same teardown, and the + * caller waiting on that answer should learn what actually went wrong. + */ + function teardown(refusal: LinkRefusal): void { + if (torn) { + return; + } + torn = true; + closed = true; + for (const pending of waiting.values()) { + pending.fail(new OwnerLinkError(refusal)); + } + waiting.clear(); + socket.removeEventListener("message", onMessage); + socket.removeEventListener("close", onClose); + socket.removeEventListener("error", onError); + try { + socket.close(); + } catch { + // Already closed, or closing threw on the way out. Either way this + // connection is over and there is nothing left to tell anybody. + } + } + + socket.addEventListener("message", onMessage); + socket.addEventListener("close", onClose); + socket.addEventListener("error", onError); + // Registered before anything can suspend, so a cancellation between here + // and `provide()` still closes the socket it just started listening to. + yield* ensure(() => { + teardown("closed"); + }); + + yield* provide({ + *ask( + id: string, + command: Record, + parse: AnswerParser, + parseRefusal: (refusal: string) => string = (refusal) => refusal, + ): Operation> { + if (closed) { + throw new OwnerLinkError("closed"); + } + // The same contract an incoming answer is held to. An id this client + // would refuse to read must never be one it sends. + if (!usableId(id)) { + throw new OwnerLinkError("malformed-request"); + } + if (waiting.has(id) || settled.has(id)) { + throw new OwnerLinkError("duplicate-answer"); + } + // Exactly what would be written, correlation and framing included, and + // measured before this request is registered as outstanding. A request + // too large to carry never becomes one the caller is waiting on. + const raw = JSON.stringify({ ...command, id }); + if (new TextEncoder().encode(raw).length > MAX_MESSAGE_BYTES) { + throw new OwnerLinkError("too-large"); + } + const settle = withResolvers>(); + waiting.set(id, { + deliver(answer: RawAnswer): Error | undefined { + if (answer.outcome === "refused") { + let refusal: string; + try { + refusal = parseRefusal(answer.refusal); + } catch (error) { + settle.reject(unreadable(error)); + return unreadable(error); + } + settle.resolve({ outcome: "refused", refusal }); + return undefined; + } + let value: T; + try { + value = parse(answer.value); + } catch (error) { + // The request that asked learns why its own answer could not be + // read. Whoever else is waiting learns the channel ended, which + // is all that is true for them. + settle.reject(unreadable(error)); + return unreadable(error); + } + settle.resolve({ outcome: "performed", value }); + return undefined; + }, + fail(error: OwnerLinkError): void { + settle.reject(error); + }, + }); + try { + socket.send(raw); + } catch { + // The socket refused the write. This request never left, and the + // connection cannot be trusted to carry the next one either. + teardown("send-failed"); + throw new OwnerLinkError("send-failed"); + } + return yield* settle.operation; + }, + }); + }); +} diff --git a/packages/workflow/src/remote/collector.ts b/packages/workflow/src/remote/collector.ts new file mode 100644 index 000000000..6803ee1f2 --- /dev/null +++ b/packages/workflow/src/remote/collector.ts @@ -0,0 +1,355 @@ +/** + * `transact()` for a run whose storage is somewhere else. + * + * A Durable Object commits synchronously and cannot hold a transaction open + * across a network wait, so the obvious reading — open a remote transaction, + * run the caller's body, commit — is not available. What is available is that + * the body does not need the transaction to be open while it runs. It needs to + * read the starting history, it needs its writes to go somewhere, and it needs + * all of them to land together or not at all. + * + * So the callback runs here, in a runner-owned scope, against a collector. The + * starting frontier is read once through an ordinary bounded request that opens + * and closes its own read on the owner. Journal appends go into a local buffer + * that `readAll()` reads back after the starting prefix, so the body sees its + * own writes. Nothing is sent while the body is running. When the body and + * everything it started have torn down successfully, one closed intent goes to + * the owner, which revalidates and applies it inside its one transaction. + * + * The callback is never serialized, interpreted, or run inside the owner. It is + * ordinary code doing ordinary work; only what it *enlisted* crosses the + * connection. That is what makes arbitrary control flow safe here — nothing + * tries to infer what the body did. + */ + +import { call, ensure, Ok, type Operation, type Result, scoped } from "effection"; +import type { CommitDecision, RetainedMapping, WorkspacePublication } from "./publication.ts"; +import { SEAL, type SealableAttempt } from "./seal.ts"; +import type { DurableEvent } from "@executablemd/durable-streams"; +import type { DurableStream } from "@executablemd/durable-streams"; +import type { WorkflowRunTransaction } from "../storage/api.ts"; + +/** Why a transaction could not be run or committed. */ +export type CollectorRefusal = + | "nested-transaction" + | "publication-already-enlisted" + | "too-many-mappings" + | "transaction-closed" + | "operation-inside-body" + | "too-many-events" + | "events-too-large" + | "malformed-event"; + +export class RemoteTransactionError extends Error { + override name = "RemoteTransactionError"; + + constructor(readonly refusal: CollectorRefusal) { + super(`this remote transaction cannot proceed (${refusal})`); + } +} + +/** The starting state a transaction is proposed against. */ +export interface StartingFrontier { + readonly workspaceRootId: string; + readonly journalEventId: string | null; + readonly events: readonly DurableEvent[]; +} + +/** + * One closed intent, as the owner will receive it. + * + * Everything the transaction decided, and nothing it did not. `publication` is + * absent for a transaction that only appended to the journal — a real case, and + * inventing a Workspace change to make the shape uniform would publish a root + * nobody asked for. + */ +export interface CommitIntent { + readonly expectedWorkspaceRootId: string; + readonly expectedJournalEventId: string | null; + readonly events: readonly DurableEvent[]; + readonly publication: WorkspacePublication | null; + readonly mappings: readonly RetainedMapping[]; + /** The sealed bytes for the pieces this proposal may have to supply. */ + readonly bytes: ReadonlyMap; +} + +/** What the collector needs from the connection. */ +export interface OwnerLink { + /** One bounded read that opens and closes its own owner-side read. */ + frontier(): Operation; + /** One closed intent, applied atomically or not at all. */ + commit(intent: CommitIntent): Operation>; +} + +/** The most events one intent may carry. */ +const MAX_EVENTS = 4096; + +/** The most serialized bytes one intent may carry. */ +const MAX_EVENT_BYTES = 4 * 1024 * 1024; + +/** The most retained mapping changes one intent may carry. */ +const MAX_MAPPINGS = 256; + +/** + * Admit one event and detach it from whoever handed it over. + * + * Cloning on the way in is not enough on its own: a caller that reads an event + * back and mutates what it received would otherwise change what this + * transaction commits. So every crossing — in, out, and into the intent — is a + * fresh copy, and the collector's own array is never handed to anybody. + */ +function admitEvent(event: DurableEvent): DurableEvent { + if (event === null || typeof event !== "object") { + throw new RemoteTransactionError("malformed-event"); + } + if (!("type" in event) || typeof event.type !== "string") { + throw new RemoteTransactionError("malformed-event"); + } + try { + return structuredClone(event); + } catch { + // A value that cannot be cloned cannot be sent either. + throw new RemoteTransactionError("malformed-event"); + } +} + +/** The serialized size of what has been collected so far. */ +function serializedBytes(events: readonly DurableEvent[]): number { + return new TextEncoder().encode(JSON.stringify(events)).length; +} + +/** + * Whether a transaction is open on this handle. + * + * Scope-local rather than global: two runs may transact at once, and what must + * not happen is a second transaction — or an ordinary operation — on the *same* + * handle from inside a body. That is the same refusal the local provider makes, + * and for the same reason: work that never received the transaction handle + * would otherwise commit on its own, outside the unit of work it appears to be + * part of. + */ +export interface TransactionGate { + open: boolean; +} + +export function createTransactionGate(): TransactionGate { + return { open: false }; +} + +/** Refuse an ordinary same-handle operation while a body is running. */ +export function requireNoOpenTransaction(gate: TransactionGate): void { + if (gate.open) { + throw new RemoteTransactionError("operation-inside-body"); + } +} + +/** + * Run `body` against a collector, then submit what it enlisted. + * + * The body may compute, suspend and perform runner-owned effects. None of that + * is reduced to an intent and none of it executes on the owner; only mutations + * made through the transaction handle enter the collector. + */ +export function transactRemotely( + link: OwnerLink, + gate: TransactionGate, + body: ( + transaction: WorkflowRunTransaction, + enlist: EnlistWorkspace, + anchor: TransactionAnchor, + ) => Operation, +): Operation> { + return call(function* (): Operation> { + // Taken synchronously, before the first suspension. Checking and then + // suspending in `frontier()` would let two calls on one handle both pass + // the check and act from the same starting frontier. + if (gate.open) { + throw new RemoteTransactionError("nested-transaction"); + } + gate.open = true; + // Released once, and only after nothing from this transaction can still + // affect the handle — which is after the commit answer, not after the body. + // Between those two the outcome is undecided, and later work must not run + // as though it had been decided. + try { + return yield* run(); + } finally { + gate.open = false; + } + + function* run(): Operation> { + const starting = yield* link.frontier(); + const appended: DurableEvent[] = []; + let live = true; + + const journal: DurableStream = { + *readAll(): Operation { + if (!live) { + throw new RemoteTransactionError("transaction-closed"); + } + // Read-your-writes, as fresh copies. The starting prefix then this + // transaction's own appends, in order. + return [...starting.events, ...appended].map((event) => structuredClone(event)); + }, + *append(event: DurableEvent): Operation { + if (!live) { + throw new RemoteTransactionError("transaction-closed"); + } + if (appended.length >= MAX_EVENTS) { + throw new RemoteTransactionError("too-many-events"); + } + const admitted = admitEvent(event); + if (serializedBytes([...appended, admitted]) > MAX_EVENT_BYTES) { + throw new RemoteTransactionError("events-too-large"); + } + appended.push(admitted); + }, + }; + + let enlisted: { attempt: SealableAttempt; mappings: readonly RetainedMapping[] } | undefined; + /** + * How a Workspace operation puts its result into this transaction. + * + * Private: it is handed to the body rather than reachable from the + * database, so work that never received it cannot publish a Workspace by + * accident. Detached on the way in, because the caller still holds the + * arrays and records it passed and a proposal that changed after it was + * admitted would not be the proposal the identity was computed over. + */ + const enlist: EnlistWorkspace = ( + attempt: SealableAttempt, + mappings: readonly RetainedMapping[] = [], + ): void => { + if (!live) { + throw new RemoteTransactionError("transaction-closed"); + } + if (enlisted !== undefined) { + throw new RemoteTransactionError("publication-already-enlisted"); + } + if (mappings.length > MAX_MAPPINGS) { + throw new RemoteTransactionError("too-many-mappings"); + } + // The mappings are detached now, because they are the caller's values. + // The Workspace itself is not read until sealing. + enlisted = { attempt, mappings: Object.freeze(mappings.map(detachMapping)) }; + }; + + let outcome: T; + try { + // A scope of its own, closed here. Everything the body started — + // spawned children, resources — has finished tearing down before the + // intent is built, so "no commit was sent" and "the body did not + // finish" are one statement. `call()` alone would let a resource whose + // teardown fails surface its failure after the commit had already gone + // out, which is the one ordering that cannot be taken back. + outcome = yield* scoped(() => + body({ journal }, enlist, { + workspaceRootId: starting.workspaceRootId, + journalEventId: starting.journalEventId, + }), + ); + } finally { + // The handle is closed before the commit goes out, so a retained + // transaction object refuses while the handle-level gate is still held. + live = false; + } + + // Sealed after teardown: the proposal is the tree as it finally is. + const sealed = + enlisted === undefined ? undefined : yield* enlisted.attempt[SEAL](enlisted.mappings); + + const committed = yield* link.commit({ + expectedWorkspaceRootId: starting.workspaceRootId, + expectedJournalEventId: starting.journalEventId, + // A private snapshot. The collector's own array never leaves. + events: appended.map((event) => structuredClone(event)), + publication: sealed?.publication ?? null, + mappings: sealed?.mappings ?? [], + bytes: sealed?.bytes ?? new Map(), + }); + if (!committed.ok) { + return committed; + } + // The owner performed this exact proposal, so the attempt that produced + // it becomes the accepted Workspace — here, inside the operation that + // received and validated the answer, rather than by handing the answer + // to a caller and trusting the sequence. + if (sealed !== undefined) { + yield* sealed.transfer(committed.value); + } + // Only now. `T` is the body's own value and never crossed the connection. + return Ok(outcome); + } + }); +} + +/** + * Exactly where this transaction began, and nothing else about it. + * + * A coordinator has to prove that the state it admitted its invocation from is + * the state this transaction will commit against. It needs the two anchors for + * that and no more — the journal prefix is the body's to read through the + * transaction, not something a route hands out. + */ +export interface TransactionAnchor { + readonly workspaceRootId: string; + readonly journalEventId: string | null; +} + +/** + * How a Workspace operation designates its attempt for publication. + * + * It names an attempt rather than handing over a proposal. What the attempt + * holds is captured when the transaction seals it — after the body and + * everything it started have finished — so the proposal always describes the + * tree as it finally is, and the tree the owner decides is the tree that gets + * transferred. There is no way to enlist a Workspace that no live attempt owns, + * which is what stops a durable commit from leaving the invocation behind. + */ +export type EnlistWorkspace = ( + attempt: SealableAttempt, + mappings?: readonly RetainedMapping[], +) => void; + +/** + * A copy nobody else holds a reference into. + * + * The caller keeps whatever it passed, and may go on using it. What the intent + * carries has to be what was admitted at the moment it was admitted — a + * publication whose inventory or manifest changed afterwards would not be the + * one its identity was computed over. + */ +/** + * One mapping, copied all the way down. + * + * A shallow copy is not enough: an Agent-session record holds its provider + * assertion as a nested object, and that assertion is part of the retained + * identity. Leaving it shared would let a caller change what the run recorded + * about a session after the transaction had sealed. + */ +function detachMapping(mapping: RetainedMapping): RetainedMapping { + if (mapping.kind === "repository") { + return Object.freeze({ + kind: mapping.kind, + locator: mapping.locator, + record: Object.freeze({ ...mapping.record }), + }); + } + if (mapping.kind === "worktree") { + return Object.freeze({ kind: mapping.kind, record: Object.freeze({ ...mapping.record }) }); + } + return Object.freeze({ + kind: mapping.kind, + record: Object.freeze({ + ...mapping.record, + assertion: Object.freeze({ ...mapping.record.assertion }), + }), + }); +} + +/** Discard a collector's work without sending it. */ +export function abandon(gate: TransactionGate): Operation { + return ensure(() => { + gate.open = false; + }); +} diff --git a/packages/workflow/src/remote/database.ts b/packages/workflow/src/remote/database.ts new file mode 100644 index 000000000..4967fb3d2 --- /dev/null +++ b/packages/workflow/src/remote/database.ts @@ -0,0 +1,434 @@ +/** + * One run's storage, when the run is owned somewhere else. + * + * The same handle the local host hands out, backed by a connection instead of a + * file. Everything the interface promises has to be true here for the same + * reasons it is true there — a snapshot is a snapshot, a transaction commits or + * it does not, and a closed handle is closed — and the differences are all + * beneath it: there is no connection to hold open across a callback, so the + * body runs on the runner and only what it enlisted crosses. + * + * Two mechanisms keep operations in order and they solve different problems. + * A *turn* serializes work so two operations do not interleave on one handle; + * unrelated work waits and then proceeds. A *marker* records that this scope is + * inside a transaction on this handle, so a nested transaction — or an ordinary + * operation called from inside the body — is refused immediately rather than + * waiting for a turn its own caller is holding and will not release. A queue + * alone would deadlock that case; a flag alone would mistake unrelated work for + * nested work. + * + * The handle is a lease. Closing it ends this handle and nothing else: the + * connection may be owned by an outer scope and shared with other handles, and + * a lease that closed it would end a run somebody else was still reading. + */ + +import { + createContext, + createSignal, + ensure, + Err, + Ok, + type Context, + type Operation, + type Result, + resource, +} from "effection"; +import type { DurableEvent, DurableStream, Json } from "@executablemd/durable-streams"; +import type { JournalEntry, WorkflowRunDatabase, WorkflowRunTransaction } from "../storage/api.ts"; +import { + WorkflowDatabaseClosedError, + WorkflowRecordMalformedError, + WorkflowRequestError, + WorkflowStorageError, + WorkflowTransactionError, +} from "../storage/errors.ts"; +import { parseJsonValue } from "../storage/members.ts"; +import type { + DefinitionRetrieval, + DocumentExecutionRecord, + WorkflowRunRecord, +} from "../storage/record.ts"; +import { createTransactionGate, type OwnerLink, transactRemotely } from "./collector.ts"; +import type { EnlistWorkspace, TransactionAnchor } from "./collector.ts"; +import type { RemoteContent, RemoteContentRequest, RemoteFrontierSnapshot } from "./read.ts"; +import type { RemoteInvocationSnapshot } from "./records.ts"; +import type { WorkspaceRootManifest } from "../workspace/root-manifest.ts"; + +/** What a remote handle needs to answer everything the interface asks. */ +export interface RemoteRunLink extends OwnerLink { + /** A fresh coherent frontier, for a read that must not use a snapshot. */ + frontierSnapshot(): Operation; + /** Replace or clear the retrieval metadata, and answer with the result. */ + replaceRetrieval( + expectedWorkspaceRootId: string, + metadata: string | null, + ): Operation>; + /** Every document execution, as one anchored snapshot. */ + readExecutions(): Operation>; +} + +/** + * Everything one remote run is reached through, as one value. + * + * The Workspace reads and the commits are the same authority, so they are the + * same object. Carried as two — a link and a read link a caller supplies + * separately — they can be taken from two owners: an invocation would then + * execute against one run's retained mappings and content and commit the + * result to another, and if the two began at the same root and anchor nothing + * downstream could notice. There is no such pair to make. + */ +export interface RemoteWorkspaceLink extends RemoteRunLink { + /** The one coherent admitted state a Workspace invocation begins from. */ + invocationSnapshot(): Operation; + root(workspaceRootId: string): Operation; + content(workspaceRootId: string, request: RemoteContentRequest): Operation; +} + +/** + * Which handles this scope is inside a transaction on. + * + * Structural and inert, exactly like the local provider's: it can only ever + * cause an operation to be refused, never authorize one. A chain rather than a + * single handle, because transactions on *different* runs may nest and + * recording only the innermost would hide the outer one. + */ +interface OpenTransaction { + readonly handle: object; + readonly enclosing: OpenTransaction | undefined; +} + +const ActiveTransaction: Context = createContext< + OpenTransaction | undefined +>("executablemd.workflow.remote.transaction", undefined); + +function* holdsTransactionOn(handle: object): Operation { + let active = yield* ActiveTransaction.get(); + while (active !== undefined) { + if (active.handle === handle) { + return true; + } + active = active.enclosing; + } + return false; +} + +/** + * The route a Workspace coordinator reaches the active transaction through. + * + * Bound to one exact handle and one exact transaction object, and live only + * inside that transaction body's descendant scope. D3c installs a coordinator + * over it; nothing about a document execution or its provenance is decided + * here, and no placeholder for either is invented. + */ +export interface WorkspaceRoute { + readonly database: WorkflowRunDatabase; + readonly transaction: WorkflowRunTransaction; + readonly enlist: EnlistWorkspace; + /** Where this transaction began, so a coordinator can prove it has not drifted. */ + readonly anchor: TransactionAnchor; +} + +const ActiveRoute: Context = createContext( + "executablemd.workflow.remote.workspace-route", + undefined, +); + +/** + * The enlistment route for this exact database and transaction, if it is live. + * + * Answers nothing for a foreign database, a substituted or stale transaction + * object, or a scope outside the body — which is the whole point: a coordinator + * that has drifted from the transaction it belongs to must not be able to + * publish into it. + */ +export function* activeWorkspaceRoute( + database: WorkflowRunDatabase, + transaction: WorkflowRunTransaction, +): Operation { + const route = yield* ActiveRoute.get(); + if (route === undefined || route.database !== database || route.transaction !== transaction) { + return undefined; + } + return route; +} + +/** + * A failure this interface can return, whatever it arrived as. + * + * The adapter beneath has already translated what it knows about; anything else + * reaching here is the body's own error, which is carried as it is. A value + * that is not an error at all becomes one rather than travelling as a thrown + * string nobody can act on. + */ +function failure(error: unknown): Error { + return error instanceof Error ? error : new WorkflowTransactionError(String(error)); +} + +/** + * What a `DurableStream` member does with a result. + * + * The interface splits these deliberately: a member returning `Result` answers + * with the failure, and a stream member raises it. Both describe the same + * condition. + */ +function* raising(result: Result): Operation { + if (!result.ok) { + throw result.error; + } + return result.value; +} + +/** One handle's cooperative turn, so two operations never interleave on it. */ +interface Turns { + take(body: () => Operation): Operation; +} + +function createTurns(): Turns { + const waiting = createSignal(); + const holder = { held: false }; + return { + *take(body: () => Operation): Operation { + while (holder.held) { + // Someone else has the handle. Wait to be told it is free rather than + // polling, and check again, because several may be waiting and only one + // of them can take the turn that was just released. + const released = yield* waiting; + yield* released.next(); + } + holder.held = true; + try { + return yield* body(); + } finally { + holder.held = false; + waiting.send(); + } + }, + }; +} + +/** Open one scope-owned lease on a run whose storage is somewhere else. */ +export function useRemoteRunDatabase( + link: RemoteRunLink, + frontier: RemoteFrontierSnapshot, +): Operation { + return resource(function* (provide) { + let closed = false; + let record: WorkflowRunRecord = frontier.record; + let retrieval: DefinitionRetrieval | undefined = frontier.retrieval; + const turns = createTurns(); + const gate = createTransactionGate(); + + /** Whether this scope may reach the handle at all, and why not. */ + function* admit(): Operation> { + if (closed) { + return Err(new WorkflowDatabaseClosedError(record.runId)); + } + if (yield* holdsTransactionOn(handle)) { + return Err( + new WorkflowTransactionError( + "this scope is inside a transaction on the same workflow run database, and an " + + "operation outside that transaction cannot run until it commits. Use the " + + "transaction handed to the body, or move the operation outside it.", + ), + ); + } + return Ok(); + } + + /** + * One turn at the handle, for an ordinary operation. + * + * A member that returns `Result` answers with the failure rather than + * raising it, so a link that raised is caught here. Cancellation is not a + * failure and is left to unwind as control flow. + */ + function* turn(body: () => Operation>): Operation> { + const admitted = yield* admit(); + if (!admitted.ok) { + return admitted; + } + return yield* turns.take(function* (): Operation> { + try { + return yield* body(); + } catch (error) { + return Err(failure(error)); + } + }); + } + + const ordinary: DurableStream = { + *readAll(): Operation { + return yield* raising( + yield* turn(function* () { + const snapshot = yield* link.frontierSnapshot(); + return Ok(snapshot.entries.map((entry) => structuredClone(entry.event))); + }), + ); + }, + + *append(event: DurableEvent): Operation { + // One journal-only transaction through the same commit path a caller's + // transaction uses. A second insertion route would be a second thing to + // keep in agreement with the first. + yield* raising( + yield* transact(function* (transaction) { + yield* transaction.journal.append(event); + }), + ); + }, + }; + + function* transact( + body: (transaction: WorkflowRunTransaction) => Operation, + ): Operation> { + if (closed) { + return Err(new WorkflowDatabaseClosedError(record.runId)); + } + if (yield* holdsTransactionOn(handle)) { + return Err( + new WorkflowTransactionError( + "a transaction on this workflow run database is already open in this scope. " + + "Nesting one inside another would commit or roll back work the outer " + + "transaction has not finished deciding about.", + ), + ); + } + return yield* turns.take(function* (): Operation> { + try { + return yield* transactRemotely(link, gate, function* (transaction, enlist, anchor) { + // The marker and the route are installed for the body's scope + // alone. Outside it neither exists, so a retained transaction + // object reaches nothing and an unrelated scope is not mistaken for + // a nested one. + yield* ActiveTransaction.set({ + handle, + enclosing: yield* ActiveTransaction.get(), + }); + yield* ActiveRoute.set({ database: handle, transaction, enlist, anchor }); + return yield* body(transaction); + }); + } catch (error) { + // A body that raised, or a resource of its that failed to tear down, + // is a failed transaction rather than a raised one: the interface + // answers with a `Result`, and nothing was committed. + return Err(failure(error)); + } + }); + } + + const handle: WorkflowRunDatabase = { + get record(): WorkflowRunRecord { + return record; + }, + + get retrieval(): DefinitionRetrieval | undefined { + return retrieval; + }, + + get journal(): DurableStream { + return ordinary; + }, + + transact, + + *readJournalEntries(): Operation> { + return yield* turn(function* () { + const snapshot = yield* link.frontierSnapshot(); + return Ok(snapshot.entries.map((entry) => Object.freeze({ ...entry }))); + }); + }, + + *replaceRetrievalMetadata(metadata: Json | undefined): Operation> { + let encoded: string | null; + try { + // Parsed by the same rules a stored value is held to, then encoded + // canonically. A value that is not JSON at all never becomes a + // request: refusing it here is what "no request" means. + encoded = + metadata === undefined + ? null + : canonical(parseJsonValue(metadata, "$", retrievalFailure)); + } catch (error) { + return Err(failure(error)); + } + const replaced = yield* turn(function* () { + const snapshot = yield* link.frontierSnapshot(); + return yield* link.replaceRetrieval(snapshot.workspaceRootId, encoded); + }); + if (!replaced.ok) { + return replaced; + } + // The answer has to describe the replacement that was asked for. An + // owner that returned different metadata would otherwise install the + // location a later fetch of the definition would use. + const answered = replaced.value; + if (encoded === null) { + if (answered !== undefined) { + return Err(contradiction()); + } + } else if (answered === undefined || canonical(answered.metadata) !== encoded) { + return Err(contradiction()); + } + // Only this handle, and only after its own successful replacement. The + // owner's revision and time are what is recorded; nothing is invented + // here. + retrieval = answered; + return Ok(); + }, + + *readDocumentExecutions(): Operation> { + return yield* turn(() => link.readExecutions()); + }, + }; + + yield* ensure(() => { + closed = true; + }); + yield* provide(handle); + }); +} + +/** How a malformed retrieval value is reported, before anything is sent. */ +function retrievalFailure(reason: string, path: string): Error { + return new WorkflowRequestError( + `this retrieval metadata is not a JSON value storage can keep at ${path}: ${reason}.`, + ); +} + +/** An answer that does not describe the replacement it answered. */ +function contradiction(): WorkflowStorageError { + return new WorkflowRecordMalformedError( + "retrieval this run's owner returned", + "it does not describe the replacement that was asked for", + ); +} + +/** + * The canonical encoding of one retrieval metadata value. + * + * Sorted keys and no incidental whitespace, so two callers writing the same + * metadata write the same bytes and a comparison of what is stored means what + * it appears to mean. + */ +function canonical(value: Json): string { + return JSON.stringify(sorted(value)); +} + +function sorted(value: Json): Json { + if (Array.isArray(value)) { + return value.map(sorted); + } + if (value === null || typeof value !== "object") { + return value; + } + const members: Record = {}; + const names = Object.keys(value); + names.sort(); + for (const key of names) { + const held = (value as Record)[key]; + if (held !== undefined) { + members[key] = sorted(held); + } + } + return members; +} diff --git a/packages/workflow/src/remote/invocation.ts b/packages/workflow/src/remote/invocation.ts new file mode 100644 index 000000000..923df77c7 --- /dev/null +++ b/packages/workflow/src/remote/invocation.ts @@ -0,0 +1,277 @@ +/** + * What a remote invocation owns on the runner, and when it lets go. + * + * Two trees, and the difference between them is the whole point. The + * *materialization* is the accepted root: what the owner last confirmed this + * run is at, restored so native tools can work in it. The *attempt* is where a + * mutation actually happens, and it is disposable by construction — until the + * owner performs the commit, nothing that happened in it has happened. + * + * Keeping them apart is what makes a documented failure ordinary. A Workspace + * effect that fails is a fact the run records against the root it started from, + * so the attempt is thrown away and the accepted tree is still exactly what the + * owner confirmed. Working directly in the accepted tree would mean a failed + * effect had already changed the only local copy of the run's Workspace, and + * the next attempt would start somewhere nobody chose. + * + * Both are Effection resources, so their lifetimes are their scopes'. Normal + * return, a raised failure, cancellation, a refusal from the owner and a lost + * response all leave nothing behind, because none of them skips teardown. Only + * a performed owner answer promotes an attempt, and promotion is a decision + * this module is told about rather than one it infers. + */ + +import { ensure, type Operation, resource } from "effection"; +import type { WorkspaceRejection } from "../workspace/root-manifest.ts"; +import { + captureWorkspace, + type CapturedWorkspace, + type HostPath, + materializeWorkspaceRoot, + type RunnerFiles, +} from "./materialize.ts"; +import type { RemoteReadLink } from "./read.ts"; +import type { CommitDecision, ProposedContent, RetainedMapping } from "./publication.ts"; +import { SEAL, type SealableAttempt, type SealedProposal } from "./seal.ts"; + +/** A directory this invocation owns for as long as it needs one. */ +export interface TemporaryTrees { + /** A fresh empty directory, removed when the calling scope ends. */ + create(purpose: string): Operation; + /** Remove one, before its scope would. */ + remove(path: string): Operation; +} + +/** + * The capability to move the accepted tree, which is not part of reading it. + * + * A symbol because a symbol cannot be written down by anyone who does not + * already have it. Everything that merely reads the Workspace receives a + * `Materialization` and can see no way to change which Workspace it is + * reading; only an attempt, created by this module, is handed the key. + */ +const ACCEPT: unique symbol = Symbol("executablemd.workflow.remote.accept"); + +/** The accepted local copy of the root the owner last confirmed. */ +export interface Materialization { + /** The root this tree is, as the owner confirmed it. */ + readonly workspaceRootId: string; + /** + * Where a logical Workspace path sits in the accepted tree. + * + * Resolved on each call rather than closed over one directory, because + * promotion replaces the tree: after it, this has to answer with the promoted + * bytes. A path captured once would keep pointing at the Workspace the run + * used to be at while the identity said otherwise. + */ + at(logical: string): string; +} + +/** The accepted materialization as this module alone sees it. */ +interface AcceptedMaterialization extends Materialization { + readonly [ACCEPT]: (next: { root: string; workspaceRootId: string }) => Operation; +} + +/** + * One disposable place to make a mutation, and the way to offer it. + * + * There is no `promote()`. Promotion is not something a caller does at the + * right moment; it is what happens inside the transaction when the owner + * performs the exact commit that proposed this attempt. An attempt offers a + * proposal, the transaction sends it, and only the transaction — holding the + * answer it just validated — transfers the tree. + */ +export interface Attempt extends SealableAttempt { + readonly at: HostPath; + /** What the attempt describes right now, captured and checked locally. */ + capture(): Operation; +} + +/** + * Restore the admitted root into a tree this invocation owns. + * + * The tree is created, filled and proved before anything else runs against it: + * `materializeWorkspaceRoot` refuses a host that cannot reproduce the retained + * modes, times or topology, so an invocation either has the Workspace the owner + * described or does not start. + */ +export function useMaterialization( + files: RunnerFiles, + trees: TemporaryTrees, + reads: RemoteReadLink, + workspaceRootId: string, + reject: WorkspaceRejection, +): Operation { + return resource(function* (provide) { + const root = yield* trees.create("accepted"); + yield* materializeWorkspaceRoot(files, reads, at(root), workspaceRootId, reject); + let accepted = { root, workspaceRootId }; + const materialization: AcceptedMaterialization = { + get workspaceRootId(): string { + return accepted.workspaceRootId; + }, + at(logical: string): string { + return at(accepted.root)(logical); + }, + *[ACCEPT](next: { root: string; workspaceRootId: string }): Operation { + const previous = accepted.root; + accepted = next; + // The tree the run used to be at is removed once nothing points at it. + // Leaving it would keep a second copy of the Workspace on disk that + // nothing can reach and nothing will clean up until the invocation ends. + yield* trees.remove(previous); + }, + }; + yield* provide(materialization); + }); +} + +/** + * The accepted materialization, with the capability an attempt needs. + * + * The declared type hides it, so this is where the two views meet. A value that + * did not come from `useMaterialization()` carries no such key and cannot be + * mistaken for one. + */ +function accepting(materialization: Materialization, reject: WorkspaceRejection) { + const accept = (materialization as Partial)[ACCEPT]; + if (accept === undefined) { + reject("this is not an accepted materialization this invocation owns"); + } + return accept; +} + +/** + * A disposable copy of the accepted tree, for one mutation. + * + * Materialized from the owner rather than copied from the accepted tree: the + * owner's copy is the one that is authoritative, and reading it again is how an + * attempt starts from what the run actually is rather than from whatever the + * last attempt happened to leave behind. + */ +export function useAttempt( + files: RunnerFiles, + trees: TemporaryTrees, + reads: RemoteReadLink, + materialization: Materialization, + reject: WorkspaceRejection, +): Operation { + return resource(function* (provide) { + const accept = accepting(materialization, reject); + const root = yield* trees.create("attempt"); + yield* materializeWorkspaceRoot( + files, + reads, + at(root), + materialization.workspaceRootId, + reject, + ); + + let transferred = false; + // Registered before the attempt is handed over, so every exit removes it — + // including the ones that never reach the end of the calling scope. + yield* ensure(function* () { + if (!transferred) { + yield* trees.remove(root); + } + }); + + yield* provide({ + at: at(root), + *capture(): Operation { + return yield* captureWorkspace(files, at(root), reject); + }, + + /** + * Seal this attempt into the proposal the owner will decide. + * + * Captured here, not earlier. The transaction calls this once the body + * and everything it started have torn down, so the proposal describes the + * tree as it finally is — a capture taken when the body enlisted could + * name one Workspace while the directory went on to hold another, and the + * owner would commit one root while the runner transferred different + * bytes under it. + */ + *[SEAL](mappings: readonly RetainedMapping[]): Operation { + if (transferred) { + reject("this attempt has already been sealed and transferred"); + } + const captured = yield* captureWorkspace(files, at(root), reject); + return { + publication: { + proposedWorkspaceRootId: captured.root.rootId, + proposedManifest: captured.root.manifest, + content: inventoryOf(captured), + }, + mappings, + bytes: bytesOf(captured), + *transfer(decision: CommitDecision): Operation { + if (transferred) { + reject("this attempt has already been transferred"); + } + if (decision.workspaceRootId !== captured.root.rootId) { + reject("the owner's decision names a root this attempt did not seal"); + } + transferred = true; + yield* accept({ root, workspaceRootId: captured.root.rootId }); + }, + }; + }, + }); + }); +} + +/** + * The exact closure a captured root names, in canonical order. + * + * Ordered by `kind:digest`, which is the order the owner reads it in: it holds + * the inventory to a strictly increasing sequence, because a proposal that + * named its pieces in another order is not the proposal whose identity the + * runner computed. Concatenating one kind after the other would produce a list + * this owner refuses, and only a real owner would say so. + */ +function inventoryOf(captured: CapturedWorkspace): ProposedContent[] { + const inventory: ProposedContent[] = [ + ...captured.root.manifests.map((digest) => ({ + kind: "manifest" as const, + digest, + size: captured.contents.get(digest)?.manifestBytes.length ?? 0, + })), + ...captured.root.blobs.map((digest) => ({ + kind: "blob" as const, + digest, + size: captured.blobs.get(digest)?.length ?? 0, + })), + ]; + return inventory.toSorted((left, right) => + orderingOf(left) < orderingOf(right) ? -1 : orderingOf(left) > orderingOf(right) ? 1 : 0, + ); +} + +function orderingOf(piece: ProposedContent): string { + return `${piece.kind}:${piece.digest}`; +} + +/** Every piece the capture can supply, by identity. */ +function bytesOf(captured: CapturedWorkspace): Map { + const bytes = new Map(); + for (const [digest, content] of captured.contents) { + bytes.set(digest, content.manifestBytes); + } + for (const [digest, blob] of captured.blobs) { + bytes.set(digest, blob); + } + return bytes; +} + +/** + * One logical Workspace path under a host directory. + * + * Kept here rather than imported from a path module because it is the only + * place the two vocabularies meet, and because a shared module may not name a + * host's path conventions. The logical root is `/` and everything under it is + * relative to the tree this invocation was given. + */ +function at(root: string): HostPath { + return (logical) => (logical === "/" ? root : `${root}/${logical.slice(1)}`); +} diff --git a/packages/workflow/src/remote/journal-route.ts b/packages/workflow/src/remote/journal-route.ts new file mode 100644 index 000000000..52755e988 --- /dev/null +++ b/packages/workflow/src/remote/journal-route.ts @@ -0,0 +1,94 @@ +/** + * Where a Workspace effect's publication goes. + * + * A durable operation publishes its result into the run's journal. When a + * Workspace effect is the thing publishing, that append has to land in the + * exact transaction the effect ran inside, so the Files change and the row + * describing it commit together or not at all. The ordinary journal would + * append outside the transaction, which is the one ordering that cannot be + * taken back. + * + * So the route is installed for one transaction's descendant scope, keyed to + * one exact database, transaction and token. Outside that scope the wrapper + * falls through to the ordinary journal, and a retained token reaches nothing. + */ + +import { type Api, createApi } from "@effectionx/context-api"; +import type { DurableEvent, DurableStream } from "@executablemd/durable-streams"; +import { ensure, type Operation, scoped } from "effection"; +import type { WorkflowRunDatabase, WorkflowRunTransaction } from "../storage/api.ts"; + +interface JournalDestinationApi { + append(database: WorkflowRunDatabase, event: DurableEvent): Operation; +} + +const RemoteJournalDestination: Api = createApi( + "executablemd.workflow.remote.journal.destination", + { + // deno-lint-ignore require-yield + *append(): Operation { + return false; + }, + }, +); + +/** + * Run `publication` with this exact transaction as the journal's destination. + * + * The transaction object is the capability. Only code inside the live + * transaction body holds one, and the caller has already proved through + * `activeWorkspaceRoute()` that this is that transaction — so there is nothing + * further to look up, and no registry to outlive the run. + * + * Scoped, so the redirection ends with the operation that needed it rather than + * outliving the transaction it names. `live` closes with that scope: an append + * arriving afterwards falls through to the ordinary journal instead of reaching + * a transaction that has closed. + */ +export function withRemoteJournalRoute( + database: WorkflowRunDatabase, + transaction: WorkflowRunTransaction, + publication: Operation, +): Operation { + return scoped(function* () { + let live = true; + yield* ensure(() => { + live = false; + }); + yield* RemoteJournalDestination.around( + { + *append([candidate, event], next): Operation { + if (candidate !== database || !live) { + return yield* next(candidate, event); + } + yield* transaction.journal.append(event); + return true; + }, + }, + { at: "min" }, + ); + return yield* publication; + }); +} + +/** + * The run's journal, willing to be redirected into an open transaction. + * + * Reads always come from the ordinary journal: what a transaction has appended + * is read back through the transaction itself, and a reader outside it is + * asking about committed history. + */ +export function routeRemoteRunJournal( + database: WorkflowRunDatabase, + ordinary: DurableStream, +): DurableStream { + return { + readAll: () => ordinary.readAll(), + + *append(event: DurableEvent): Operation { + if (!(yield* RemoteJournalDestination.operations.append(database, event))) { + yield* ordinary.append(event); + } + }, + }; +} diff --git a/packages/workflow/src/remote/mappings.ts b/packages/workflow/src/remote/mappings.ts new file mode 100644 index 000000000..495c858e2 --- /dev/null +++ b/packages/workflow/src/remote/mappings.ts @@ -0,0 +1,255 @@ +/** + * Retained mappings, as one invocation on the runner sees them. + * + * The runner has no database. What it has is the coherent snapshot the owner + * admitted this invocation from, and whatever this invocation has staged since. + * That is enough to answer every question the shared composition rules ask, + * because those rules only ever read a mapping back and compare it — and this + * answers with the retained row when there is one, and with what this + * invocation staged when there is not. + * + * Read-your-writes without durability. A document that creates a Repository and + * then asks for it again is asking about its own work, and must see it; nothing + * about that makes it committed. Only the exact list handed through the live + * enlistment capability reaches an intent, and only the owner's transaction + * makes any of it authoritative. + * + * The reconciliation rules are not restated here. A same-name Repository is + * compared by the composition provider that already knows what compatible + * means, and an Agent session by `resolveAgentSession()`; this module decides + * only where a record comes from and what a new one stages. + */ + +import { + type AgentSessionRecord, + type AgentSessions, + WorkflowAgentSessionError, +} from "../storage/agent-session.ts"; +import type { WorktreeRecord } from "../composition/records.ts"; +import { parseCheckoutPath } from "../composition/records.ts"; +import { locatorFingerprintOf } from "../composition/locator.ts"; +import type { StoredRepository, WorkspaceMetadata } from "../workspace/metadata.ts"; +import type { RemoteInvocationSnapshot } from "./records.ts"; +import type { RetainedMapping } from "./publication.ts"; +import { WorkflowRecordMalformedError } from "../storage/errors.ts"; + +/** The most mappings one invocation may stage before it is refused. */ +const MAX_STAGED_MAPPINGS = 256; + +/** The most serialized bytes one invocation may stage before it is refused. */ +const MAX_STAGED_BYTES = 256 * 1024; + +/** + * What an invocation may reach, and what it has decided to retain. + * + * `deltas()` is the whole of what may be enlisted. It is a fresh array each + * time, deterministically ordered, so the caller cannot reach back into what + * the view is still holding. + */ +export interface InvocationMappings { + readonly metadata: WorkspaceMetadata; + readonly agentSessions: AgentSessions; + deltas(): readonly RetainedMapping[]; +} + +function refuse(reason: string): never { + throw new WorkflowRecordMalformedError("this run's retained mappings", reason); +} + +/** + * A copy that shares nothing with what it was given. + * + * These records are handed to a document and staged for a commit, and both hold + * them for longer than the call. A structural clone is what makes "the delta is + * what was staged" true rather than a description of what nobody mutated. + */ +function detach(value: T): T { + return structuredClone(value) as T; +} + +export function createInvocationMappings( + snapshot: RemoteInvocationSnapshot, + live: () => void, +): InvocationMappings { + const repositories = new Map(); + const worktrees = new Map(); + const sessions = new Map(); + for (const stored of snapshot.repositories) { + repositories.set(stored.record.name, stored); + } + for (const record of snapshot.worktrees) { + worktrees.set(worktreeKey(record.repositoryName, record.name), record); + } + for (const record of snapshot.agentSessions) { + sessions.set(record.sessionKey, record); + } + + const staged: RetainedMapping[] = []; + function stage(mapping: RetainedMapping): void { + if (staged.length >= MAX_STAGED_MAPPINGS) { + refuse("this invocation stages more retained mappings than one commit may carry"); + } + const next = [...staged, mapping]; + if (new TextEncoder().encode(JSON.stringify(next)).length > MAX_STAGED_BYTES) { + refuse("this invocation stages more retained mapping bytes than one commit may carry"); + } + staged.push(mapping); + } + + return { + metadata: { + readRepository(name: string): StoredRepository | undefined { + live(); + const found = repositories.get(name); + return found === undefined ? undefined : detach(found); + }, + + readRepositories(): StoredRepository[] { + live(); + return [...repositories.values()] + .toSorted((left, right) => compare(left.record.name, right.record.name)) + .map(detach); + }, + + insertRepository(stored: StoredRepository): void { + live(); + if (locatorFingerprintOf(stored.locator) !== stored.record.locatorFingerprint) { + refuse("a Repository was retained with a fingerprint its locator does not produce"); + } + if (parseCheckoutPath(stored.record.checkoutPath) === undefined) { + refuse("a Repository was retained with a checkout path this build does not admit"); + } + const existing = repositories.get(stored.record.name); + if (existing !== undefined) { + // The same insert twice in one invocation is the same fact stated + // twice. Anything else is a conflict, and a conflict never replaces + // what is already there. + if (!sameRepository(existing, stored)) { + refuse("a Repository name was retained twice under different identities"); + } + // Either the owner already holds it, or this invocation staged it + // earlier. Both mean there is nothing new to retain: a snapshot row + // re-sent as a mutation would ask the owner to insert what it has. + return; + } + const admitted = detach(stored); + repositories.set(admitted.record.name, admitted); + stage({ kind: "repository", record: admitted.record, locator: admitted.locator }); + }, + + readWorktree(repositoryName: string, name: string): WorktreeRecord | undefined { + live(); + const found = worktrees.get(worktreeKey(repositoryName, name)); + return found === undefined ? undefined : detach(found); + }, + + readWorktreesForRepository(repositoryName: string): WorktreeRecord[] { + live(); + return [...worktrees.values()] + .filter((record) => record.repositoryName === repositoryName) + .toSorted((left, right) => compare(left.name, right.name)) + .map(detach); + }, + + insertWorktree(record: WorktreeRecord): void { + live(); + if (parseCheckoutPath(record.checkoutPath) === undefined) { + refuse("a Worktree was retained with a checkout path this build does not admit"); + } + if (!repositories.has(record.repositoryName)) { + refuse("a Worktree was retained for a Repository this run does not hold"); + } + const key = worktreeKey(record.repositoryName, record.name); + const existing = worktrees.get(key); + if (existing !== undefined) { + if (!sameWorktree(existing, record)) { + refuse("a Worktree name was retained twice under different identities"); + } + return; + } + const admitted = detach(record); + worktrees.set(key, admitted); + stage({ kind: "worktree", record: admitted }); + }, + }, + + agentSessions: { + read(sessionKey: string): AgentSessionRecord | undefined { + live(); + const found = sessions.get(sessionKey); + return found === undefined ? undefined : detach(found); + }, + + commit(record: AgentSessionRecord): void { + live(); + const existing = sessions.get(record.sessionKey); + if (existing !== undefined) { + if (!sameSession(existing, record)) { + throw new WorkflowAgentSessionError( + "this run already retains a different Agent session under this identity, and a " + + "session established under one ceiling is not continued under another.", + ); + } + return; + } + const admitted = detach(record); + sessions.set(admitted.sessionKey, admitted); + stage({ kind: "agent-session", record: admitted }); + }, + }, + + deltas(): readonly RetainedMapping[] { + // Parents before children, then by name: the owner applies them in + // dependency order, and a deterministic list is what makes one + // invocation's proposal the same proposal on a retry. + const order = { repository: 0, worktree: 1, "agent-session": 2 } as const; + return staged + .map((mapping, index) => ({ mapping, index })) + .toSorted((left, right) => { + const kinds = order[left.mapping.kind] - order[right.mapping.kind]; + return kinds !== 0 ? kinds : left.index - right.index; + }) + .map((entry) => detach(entry.mapping)); + }, + }; +} + +function worktreeKey(repositoryName: string, name: string): string { + return `${repositoryName}\u0000${name}`; +} + +function compare(left: string, right: string): number { + return left < right ? -1 : left > right ? 1 : 0; +} + +function sameRepository(left: StoredRepository, right: StoredRepository): boolean { + return ( + left.locator === right.locator && + left.record.locatorFingerprint === right.record.locatorFingerprint && + left.record.requestedBase === right.record.requestedBase && + left.record.creationCommit === right.record.creationCommit && + left.record.primaryBranch === right.record.primaryBranch && + left.record.objectFormat === right.record.objectFormat && + left.record.checkoutPath === right.record.checkoutPath + ); +} + +function sameWorktree(left: WorktreeRecord, right: WorktreeRecord): boolean { + return ( + left.requestedBranch === right.requestedBranch && + left.requestedBase === right.requestedBase && + left.creationCommit === right.creationCommit && + left.checkoutPath === right.checkoutPath + ); +} + +function sameSession(left: AgentSessionRecord, right: AgentSessionRecord): boolean { + return ( + left.provider === right.provider && + left.agentCommand === right.agentCommand && + left.sessionIdentity === right.sessionIdentity && + left.policy === right.policy && + left.assertion.kind === right.assertion.kind && + left.assertion.value === right.assertion.value + ); +} diff --git a/packages/workflow/src/remote/materialize.ts b/packages/workflow/src/remote/materialize.ts new file mode 100644 index 000000000..114bb00ed --- /dev/null +++ b/packages/workflow/src/remote/materialize.ts @@ -0,0 +1,360 @@ +/** + * Putting one retained Workspace root on a runner, and reading it back. + * + * The owner holds the run and cannot run anything. Git, an Agent, an evidence + * command — all of it needs real files, and real files are the runner's. So a + * root is materialized into a temporary tree the invocation owns, worked in, + * and captured back into a proposal the owner validates and publishes. + * + * Two things this module refuses to know. It does not know a runtime: every + * native operation arrives as an injected Effection operation, so the same code + * materializes onto whatever filesystem the host adapter wrapped. And it does + * not treat the host path as identity — the logical Workspace root is `/`, the + * temporary directory is an implementation detail of this invocation, and no + * part of the host path reaches a manifest, a journal event, a proposal or an + * error. A run that recorded where it happened to be unpacked would be a run + * that could not be resumed anywhere else. + * + * Everything is verified twice. The owner validated the root before sending it + * and the connection verified each piece on arrival; this verifies again on the + * way to disk, because what must be true is not "the owner was honest" but + * "these bytes are the bytes this root names". The same holds coming back: a + * capture is checked against the rules a stored root is read through before it + * is ever proposed. + */ + +import { type Operation } from "effection"; +import { + captureContent, + type CapturedContent, + type CapturedNode, + type CapturedRoot, + captureWorkspaceRoot, +} from "../workspace/capture.ts"; +import { + compareUtf8, + type WorkspaceRejection, + type WorkspaceRootManifest, +} from "../workspace/root-manifest.ts"; +import { decodeContentManifest } from "../workspace/content-manifest.ts"; +import { sha256Hex } from "../workspace/sha256.ts"; +import type { RemoteReadLink } from "./read.ts"; + +/** One node the runner found, as its host describes one. */ +export interface RunnerNode { + readonly name: string; + readonly kind: "directory" | "file" | "symlink"; + readonly mode: number; + /** Whole seconds, matching what a retained entry carries. */ + readonly mtime: number; + readonly size: number; + /** + * What makes two paths one file. + * + * The host's own answer — an inode, or whatever stands in for one. Absent + * means the host cannot say, and every file is then its own. + */ + readonly identity: string | undefined; + /** Present only for a symbolic link, and never followed. */ + readonly target: string | undefined; +} + +/** + * The native operations materialization needs, and only those. + * + * Deliberately small and deliberately injected. Nothing here opens a process, + * resolves a symbolic link, or reaches outside the directory it was given. + */ +export interface RunnerFiles { + makeDirectory(path: string, mode: number): Operation; + writeFile(path: string, bytes: Uint8Array, mode: number): Operation; + makeSymlink(target: string, path: string): Operation; + makeHardlink(existing: string, path: string): Operation; + /** + * Set permissions exactly, after creation. + * + * Creation modes are narrowed by the process umask, and a retained mode is + * durable identity rather than a preference. Applied to a directory only once + * its children exist, because a mode that forbids writing would otherwise + * forbid filling it. + */ + setMode(path: string, mode: number): Operation; + /** Applied last, because writing into a directory moves its own time. */ + setModifiedAt(path: string, mtime: number): Operation; + /** + * Set a link's own time without following it. + * + * Separate because a link's target may not exist, may be outside the tree, or + * may be something this code must never touch. `undefined` when the host + * cannot do it at all, which materialization reports rather than works around. + */ + readonly setLinkModifiedAt: ((path: string, mtime: number) => Operation) | undefined; + /** The same, for a link's own permissions. `undefined` where unsupported. */ + readonly setLinkMode: ((path: string, mode: number) => Operation) | undefined; + readFile(path: string): Operation; + /** One directory's entries, described without following a link. */ + list(path: string): Operation; + /** One path, described without following a link. */ + describe(path: string): Operation; +} + +/** Where one logical Workspace path sits on this host, for this invocation. */ +export type HostPath = (logical: string) => string; + +/** + * Materialize the exact root, one bounded piece at a time. + * + * Entries are created in canonical order, which is also parent-before-child + * order for everything but the depth ordering a restore needs — so directories + * are created as they are met and a file never arrives before the directory + * holding it. A hardlink group's first member is written and the rest are + * linked to it, which is what makes them one file again rather than copies. + * + * Times are set after the tree exists. Writing a file into a directory updates + * that directory's own time, so setting times as we went would leave every + * directory carrying the moment it was filled rather than the moment the root + * records. + */ +export function* materializeWorkspaceRoot( + files: RunnerFiles, + reads: RemoteReadLink, + at: HostPath, + workspaceRootId: string, + reject: WorkspaceRejection, +): Operation { + const manifest = yield* reads.root(workspaceRootId); + /** + * The first path written for each hardlink group. + * + * Keyed by the group the root declares, never by the content digest. Two + * groups may legally hold identical bytes and therefore share one manifest, + * and linking the second to the first would merge two files into one — a + * different Workspace, arriving under the identity of this one. + */ + const groups = new Map(); + const modes: { path: string; mode: number; link: boolean }[] = []; + const times: { path: string; mtime: number; link: boolean }[] = []; + + for (const entry of manifest.entries) { + const path = at(entry.path); + if (entry.kind === "directory") { + if (entry.path !== "/") { + yield* files.makeDirectory(path, entry.mode); + } + modes.push({ path, mode: entry.mode, link: false }); + times.push({ path, mtime: entry.mtime, link: false }); + continue; + } + if (entry.kind === "symlink") { + // Created, never followed. A retained link may point anywhere, including + // outside the tree, and resolving one here would be this code deciding to + // read something the Workspace merely mentions. + yield* files.makeSymlink(entry.target, path); + modes.push({ path, mode: entry.mode, link: true }); + times.push({ path, mtime: entry.mtime, link: true }); + continue; + } + + const first = entry.hardlink === null ? undefined : groups.get(entry.hardlink); + if (first !== undefined) { + // One inode reached by a second name. Its mode and time belong to the + // file, which the first member already carries. + yield* files.makeHardlink(first, path); + continue; + } + const bytes = yield* fetchFile(reads, workspaceRootId, entry.manifest, entry.size, reject); + yield* files.writeFile(path, bytes, entry.mode); + if (entry.hardlink !== null) { + groups.set(entry.hardlink, path); + } + modes.push({ path, mode: entry.mode, link: false }); + times.push({ path, mtime: entry.mtime, link: false }); + } + + // Modes before times and both deepest-first: a mode that forbids writing must + // not be applied while children are still arriving, and filling a directory + // moves a time that was already restored. + for (const entry of modes.toReversed()) { + if (!entry.link) { + yield* files.setMode(entry.path, entry.mode); + continue; + } + if (files.setLinkMode !== undefined) { + yield* files.setLinkMode(entry.path, entry.mode); + } + } + for (const entry of times.toReversed()) { + if (!entry.link) { + yield* files.setModifiedAt(entry.path, entry.mtime); + continue; + } + if (files.setLinkModifiedAt !== undefined) { + yield* files.setLinkModifiedAt(entry.path, entry.mtime); + } + } + + // Proved rather than assumed. A host that cannot represent a legal retained + // mode or time must say so here, before anything executes against this tree — + // silently normalizing one would hand the run a Workspace with a different + // durable identity than the history it accepted. + yield* requireExactMaterialization(files, at, manifest, reject); + return manifest; +} + +/** + * Whether what is on disk is what the root said. + * + * Every entry's kind, mode and time, read back without following a link. This + * is not defensive duplication: umask, platform link semantics and filesystem + * timestamp granularity are all real, and each of them turns one retained root + * into a different one quietly. The refusal names what disagreed, not where the + * tree happens to live. + */ +function* requireExactMaterialization( + files: RunnerFiles, + at: HostPath, + manifest: WorkspaceRootManifest, + reject: WorkspaceRejection, +): Operation { + for (const entry of manifest.entries) { + const found = yield* files.describe(at(entry.path)); + if (found.kind !== entry.kind) { + reject(`this host materialized a ${entry.kind} as a ${found.kind}`); + } + if (found.mode !== entry.mode) { + reject(`this host cannot preserve the retained mode of a ${entry.kind}`); + } + if (found.mtime !== entry.mtime) { + reject(`this host cannot preserve the retained modification time of a ${entry.kind}`); + } + } +} + +/** One file's bytes, assembled from the chunks its manifest names. */ +function* fetchFile( + reads: RemoteReadLink, + workspaceRootId: string, + manifestDigest: string, + size: number, + reject: WorkspaceRejection, +): Operation { + const encoded = yield* reads.content(workspaceRootId, { + kind: "manifest", + digest: manifestDigest, + }); + const manifest = decodeContentManifest(encoded.bytes, reject); + if (manifest.size !== size) { + reject("a retained Workspace file size disagrees with the manifest it names"); + } + const bytes = new Uint8Array(size); + let offset = 0; + for (const chunk of manifest.chunks) { + const piece = yield* reads.content(workspaceRootId, { + kind: "blob", + digest: chunk.hash, + manifestDigest, + }); + if (piece.bytes.length !== chunk.size) { + reject("a retained content piece is not the size its manifest declares"); + } + bytes.set(piece.bytes, offset); + offset += piece.bytes.length; + } + if (offset !== size) { + reject("a retained Workspace file is not the size its entry declares"); + } + return bytes; +} + +/** What a capture produced, and the content it must be able to supply. */ +export interface CapturedWorkspace { + readonly root: CapturedRoot; + readonly contents: ReadonlyMap; + /** Every blob identity, with the bytes to send if the owner lacks it. */ + readonly blobs: ReadonlyMap; +} + +/** + * Read the tree back as the root it now describes. + * + * A walk, then the shared rules. Nothing here decides ordering, numbering or + * encoding — those belong to the capture rules both hosts share, so that this + * walk and the local provider's walk of its own tables cannot drift apart. + */ +export function* captureWorkspace( + files: RunnerFiles, + at: HostPath, + reject: WorkspaceRejection, +): Operation { + const nodes: CapturedNode[] = []; + const contents = new Map(); + const blobs = new Map(); + + function* visit(logical: string): Operation { + const found = yield* files.list(at(logical)); + for (const node of found.toSorted((left, right) => compareUtf8(left.name, right.name))) { + const path = logical === "/" ? `/${node.name}` : `${logical}/${node.name}`; + if (node.kind === "directory") { + nodes.push({ path, kind: "directory", mode: node.mode, mtime: node.mtime }); + yield* visit(path); + continue; + } + if (node.kind === "symlink") { + if (node.target === undefined) { + reject("a Workspace symbolic link has no target"); + } + nodes.push({ + path, + kind: "symlink", + mode: node.mode, + mtime: node.mtime, + target: node.target, + }); + continue; + } + const bytes = yield* files.readFile(at(path)); + if (bytes.length !== node.size) { + reject("a Workspace file changed size while it was being captured"); + } + const content = captureContent(bytes); + if (!contents.has(content.manifest)) { + contents.set(content.manifest, content); + let offset = 0; + for (const chunk of content.chunks) { + blobs.set(chunk.hash, bytes.slice(offset, offset + chunk.size)); + offset += chunk.size; + } + } + nodes.push({ + path, + kind: "file", + mode: node.mode, + mtime: node.mtime, + size: node.size, + manifest: content.manifest, + identity: node.identity, + }); + } + } + + // The root directory is part of the root's identity like any other entry, so + // its own mode and time are read rather than assumed. + const top = yield* files.describe(at("/")); + if (top.kind !== "directory") { + reject("a Workspace root is not a directory"); + } + nodes.push({ path: "/", kind: "directory", mode: top.mode, mtime: top.mtime }); + yield* visit("/"); + + return { root: captureWorkspaceRoot(nodes, contents, reject), contents, blobs }; +} + +/** Whether a captured root is the one it was materialized from. */ +export function unchangedFrom(captured: CapturedRoot, workspaceRootId: string): boolean { + return captured.rootId === workspaceRootId; +} + +/** The digest of a piece the runner is about to offer. */ +export function pieceDigest(bytes: Uint8Array): string { + return sha256Hex(bytes); +} diff --git a/packages/workflow/src/remote/publication.ts b/packages/workflow/src/remote/publication.ts new file mode 100644 index 000000000..2c7371b5b --- /dev/null +++ b/packages/workflow/src/remote/publication.ts @@ -0,0 +1,93 @@ +/** + * What a runner proposes when a transaction changed the Workspace. + * + * A transaction that only appended to the journal proposes nothing here: the + * run's Workspace is where it was, and saying otherwise would invent a mutation + * to make the shape uniform. When the Workspace did change, exactly one of + * these describes the whole change — the root that was started from, the + * canonical root now proposed, the content that root closes over, and the + * retained mappings the same operation produced. + * + * Everything here is semantic. There is no command, no correlation id, no + * base64, no staged row, no SQL, no socket and no path on the runner. The + * adapter beneath translates this into whatever its owner speaks; a neutral + * value that carried transport vocabulary would make every other host implement + * this one's transport. + * + * The inventory is exact rather than advisory. It names every manifest and blob + * the proposed root closes over, once each, in canonical order — not the pieces + * that happen to be new. The owner resolves each identity from content it + * already holds or from what this acquisition staged, and an inventory that + * named more or fewer would be a root whose content nobody agreed on. + */ + +import type { RepositoryRecord, WorktreeRecord } from "../composition/records.ts"; +import type { AgentSessionRecord } from "../storage/agent-session.ts"; + +/** One content identity a proposed root closes over. */ +export interface ProposedContent { + readonly kind: "manifest" | "blob"; + readonly digest: string; + readonly size: number; +} + +/** + * One complete Workspace change, as the owner will receive it. + * + * `proposedWorkspaceRootId` is not taken on trust: it is what the runner + * computed, and the owner recomputes it from the manifest before anything is + * adopted. Carrying it makes the disagreement detectable rather than making the + * owner guess what the runner thought it was proposing. + */ +export interface WorkspacePublication { + readonly proposedWorkspaceRootId: string; + /** The canonical root manifest, exactly as it was encoded and hashed. */ + readonly proposedManifest: string; + readonly content: readonly ProposedContent[]; +} + +/** + * One retained mapping the same operation produced. + * + * A Repository or Worktree row and the Workspace bytes that make its checkout + * true are one proposal: a mapping naming a checkout that does not exist, or a + * checkout no mapping accounts for, is a Workspace that only half happened. + * + * An Agent-session mapping carries the provider's canonical assertion and the + * derived key, and nothing of the conversation itself. The owner retains what + * the run established; it never contacts or impersonates an Agent provider. + */ +export type RetainedMapping = + | { + readonly kind: "repository"; + readonly record: RepositoryRecord; + /** + * The admitted locator, which the record deliberately does not carry. + * + * A record is journal-safe and names only the fingerprint, because a + * locator can carry a credential and a journal is history. Storage needs + * the real thing to reattach, so it travels beside the record and its + * fingerprint must follow from it. + */ + readonly locator: string; + } + | { readonly kind: "worktree"; readonly record: WorktreeRecord } + | { readonly kind: "agent-session"; readonly record: AgentSessionRecord }; + +/** + * What the owner did, as the runner is allowed to know it. + * + * Returned by a performed commit and by nothing else. It names the root the + * commit selected and the identities the owner minted for the events it + * retained, which is what makes it checkable: a runner can compare the answer + * with the proposal it sent and refuse an owner that agreed to something else. + * + * It is also the only thing that authorizes a local promotion. Passing it is + * how an attempt proves the owner published *that* Workspace — a promotion + * that took no evidence would be the runner deciding on the owner's behalf, + * and the two would disagree the first time a commit was refused. + */ +export interface CommitDecision { + readonly workspaceRootId: string; + readonly journalEventIds: readonly string[]; +} diff --git a/packages/workflow/src/remote/read.ts b/packages/workflow/src/remote/read.ts new file mode 100644 index 000000000..cd2870968 --- /dev/null +++ b/packages/workflow/src/remote/read.ts @@ -0,0 +1,60 @@ +/** + * What a runner may read from the owner of its run. + * + * Semantic values, not messages. The seam speaks in workflow records, Workspace + * roots and content identities; how those are asked for, what a page is, and + * which refusals exist are the adapter's, below this line. That division is + * what lets a second host implement the same reads without this module learning + * anything about it — and what stops paging mechanics leaking into the code + * that only wanted the frontier. + * + * The frontier snapshot is deliberately richer than `StartingFrontier`. A + * transaction needs the root, the anchor and the events; a database handle will + * also need the run record and its retrieval snapshot. Modelling both as one + * value would make the collector carry members it has no business reading, so + * the richer value is separate and maps down to the smaller one. + * + * Nothing here is exported from the package. A read seam a document or a runner + * could name would be a second place deciding what a run may see. + */ + +import type { Operation } from "effection"; +import type { RemoteInvocationSnapshot } from "./records.ts"; +import type { JournalEntry } from "../storage/api.ts"; +import type { DefinitionRetrieval, WorkflowRunRecord } from "../storage/record.ts"; +import type { WorkspaceRootManifest } from "../workspace/root-manifest.ts"; +import type { StartingFrontier } from "./collector.ts"; + +export interface RemoteFrontierSnapshot { + readonly record: WorkflowRunRecord; + readonly retrieval: DefinitionRetrieval | undefined; + readonly workspaceRootId: string; + readonly journalEventId: string | null; + readonly entries: readonly JournalEntry[]; +} + +export interface RemoteContent { + readonly kind: "manifest" | "blob"; + readonly digest: string; + readonly bytes: Uint8Array; +} + +export type RemoteContentRequest = + | { readonly kind: "manifest"; readonly digest: string } + | { readonly kind: "blob"; readonly digest: string; readonly manifestDigest: string }; + +export interface RemoteReadLink { + frontier(): Operation; + /** The one coherent admitted state a Workspace invocation begins from. */ + invocationSnapshot(): Operation; + root(workspaceRootId: string): Operation; + content(workspaceRootId: string, request: RemoteContentRequest): Operation; +} + +export function startingFrontier(snapshot: RemoteFrontierSnapshot): StartingFrontier { + return { + workspaceRootId: snapshot.workspaceRootId, + journalEventId: snapshot.journalEventId, + events: snapshot.entries.map((entry) => structuredClone(entry.event)), + }; +} diff --git a/packages/workflow/src/remote/records.ts b/packages/workflow/src/remote/records.ts new file mode 100644 index 000000000..3bb3383c8 --- /dev/null +++ b/packages/workflow/src/remote/records.ts @@ -0,0 +1,334 @@ +/** + * Reading a workflow record that arrived over a connection. + * + * The owner is the same build and is trusted to be honest; it is not trusted to + * be correct, and neither is the wire between them. A performed answer is an + * answer the owner labelled performed — that is all it is — so nothing here + * turns an `unknown` into a run record, a retrieval or a journal entry without + * checking every member first. + * + * The parsers the local host holds its own rows to are the parsers used here. + * Two readings of one record is how the two hosts would stop agreeing about + * what a run is, and the second reading is always the more permissive one. + * + * A failure names the member and never the value. What crossed the connection + * is retained history, and a record that does not parse is not a reason to + * repeat what it held. + */ + +import { parseDurableEvent } from "@executablemd/durable-streams"; +import type { JournalEntry } from "../storage/api.ts"; +import { parseWorkflowDefinition } from "../storage/definition.ts"; +import { + parseJsonObject, + parseJsonValue, + parseMembers, + parseStringMember, + requireMemberNames, +} from "../storage/members.ts"; +import { + type DefinitionRetrieval, + type DocumentExecutionRecord, + parseRunId, + parseWorkflowRunStatus, + parseWorkflowStopReason, + type WorkflowRunRecord, +} from "../storage/record.ts"; +import { SHA256 } from "../workspace/root-manifest.ts"; +import { admitLocator, locatorFingerprintOf } from "../composition/locator.ts"; +import { + parseRepositoryRecord, + parseWorktreeRecord, + type WorktreeRecord, +} from "../composition/records.ts"; +import { type AgentSessionRecord, parseAgentSessionRecord } from "../storage/agent-session.ts"; +import type { StoredRepository } from "../workspace/metadata.ts"; + +export class RemoteRecordError extends Error { + override name = "RemoteRecordError"; +} + +function fail(reason: string, path: string): Error { + return new RemoteRecordError( + `the owner returned a malformed workflow record at ${path}: ${reason}`, + ); +} + +function instant(value: unknown, path: string): string { + if (typeof value !== "string") { + throw fail("expected an instant", path); + } + const parsed = new Date(value); + if (Number.isNaN(parsed.getTime()) || parsed.toISOString() !== value) { + throw fail("expected an instant", path); + } + return value; +} + +function positiveInteger(value: unknown, path: string): number { + if (typeof value !== "number" || !Number.isSafeInteger(value) || value < 1) { + throw fail("expected a positive whole number", path); + } + return value; +} + +function rootId(value: unknown, path: string): string { + if (typeof value !== "string" || !SHA256.test(value)) { + throw fail("expected a Workspace root identity", path); + } + return value; +} + +export function parseRemoteRunRecord(value: unknown): WorkflowRunRecord { + const members = parseMembers(value, "$", fail); + requireMemberNames( + members, + ["runId", "definition", "base", "props", "status", "stopReason", "createdAt", "updatedAt"], + "$", + fail, + ); + const definition = parseWorkflowDefinition(members.get("definition")); + if (!definition.ok) { + throw fail("expected a workflow definition", "$.definition"); + } + const base = parseStringMember(members, "base", "$", fail); + if (base === "") { + throw fail("expected a non-empty string", "$.base"); + } + const record: WorkflowRunRecord = { + runId: parseRunId(members.get("runId"), "$.runId", fail), + definition: definition.value, + base, + props: parseJsonObject(members.get("props"), "$.props", fail), + status: parseWorkflowRunStatus(members.get("status"), "$.status", fail), + createdAt: instant(members.get("createdAt"), "$.createdAt"), + updatedAt: instant(members.get("updatedAt"), "$.updatedAt"), + }; + if (!members.has("stopReason")) { + return Object.freeze(record); + } + return Object.freeze({ + ...record, + stopReason: parseWorkflowStopReason(members.get("stopReason"), "$.stopReason", fail), + }); +} + +export function parseRemoteRetrieval(value: unknown): DefinitionRetrieval | undefined { + if (value === null) { + return undefined; + } + const members = parseMembers(value, "$", fail); + requireMemberNames(members, ["metadata", "revision", "updatedAt"], "$", fail); + if (members.size !== 3) { + throw fail("expected every retrieval member", "$ "); + } + return Object.freeze({ + metadata: parseJsonValue(members.get("metadata"), "$.metadata", fail), + revision: positiveInteger(members.get("revision"), "$.revision"), + updatedAt: instant(members.get("updatedAt"), "$.updatedAt"), + }); +} + +export function parseRemoteJournalEntry(value: unknown): JournalEntry { + const members = parseMembers(value, "$", fail); + requireMemberNames(members, ["eventId", "record", "workspaceRootId"], "$", fail); + if (members.size !== 3) { + throw fail("expected every journal member", "$ "); + } + const eventId = parseStringMember(members, "eventId", "$", fail); + if (eventId === "") { + throw fail("expected a non-empty identity", "$.eventId"); + } + const record = parseStringMember(members, "record", "$", fail); + const event = parseDurableEvent(record); + if (!event.ok) { + throw fail("expected a durable event", "$.record"); + } + return Object.freeze({ + eventId, + event: event.value, + workspaceRootId: rootId(members.get("workspaceRootId"), "$.workspaceRootId"), + }); +} + +/** + * One document execution, read out of a value nothing has checked. + * + * The shared rules the local host holds its own rows to, applied to what + * arrived. A stopped execution has to carry its status, and a stop reason has + * to agree with the way the record spells one — an execution that stopped for a + * reason the shape does not admit is not a record this build can act on. + */ +export function parseRemoteExecution(value: unknown): DocumentExecutionRecord { + const found = parseMembers(value, "$", fail); + // One of exactly two legal shapes. A record carrying a stop status without + // having stopped, or an undeclared member, is a shape this build does not + // understand — and reading it leniently would make a history that means one + // thing here and another where it was written. + const active = ["executionId", "startedAt"]; + const stopped = [...active, "stoppedAt", "stopStatus"]; + const declared = found.has("stoppedAt") + ? found.has("stopReason") + ? [...stopped, "stopReason"] + : stopped + : active; + requireMemberNames(found, declared, "$", fail); + if (found.size !== declared.length) { + throw fail("expected exactly the members this shape declares", "$"); + } + + const executionId = parseStringMember(found, "executionId", "$", fail); + if (executionId === "") { + throw fail("expected a non-empty identity", "$.executionId"); + } + const record: DocumentExecutionRecord = { + executionId, + startedAt: instant(found.get("startedAt"), "$.startedAt"), + }; + if (!found.has("stoppedAt")) { + return Object.freeze(record); + } + const halted: DocumentExecutionRecord = { + ...record, + stoppedAt: instant(found.get("stoppedAt"), "$.stoppedAt"), + stopStatus: parseWorkflowRunStatus(found.get("stopStatus"), "$.stopStatus", fail), + }; + if (!found.has("stopReason")) { + return Object.freeze(halted); + } + return Object.freeze({ + ...halted, + stopReason: parseWorkflowStopReason(found.get("stopReason"), "$.stopReason", fail), + }); +} + +/** + * One admitted invocation snapshot, as the runner is allowed to read it. + * + * The root and journal anchor travel with the mappings because they are one + * fact, and the runner holds the whole answer to that: before a document runs, + * the transaction it runs inside has to start from exactly this root and this + * anchor. + */ +export interface RemoteInvocationSnapshot { + readonly workspaceRootId: string; + readonly journalEventId: string | null; + readonly repositories: readonly StoredRepository[]; + readonly worktrees: readonly WorktreeRecord[]; + readonly agentSessions: readonly AgentSessionRecord[]; +} + +/** The most mapping entries one admitted snapshot may carry. */ +const MAX_SNAPSHOT_ENTRIES = 256; + +export function parseRemoteInvocationSnapshot(value: unknown): RemoteInvocationSnapshot { + const found = parseMembers(value, "$", fail); + requireMemberNames( + found, + ["workspaceRootId", "journalEventId", "repositories", "worktrees", "agentSessions"], + "$", + fail, + ); + const workspaceRootId = parseStringMember(found, "workspaceRootId", "$", fail); + if (!SHA256.test(workspaceRootId)) { + throw fail("expected a Workspace root identity", "$.workspaceRootId"); + } + const anchor = found.get("journalEventId"); + if (anchor !== null && (typeof anchor !== "string" || anchor === "")) { + throw fail("expected a journal event identity or an explicit empty anchor", "$.journalEventId"); + } + + const repositories = list(found.get("repositories"), "$.repositories").map((entry, index) => + parseStoredRepository(entry, `$.repositories[${index}]`), + ); + const worktrees = list(found.get("worktrees"), "$.worktrees").map((entry, index) => + admitted(parseWorktreeRecord(entry), `$.worktrees[${index}]`, "a Worktree"), + ); + const agentSessions = list(found.get("agentSessions"), "$.agentSessions").map((entry, index) => + admitted(parseAgentSessionRecord(entry), `$.agentSessions[${index}]`, "an Agent session"), + ); + + if (repositories.length + worktrees.length + agentSessions.length > MAX_SNAPSHOT_ENTRIES) { + throw fail("expected fewer retained mappings than one snapshot may carry", "$"); + } + requireOrdered( + repositories.map((stored) => stored.record.name), + "$.repositories", + ); + requireOrdered( + worktrees.map((record) => `${record.repositoryName} ${record.name}`), + "$.worktrees", + ); + requireOrdered( + agentSessions.map((record) => record.sessionKey), + "$.agentSessions", + ); + // Every Worktree names a Repository this snapshot also carries. A checkout + // whose Repository is missing is not a state this run was ever in. + const names = new Set(repositories.map((stored) => stored.record.name)); + for (const [index, record] of worktrees.entries()) { + if (!names.has(record.repositoryName)) { + throw fail( + "expected a Worktree whose Repository this snapshot holds", + `$.worktrees[${index}]`, + ); + } + } + return Object.freeze({ + workspaceRootId, + journalEventId: anchor === null ? null : anchor, + repositories: Object.freeze(repositories), + worktrees: Object.freeze(worktrees), + agentSessions: Object.freeze(agentSessions), + }); +} + +function list(value: unknown, path: string): unknown[] { + if (!Array.isArray(value)) { + throw fail("expected an array", path); + } + return value; +} + +function admitted(parsed: T | undefined, path: string, expectation: string): T { + if (parsed === undefined) { + throw fail(`expected ${expectation}`, path); + } + return parsed; +} + +/** + * Deterministic and without repeats, checked rather than assumed. + * + * The owner reads these in one order; a snapshot that arrived in another, or + * twice under one name, is not the state it claims to describe — and a mapping + * view built from it would answer differently depending on which copy it read. + */ +function requireOrdered(keys: readonly string[], path: string): void { + for (const [index, key] of keys.entries()) { + const previous = keys[index - 1]; + if (previous !== undefined && previous >= key) { + throw fail("expected retained mappings in one deterministic order, without repeats", path); + } + } +} + +function parseStoredRepository(value: unknown, path: string): StoredRepository { + const found = parseMembers(value, path, fail); + requireMemberNames(found, ["record", "locator"], path, fail); + const locator = parseStringMember(found, "locator", path, fail); + // Admitted by the same rule the local host admits one by, so a locator this + // build would refuse to use never becomes one it reconciles against. + if (admitLocator(locator) === undefined) { + throw fail("expected a Repository locator this build admits", `${path}.locator`); + } + const record = admitted( + parseRepositoryRecord(found.get("record")), + `${path}.record`, + "a Repository", + ); + if (locatorFingerprintOf(locator) !== record.locatorFingerprint) { + throw fail("expected a locator the record's fingerprint follows from", `${path}.locator`); + } + return Object.freeze({ record, locator }); +} diff --git a/packages/workflow/src/remote/seal.ts b/packages/workflow/src/remote/seal.ts new file mode 100644 index 000000000..07f86f46b --- /dev/null +++ b/packages/workflow/src/remote/seal.ts @@ -0,0 +1,41 @@ +/** + * How a transaction reaches into the attempt it was given, and nothing else can. + * + * A transaction needs two things from a disposable attempt: to seal it into a + * proposal once the body has finished, and to make it the accepted Workspace + * once the owner has performed that exact proposal. Neither may be offered to + * whoever is running the body — a capability handed out is a capability that + * can be used at the wrong moment, and the wrong moment here is any moment + * before the owner has decided. + * + * So they hang off a symbol. A symbol cannot be written down by code that does + * not already have it, this module is reachable from no package entrypoint, and + * the declared `Attempt` says nothing about it. What a caller receives is a + * place to work and a way to read what it did. + */ + +import type { Operation } from "effection"; +import type { CommitDecision, RetainedMapping, WorkspacePublication } from "./publication.ts"; + +/** The key a transaction reaches an attempt's own machinery through. */ +export const SEAL: unique symbol = Symbol("executablemd.workflow.remote.seal"); + +/** One attempt, sealed into the proposal the owner will decide. */ +export interface SealedProposal { + readonly publication: WorkspacePublication; + readonly mappings: readonly RetainedMapping[]; + readonly bytes: ReadonlyMap; + /** + * Make the sealed attempt the accepted Workspace. + * + * Called once, by the transaction, after the owner performed this exact + * proposal and the answer was checked against it. The decision is compared + * with what was sealed, so an answer about another Workspace moves nothing. + */ + transfer(decision: CommitDecision): Operation; +} + +/** What an attempt privately offers the transaction that was given it. */ +export interface SealableAttempt { + readonly [SEAL]: (mappings: readonly RetainedMapping[]) => Operation; +} diff --git a/packages/workflow/src/remote/workspace.ts b/packages/workflow/src/remote/workspace.ts new file mode 100644 index 000000000..14f477e54 --- /dev/null +++ b/packages/workflow/src/remote/workspace.ts @@ -0,0 +1,473 @@ +/** + * Running Workspace work on the runner, against a run the owner holds. + * + * The Deno coordinator opens a transaction, hands a mutation the authoritative + * filesystem and the retained metadata, and commits both together. This is the + * same shape with the storage somewhere else: the Workspace is a real directory + * this invocation materialized from the exact admitted root, the metadata is a + * detached snapshot of that same admitted state, and "commit" is one intent the + * owner performs atomically or not at all. + * + * The ordering is the whole of the correctness argument, so it is written out + * rather than implied: + * + * 1. The execution identity is claimed by one database before any effect is + * created, so a foreign or reused one cannot coordinate. + * 2. One coherent snapshot is admitted: root, journal anchor and mappings + * together, because they are one state. + * 3. The accepted tree and the disposable attempt are created *outside* the + * transaction, because the collector seals the attempt after the transaction + * body has torn down — an attempt scoped to the body would be gone by then. + * 4. Inside the exact transaction callback, the route proves it starts from the + * admitted state. Drift refuses here, before the document runs and before + * anything is sent. + * 5. The document runs once. A documented Workspace failure is the effect's own + * result and journals against the unchanged root; everything else is the run + * failing and publishes nothing. + * 6. Only a successful result enlists the attempt and its staged deltas, and + * the publication is routed into this exact transaction's journal. + * 7. The collector seals, sends, and transfers the tree only on the exact + * performed answer. + */ + +import { type Api, createApi } from "@effectionx/context-api"; +import { + createOwnedDurableWorkspaceOperation, + type WorkspaceCoordinationAuthority, + type WorkspaceCoordinationProvider, + withWorkspaceCoordinationProvider, +} from "../workspace/effect.ts"; +import { + type DurableEffect, + type EffectDescription, + type Json, + type JournalProvenance, + type Result as DurableResult, + serializeError, +} from "@executablemd/durable-streams"; +import { ensure, type Operation, scoped } from "effection"; +import type { WorkflowRunDatabase, WorkflowRunTransaction } from "../storage/api.ts"; +import { WorkflowTransactionError } from "../storage/errors.ts"; +import type { WorkspaceFilesystem } from "../workspace/filesystem.ts"; +import { isJournaledEffectFailure } from "../workspace/failure.ts"; +import type { WorkspaceMetadata } from "../workspace/metadata.ts"; +import type { AgentSessions } from "../storage/agent-session.ts"; +import { activeWorkspaceRoute, type WorkspaceRoute } from "./database.ts"; +import { createInvocationMappings } from "./mappings.ts"; +import { + type Attempt, + type Materialization, + useAttempt, + useMaterialization, +} from "./invocation.ts"; +import type { HostPath, RunnerFiles } from "./materialize.ts"; +import type { RemoteReadLink } from "./read.ts"; +import type { RemoteInvocationSnapshot } from "./records.ts"; +import { withRemoteJournalRoute } from "./journal-route.ts"; +import { resource } from "effection"; +import { establishJournalProvenance, type DurableStream } from "@executablemd/durable-streams"; +import { useRemoteRunDatabase, type RemoteWorkspaceLink } from "./database.ts"; +import { routeRemoteRunJournal } from "./journal-route.ts"; + +import type { TemporaryTrees } from "./invocation.ts"; + +/** + * What a Workspace mutation is given. + * + * The same two things the Deno coordinator hands one, plus the Agent-session + * mappings, so one contract describes both hosts' work. + */ +export type RemoteWorkspaceMutation = ( + filesystem: WorkspaceFilesystem, + metadata: WorkspaceMetadata, + agentSessions: AgentSessions, +) => Operation; + +/** + * The host facts the coordinator cannot know. + * + * Where temporary trees come from, how bytes are written, and how a Workspace + * filesystem is built over a directory. Supplied by a runtime-named adapter, so + * nothing here names a host. + */ +export interface RemoteWorkspaceRuntime { + readonly files: RunnerFiles; + readonly trees: TemporaryTrees; + readonly reads: RemoteReadLink; + createFilesystem(at: HostPath, authorize: () => void): WorkspaceFilesystem; +} + +interface WorkspaceMutationApi { + run( + database: WorkflowRunDatabase, + mutate: RemoteWorkspaceMutation, + ): Operation; +} + +function unavailable(reason: string): never { + throw new WorkflowTransactionError(reason); +} + +const WorkspaceMutation: Api = createApi( + "executablemd.workflow.remote.workspace.effect.mutation", + { + // deno-lint-ignore require-yield + *run(): Operation { + return unavailable( + "the Workspace effect is not bound to an active remote WorkflowRun transaction.", + ); + }, + }, +); + +/** + * Which database claimed which execution identity. + * + * A `WeakMap` keyed by the identity object, so the claim is the object itself + * rather than anything written down. A second loaded copy of this module has + * its own map and its own identities, and neither can answer for the other's. + */ +const workspaceEffectOwners = (() => { + const owners = new WeakMap(); + return { + claim(identity: object, run: object): void { + owners.set(identity, run); + }, + get(identity: object): object | undefined { + return owners.get(identity); + }, + }; +})(); + +/** + * One remote run, as one thing. + * + * The pieces a Workspace invocation needs — the database handle, the owner link + * its reads and commits go through, the runtime adapters that materialize from + * that owner, the routed journal and the provenance taken over it — describe + * one run only when they came from the same one. Supplied separately they can + * be recombined: pair run B's database with run A's link and journal, and if + * both happen to start at the same root and anchor, one effect journals in A + * and publishes its Workspace in B. + * + * So they are not supplied separately. This constructs them together and hands + * back one opaque value. There is nothing to recombine, and nothing structural + * to forge: the coordinator compares the object it was given, not the run id, + * root or anchor inside it. + */ +export interface RemoteRun { + /** The run's storage handle, for work that is not a Workspace effect. */ + readonly database: WorkflowRunDatabase; + /** + * The run's journal, routed so a Workspace publication lands in its + * transaction. This exact stream is the one provenance was taken over. + */ + readonly journal: DurableStream; +} + +/** What only this module may read off a binding. */ +interface BoundRun extends RemoteRun { + readonly runtime: RemoteWorkspaceRuntime; + readonly provenance: JournalProvenance; +} + +/** + * The private view of a binding, keyed by the binding itself. + * + * A `WeakSet` would answer "did this module make it"; this answers "and here is + * what it was made from", without putting either on the value a host holds. A + * second loaded copy of this module has its own map and cannot answer for one + * of these, which is the loaded-copy contract. + */ +const bindings = (() => { + const held = new WeakMap(); + return { + bind(run: BoundRun): RemoteRun { + const handle: RemoteRun = Object.freeze({ database: run.database, journal: run.journal }); + held.set(handle, run); + return handle; + }, + of(run: RemoteRun | undefined): BoundRun | undefined { + return run === undefined ? undefined : held.get(run); + }, + }; +})(); + +/** What a host supplies to open one remote run. */ +export interface RemoteRunOptions { + /** + * The one owner link this run's database, reads and commits go through. + * + * Deliberately one member. A separate read link could be another owner's, + * and an invocation admitted from one run would commit to the other. + */ + readonly link: RemoteWorkspaceLink; + readonly files: RunnerFiles; + readonly trees: TemporaryTrees; + createFilesystem(at: HostPath, authorize: () => void): WorkspaceFilesystem; + /** The run's ordinary journal, which this routes and takes provenance over. */ + readonly journal: DurableStream; +} + +/** + * Open one remote run: its database, its routed journal and its provenance. + * + * The database is created here from the same link the runtime reads through, so + * "this runtime belongs to this handle" is true by construction rather than by + * a check that could be passed with another handle. + */ +export function useRemoteRun(options: RemoteRunOptions): Operation { + return resource(function* (provide) { + const database = yield* useRemoteRunDatabase( + options.link, + yield* options.link.frontierSnapshot(), + ); + const journal = routeRemoteRunJournal(database, options.journal); + yield* provide( + bindings.bind({ + database, + journal, + provenance: establishJournalProvenance(journal), + runtime: { + files: options.files, + trees: options.trees, + // A view of the same object the database and the commits came from, + // not a second link: `frontier` names two different reads on the two + // contracts, and materialization wants the coherent one. + reads: readsOf(options.link), + createFilesystem: options.createFilesystem, + }, + }), + ); + }); +} + +/** The read half of one owner link, presented the way materialization reads it. */ +function readsOf(link: RemoteWorkspaceLink): RemoteReadLink { + return { + frontier: () => link.frontierSnapshot(), + root: (workspaceRootId) => link.root(workspaceRootId), + content: (workspaceRootId, request) => link.content(workspaceRootId, request), + invocationSnapshot: () => link.invocationSnapshot(), + }; +} + +interface ProviderApi { + readonly provider: object | undefined; +} + +const RemoteWorkspaceProvider: Api = createApi( + "executablemd.workflow.remote.workspace.effect.provider", + { provider: undefined }, +); + +interface Registration { + open: boolean; + readonly run: BoundRun; +} + +const registrations = (() => { + const held = new WeakMap(); + return { + register(run: BoundRun) { + const selection = Object.freeze({}); + const registration: Registration = { open: true, run }; + held.set(selection, registration); + return { + selection, + close(): void { + registration.open = false; + held.delete(selection); + }, + }; + }, + get(selection: object): Registration | undefined { + const registration = held.get(selection); + return registration?.open === true ? registration : undefined; + }, + }; +})(); + +/** Install the runner's Workspace coordination for this run, in this scope. */ +export function* useRemoteWorkspaceEffects(run: RemoteRun): Operation { + const bound = bindings.of(run); + if (bound === undefined) { + unavailable("this is not a remote run this build opened."); + } + const registration = registrations.register(bound); + yield* ensure(registration.close); + yield* RemoteWorkspaceProvider.around({ provider: () => registration.selection }, { at: "min" }); +} + +export function withRemoteWorkspaceEffects( + run: RemoteRun, + operation: Operation, +): Operation { + return scoped(function* () { + const selection = yield* RemoteWorkspaceProvider.operations.provider; + const registration = selection === undefined ? undefined : registrations.get(selection); + // The exact binding, not one that describes the same run. Two handles on + // two owners can hold identical records; only one of them is this one. + if (registration === undefined || registration.run !== bindings.of(run)) { + return unavailable("no remote Workspace coordinator is installed for this run."); + } + return yield* withWorkspaceCoordinationProvider(coordinator(registration.run), operation); + }); +} + +export function createRemoteWorkspaceEffect( + run: RemoteRun, + description: EffectDescription, + mutate: RemoteWorkspaceMutation, +): DurableEffect { + const bound = bindings.of(run); + if (bound === undefined) { + unavailable("this is not a remote run this build opened."); + } + const execute = () => WorkspaceMutation.operations.run(bound.database, mutate); + const executionIdentity = Object.freeze({}); + // Claimed for the binding rather than for a database, so an effect cannot be + // created against one run and coordinated by another that holds it. + workspaceEffectOwners.claim(executionIdentity, bound); + return createOwnedDurableWorkspaceOperation(description, execute, executionIdentity); +} + +function coordinator(run: BoundRun): WorkspaceCoordinationProvider { + return { + *run(authority: WorkspaceCoordinationAuthority): Operation { + let transacted; + try { + // Both against the same binding, so there is no pair of checks that a + // recombination could satisfy one at a time. + if (workspaceEffectOwners.get(authority.executionIdentity) !== run) { + unavailable( + "the live Workspace effect is missing, foreign, completed, or stale for this " + + "remote run.", + ); + } + if ( + authority.journalProvenance === undefined || + authority.journalProvenance !== run.provenance + ) { + unavailable( + "the live Workspace journal does not have the provenance of the selected remote run.", + ); + } + transacted = yield* invoke(run, authority); + } catch (error) { + throw yield* authority.activateFailure(error); + } + if (!transacted.ok) { + throw yield* authority.activateFailure(transacted.error); + } + return transacted.value; + }, + }; +} + +function* invoke(run: BoundRun, authority: WorkspaceCoordinationAuthority) { + const { runtime, database } = run; + const reject = (reason: string): never => unavailable(reason); + const snapshot = yield* runtime.reads.invocationSnapshot(); + + // Outside the transaction, deliberately. The collector seals the attempt + // after the transaction body and everything it started have torn down, so an + // attempt owned by the body would already be gone when its proposal is taken. + const materialization: Materialization = yield* useMaterialization( + runtime.files, + runtime.trees, + runtime.reads, + snapshot.workspaceRootId, + reject, + ); + const attempt: Attempt = yield* useAttempt( + runtime.files, + runtime.trees, + runtime.reads, + materialization, + reject, + ); + + return yield* database.transact(function* (transaction) { + const route = yield* activeWorkspaceRoute(database, transaction); + if (route === undefined) { + unavailable( + "the live Workspace coordinator is not inside this WorkflowRun's active transaction.", + ); + } + // Before the document runs, and before anything is sent. If the run moved + // between the snapshot and this transaction, everything admitted describes + // a state this commit would not be against. + if ( + route.anchor.workspaceRootId !== snapshot.workspaceRootId || + route.anchor.journalEventId !== snapshot.journalEventId + ) { + unavailable( + "this Workspace invocation was admitted from a state this run has since moved past.", + ); + } + return yield* coordinateTransaction(run, transaction, route, snapshot, attempt, authority); + }); +} + +function* coordinateTransaction( + run: BoundRun, + transaction: WorkflowRunTransaction, + route: WorkspaceRoute, + snapshot: RemoteInvocationSnapshot, + attempt: Attempt, + authority: WorkspaceCoordinationAuthority, +): Operation { + const { database, runtime } = run; + return yield* scoped(function* () { + let live = true; + // The capabilities exist while this invocation does and no longer. A + // filesystem or mapping view captured for later is asking about a + // Workspace that has already been committed or discarded. + yield* ensure(() => { + live = false; + }); + const authorize = (): void => { + if (!live) { + unavailable("this Workspace capability is completed, cancelled, or stale."); + } + }; + const mappings = createInvocationMappings(snapshot, authorize); + const filesystem = runtime.createFilesystem(attempt.at, authorize); + + let result: DurableResult; + try { + const value = yield* scoped(function* () { + yield* WorkspaceMutation.around( + { + *run([candidate, mutate]): Operation { + if (candidate !== database) { + unavailable( + "the Workspace effect is not bound to an active remote WorkflowRun transaction.", + ); + } + return yield* mutate(filesystem, mappings.metadata, mappings.agentSessions); + }, + }, + { at: "min" }, + ); + return yield* authority.execute(); + }); + result = { status: "ok", value }; + // Only a successful result publishes a Workspace. The attempt is named + // rather than captured: the collector seals it after this body tears + // down, so what the owner decides is the tree as it finally is. + route.enlist(attempt, mappings.deltas()); + } catch (error) { + if (!isJournaledEffectFailure(error)) { + throw error; + } + // The effect's own outcome. Nothing is enlisted, so the commit carries + // only this row and the root stays exactly where it was. + result = { status: "err", error: serializeError(error) }; + } + + yield* withRemoteJournalRoute(database, transaction, authority.publish(result)); + return result; + }); +} diff --git a/packages/workflow/src/software-factory/run-id.ts b/packages/workflow/src/software-factory/run-id.ts new file mode 100644 index 000000000..16b5e8b54 --- /dev/null +++ b/packages/workflow/src/software-factory/run-id.ts @@ -0,0 +1,252 @@ +/** + * The run id a software-factory run is addressed by. + * + * One GitHub issue is one durable run, so the id has to be a function of the + * issue and of nothing that can change while the work is going on. Repository + * names get renamed, issue numbers move between deployments, Project items and + * their statuses are edited constantly, branches and revisions are the point of + * the exercise, and delivery ids and actors differ on every request. None of + * them takes part. What is left is the deployment the issue lives in and the + * opaque node id that deployment gave it, and those two are what this hashes. + * + * Because every input is immutable, admitting one issue twice derives one id and + * reaches one run through ordinary compatible reuse, and no separate idempotency + * concept appears anywhere above it. Two independent implementations handed the + * same authority and node id produce the same 52 characters. + * + * The derivation is specified in `specs/github-actions-software-factory-spec.md` + * §1.1 and restated in `specs/workflow-spec.md` §9.1. It is host-selected public + * run id and nothing more: opaque to everything but equality and lifecycle + * addressing, and a legal one under the storage rule, which wants a non-empty + * string containing no NUL. + */ + +import { until } from "effection"; +import type { Operation } from "effection"; + +/** The version tag the digest opens with. A different scheme takes a different tag. */ +const SCHEME = "github-issue-v1"; + +/** Lowercase RFC 4648 Base32. No padding is ever emitted, so `=` is absent. */ +const BASE32_ALPHABET = "abcdefghijklmnopqrstuvwxyz234567"; + +/** + * How many characters a full SHA-256 becomes. + * + * 32 bytes is 256 bits, and Base32 carries five bits per character, so the + * unpadded encoding is `ceil(256 / 5)` characters. Stated rather than computed + * because it is a contract a second implementation is held to. + */ +const FACTORY_RUN_ID_LENGTH = 52; + +/** Why a subject could not be turned into a run id. */ +export type FactoryRunSubjectFailure = + | "authority-empty" + | "authority-has-scheme" + | "authority-has-userinfo" + | "authority-has-path" + | "authority-has-query" + | "authority-has-fragment" + | "authority-has-whitespace" + | "authority-malformed-host" + | "authority-malformed-port" + | "authority-default-port" + | "node-id-empty" + | "node-id-has-nul"; + +/** A subject this build cannot derive an id from, named by what was wrong with it. */ +export class FactoryRunSubjectError extends Error { + override name = "FactoryRunSubjectError"; + + constructor( + readonly reason: FactoryRunSubjectFailure, + detail: string, + ) { + super(`this GitHub subject cannot address a factory run: ${detail}`); + } +} + +/** The exact GitHub subject one factory run is a run of. */ +export interface FactoryRunSubject { + /** + * The canonical GitHub authority: a lowercase DNS hostname, plus `:` and a + * port when that port is not the scheme's default. + */ + readonly authority: string; + /** + * The exact string GitHub's GraphQL API returned for this issue. + * + * Compared byte for byte. It is an opaque provider identity, and normalizing + * one would be inventing a second. + */ + readonly issueNodeId: string; +} + +/** The port `https` implies, and therefore the one an authority may not spell out. */ +const DEFAULT_PORT = 443; + +/** + * A hostname the DNS grammar admits: labels of letters, digits and hyphens, + * each starting and ending with an alphanumeric, separated by dots. + * + * Deliberately not a URL parse. A parser would accept — and silently discard — + * the parts an authority may not carry, and the point here is to refuse them. + */ +const HOSTNAME = + /^(?=.{1,253}$)[a-z0-9](?:[a-z0-9-]*[a-z0-9])?(?:\.[a-z0-9](?:[a-z0-9-]*[a-z0-9])?)*$/; + +/** + * Normalize what an operator configured into the one spelling this hash uses. + * + * Case folding is the only transformation. Everything else an authority must not + * contain is refused rather than stripped: a value that had to be repaired to be + * usable is a value somebody meant differently, and two spellings that both + * became one authority would be two runs quietly becoming one. + */ +function canonicalGitHubAuthority(value: string): string { + if (value === "") { + throw new FactoryRunSubjectError("authority-empty", "the authority is empty"); + } + if (/\s/.test(value)) { + throw new FactoryRunSubjectError( + "authority-has-whitespace", + "the authority contains whitespace", + ); + } + if (value.includes("//") || /^[a-zA-Z][a-zA-Z0-9+.-]*:\/\//.test(value)) { + throw new FactoryRunSubjectError( + "authority-has-scheme", + "the authority carries a scheme; write the host alone", + ); + } + if (value.includes("@")) { + throw new FactoryRunSubjectError( + "authority-has-userinfo", + "the authority carries user information", + ); + } + if (value.includes("#")) { + throw new FactoryRunSubjectError("authority-has-fragment", "the authority carries a fragment"); + } + if (value.includes("?")) { + throw new FactoryRunSubjectError("authority-has-query", "the authority carries a query"); + } + if (value.includes("/")) { + throw new FactoryRunSubjectError( + "authority-has-path", + "the authority carries a path or a trailing separator", + ); + } + + const folded = value.toLowerCase(); + const separator = folded.lastIndexOf(":"); + const host = separator === -1 ? folded : folded.slice(0, separator); + const port = separator === -1 ? undefined : folded.slice(separator + 1); + + if (!HOSTNAME.test(host)) { + throw new FactoryRunSubjectError("authority-malformed-host", "the host is not a DNS hostname"); + } + if (port === undefined) { + return host; + } + if (!/^[0-9]{1,5}$/.test(port)) { + throw new FactoryRunSubjectError("authority-malformed-port", "the port is not a number"); + } + const numeric = Number(port); + if (numeric < 1 || numeric > 65535) { + throw new FactoryRunSubjectError("authority-malformed-port", "the port is out of range"); + } + if (numeric === DEFAULT_PORT) { + throw new FactoryRunSubjectError( + "authority-default-port", + "the default port is written out; omit it so one deployment has one spelling", + ); + } + return `${host}:${numeric}`; +} + +/** Hold a node id to what a retained identity has to be, and change nothing about it. */ +function admitIssueNodeId(value: string): string { + if (value === "") { + throw new FactoryRunSubjectError("node-id-empty", "the issue node id is empty"); + } + if (value.includes("\0")) { + throw new FactoryRunSubjectError("node-id-has-nul", "the issue node id contains a NUL"); + } + return value; +} + +/** + * The exact bytes the digest is taken over. + * + * `github-issue-v1`, NUL, the canonical authority, NUL, the node id — all + * UTF-8. The NULs are separators the inputs cannot contain, so no pair of + * (authority, node id) can be rearranged into another pair with the same bytes. + */ +export function factoryRunIdPreimage(subject: FactoryRunSubject): ArrayBuffer { + const encoder = new TextEncoder(); + const scheme = encoder.encode(SCHEME); + const authority = encoder.encode(subject.authority); + const node = encoder.encode(subject.issueNodeId); + // An `ArrayBuffer` rather than a view, because that is what `crypto.subtle` + // accepts without anything having to assert a type at the call site. + const buffer = new ArrayBuffer(scheme.length + 1 + authority.length + 1 + node.length); + const bytes = new Uint8Array(buffer); + let at = 0; + bytes.set(scheme, at); + at += scheme.length; + bytes[at] = 0; + at += 1; + bytes.set(authority, at); + at += authority.length; + bytes[at] = 0; + at += 1; + bytes.set(node, at); + return buffer; +} + +/** Lowercase unpadded RFC 4648 Base32 of exactly these bytes. */ +export function base32Unpadded(bytes: Uint8Array): string { + let out = ""; + let buffer = 0; + let bits = 0; + for (const byte of bytes) { + buffer = (buffer << 8) | byte; + bits += 8; + while (bits >= 5) { + bits -= 5; + out += BASE32_ALPHABET[(buffer >> bits) & 31]; + } + } + if (bits > 0) { + out += BASE32_ALPHABET[(buffer << (5 - bits)) & 31]; + } + return out; +} + +/** + * Normalize a subject, refusing anything this build cannot address a run from. + * + * Separate from the derivation so a caller can admit a subject before it has + * anywhere to put the answer — which is what an admission check needs, and what + * a later story comparing a reread subject against a retained one needs too. + */ +export function admitFactoryRunSubject(subject: FactoryRunSubject): FactoryRunSubject { + return { + authority: canonicalGitHubAuthority(subject.authority), + issueNodeId: admitIssueNodeId(subject.issueNodeId), + }; +} + +/** + * The public run id for one GitHub issue. + * + * The subject is admitted first, so a malformed authority or node id is refused + * before any digest exists and long before anything looks for an owner to route + * it to. + */ +export function* deriveFactoryRunId(subject: FactoryRunSubject): Operation { + const admitted = admitFactoryRunSubject(subject); + const digest = yield* until(crypto.subtle.digest("SHA-256", factoryRunIdPreimage(admitted))); + return base32Unpadded(new Uint8Array(digest)); +} diff --git a/packages/workflow/src/deno/rows.ts b/packages/workflow/src/sqlite/rows.ts similarity index 96% rename from packages/workflow/src/deno/rows.ts rename to packages/workflow/src/sqlite/rows.ts index 30951826c..b09889e03 100644 --- a/packages/workflow/src/deno/rows.ts +++ b/packages/workflow/src/sqlite/rows.ts @@ -10,6 +10,12 @@ * A failure names the column and never the value. Props and journal payloads * are retained history, and a row that does not parse is not a reason to print * what it held. + * + * It lives beside the schema rather than under a host because two adapters read + * the same rows back. The Deno host opens a file with `node:sqlite`; the + * Cloudflare owner reads the storage of one Durable Object. What a stored row + * *means* is the same question in both, and a second copy of these parsers + * would be the place the two hosts quietly stopped agreeing. */ import type { Json } from "@executablemd/durable-streams"; diff --git a/packages/workflow/src/sqlite/workflow-schema.ts b/packages/workflow/src/sqlite/workflow-schema.ts new file mode 100644 index 000000000..df89b9e9e --- /dev/null +++ b/packages/workflow/src/sqlite/workflow-schema.ts @@ -0,0 +1,617 @@ +/** + * The version-1 WorkflowRun schema, as SQLite holds it. + * + * Two adapters keep a run in an embedded SQLite database — the Deno host in a + * file it opens with `node:sqlite`, the Cloudflare owner in the storage of one + * Durable Object — and they must agree about what version 1 *is*. A second copy + * of this DDL under a second adapter would be two schemas that happen to look + * alike, and the first amendment either of them missed would be a run neither + * could recognize. + * + * So the declaration lives here once, and each adapter keeps what is genuinely + * its own: how a connection is opened, how an error is translated, and how the + * identity of the schema is carried. That last one differs because it has to. + * Deno writes `PRAGMA application_id` and `PRAGMA user_version` into the SQLite + * header; Cloudflare's Durable Object storage refuses both pragmas outright, so + * that adapter carries the same two values in a table of its own. The logical + * version is one; only its physical carrier is per-adapter. + * + * Nothing here owns a connection, a path, a transaction or any lifecycle + * authority, and nothing here names a runtime. It is a description of a shape + * and the arithmetic for comparing a database against it. + */ + +/** + * The bytes `XMD1` as a 32-bit integer, written into the SQLite header. + * + * A database carries what wrote it, so a file that is perfectly valid SQLite + * and belongs to something else is refused on sight rather than through the + * confusing shape of its missing tables. + */ +export const APPLICATION_ID = 0x584d4431; + +/** The only schema version this build reads or writes. */ +export const SCHEMA_VERSION = 1; + +/** + * The largest value the schema version can be carried in. + * + * The logical carrier is SQLite's `user_version`, a signed 32-bit integer. Any + * host holding this schema has to represent the same versions, so the bound is + * the carrier's rather than one adapter's. + */ +export const MAX_SCHEMA_VERSION = 0x7fffffff; + +/** + * Whether a retained value could name a schema version at all. + * + * Version numbering starts at 1 and rises. Zero is a database carrying the XMD + * identity without a complete schema, which is a partial initialization and so + * damage; a negative or out-of-range value is retained data that no build of + * this project ever wrote. Neither is a version this build has not learned, so + * neither may travel as one. + */ +export function isSchemaVersion(value: number): boolean { + return Number.isInteger(value) && value >= 1 && value <= MAX_SCHEMA_VERSION; +} + +const STATUSES = "'running', 'suspended', 'interrupted', 'completed', 'failed', 'cancelled'"; + +/** + * A stop reason is three columns wide and has three legal shapes. + * + * Spreading the variant across columns is what lets SQLite hold the invariant + * rather than the code that writes rows: a host reason with an event id, or a + * journal reason with a code, is refused by the database itself. + */ +function coherentStopReason(): string { + return `CHECK ( + (stop_reason_kind IS NULL AND stop_reason_code IS NULL AND stop_reason_event_id IS NULL) + OR (stop_reason_kind = 'host' AND stop_reason_code IS NOT NULL AND stop_reason_event_id IS NULL) + OR (stop_reason_kind = 'journal' AND stop_reason_code IS NULL AND stop_reason_event_id IS NOT NULL) + )`; +} + +/** + * Version 1, one table at a time. + * + * Kept as separate definitions so verification can compare what a file holds + * with what this build writes, rather than settling for the table's name. + * + * The complete version-1 shape includes the pinned DOFS objects, retained + * Workspace roots, journal and metadata. Dependency order is explicit: DOFS + * content precedes root references, and roots precede the journal rows that + * name them. + */ +interface DeclaredObject { + readonly type: "table" | "index"; + readonly sql: string; +} + +export const OBJECTS: ReadonlyMap = new Map([ + [ + "vfs_meta", + { + type: "table", + sql: `CREATE TABLE vfs_meta ( + k TEXT PRIMARY KEY, + v INTEGER NOT NULL + )`, + }, + ], + [ + "vfs_nodes", + { + type: "table", + sql: `CREATE TABLE vfs_nodes ( + inode INTEGER PRIMARY KEY AUTOINCREMENT, + type TEXT NOT NULL CHECK(type IN ('file','dir','symlink')), + mode INTEGER NOT NULL DEFAULT 493, + mtime INTEGER NOT NULL, + rev INTEGER NOT NULL DEFAULT 0, + mount_root TEXT, + stub_size INTEGER, + manifest_hash BLOB, + link_target TEXT, + size INTEGER NOT NULL DEFAULT 0 + )`, + }, + ], + [ + "vfs_dirents", + { + type: "table", + sql: `CREATE TABLE vfs_dirents ( + parent_inode INTEGER NOT NULL, + name TEXT NOT NULL, + child_inode INTEGER NOT NULL, + PRIMARY KEY (parent_inode, name) + ) WITHOUT ROWID`, + }, + ], + [ + "vfs_dirents_by_child", + { + type: "index", + sql: "CREATE INDEX vfs_dirents_by_child ON vfs_dirents(child_inode)", + }, + ], + [ + "vfs_nodes_by_rev", + { + type: "index", + sql: "CREATE INDEX vfs_nodes_by_rev ON vfs_nodes(rev)", + }, + ], + [ + "vfs_nodes_by_manifest_hash", + { + type: "index", + sql: `CREATE INDEX vfs_nodes_by_manifest_hash + ON vfs_nodes(manifest_hash) WHERE manifest_hash IS NOT NULL`, + }, + ], + [ + "vfs_blobs", + { + type: "table", + sql: `CREATE TABLE vfs_blobs ( + hash BLOB PRIMARY KEY, + size INTEGER NOT NULL, + last_seen INTEGER NOT NULL + )`, + }, + ], + [ + "vfs_blob_bytes", + { + type: "table", + sql: `CREATE TABLE vfs_blob_bytes ( + hash BLOB PRIMARY KEY REFERENCES vfs_blobs(hash) ON DELETE CASCADE, + bytes BLOB NOT NULL + )`, + }, + ], + [ + "vfs_chunks", + { + type: "table", + sql: `CREATE TABLE vfs_chunks ( + inode INTEGER NOT NULL, + idx INTEGER NOT NULL, + hash BLOB NOT NULL, + size INTEGER NOT NULL, + PRIMARY KEY (inode, idx) + ) WITHOUT ROWID`, + }, + ], + [ + "vfs_chunks_by_hash", + { + type: "index", + sql: "CREATE INDEX vfs_chunks_by_hash ON vfs_chunks(hash)", + }, + ], + [ + "vfs_manifests", + { + type: "table", + sql: `CREATE TABLE vfs_manifests ( + hash BLOB PRIMARY KEY, + size INTEGER NOT NULL, + encoded BLOB NOT NULL, + last_seen INTEGER NOT NULL DEFAULT 0 + )`, + }, + ], + [ + "vfs_changes", + { + type: "table", + sql: `CREATE TABLE vfs_changes ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + rev INTEGER NOT NULL, + path TEXT NOT NULL, + op TEXT NOT NULL CHECK(op IN ('delete')) + )`, + }, + ], + [ + "vfs_changes_by_rev", + { + type: "index", + sql: "CREATE INDEX vfs_changes_by_rev ON vfs_changes(rev)", + }, + ], + [ + "vfs_changes_by_path", + { + type: "index", + sql: "CREATE INDEX vfs_changes_by_path ON vfs_changes(path, id DESC)", + }, + ], + [ + "_vfs_watermark", + { + type: "table", + sql: `CREATE TABLE _vfs_watermark ( + k TEXT NOT NULL, + backend TEXT NOT NULL DEFAULT 'default', + v INTEGER NOT NULL, + PRIMARY KEY (k, backend) + )`, + }, + ], + [ + "_vfs_fetch_cursor", + { + type: "table", + sql: `CREATE TABLE _vfs_fetch_cursor ( + k TEXT NOT NULL CHECK(k = 'fetch'), + backend TEXT NOT NULL DEFAULT 'default', + path TEXT, + PRIMARY KEY (k, backend) + )`, + }, + ], + [ + "_vfs_mounts", + { + type: "table", + sql: `CREATE TABLE _vfs_mounts ( + root TEXT PRIMARY KEY, + kind TEXT NOT NULL, + indexed INTEGER NOT NULL DEFAULT 0, + mode TEXT NOT NULL DEFAULT 'read-only' + CHECK(mode IN ('read-only', 'read-write')) + )`, + }, + ], + [ + "workspace_roots", + { + type: "table", + sql: `CREATE TABLE workspace_roots ( + root_id TEXT PRIMARY KEY CHECK ( + length(root_id) = 64 AND root_id NOT GLOB '*[^0-9a-f]*' + ), + format_version INTEGER NOT NULL CHECK (format_version = 1), + manifest TEXT NOT NULL CHECK (json_valid(manifest)) +) STRICT`, + }, + ], + [ + "workspace_root_manifest_refs", + { + type: "table", + sql: `CREATE TABLE workspace_root_manifest_refs ( + root_id TEXT NOT NULL REFERENCES workspace_roots(root_id) ON DELETE CASCADE, + manifest_hash BLOB NOT NULL REFERENCES vfs_manifests(hash) ON DELETE RESTRICT, + PRIMARY KEY (root_id, manifest_hash) +) STRICT, WITHOUT ROWID`, + }, + ], + [ + "workspace_root_blob_refs", + { + type: "table", + sql: `CREATE TABLE workspace_root_blob_refs ( + root_id TEXT NOT NULL REFERENCES workspace_roots(root_id) ON DELETE CASCADE, + blob_hash BLOB NOT NULL, + PRIMARY KEY (root_id, blob_hash), + FOREIGN KEY (blob_hash) REFERENCES vfs_blobs(hash) ON DELETE RESTRICT, + FOREIGN KEY (blob_hash) REFERENCES vfs_blob_bytes(hash) ON DELETE RESTRICT +) STRICT, WITHOUT ROWID`, + }, + ], + [ + "agent_sessions", + { + type: "table", + sql: `CREATE TABLE agent_sessions ( + session_key TEXT PRIMARY KEY, + provider TEXT NOT NULL, + agent_command TEXT NOT NULL, + session_identity TEXT NOT NULL, + policy TEXT NOT NULL, + assertion_kind TEXT NOT NULL, + assertion_value TEXT NOT NULL, + created_at TEXT NOT NULL +) STRICT`, + }, + ], + [ + "workspace_state", + { + type: "table", + sql: `CREATE TABLE workspace_state ( + singleton_id INTEGER PRIMARY KEY CHECK (singleton_id = 1), + current_root_id TEXT NOT NULL REFERENCES workspace_roots(root_id) ON DELETE RESTRICT +) STRICT`, + }, + ], + [ + "journal_events", + { + type: "table", + sql: `CREATE TABLE journal_events ( + sequence INTEGER PRIMARY KEY AUTOINCREMENT, + event_id TEXT NOT NULL UNIQUE, + record TEXT NOT NULL CHECK (json_valid(record)), + workspace_root_id TEXT NOT NULL REFERENCES workspace_roots(root_id) ON DELETE RESTRICT +) STRICT`, + }, + ], + [ + "workflow_run", + { + type: "table", + sql: `CREATE TABLE workflow_run ( + id INTEGER PRIMARY KEY CHECK (id = 1), + run_id TEXT NOT NULL, + definition TEXT NOT NULL CHECK (json_valid(definition)), + base TEXT NOT NULL, + props TEXT NOT NULL CHECK (json_valid(props) AND json_type(props) = 'object'), + status TEXT NOT NULL CHECK (status IN (${STATUSES})), + stop_reason_kind TEXT CHECK (stop_reason_kind IS NULL OR stop_reason_kind IN ('host', 'journal')), + stop_reason_code TEXT, + stop_reason_event_id TEXT REFERENCES journal_events (event_id), + created_at TEXT NOT NULL, + updated_at TEXT NOT NULL, + ${coherentStopReason()} +) STRICT`, + }, + ], + [ + "definition_retrieval", + { + type: "table", + sql: `CREATE TABLE definition_retrieval ( + id INTEGER PRIMARY KEY CHECK (id = 1), + metadata TEXT NOT NULL CHECK (json_valid(metadata)), + revision INTEGER NOT NULL CHECK (revision >= 1 AND revision <= 9007199254740991), + updated_at TEXT NOT NULL +) STRICT`, + }, + ], + [ + "document_executions", + { + type: "table", + sql: `CREATE TABLE document_executions ( + sequence INTEGER PRIMARY KEY AUTOINCREMENT, + execution_id TEXT NOT NULL UNIQUE, + started_at TEXT NOT NULL, + stopped_at TEXT, + stop_status TEXT CHECK (stop_status IS NULL OR stop_status IN (${STATUSES})), + stop_reason_kind TEXT CHECK (stop_reason_kind IS NULL OR stop_reason_kind IN ('host', 'journal')), + stop_reason_code TEXT, + stop_reason_event_id TEXT REFERENCES journal_events (event_id), + CHECK ((stopped_at IS NULL) = (stop_status IS NULL)), + CHECK (stop_status IS NOT NULL OR stop_reason_kind IS NULL), + ${coherentStopReason()} +) STRICT`, + }, + ], + [ + "workspace_repositories", + { + type: "table", + sql: `CREATE TABLE workspace_repositories ( + name TEXT PRIMARY KEY CHECK (length(name) > 0), + locator TEXT NOT NULL CHECK (length(locator) > 0), + locator_fingerprint TEXT NOT NULL CHECK ( + length(locator_fingerprint) = 64 AND locator_fingerprint NOT GLOB '*[^0-9a-f]*' + ), + requested_base TEXT CHECK (requested_base IS NULL OR length(requested_base) > 0), + creation_commit TEXT NOT NULL CHECK (length(creation_commit) > 0), + primary_branch TEXT NOT NULL CHECK (length(primary_branch) > 0), + object_format TEXT NOT NULL CHECK (object_format IN ('sha1', 'sha256')), + checkout_path TEXT NOT NULL UNIQUE CHECK ( + length(checkout_path) > 0 AND substr(checkout_path, 1, 1) = '/' + ) +) STRICT`, + }, + ], + [ + "workspace_worktrees", + { + type: "table", + sql: `CREATE TABLE workspace_worktrees ( + repository_name TEXT NOT NULL REFERENCES workspace_repositories(name) ON DELETE RESTRICT, + name TEXT NOT NULL CHECK (length(name) > 0), + requested_branch TEXT NOT NULL CHECK (length(requested_branch) > 0), + requested_base TEXT CHECK (requested_base IS NULL OR length(requested_base) > 0), + creation_commit TEXT NOT NULL CHECK (length(creation_commit) > 0), + checkout_path TEXT NOT NULL UNIQUE CHECK ( + length(checkout_path) > 0 AND substr(checkout_path, 1, 1) = '/' + ), + PRIMARY KEY (repository_name, name) +) STRICT, WITHOUT ROWID`, + }, + ], + [ + "workflow_suspension_answers", + { + type: "table", + sql: `CREATE TABLE workflow_suspension_answers ( + suspension_id TEXT PRIMARY KEY, + request_event_id TEXT NOT NULL REFERENCES journal_events(event_id) ON DELETE RESTRICT, + request_fingerprint TEXT NOT NULL CHECK ( + length(request_fingerprint) = 64 AND request_fingerprint NOT GLOB '*[^0-9a-f]*' + ), + answer TEXT NOT NULL CHECK (json_valid(answer)), + state TEXT NOT NULL CHECK (state IN ('pending', 'consumed')), + created_at TEXT NOT NULL, + consumed_at TEXT, + CHECK ((state = 'consumed') = (consumed_at IS NOT NULL)) +) STRICT`, + }, + ], + [ + "workflow_fork_lineage", + { + type: "table", + sql: `CREATE TABLE workflow_fork_lineage ( + id INTEGER PRIMARY KEY CHECK (id = 1), + source_run_id TEXT NOT NULL CHECK (length(source_run_id) > 0), + checkpoint_event_id TEXT NOT NULL CHECK (length(checkpoint_event_id) > 0), + checkpoint_workspace_root_id TEXT NOT NULL + REFERENCES workspace_roots(root_id) ON DELETE RESTRICT, + created_at TEXT NOT NULL +) STRICT`, + }, + ], + [ + "journal_event_provenance", + { + type: "table", + sql: `CREATE TABLE journal_event_provenance ( + event_id TEXT PRIMARY KEY REFERENCES journal_events(event_id) ON DELETE RESTRICT, + source_run_id TEXT NOT NULL CHECK (length(source_run_id) > 0), + source_event_id TEXT NOT NULL CHECK (length(source_event_id) > 0) +) STRICT, WITHOUT ROWID`, + }, + ], +]); + +export const EXPECTED_SCHEMA = Object.freeze( + [...OBJECTS.entries()].map(([name, object]) => + Object.freeze({ name, type: object.type, sql: normalize(object.sql) }), + ), +); + +/** Objects version 1 declares, including the pinned Cloudflare structure. */ +export const REQUIRED_OBJECTS: readonly string[] = Object.freeze([...OBJECTS.keys()]); + +/** Tables version 1 declares. */ +export const REQUIRED_TABLES: readonly string[] = Object.freeze( + [...OBJECTS.entries()].filter(([, object]) => object.type === "table").map(([name]) => name), +); + +/** Version 1 in full. */ +export const SCHEMA_SQL = [...OBJECTS.values()] + .filter((object) => object.type === "table" && !object.sql.startsWith("CREATE TABLE vfs_")) + .filter((object) => !object.sql.startsWith("CREATE TABLE _vfs_")) + .map((object) => `${object.sql};`) + .join("\n\n"); + +/** One object a database declares, as `sqlite_schema` reports it. */ +export interface SchemaObject { + readonly type: string; + readonly name: string; + readonly sql: string; +} + +/** One statement's shape, independent of how it was laid out. */ +export function normalize(sql: string): string { + return sql.replace(/\s+/g, " ").trim(); +} + +/** + * Every in-place amendment to version 1, newest first. + * + * Each entry names what that amendment added. Peeling them off in order is what + * reconstructs the shapes that once claimed to be a complete version 1, so a + * database an earlier build produced is refused as an incomplete pre-release + * rather than as arbitrary damage. + */ +const AMENDMENTS: readonly (readonly string[])[] = Object.freeze([ + Object.freeze(["workflow_fork_lineage", "journal_event_provenance"]), + Object.freeze(["workflow_suspension_answers"]), + Object.freeze(["workspace_repositories", "workspace_worktrees"]), +]); + +/** What the newest amendment added. Its presence marks a current-shape database. */ +const LATEST_AMENDMENT: readonly string[] = AMENDMENTS[0] ?? []; + +/** The very first pre-release shape, before Workspace root retention existed. */ +const EARLIEST_PRE_RELEASE_SHAPE: readonly string[] = [ + "definition_retrieval", + "document_executions", + "journal_events", + "workflow_run", +]; + +/** + * Every later shape that once claimed to be a complete version 1. + * + * Newest first: version 1 minus the newest amendment, then minus the one before + * it, and so on. + */ +const PRIOR_COMPLETE_SHAPES: readonly (readonly string[])[] = Object.freeze( + AMENDMENTS.map((_, index) => { + const removed = new Set(AMENDMENTS.slice(0, index + 1).flat()); + return Object.freeze(REQUIRED_OBJECTS.filter((name) => !removed.has(name))); + }), +); + +/** + * Whether these declarations describe an earlier shape that once claimed to be + * a complete version 1. + * + * The very first pre-release held only the run, journal and execution tables. + * Every shape after it is version 1 minus whichever amendments had not been + * made yet, and each is named here so the refusal reads as an incomplete + * pre-release rather than as corruption. + */ +export function isIncompletePreReleaseShape(objects: readonly SchemaObject[]): boolean { + const present = new Set(objects.map((object) => object.name)); + if (LATEST_AMENDMENT.some((name) => present.has(name))) { + return false; + } + const earliest = new Set(EARLIEST_PRE_RELEASE_SHAPE); + if (present.size === earliest.size && [...present].every((name) => earliest.has(name))) { + return objects.every((object) => object.type === "table"); + } + return PRIOR_COMPLETE_SHAPES.some((shape) => { + const expected = new Set(shape); + return present.size === expected.size && [...present].every((name) => expected.has(name)); + }); +} + +/** What a structural disagreement is, without either adapter's error types. */ +export type StructureFailure = + | { readonly kind: "incomplete-pre-release" } + | { readonly kind: "undeclared-object"; readonly name: string } + | { readonly kind: "misshapen-object"; readonly name: string } + | { readonly kind: "missing-objects"; readonly names: readonly string[] }; + +/** + * Compare what a database declares with what this build writes. + * + * Answers with the disagreement rather than raising one, because the two + * adapters report the same finding as different failures: a path names the + * file the Deno host refused, and a Durable Object has no path to name. + * + * Recognizing a schema is not reading its table names. A dropped constraint and + * a column that is gone both leave the name intact, so the stored definition of + * every object is compared with the definition version 1 declares. + */ +export function declaredStructureFailure( + objects: readonly SchemaObject[], +): StructureFailure | undefined { + if (isIncompletePreReleaseShape(objects)) { + return { kind: "incomplete-pre-release" }; + } + for (const object of objects) { + const expected = OBJECTS.get(object.name); + if (expected === undefined) { + return { kind: "undeclared-object", name: object.name }; + } + if (object.type !== expected.type || normalize(object.sql) !== normalize(expected.sql)) { + return { kind: "misshapen-object", name: object.name }; + } + } + const present = new Set(objects.map((object) => object.name)); + const missing = REQUIRED_OBJECTS.filter((name) => !present.has(name)); + if (missing.length > 0) { + return { kind: "missing-objects", names: missing }; + } + return undefined; +} + +/** Whether any object version 1 declares is present at all. */ +export function hasAnyDeclaredObject(objects: readonly SchemaObject[]): boolean { + return objects.some((object) => OBJECTS.has(object.name)); +} diff --git a/packages/workflow/src/storage/agent-session.ts b/packages/workflow/src/storage/agent-session.ts new file mode 100644 index 000000000..5fd371494 --- /dev/null +++ b/packages/workflow/src/storage/agent-session.ts @@ -0,0 +1,215 @@ +/** + * What one retained Agent session is, independent of who stores it. + * + * A run remembers that a `` element was attached to a provider's + * conversation so a later execution can reattach to the same one. What it + * remembers is deliberately thin: which provider, which resolved command, the + * engine-derived expansion identity, the policy in force, and the provider's + * own assertion about the session. The conversation is the provider's and is + * never retained, sent, or reconstructed. + * + * Both hosts retain this, so the shape and the key derivation live here rather + * than inside either one. A second derivation would be two keys for one + * session, and reattachment would silently start a new conversation. + */ + +import { sha256Hex } from "../workspace/sha256.ts"; + +/** + * What a provider says about a session it created. + * + * Tagged, because "the adapter's own session id", "an ACP session id" and "a + * record id in some store" are different claims that happen to be strings. A + * host comparing them without the tag would accept one for another. + */ +export interface ProviderAssertion { + readonly kind: string; + readonly value: string; +} + +/** What identifies one logical Agent session. */ +export interface AgentSessionIdentity { + /** Which provider holds the conversation, as that provider names itself. */ + readonly provider: string; + /** The resolved agent command, not the name a document wrote. */ + readonly agentCommand: string; + /** The engine-derived Agent/Session expansion identity. Never authored. */ + readonly sessionIdentity: string; +} + +/** One retained mapping, as a run's storage holds it. */ +export interface AgentSessionRecord extends AgentSessionIdentity { + readonly sessionKey: string; + /** The session policy in force when the provider created this session. */ + readonly policy: string; + readonly assertion: ProviderAssertion; + readonly createdAt: string; +} + +/** + * The key one logical session is retained under, within this run. + * + * The engine-derived Session expansion identity and nothing else. The provider + * and the resolved agent command are compatibility attributes stored beside it: + * changing either refuses reattachment rather than addressing a second mapping, + * because a `` element that changed agent is the same element asking + * for something this run cannot give it. + * + * Digested so it stays bounded, and namespaced so a row is recognizable. + */ +export function agentSessionKey(identity: AgentSessionIdentity): string { + return ["xmd", "workflow", "v1", sha256Hex(identity.sessionIdentity).slice(0, 32)].join(":"); +} + +function text(found: Map, name: string): string | undefined { + const value = found.get(name); + return typeof value === "string" && value !== "" ? value : undefined; +} + +function members(value: unknown): Map | undefined { + if (value === null || typeof value !== "object" || Array.isArray(value)) { + return undefined; + } + return new Map(Object.entries(value)); +} + +/** + * Read one retained mapping out of a value nothing has checked. + * + * Every member, and the key recomputed from the identity rather than believed. + * A record whose key does not follow from its own identity is a record that + * would be retained under a name nothing could look it up by. + */ +export function parseAgentSessionRecord(value: unknown): AgentSessionRecord | undefined { + const found = members(value); + if (found === undefined || found.size !== 7) { + return undefined; + } + const provider = text(found, "provider"); + const agentCommand = text(found, "agentCommand"); + const sessionIdentity = text(found, "sessionIdentity"); + const sessionKey = text(found, "sessionKey"); + const policy = text(found, "policy"); + const assertion = members(found.get("assertion")); + if ( + provider === undefined || + agentCommand === undefined || + sessionIdentity === undefined || + sessionKey === undefined || + policy === undefined || + assertion === undefined || + assertion.size !== 2 + ) { + return undefined; + } + const kind = text(assertion, "kind"); + const asserted = text(assertion, "value"); + if (kind === undefined || asserted === undefined) { + return undefined; + } + const identity: AgentSessionIdentity = { provider, agentCommand, sessionIdentity }; + if (agentSessionKey(identity) !== sessionKey) { + return undefined; + } + const createdAt = text(found, "createdAt"); + if (createdAt === undefined || new Date(createdAt).toISOString() !== createdAt) { + return undefined; + } + return Object.freeze({ + ...identity, + sessionKey, + policy, + assertion: Object.freeze({ kind, value: asserted }), + createdAt, + }); +} + +/** A retained Agent session this host will not continue under. */ +export class WorkflowAgentSessionError extends Error { + override name = "WorkflowAgentSessionError"; +} + +/** Every retained mapping one run holds, as a coordinator may reach it. */ +export interface AgentSessions { + read(sessionKey: string): AgentSessionRecord | undefined; + commit(record: AgentSessionRecord): void; +} + +/** What a continuation may do with the session a key names. */ +export type AgentSessionResolution = + | { readonly kind: "create"; readonly sessionKey: string } + | { readonly kind: "reattach"; readonly record: AgentSessionRecord }; + +/** + * Decide what this attachment may do with the session this identity names. + * + * `asserted` is every canonical identity the provider currently asserts for that + * key — none, one, or more than one. It is deliberately not "does the provider + * hold this key": occupancy says something is there, not what conversation it + * is, and adopting one on that basis is how a run continues a session it cannot + * name. + */ +export function resolveAgentSession( + retained: AgentSessionRecord | undefined, + policy: string, + asserted: readonly ProviderAssertion[], + identity: AgentSessionIdentity, +): AgentSessionResolution { + const sessionKey = agentSessionKey(identity); + if (asserted.length > 1) { + throw new WorkflowAgentSessionError( + "the provider asserts more than one durable identity for this run's Agent session, so " + + "this host cannot tell which conversation it would be continuing. Start a new run " + + "rather than continuing this one.", + ); + } + const current = asserted[0]; + + if (retained === undefined) { + if (current === undefined) { + // Neither side holds anything: nothing was ever established here. + return { kind: "create", sessionKey }; + } + // The pre-commit window. An attempt was interrupted between the provider + // asserting an identity and this run recording it, and exactly one + // canonical assertion is what reconciles it — nothing else may. + return { + kind: "reattach", + record: { + sessionKey, + ...identity, + policy, + assertion: current, + createdAt: new Date().toISOString(), + }, + }; + } + + if ( + retained.provider !== identity.provider || + retained.agentCommand !== identity.agentCommand || + retained.sessionIdentity !== identity.sessionIdentity || + retained.policy !== policy + ) { + throw new WorkflowAgentSessionError( + "this run's Agent session was established under a different provider, agent or session " + + "policy than this host states, and a session created under one ceiling is not " + + "continued under another. Start a new run rather than continuing this one.", + ); + } + if (current === undefined) { + throw new WorkflowAgentSessionError( + "the provider asserts no durable identity for the Agent session this run retained, and " + + "this host does not reconstruct a conversation by replaying it into a new session. " + + "Start a new run rather than continuing this one.", + ); + } + if (current.kind !== retained.assertion.kind || current.value !== retained.assertion.value) { + throw new WorkflowAgentSessionError( + "the provider asserts a different durable identity than the Agent session this run " + + "retained, so it did not resume the conversation this run was having. This host does " + + "not continue under a replacement session.", + ); + } + return { kind: "reattach", record: retained }; +} diff --git a/packages/workflow/src/storage/definition.ts b/packages/workflow/src/storage/definition.ts index cda336a5b..935e038b7 100644 --- a/packages/workflow/src/storage/definition.ts +++ b/packages/workflow/src/storage/definition.ts @@ -16,7 +16,11 @@ */ import { Err, Ok, type Result } from "effection"; -import { isCanonicalDocumentTarget, isComponentName } from "@executablemd/core"; +// The node-free subpaths: this module is reached from a Cloudflare Worker, and +// the package root's barrel resolves host modules a Worker cannot load. Both +// predicates are the same public functions, selected through a narrower path. +import { isCanonicalDocumentTarget } from "@executablemd/core/document-target"; +import { isComponentName } from "@executablemd/core/component-name"; import type { Json } from "@executablemd/durable-streams"; import { WorkflowDefinitionError } from "./errors.ts"; import { diff --git a/packages/workflow/src/storage/record.ts b/packages/workflow/src/storage/record.ts index 37c78d0f5..c626fa958 100644 --- a/packages/workflow/src/storage/record.ts +++ b/packages/workflow/src/storage/record.ts @@ -14,7 +14,10 @@ */ import { Err, Ok, type Result } from "effection"; -import { canonicalize } from "@executablemd/core"; +// The node-free subpath: this module is reached from a Cloudflare Worker, and +// the package root's barrel resolves `node:crypto` and the rest of the host +// surface. Same function, narrower resolution path. +import { canonicalize } from "@executablemd/core/canonicalize"; import type { Json } from "@executablemd/durable-streams"; import type { WorkflowDefinition } from "./definition.ts"; import { WorkflowRequestError } from "./errors.ts"; diff --git a/packages/workflow/src/workspace/capture.ts b/packages/workflow/src/workspace/capture.ts new file mode 100644 index 000000000..a49b0b21f --- /dev/null +++ b/packages/workflow/src/workspace/capture.ts @@ -0,0 +1,203 @@ +/** + * Turning a tree of nodes into the canonical root that names it. + * + * Capture happens in two places that share nothing else. The local host walks + * the DOFS tables inside its SQLite file; a remote runner walks a real + * directory it materialized on disk. Neither walk is shareable — one reads rows + * and the other reads a filesystem — but what the walk *means* has to be + * identical, because the root identity is a digest of the encoding and two + * hosts that encoded differently would produce two roots for one Workspace. + * + * So the walk stays with whoever can perform it, and everything after the walk + * lives here: ordering, hardlink numbering, manifest encoding, chunk identity + * and the root digest. A caller hands over what it found and receives the root + * that describes it, or a refusal saying it does not describe one. + * + * The rule this exists to protect is narrow and worth stating plainly: an + * untouched materialization must capture back to the exact root it came from. + * If it did not, every no-op Workspace operation would propose a new root, and + * a run would appear to change its Workspace by looking at it. + */ + +import { + compareUtf8, + type WorkspaceRejection, + type WorkspaceRootEntry, + validateWorkspaceRootEntries, + WORKSPACE_ROOT_DOMAIN, + WORKSPACE_ROOT_FORMAT, +} from "./root-manifest.ts"; +import { + CHUNK_SIZE, + type ContentChunkReference, + encodeContentManifest, +} from "./content-manifest.ts"; +import { sha256Hex } from "./sha256.ts"; + +/** One node a walk found, before anything is ordered or numbered. */ +export type CapturedNode = + | { + readonly path: string; + readonly kind: "directory"; + readonly mode: number; + readonly mtime: number; + } + | { + readonly path: string; + readonly kind: "symlink"; + readonly mode: number; + readonly mtime: number; + readonly target: string; + } + | { + readonly path: string; + readonly kind: "file"; + readonly mode: number; + readonly mtime: number; + readonly size: number; + /** The content manifest identity of this file's bytes. */ + readonly manifest: string; + /** + * What makes two paths the same file rather than two copies. + * + * An inode on a real filesystem, an inode number in DOFS. Two entries + * sharing one are a hardlink group; `undefined` is a file reached by one + * path. Identical bytes are deliberately *not* enough — two independent + * files that happen to match are two files, and a capture that merged + * them would materialize back as something the run never had. + */ + readonly identity: string | undefined; + }; + +/** What one file's bytes are, once chunked. */ +export interface CapturedContent { + readonly manifest: string; + readonly manifestBytes: Uint8Array; + readonly chunks: readonly ContentChunkReference[]; +} + +/** The root one capture describes, and the content it closes over. */ +export interface CapturedRoot { + readonly rootId: string; + readonly manifest: string; + readonly entries: readonly WorkspaceRootEntry[]; + /** Every content manifest identity this root names, in canonical order. */ + readonly manifests: readonly string[]; + /** Every blob identity those manifests name, in canonical order. */ + readonly blobs: readonly string[]; +} + +const encoder = new TextEncoder(); + +/** + * Split one file's bytes the way the content store splits them. + * + * An empty file has no chunks, which is not the same as having one chunk of + * nothing: its manifest names zero bytes and is still a manifest, and every + * empty file in a Workspace shares it. + */ +export function captureContent(bytes: Uint8Array): CapturedContent { + const chunks: ContentChunkReference[] = []; + for (let offset = 0; offset < bytes.length; offset += CHUNK_SIZE) { + const slice = bytes.subarray(offset, Math.min(offset + CHUNK_SIZE, bytes.length)); + chunks.push({ hash: sha256Hex(slice), size: slice.length }); + } + const manifestBytes = encodeContentManifest(chunks); + return { manifest: sha256Hex(manifestBytes), manifestBytes, chunks }; +} + +/** The identity a canonical root manifest has. */ +export function workspaceRootIdOf(manifest: string): string { + return sha256Hex(`${WORKSPACE_ROOT_DOMAIN}${manifest}`); +} + +/** + * Order the nodes, number the hardlink groups, and encode the root. + * + * Ordering is by UTF-8 bytes because that is what the format declares, and + * hardlink groups are numbered by the byte order of their first path so that + * the same tree numbers the same way whoever walked it — a group numbered by + * discovery order would depend on the walk, and the two walks are different. + * + * The result is validated against the same entry rules a stored root is read + * back through. A capture that produced something the reader would refuse is a + * bug worth finding here rather than at the owner. + */ +export function captureWorkspaceRoot( + nodes: readonly CapturedNode[], + contents: ReadonlyMap, + reject: WorkspaceRejection, +): CapturedRoot { + const ordered = nodes.toSorted((left, right) => compareUtf8(left.path, right.path)); + + const shared = new Map(); + for (const node of ordered) { + if (node.kind === "file" && node.identity !== undefined) { + shared.set(node.identity, [...(shared.get(node.identity) ?? []), node.path]); + } + } + const group = new Map(); + const groups = [...shared.values()] + .filter((paths) => paths.length > 1) + .map((paths) => paths.toSorted(compareUtf8)) + .toSorted((left, right) => compareUtf8(left[0] ?? "", right[0] ?? "")); + for (const [index, paths] of groups.entries()) { + for (const path of paths) { + group.set(path, `h${index}`); + } + } + + const entries: WorkspaceRootEntry[] = ordered.map((node) => { + if (node.kind === "directory") { + return { path: node.path, kind: node.kind, mode: node.mode, mtime: node.mtime }; + } + if (node.kind === "symlink") { + return { + path: node.path, + kind: node.kind, + mode: node.mode, + mtime: node.mtime, + target: node.target, + }; + } + return { + path: node.path, + kind: node.kind, + mode: node.mode, + mtime: node.mtime, + size: node.size, + manifest: node.manifest, + hardlink: group.get(node.path) ?? null, + }; + }); + + validateWorkspaceRootEntries(entries, reject); + const manifest = JSON.stringify({ format: WORKSPACE_ROOT_FORMAT, entries }); + + const manifests = new Set(); + const blobs = new Set(); + for (const entry of entries) { + if (entry.kind !== "file") { + continue; + } + const content = contents.get(entry.manifest); + if (content === undefined) { + reject("a captured Workspace file names content the capture did not produce"); + } + if (entry.size !== content.chunks.reduce((total, chunk) => total + chunk.size, 0)) { + reject("a captured Workspace file size disagrees with its content"); + } + manifests.add(entry.manifest); + for (const chunk of content.chunks) { + blobs.add(chunk.hash); + } + } + + return { + rootId: workspaceRootIdOf(manifest), + manifest, + entries, + manifests: [...manifests].toSorted(compareUtf8), + blobs: [...blobs].toSorted(compareUtf8), + }; +} diff --git a/packages/workflow/src/workspace/content-manifest.ts b/packages/workflow/src/workspace/content-manifest.ts new file mode 100644 index 000000000..82c04b836 --- /dev/null +++ b/packages/workflow/src/workspace/content-manifest.ts @@ -0,0 +1,129 @@ +/** + * How a file's bytes are described, once they are in the content store. + * + * A Workspace root names a file's content by one identity; it says nothing + * about how those bytes are kept. That is this format's job: an ordered list of + * chunks, each named by its own digest, encoded canonically so that identical + * bytes always produce one identity. + * + * Every host keeps content this way, so the rules are shared and name no host. + * Which store implements them, and in what tables, is the storage adapter's + * business and stays there — a neutral module that named one would be the + * Workspace surface learning where it happened to be kept. + * + * Nothing here opens a store, hashes anything or names a runtime. It decides + * whether a sequence of bytes is a canonically encoded manifest, and produces + * the bytes one ought to be. + */ + +import { SHA256, type WorkspaceRejection } from "./root-manifest.ts"; + +/** + * The size a file's bytes are split at. + * + * Pinned to what the vendored content layer uses. A writer that chunked + * differently would compute different manifest identities for identical bytes, + * and the store would then hold two names for one file. + */ +export const CHUNK_SIZE = 512 * 1024; + +/** One chunk a file's bytes are stored as. */ +export interface ContentChunkReference { + readonly hash: string; + readonly size: number; +} + +/** One file's bytes, as the store describes them. */ +export interface ContentManifest { + readonly size: number; + readonly chunks: readonly ContentChunkReference[]; +} + +const encoder = new TextEncoder(); +const decoder = new TextDecoder("utf-8", { fatal: true, ignoreBOM: false }); + +/** The bytes a content manifest is stored and identified as. */ +export function encodeContentManifest(chunks: readonly ContentChunkReference[]): Uint8Array { + return encoder.encode( + JSON.stringify({ + version: 1, + chunks: chunks.map((chunk) => ({ hash: chunk.hash, size: chunk.size })), + }), + ); +} + +function isSafeInteger(value: unknown): value is number { + return typeof value === "number" && Number.isSafeInteger(value); +} + +function members(value: unknown): Map | undefined { + if (value === null || typeof value !== "object" || Array.isArray(value)) { + return undefined; + } + return new Map(Object.entries(value)); +} + +function declares(found: Map, expected: readonly string[]): boolean { + return found.size === expected.length && expected.every((name) => found.has(name)); +} + +/** + * The manifest one encoding describes, without a store to look anything up + * in. + * + * The same bytes are validated in more than one place — by a live run reading + * its own content store, by a reader checking a detached copy, and by an owner + * about to send a copy to a runner. What is decided here is only whether these + * bytes are a canonically encoded content manifest at all, and what size the + * chunks it names add up to. That the chunks exist is whoever called's to prove. + */ +export function decodeContentManifest( + encoded: Uint8Array, + reject: WorkspaceRejection, +): ContentManifest { + let text: string; + let offered: unknown; + try { + text = decoder.decode(encoded); + offered = JSON.parse(text); + } catch { + reject("a content manifest is not canonical UTF-8 JSON"); + } + const found = members(offered); + const chunks = found?.get("chunks"); + if ( + found === undefined || + !declares(found, ["version", "chunks"]) || + found.get("version") !== 1 || + !Array.isArray(chunks) + ) { + reject("a content manifest is not canonically encoded"); + } + const references: ContentChunkReference[] = []; + for (const chunk of chunks) { + const entry = members(chunk); + const hash = entry?.get("hash"); + const size = entry?.get("size"); + if ( + entry === undefined || + !declares(entry, ["hash", "size"]) || + typeof hash !== "string" || + !SHA256.test(hash) || + !isSafeInteger(size) || + size < 1 + ) { + // A zero-length chunk names no bytes, so a manifest that lists one is + // describing content it does not have. + reject("a content manifest is not canonically encoded"); + } + references.push({ hash, size }); + } + if (JSON.stringify({ version: 1, chunks: references }) !== text) { + reject("a content manifest is not canonically encoded"); + } + const total = references.reduce((sum, chunk) => sum + chunk.size, 0); + if (!Number.isSafeInteger(total)) { + reject("a content manifest names more bytes than a size can hold"); + } + return Object.freeze({ size: total, chunks: Object.freeze(references) }); +} diff --git a/packages/workflow/src/workspace/failure.ts b/packages/workflow/src/workspace/failure.ts new file mode 100644 index 000000000..d13f65df7 --- /dev/null +++ b/packages/workflow/src/workspace/failure.ts @@ -0,0 +1,28 @@ +/** + * A failure the Workspace effect publishes instead of raising. + * + * The distinction is not "what went wrong" but "who this belongs to". A failure + * of this kind is part of what the effect *did*: it is written into the journal + * as the effect's result, the Workspace root stays where it was, and a replay + * reproduces it without performing anything. Every other failure is the run + * failing, and travels as an ordinary raise. + * + * It is a base class rather than a predicate over shapes so that being + * publishable is something a failure declares by construction. A module that + * wants its own refusal published extends this; nothing acquires the property + * by resembling something. + * + * Shared because both coordinators have to make the same choice, and two + * classifiers would eventually disagree about which failures are the run's. + */ +export abstract class JournaledEffectFailure extends Error {} + +/** + * Whether this failure is the effect's outcome rather than the run's failure. + * + * Asked by the one place in each host that has to choose between writing a + * result and letting a failure through. + */ +export function isJournaledEffectFailure(error: unknown): error is Error { + return error instanceof JournaledEffectFailure; +} diff --git a/packages/workflow/src/workspace/filesystem.ts b/packages/workflow/src/workspace/filesystem.ts new file mode 100644 index 000000000..07bfc57bc --- /dev/null +++ b/packages/workflow/src/workspace/filesystem.ts @@ -0,0 +1,43 @@ +/** + * The Workspace filesystem, as an operation rather than a place. + * + * Both hosts run the same Workspace work and neither one's storage is the + * contract. The Deno host's Workspace is rows in the run's own SQLite database + * reached through DOFS; the runner's is a real directory it materialized from + * the owner. What a mutation is allowed to ask for is the same either way, so + * it is stated here and implemented twice. + * + * Every member is an `Operation`. That is not decoration: one implementation is + * synchronous by necessity and the other is asynchronous by necessity, and a + * caller written against either shape would only work against that one. + */ + +import type { Operation } from "effection"; + +export interface WorkspaceEntry { + readonly name: string; + readonly kind: "file" | "directory" | "symlink"; +} + +export interface WorkspaceStat { + readonly kind: "file" | "directory" | "symlink"; + readonly mode: number; + readonly mtime: number; + readonly size: number; +} + +export interface WorkspaceFilesystem { + readFile(path: string): Operation; + readTextFile(path: string): Operation; + stat(path: string): Operation; + lstat(path: string): Operation; + readlink(path: string): Operation; + readdir(path: string): Operation; + writeFile(path: string, content: string | Uint8Array, mode?: number): Operation; + mkdir(path: string, options?: { recursive?: boolean; mode?: number }): Operation; + remove(path: string, options?: { recursive?: boolean; force?: boolean }): Operation; + rename(from: string, to: string): Operation; + chmod(path: string, mode: number): Operation; + symlink(target: string, path: string): Operation; + link(existingPath: string, newPath: string): Operation; +} diff --git a/packages/workflow/src/workspace/metadata.ts b/packages/workflow/src/workspace/metadata.ts new file mode 100644 index 000000000..905ac9803 --- /dev/null +++ b/packages/workflow/src/workspace/metadata.ts @@ -0,0 +1,33 @@ +/** + * Retained Repository and Worktree identity, as a mutation may reach it. + * + * These rows are immutable creation identity. Insertion adds one and never + * mutates one; a reused name is answered by reading the row back and comparing + * it. Where the rows live is the host's business — SQLite rows inside the Deno + * transaction's savepoint, a detached invocation snapshot and staged deltas on + * the runner — and the rules that decide whether a reused name is the same + * repository are shared, so they are stated against this interface rather than + * against either store. + * + * The locator is retained beside the record rather than inside it. Deciding + * whether a reused name asks for the same repository needs the bytes; the + * journal and the document need only the fingerprint, and a URL that turned out + * to carry a credential is then one column rather than one history. + */ + +import type { RepositoryRecord, WorktreeRecord } from "../composition/records.ts"; + +/** A Repository row: its journal-safe record, and the locator only storage sees. */ +export interface StoredRepository { + readonly record: RepositoryRecord; + readonly locator: string; +} + +export interface WorkspaceMetadata { + readRepository(name: string): StoredRepository | undefined; + readRepositories(): StoredRepository[]; + insertRepository(stored: StoredRepository): void; + readWorktree(repositoryName: string, name: string): WorktreeRecord | undefined; + readWorktreesForRepository(repositoryName: string): WorktreeRecord[]; + insertWorktree(record: WorktreeRecord): void; +} diff --git a/packages/workflow/src/workspace/root-manifest.ts b/packages/workflow/src/workspace/root-manifest.ts new file mode 100644 index 000000000..90090d74f --- /dev/null +++ b/packages/workflow/src/workspace/root-manifest.ts @@ -0,0 +1,356 @@ +/** + * What a retained Workspace root *is*, independent of who stored it. + * + * A root is a canonical JSON manifest and the content-addressed objects its + * entries name. Its identity is the SHA-256 of a domain-separated encoding of + * that manifest, so two hosts holding the same bytes hold the same root and + * neither has to be asked. + * + * Two adapters retain roots — the Deno host in a SQLite file, the Cloudflare + * owner in the storage of one Durable Object — and a second copy of these rules + * under a second adapter would be the place they stopped agreeing. Whichever + * one is looser decides what the other must accept, and the looser one is + * always the newer one. So the rules live here once. + * + * Nothing here opens a database, hashes anything, or names a runtime. Hashing + * is deliberately absent: each host has its own primitive for it, and this + * module has no business choosing between them. What it decides is whether a + * sequence of bytes is a canonically encoded manifest at all, and what that + * manifest says. + * + * A caller supplies `reject`, because the same disagreement is reported very + * differently depending on who found it: a path names the file the Deno host + * refused, a Durable Object has no path to name, and a sealed artifact is not a + * run database at all. + */ + +/** The only root format this build reads or writes. */ +export const WORKSPACE_ROOT_FORMAT = 1; + +/** + * What a root identity is taken over, before the manifest itself. + * + * Domain separation, so a digest of a Workspace root can never collide with a + * digest of anything else this system hashes. + */ +export const WORKSPACE_ROOT_DOMAIN = "xmd-workspace-root\0v1\0"; + +/** A lowercase SHA-256 identity, which is the only spelling any of this uses. */ +export const SHA256 = /^[0-9a-f]{64}$/; + +/** How a reader says these bytes are not a root it can accept. */ +export type WorkspaceRejection = (reason: string) => never; + +/** One directory in a root. */ +export interface WorkspaceDirectoryEntry { + readonly path: string; + readonly kind: "directory"; + readonly mode: number; + readonly mtime: number; +} + +/** One file in a root, named by the DOFS manifest holding its bytes. */ +export interface WorkspaceFileEntry { + readonly path: string; + readonly kind: "file"; + readonly mode: number; + readonly mtime: number; + readonly size: number; + readonly manifest: string; + readonly hardlink: string | null; +} + +/** One symbolic link in a root. */ +export interface WorkspaceSymlinkEntry { + readonly path: string; + readonly kind: "symlink"; + readonly mode: number; + readonly mtime: number; + readonly target: string; +} + +export type WorkspaceRootEntry = + | WorkspaceDirectoryEntry + | WorkspaceFileEntry + | WorkspaceSymlinkEntry; + +export interface WorkspaceRootManifest { + readonly format: typeof WORKSPACE_ROOT_FORMAT; + readonly entries: readonly WorkspaceRootEntry[]; +} + +const encoder = new TextEncoder(); +const decoder = new TextDecoder("utf-8", { fatal: true, ignoreBOM: false }); + +/** + * Compare two paths by their UTF-8 bytes. + * + * Byte order rather than `String` order, because a root's canonical ordering is + * a property of its encoding: two hosts that sorted differently would disagree + * about whether the same entries are the same root. + */ +export function compareUtf8(left: string, right: string): number { + const a = encoder.encode(left); + const b = encoder.encode(right); + const shared = Math.min(a.length, b.length); + for (let index = 0; index < shared; index += 1) { + const first = a[index] ?? 0; + const second = b[index] ?? 0; + if (first !== second) { + return first < second ? -1 : 1; + } + } + return a.length === b.length ? 0 : a.length < b.length ? -1 : 1; +} + +/** The directory one canonical path sits in. */ +export function parentPath(path: string): string { + const boundary = path.lastIndexOf("/"); + return boundary === 0 ? "/" : path.slice(0, boundary); +} + +/** Depth first, then byte order — the order a restore creates entries in. */ +export function parentFirst(left: WorkspaceRootEntry, right: WorkspaceRootEntry): number { + const depth = left.path.split("/").length - right.path.split("/").length; + return depth === 0 ? compareUtf8(left.path, right.path) : depth; +} + +/** + * Whether text contains a code unit that is not part of a valid pair. + * + * An unpaired surrogate survives a round trip through JSON and does not survive + * one through UTF-8, so a manifest carrying one is a manifest whose bytes + * cannot be reproduced. + */ +export function hasUnpairedSurrogate(value: string): boolean { + for (let index = 0; index < value.length; index += 1) { + const code = value.charCodeAt(index); + if (code >= 0xd800 && code <= 0xdbff) { + const next = value.charCodeAt(index + 1); + if (next < 0xdc00 || next > 0xdfff) { + return true; + } + index += 1; + } else if (code >= 0xdc00 && code <= 0xdfff) { + return true; + } + } + return false; +} + +function isSafeInteger(value: unknown): value is number { + return typeof value === "number" && Number.isSafeInteger(value); +} + +function members(value: unknown): Map | undefined { + if (value === null || typeof value !== "object" || Array.isArray(value)) { + return undefined; + } + return new Map(Object.entries(value)); +} + +/** + * Whether an object declares exactly these members and no others. + * + * A manifest is compared with its own re-encoding further down, so an extra + * member would already be caught. It is refused here as well because the reason + * matters: an unknown member is a manifest this build does not understand, + * which is a different thing from bytes that were laid out differently. + */ +function declares(found: Map, expected: readonly string[]): boolean { + if (found.size !== expected.length) { + return false; + } + return expected.every((name) => found.has(name)); +} + +function mode(value: unknown): boolean { + return isSafeInteger(value) && value >= 0 && value <= 0o7777; +} + +/** + * Read one entry, in the exact member order this build writes. + * + * The order matters and is not a style choice: a manifest is compared with its + * own re-encoding, and a re-encoding that named the same members in a different + * order would be refused as noncanonical. So each branch builds its object + * literally rather than by spreading a shared prefix. + */ +function entryOf(value: unknown): WorkspaceRootEntry | undefined { + const found = members(value); + if (found === undefined) { + return undefined; + } + const path = found.get("path"); + const kind = found.get("kind"); + const entryMode = found.get("mode"); + const mtime = found.get("mtime"); + if ( + typeof path !== "string" || + !mode(entryMode) || + !isSafeInteger(entryMode) || + !isSafeInteger(mtime) + ) { + return undefined; + } + + if (kind === "directory") { + return declares(found, ["path", "kind", "mode", "mtime"]) + ? { path, kind, mode: entryMode, mtime } + : undefined; + } + if (kind === "symlink") { + const target = found.get("target"); + if (typeof target !== "string") { + return undefined; + } + return declares(found, ["path", "kind", "mode", "mtime", "target"]) + ? { path, kind, mode: entryMode, mtime, target } + : undefined; + } + if (kind !== "file") { + return undefined; + } + const size = found.get("size"); + const manifest = found.get("manifest"); + const hardlink = found.get("hardlink"); + if (!isSafeInteger(size) || size < 0 || typeof manifest !== "string" || !SHA256.test(manifest)) { + return undefined; + } + if (hardlink !== null && (typeof hardlink !== "string" || !/^h[0-9]+$/.test(hardlink))) { + return undefined; + } + return declares(found, ["path", "kind", "mode", "mtime", "size", "manifest", "hardlink"]) + ? { path, kind, mode: entryMode, mtime, size, manifest, hardlink } + : undefined; +} + +/** + * Read one root manifest out of the exact text a store retained. + * + * Three separate questions, in order: is it JSON, is it a manifest this build + * declares, and are these the exact bytes this build would have written for + * that manifest. The last one is what makes the identity meaningful — a root ID + * is a digest of these bytes, so a manifest that means the same thing and is + * spelled differently is a different root and must not be admitted as this one. + */ +export function parseWorkspaceRootManifest( + manifest: string, + reject: WorkspaceRejection, +): WorkspaceRootManifest { + let offered: unknown; + try { + offered = JSON.parse(manifest); + } catch { + reject("one of its retained Workspace roots is not JSON"); + } + const found = members(offered); + const declared = found !== undefined && declares(found, ["format", "entries"]); + const entries = found?.get("entries"); + if (!declared || found?.get("format") !== WORKSPACE_ROOT_FORMAT || !Array.isArray(entries)) { + reject("one of its retained Workspace roots has an invalid manifest"); + } + const parsed: WorkspaceRootEntry[] = []; + for (const entry of entries) { + const admitted = entryOf(entry); + if (admitted === undefined) { + reject("one of its retained Workspace roots has an invalid manifest"); + } + parsed.push(admitted); + } + const root: WorkspaceRootManifest = { format: WORKSPACE_ROOT_FORMAT, entries: parsed }; + validateWorkspaceRootEntries(parsed, reject); + if (JSON.stringify(root) !== manifest) { + reject("one of its retained Workspace roots is not canonically encoded"); + } + return root; +} + +/** + * Whether these entries describe a Workspace at all. + * + * Shape is not enough. A root is a tree, and its manifest is a flat list, so + * the tree lives in these rules: the list starts at the root directory, every + * path is canonical, order is total and by bytes, every entry has a parent that + * was already declared, and a hardlink group is numbered in the order it first + * appears and agrees with itself. + */ +export function validateWorkspaceRootEntries( + entries: readonly WorkspaceRootEntry[], + reject: WorkspaceRejection, +): void { + if (entries.length === 0 || entries[0]?.path !== "/" || entries[0]?.kind !== "directory") { + reject("a Workspace root does not begin with its root directory"); + } + + let previous: string | undefined; + let nextHardlink = 0; + const directories = new Set(); + const hardlinkMembers = new Map(); + const hardlinkFirst = new Map(); + + for (const entry of entries) { + validateCanonicalWorkspacePath(entry.path, reject); + if (previous !== undefined && compareUtf8(previous, entry.path) >= 0) { + reject("a Workspace root's paths are duplicated or out of canonical order"); + } + previous = entry.path; + + if (entry.path !== "/" && !directories.has(parentPath(entry.path))) { + reject("a Workspace root contains an entry without a parent directory"); + } + if (entry.kind === "directory") { + directories.add(entry.path); + } + if ( + entry.kind === "symlink" && + (entry.target.includes("\0") || hasUnpairedSurrogate(entry.target)) + ) { + reject("a Workspace root contains an invalid symbolic-link target"); + } + if (entry.kind === "file" && entry.hardlink !== null) { + const first = hardlinkFirst.get(entry.hardlink); + if (first === undefined) { + if (entry.hardlink !== `h${nextHardlink}`) { + reject("a Workspace root's hardlinks are not canonically numbered"); + } + nextHardlink += 1; + hardlinkFirst.set(entry.hardlink, entry); + } else if ( + first.mode !== entry.mode || + first.mtime !== entry.mtime || + first.size !== entry.size || + first.manifest !== entry.manifest + ) { + reject("a Workspace root's hardlink group has inconsistent metadata"); + } + hardlinkMembers.set(entry.hardlink, (hardlinkMembers.get(entry.hardlink) ?? 0) + 1); + } + } + + for (const count of hardlinkMembers.values()) { + if (count < 2) { + reject("a Workspace root contains a one-member hardlink group"); + } + } +} + +/** One absolute path with no traversal, no empty component and no surprises. */ +export function validateCanonicalWorkspacePath(value: string, reject: WorkspaceRejection): void { + if (value === "/") { + return; + } + if ( + !value.startsWith("/") || + value.endsWith("/") || + value.includes("\0") || + hasUnpairedSurrogate(value) + ) { + reject("a Workspace root contains a noncanonical path"); + } + for (const part of value.slice(1).split("/")) { + if (part === "" || part === "." || part === "..") { + reject("a Workspace root contains a noncanonical path component"); + } + } +} diff --git a/packages/workflow/src/workspace/sha256.ts b/packages/workflow/src/workspace/sha256.ts new file mode 100644 index 000000000..957601942 --- /dev/null +++ b/packages/workflow/src/workspace/sha256.ts @@ -0,0 +1,119 @@ +/** + * SHA-256, in the language itself. + * + * Every host this package runs on has a SHA-256 already, and none of them has + * one this code can use. `node:crypto` is a host specifier, and the whole point + * of a shared module is that it names no host. `crypto.subtle.digest()` is + * asynchronous, and the place this is needed most is inside a Durable Object's + * synchronous transaction, where there is nothing to await into. + * + * So the arithmetic lives here. A content identity is what decides whether two + * hosts are holding the same Workspace root, and a digest that differed between + * them would be two systems quietly disagreeing about history. FIPS 180-4 is + * fixed, small, and has published answers, which is why this is a reasonable + * thing to carry: the tests hold it to those answers and to the identity the + * Deno host computes with its own primitive. + * + * It hashes bytes already in memory. It is not a streaming interface and is not + * for anything large; the private protocol bounds every piece it is used on. + */ + +const INITIAL = new Uint32Array([ + 0x6a09e667, 0xbb67ae85, 0x3c6ef372, 0xa54ff53a, 0x510e527f, 0x9b05688c, 0x1f83d9ab, 0x5be0cd19, +]); + +const ROUND = new Uint32Array([ + 0x428a2f98, 0x71374491, 0xb5c0fbcf, 0xe9b5dba5, 0x3956c25b, 0x59f111f1, 0x923f82a4, 0xab1c5ed5, + 0xd807aa98, 0x12835b01, 0x243185be, 0x550c7dc3, 0x72be5d74, 0x80deb1fe, 0x9bdc06a7, 0xc19bf174, + 0xe49b69c1, 0xefbe4786, 0x0fc19dc6, 0x240ca1cc, 0x2de92c6f, 0x4a7484aa, 0x5cb0a9dc, 0x76f988da, + 0x983e5152, 0xa831c66d, 0xb00327c8, 0xbf597fc7, 0xc6e00bf3, 0xd5a79147, 0x06ca6351, 0x14292967, + 0x27b70a85, 0x2e1b2138, 0x4d2c6dfc, 0x53380d13, 0x650a7354, 0x766a0abb, 0x81c2c92e, 0x92722c85, + 0xa2bfe8a1, 0xa81a664b, 0xc24b8b70, 0xc76c51a3, 0xd192e819, 0xd6990624, 0xf40e3585, 0x106aa070, + 0x19a4c116, 0x1e376c08, 0x2748774c, 0x34b0bcb5, 0x391c0cb3, 0x4ed8aa4a, 0x5b9cca4f, 0x682e6ff3, + 0x748f82ee, 0x78a5636f, 0x84c87814, 0x8cc70208, 0x90befffa, 0xa4506ceb, 0xbef9a3f7, 0xc67178f2, +]); + +function rotate(value: number, bits: number): number { + return (value >>> bits) | (value << (32 - bits)); +} + +function padded(input: Uint8Array): Uint8Array { + const length = Math.ceil((input.length + 9) / 64) * 64; + const bytes = new Uint8Array(length); + bytes.set(input); + bytes[input.length] = 0x80; + const bits = BigInt(input.length) * 8n; + for (let index = 0; index < 8; index += 1) { + bytes[length - 1 - index] = Number((bits >> BigInt(index * 8)) & 0xffn); + } + return bytes; +} + +export function sha256(value: Uint8Array | string): Uint8Array { + const input = typeof value === "string" ? new TextEncoder().encode(value) : value; + const bytes = padded(input); + const state = new Uint32Array(INITIAL); + const words = new Uint32Array(64); + for (let offset = 0; offset < bytes.length; offset += 64) { + for (let index = 0; index < 16; index += 1) { + const at = offset + index * 4; + words[index] = + ((bytes[at] ?? 0) << 24) | + ((bytes[at + 1] ?? 0) << 16) | + ((bytes[at + 2] ?? 0) << 8) | + (bytes[at + 3] ?? 0); + } + for (let index = 16; index < 64; index += 1) { + const x = words[index - 15] ?? 0; + const y = words[index - 2] ?? 0; + const sigma0 = rotate(x, 7) ^ rotate(x, 18) ^ (x >>> 3); + const sigma1 = rotate(y, 17) ^ rotate(y, 19) ^ (y >>> 10); + words[index] = ((words[index - 16] ?? 0) + sigma0 + (words[index - 7] ?? 0) + sigma1) >>> 0; + } + let a = state[0] ?? 0; + let b = state[1] ?? 0; + let c = state[2] ?? 0; + let d = state[3] ?? 0; + let e = state[4] ?? 0; + let f = state[5] ?? 0; + let g = state[6] ?? 0; + let h = state[7] ?? 0; + for (let index = 0; index < 64; index += 1) { + const sum1 = rotate(e, 6) ^ rotate(e, 11) ^ rotate(e, 25); + const choice = (e & f) ^ (~e & g); + const first = (h + sum1 + choice + (ROUND[index] ?? 0) + (words[index] ?? 0)) >>> 0; + const sum0 = rotate(a, 2) ^ rotate(a, 13) ^ rotate(a, 22); + const majority = (a & b) ^ (a & c) ^ (b & c); + const second = (sum0 + majority) >>> 0; + h = g; + g = f; + f = e; + e = (d + first) >>> 0; + d = c; + c = b; + b = a; + a = (first + second) >>> 0; + } + state[0] = ((state[0] ?? 0) + a) >>> 0; + state[1] = ((state[1] ?? 0) + b) >>> 0; + state[2] = ((state[2] ?? 0) + c) >>> 0; + state[3] = ((state[3] ?? 0) + d) >>> 0; + state[4] = ((state[4] ?? 0) + e) >>> 0; + state[5] = ((state[5] ?? 0) + f) >>> 0; + state[6] = ((state[6] ?? 0) + g) >>> 0; + state[7] = ((state[7] ?? 0) + h) >>> 0; + } + const digest = new Uint8Array(32); + for (let index = 0; index < state.length; index += 1) { + const word = state[index] ?? 0; + digest[index * 4] = word >>> 24; + digest[index * 4 + 1] = word >>> 16; + digest[index * 4 + 2] = word >>> 8; + digest[index * 4 + 3] = word; + } + return digest; +} + +export function sha256Hex(value: Uint8Array | string): string { + return Array.from(sha256(value), (byte) => byte.toString(16).padStart(2, "0")).join(""); +} diff --git a/packages/workflow/tests/cloudflare/env.d.ts b/packages/workflow/tests/cloudflare/env.d.ts new file mode 100644 index 000000000..e0dfafaa6 --- /dev/null +++ b/packages/workflow/tests/cloudflare/env.d.ts @@ -0,0 +1,13 @@ +import type { ExecutorObject } from "./support/executor-object.ts"; +import type { OwnerObject } from "./support/owner-object.ts"; +import type { StorageProbeObject } from "./support/probe-object.ts"; + +declare global { + namespace Cloudflare { + interface Env { + STORAGE_PROBE: DurableObjectNamespace; + OWNER: DurableObjectNamespace; + EXECUTOR: DurableObjectNamespace; + } + } +} diff --git a/packages/workflow/tests/cloudflare/executor-acquisition.vitest.ts b/packages/workflow/tests/cloudflare/executor-acquisition.vitest.ts new file mode 100644 index 000000000..86c14bfe2 --- /dev/null +++ b/packages/workflow/tests/cloudflare/executor-acquisition.vitest.ts @@ -0,0 +1,386 @@ +/** + * Who may execute a run, on real workerd. + * + * Admission order and acquisition lifetime are the two things this suite is + * about. The order matters because a mismatched build must not reach a token + * and a bad token must not reach run state; the lifetime matters because the + * connection *is* the acquisition, with no lease to expire and no heartbeat to + * miss, so the only proof that ownership ended is that the socket did. + */ + +import { env, runInDurableObject } from "cloudflare:test"; +import { beforeAll, describe, expect, it } from "vitest"; +import { serializeDurableEvent } from "@executablemd/durable-streams"; +import type { ExecutorObject } from "./support/executor-object.ts"; +import { POLICY, VALID_CLAIMS } from "./support/executor-object.ts"; +import { generateKeys, signToken, tamper, type TestKeys } from "./support/tokens.ts"; + +let unique = 0; + +function executor() { + unique += 1; + return env.EXECUTOR.get(env.EXECUTOR.idFromName(`executor-${unique}-${Math.random()}`)); +} + +function on( + stub: ReturnType, + body: (instance: ExecutorObject) => T, +): Promise> { + return runInDurableObject(stub, body) as Promise>; +} + +const RUN_ID = "5cktgrv2zyutngh7bbddr2tyg2b5a567cg725hu5e7u42orerxaa"; + +/** The clock the owner is configured with, so expiry is exact. */ +const NOW = 1_800_000_000; + +let keys: TestKeys; +let otherKeys: TestKeys; + +beforeAll(async () => { + keys = await generateKeys(); + otherKeys = await generateKeys("other-key"); +}); + +/** Claims a correctly issued token carries, plus any override. */ +function claims(overrides: Record = {}): Record { + return { ...VALID_CLAIMS, iat: NOW - 10, nbf: NOW - 10, exp: NOW + 600, ...overrides }; +} + +/** An owner configured with the real public key, ready to be connected to. */ +async function admitted( + stub: ReturnType, + request: Record = {}, + signWith: TestKeys = keys, + header: Record = {}, +): Promise { + await on(stub, (o) => o.configure([{ kid: keys.kid, jwk: keys.publicJwk }], NOW)); + const token = "token" in request ? request["token"] : await signToken(signWith, claims(), header); + return await on(stub, (o) => o.admitConnection({ ...request, token })); +} + +describe("admitting an executor", () => { + it("admits a matching build with authenticated claims", async () => { + const stub = executor(); + expect(await admitted(stub)).toBe("admitted"); + expect(await on(stub, (o) => o.holders())).toBe(1); + }); + + it("refuses a build the owner did not agree to, before reading the token", async () => { + const stub = executor(); + // The token is deliberately unusable. If the release were checked after it, + // the refusal would name the token rather than the build. + expect(await admitted(stub, { release: "other-build", token: "not a token" })).toBe( + "release:release-mismatch", + ); + expect(await on(stub, (o) => o.holders())).toBe(0); + }); + + it("refuses an absent or malformed build identity", async () => { + const stub = executor(); + expect(await on(stub, (o) => o.admitConnection({ release: undefined }))).toBe( + "release:release-absent", + ); + expect(await on(stub, (o) => o.admitConnection({ release: "not a fingerprint" }))).toBe( + "release:release-malformed", + ); + expect(await on(stub, (o) => o.holders())).toBe(0); + }); + + it("refuses every claim the policy names, one at a time", async () => { + const cases: [string, Record][] = [ + ["admission:issuer", { iss: "https://evil.example" }], + ["admission:audience", { aud: "https://somebody-else" }], + ["admission:repository-id", { repository_id: "999" }], + ["admission:repository-owner-id", { repository_owner_id: "999" }], + ["admission:event-name", { event_name: "push" }], + [ + "admission:workflow-ref", + { + workflow_ref: "octo/repo/.github/workflows/other.yml@refs/heads/main", + }, + ], + [ + "admission:workflow-sha", + { + workflow_sha: "1111111111111111111111111111111111111111", + }, + ], + [ + "admission:workflow-identity", + { + job_workflow_ref: "octo/repo/.github/workflows/other.yml@refs/heads/main", + }, + ], + ]; + for (const [expected, overrides] of cases) { + const stub = executor(); + const token = await signToken(keys, claims(overrides)); + expect(await admitted(stub, { token })).toBe(expected); + expect(await on(stub, (o) => o.holders())).toBe(0); + } + }); + + it("accepts an audience array containing the configured one", async () => { + const stub = executor(); + const token = await signToken(keys, claims({ aud: ["https://other", POLICY.audience] })); + expect(await admitted(stub, { token })).toBe("admitted"); + }); + + it("refuses a token whose payload was edited after signing", async () => { + const stub = executor(); + const token = tamper(await signToken(keys, claims()), claims({ repository_id: "999" })); + expect(await admitted(stub, { token })).toBe("token:bad-signature"); + expect(await on(stub, (o) => o.holders())).toBe(0); + }); + + it("refuses a token naming a key the deployment does not hold", async () => { + const stub = executor(); + // Signed by another issuer, and saying so: no configured key is even a + // candidate, which is a different refusal from one that failed to verify. + expect(await admitted(stub, {}, otherKeys)).toBe("token:unknown-key"); + }); + + it("refuses a token signed with the wrong key under a configured key id", async () => { + const stub = executor(); + const token = await signToken(otherKeys, claims(), { kid: keys.kid }); + expect(await admitted(stub, { token })).toBe("token:bad-signature"); + expect(await on(stub, (o) => o.holders())).toBe(0); + }); + + it("refuses an algorithm it does not support", async () => { + const stub = executor(); + const token = await signToken(keys, claims(), { alg: "none" }); + expect(await admitted(stub, { token })).toBe("token:unsupported-algorithm"); + }); + + it("refuses a token that is absent or not a compact JWS", async () => { + const stub = executor(); + expect(await admitted(stub, { token: undefined })).toBe("token:token-absent"); + expect(await admitted(stub, { token: "one.two" })).toBe("token:token-malformed"); + }); + + it("requires every temporal claim, rather than treating an absent one as met", async () => { + for (const missing of ["exp", "iat", "nbf"]) { + const stub = executor(); + const without = claims(); + delete without[missing]; + expect(await admitted(stub, { token: await signToken(keys, without) })).toBe( + "token:malformed-claims", + ); + } + // And a claim that is present but not a NumericDate. + for (const wrong of [{ exp: "soon" }, { iat: 1.5 }, { nbf: null }]) { + const stub = executor(); + expect(await admitted(stub, { token: await signToken(keys, claims(wrong)) })).toBe( + "token:malformed-claims", + ); + } + }); + + it("treats the expiration boundary itself as expired", async () => { + // RFC 7519 wants the current time strictly before `exp`. With no skew, a + // token expiring exactly now is spent. + const exact = executor(); + await on(exact, (o) => o.configure([{ kid: keys.kid, jwk: keys.publicJwk }], NOW)); + const boundary = await signToken(keys, claims({ exp: NOW })); + expect( + await on(exact, (o) => o.admitConnection({ token: boundary, release: POLICY.release })), + ).toBe("token:expired"); + }); + + it("requires a key id naming exactly one configured key", async () => { + const absent = executor(); + expect( + await admitted(absent, { token: await signToken(keys, claims(), { kid: undefined }) }), + ).toBe("token:unknown-key"); + const empty = executor(); + expect(await admitted(empty, { token: await signToken(keys, claims(), { kid: "" }) })).toBe( + "token:unknown-key", + ); + const unknown = executor(); + expect( + await admitted(unknown, { token: await signToken(keys, claims(), { kid: "nope" }) }), + ).toBe("token:unknown-key"); + }); + + it("requires the header to say it is a JWT", async () => { + const stub = executor(); + const token = await signToken(keys, claims(), { typ: "at+jwt" }); + expect(await admitted(stub, { token })).toBe("token:unsupported-type"); + }); + + it("refuses a clock configuration it cannot trust", async () => { + const negative = executor(); + await on(negative, (o) => o.configure([{ kid: keys.kid, jwk: keys.publicJwk }], NOW, -1)); + expect( + await on(negative, (o) => o.admitConnection({ release: POLICY.release, token: "a.b.c" })), + ).toBe("token:misconfigured-clock"); + + const huge = executor(); + await on(huge, (o) => o.configure([{ kid: keys.kid, jwk: keys.publicJwk }], NOW, 86_400)); + expect( + await on(huge, (o) => o.admitConnection({ release: POLICY.release, token: "a.b.c" })), + ).toBe("token:misconfigured-clock"); + }); + + it("refuses a token outside its validity window", async () => { + const expired = executor(); + expect( + await admitted(expired, { token: await signToken(keys, claims({ exp: NOW - 3600 })) }), + ).toBe("token:expired"); + const early = executor(); + expect( + await admitted(early, { token: await signToken(keys, claims({ nbf: NOW + 3600 })) }), + ).toBe("token:not-yet-valid"); + }); + + it("refuses a run id that could not address an owner", async () => { + const stub = executor(); + expect(await admitted(stub, { runId: "" })).toBe("run-id:run-id-empty"); + expect(await admitted(stub, { runId: 42 })).toBe("run-id:run-id-absent"); + expect(await on(stub, (o) => o.holders())).toBe(0); + }); +}); + +/** A record the owner will accept: exactly what the serializer produces. */ +function serializedEvent(name: string): string { + return serializeDurableEvent({ + type: "yield", + coroutineId: "root", + description: { type: "test", name }, + result: { status: "ok", value: name }, + }); +} + +describe("holding an acquisition", () => { + it("refuses a second healthy executor rather than following it", async () => { + const stub = executor(); + expect(await admitted(stub)).toBe("admitted"); + expect(await admitted(stub)).toBe("acquisition:already-running"); + expect(await on(stub, (o) => o.holders())).toBe(1); + }); + + it("mints its own correlation, which no caller can select or reuse", async () => { + const first = executor(); + await admitted(first); + const one = await on(first, (o) => o.acquisitionId()); + await on(first, (o) => o.closeConnection(1)); + await admitted(first); + const two = await on(first, (o) => o.acquisitionId()); + + // Bounded, unpredictable, and different for a second acquisition of the + // same run — so private staging belonging to the first cannot be addressed + // by the second. + expect(one).toMatch(/^[0-9a-f]{32}$/); + expect(two).toMatch(/^[0-9a-f]{32}$/); + expect(two).not.toBe(one); + }); + + it("lets the admitted connection send, and answers what it performed", async () => { + const stub = executor(); + await on(stub, (o) => o.initialize()); + await admitted(stub); + const answer = await on(stub, (o) => + o.send(1, JSON.stringify({ id: "1", command: "frontier" })), + ); + expect(answer).toMatchObject({ id: "1", outcome: "performed" }); + }); + + it("refuses a socket it never admitted", async () => { + const stub = executor(); + await admitted(stub); + expect( + await on(stub, (o) => o.sendAsStranger(JSON.stringify({ id: "1", command: "frontier" }))), + ).toEqual({ id: "", outcome: "refused", refusal: "acquisition:foreign-connection" }); + }); + + it("does not treat copied attachment bytes as an acquisition", async () => { + const stub = executor(); + await admitted(stub); + expect( + await on(stub, (o) => + o.sendWithCopiedAttachment(JSON.stringify({ id: "1", command: "frontier" })), + ), + ).toEqual({ id: "", outcome: "refused", refusal: "acquisition:foreign-connection" }); + }); + + it("owns nothing once the connection ends, and rolls nothing back", async () => { + const stub = executor(); + await admitted(stub); + await on(stub, (o) => o.closeConnection(1)); + expect(await on(stub, (o) => o.holders())).toBe(0); + // And the next executor may take it, with no lease having expired. + expect(await admitted(stub)).toBe("admitted"); + }); + + it("proves the acquisition before it reads a command", async () => { + const stub = executor(); + // Nothing is admitted, so even a well-formed command is refused for + // ownership rather than for its shape. + expect( + await on(stub, (o) => o.sendAsStranger(JSON.stringify({ id: "1", command: "frontier" }))), + ).toEqual({ id: "", outcome: "refused", refusal: "acquisition:not-acquired" }); + }); +}); + +describe("reading a runner command", () => { + it("refuses what it cannot read as one", async () => { + const stub = executor(); + await admitted(stub); + const refuse = async (raw: string) => + (await on(stub, (o) => o.send(1, raw))) as { refusal: string }; + expect((await refuse("not json")).refusal).toBe("command:not-an-object"); + expect((await refuse(JSON.stringify([1, 2]))).refusal).toBe("command:not-an-object"); + expect((await refuse(JSON.stringify({ id: "1", command: "explode" }))).refusal).toBe( + "command:unknown-command", + ); + expect((await refuse(JSON.stringify({ id: "1", command: "frontier", extra: 1 }))).refusal).toBe( + "command:unknown-member", + ); + expect((await refuse(JSON.stringify({ command: "frontier" }))).refusal).toBe( + "command:malformed-member", + ); + expect((await refuse(JSON.stringify({ id: "1", command: "root" }))).refusal).toBe( + "command:malformed-member", + ); + }); + + it("reads a commit intent whole, then refuses one proposed against a moved frontier", async () => { + const stub = executor(); + await on(stub, (o) => o.initialize()); + await admitted(stub); + const raw = JSON.stringify({ + id: "7", + command: "commit", + expectedWorkspaceRootId: `a${"0".repeat(63)}`, + expectedJournalEventId: null, + publication: { + proposedWorkspaceRootId: `b${"1".repeat(63)}`, + proposedManifest: "{}", + content: [], + }, + mappings: [], + events: [serializedEvent("read whole")], + }); + // The shape is read — an unknown member or a malformed root would refuse + // differently — and then declined on its merits: this run's frontier is not + // the root the proposal says it started from. + expect(await on(stub, (o) => o.send(1, raw))).toEqual({ + id: "7", + outcome: "refused", + refusal: "command:stale-root", + }); + expect( + await on(stub, (o) => o.send(1, JSON.stringify({ ...JSON.parse(raw), id: "8", extra: 1 }))), + ).toEqual({ id: "", outcome: "refused", refusal: "command:unknown-member" }); + }); +}); + +describe("routing a run to its owner", () => { + it("reaches one object for one run id, without a registry", () => { + const first = env.EXECUTOR.idFromName(RUN_ID).toString(); + expect(env.EXECUTOR.idFromName(RUN_ID).toString()).toBe(first); + expect(env.EXECUTOR.idFromName(`${RUN_ID}x`).toString()).not.toBe(first); + }); +}); diff --git a/packages/workflow/tests/cloudflare/owner-storage.vitest.ts b/packages/workflow/tests/cloudflare/owner-storage.vitest.ts new file mode 100644 index 000000000..1a28ea72c --- /dev/null +++ b/packages/workflow/tests/cloudflare/owner-storage.vitest.ts @@ -0,0 +1,179 @@ +/** + * The owner's storage, on real workerd. + * + * Initialization, recognition and the one transaction an owner commit runs + * inside are all properties of the runtime rather than of a model of it: the + * marker exists because the pragmas are refused, and the direct DOFS enlistment + * exists because a reentrant transaction is refused. Each object below gets a + * fresh name so its storage starts pristine. + */ + +import { env, runInDurableObject } from "cloudflare:test"; +import { describe, expect, it } from "vitest"; +import type { OwnerObject } from "./support/owner-object.ts"; + +let unique = 0; + +function owner() { + unique += 1; + const name = `owner-${unique}-${Math.random().toString(36).slice(2)}`; + return env.OWNER.get(env.OWNER.idFromName(name)); +} + +function on( + stub: ReturnType, + body: (instance: OwnerObject) => T, +): Promise> { + return runInDurableObject(stub, body) as Promise>; +} + +describe("initializing an owner object", () => { + it("creates the schema, the DOFS schema, the run row and the marker together", async () => { + const stub = owner(); + expect(await on(stub, (o) => o.initialize())).toBe("initialized"); + expect(await on(stub, (o) => o.marker())).toEqual([ + { application_id: 0x584d4431, schema_version: 1 }, + ]); + expect(await on(stub, (o) => o.recognize())).toBe("recognized"); + }); + + it("refuses storage that already holds something", async () => { + const stub = owner(); + await on(stub, (o) => o.addForeignObject()); + expect(await on(stub, (o) => o.initialize())).toBe("refused:foreign"); + }); +}); + +describe("recognizing an owner object", () => { + it("refuses storage that holds nothing at all", async () => { + expect(await on(owner(), (o) => o.recognize())).toBe("refused:foreign"); + }); + + it("refuses storage carrying objects but no marker", async () => { + const stub = owner(); + await on(stub, (o) => o.addForeignObject()); + expect(await on(stub, (o) => o.recognize())).toBe("refused:foreign"); + }); + + it("refuses another application's identity", async () => { + const stub = owner(); + await on(stub, (o) => o.initialize()); + await on(stub, (o) => o.rewriteMarker(0x11111111, 1)); + expect(await on(stub, (o) => o.recognize())).toBe("refused:foreign"); + }); + + it("refuses a version this build does not implement", async () => { + const stub = owner(); + await on(stub, (o) => o.initialize()); + await on(stub, (o) => o.rewriteMarker(0x584d4431, 2)); + expect(await on(stub, (o) => o.recognize())).toBe("refused:unsupported-version"); + }); + + it("calls version zero a partial initialization rather than an old version", async () => { + // This project's identity with nothing finished under it. There has never + // been a version zero to be behind, so reporting one would send a host + // looking for a migration that cannot exist. + const stub = owner(); + await on(stub, (o) => o.initialize()); + await on(stub, (o) => o.rewriteMarker(0x584d4431, 0)); + expect(await on(stub, (o) => o.recognize())).toBe("refused:corrupt"); + }); + + it("carries a version wider than the refusal's old grammar", async () => { + const stub = owner(); + await on(stub, (o) => o.initialize()); + await on(stub, (o) => o.rewriteMarker(0x584d4431, 1_000_000)); + expect(await on(stub, (o) => o.recognize())).toBe("refused:unsupported-version"); + }); + + it("calls a version the carrier could never hold damaged retained data", async () => { + // Negative, and past the signed 32-bit carrier: no build of this project + // wrote either. A version this build has not learned and a row that cannot + // be a version are different facts. + for (const version of [-1, 0x80000000]) { + const stub = owner(); + await on(stub, (o) => o.initialize()); + await on(stub, (o) => o.rewriteMarker(0x584d4431, version)); + expect([version, await on(stub, (o) => o.recognize())]).toEqual([version, "refused:corrupt"]); + } + }); + + it("refuses a shape that disagrees with what version 1 declares", async () => { + const stub = owner(); + await on(stub, (o) => o.initialize()); + await on(stub, (o) => o.damage("workflow_suspension_answers")); + expect(await on(stub, (o) => o.recognize())).toBe("refused:corrupt"); + }); + + it("refuses a missing Cloudflare-private protocol table", async () => { + const stub = owner(); + await on(stub, (o) => o.initialize()); + await on(stub, (o) => o.damage("_xmd_executor_commands")); + expect(await on(stub, (o) => o.recognize())).toBe("refused:corrupt"); + }); +}); + +describe("an owner commit", () => { + it("publishes DOFS content and WorkflowRun rows together", async () => { + const stub = owner(); + await on(stub, (o) => o.initialize()); + expect(await on(stub, (o) => o.frontier())).toEqual({ status: "running", publishedPaths: 0 }); + + expect(await on(stub, (o) => o.commitMixedChange(false))).toBe("committed"); + expect(await on(stub, (o) => o.frontier())).toEqual({ + status: "suspended", + publishedPaths: 1, + }); + }); + + it("rolls both categories back when the body fails after changing each", async () => { + const stub = owner(); + await on(stub, (o) => o.initialize()); + expect(await on(stub, (o) => o.commitMixedChange(true))).toContain("threw:"); + + // Neither the filesystem write nor the row update may survive, and the next + // operation must not read either of them out of a cache the failed + // transaction populated. + expect(await on(stub, (o) => o.frontier())).toEqual({ status: "running", publishedPaths: 0 }); + expect(await on(stub, (o) => o.recognize())).toBe("recognized"); + }); + + it("commits after a failed attempt, from the frontier the failure left", async () => { + const stub = owner(); + await on(stub, (o) => o.initialize()); + await on(stub, (o) => o.commitMixedChange(true)); + expect(await on(stub, (o) => o.commitMixedChange(false))).toBe("committed"); + expect(await on(stub, (o) => o.frontier())).toEqual({ + status: "suspended", + publishedPaths: 1, + }); + }); +}); + +describe("owner transaction ownership", () => { + it("refuses a nested transaction on the same storage", async () => { + const stub = owner(); + await on(stub, (o) => o.initialize()); + expect(await on(stub, (o) => o.nestOnSameStorage())).toBe("refused:nested"); + }); + + it("does not couple a transaction on one storage to another storage", async () => { + // A module-level flag would refuse the second transaction because the first + // was open. Every Durable Object in an isolate shares this module and + // shares nothing else, so the guard is keyed by the storage it governs. + const stub = owner(); + await on(stub, (o) => o.initialize()); + expect(await on(stub, (o) => o.transactOnADifferentStorage())).toBe( + "committed while another storage transacted", + ); + }); + + it("releases the storage however its transaction ended", async () => { + const stub = owner(); + await on(stub, (o) => o.initialize()); + // A throwing transaction must leave the storage free for the next one. + await on(stub, (o) => o.commitMixedChange(true)); + expect(await on(stub, (o) => o.commitMixedChange(false))).toBe("committed"); + expect(await on(stub, (o) => o.nestOnSameStorage())).toBe("refused:nested"); + }); +}); diff --git a/packages/workflow/tests/cloudflare/remote-owner.vitest.ts b/packages/workflow/tests/cloudflare/remote-owner.vitest.ts new file mode 100644 index 000000000..1138f2bab --- /dev/null +++ b/packages/workflow/tests/cloudflare/remote-owner.vitest.ts @@ -0,0 +1,783 @@ +/** + * The owner's half of the private protocol, on real workerd. + * + * Almost nothing here would be worth proving against a model. Hibernation is a + * property of the runtime: the object is evicted, its fields are gone, and what + * comes back is whatever the storage and the live sockets say. Acquisition + * replacement is a property of the runtime's socket list. Transaction + * atomicity, `WITHOUT ROWID` constraints and blob round-trips are properties of + * the Durable Object's SQLite. A map standing in for any of those would prove + * that the map behaves, which is not the claim. + * + * So these run against a real namespace, real storage, real Hibernation + * WebSockets and a real `evictDurableObject()`, and the assertions are about + * what survived, what was refused, and what was left untouched. + */ + +import { env, evictDurableObject, runInDurableObject } from "cloudflare:test"; +import { beforeAll, describe, expect, it } from "vitest"; +import { MAX_COMMANDS, MAX_CONTENT_BYTES } from "../../src/cloudflare/commands.ts"; +import { encodeBase64 } from "../../src/cloudflare/encoding.ts"; +import { sha256Hex } from "../../src/workspace/sha256.ts"; +import type { ExecutorObject } from "./support/executor-object.ts"; +import { + BLOB_ID, + DOFS_MANIFEST, + FILE_BYTES, + MANIFEST_ID, + POLICY, + ROOT_ID, + ROOT_MANIFEST, + RUN_ID, + VALID_CLAIMS, +} from "./support/executor-object.ts"; +import { generateKeys, signToken, type TestKeys } from "./support/tokens.ts"; +import { run } from "effection"; +import { + type OwnerSocket, + type SocketListener, + useOwnerConnection, +} from "../../src/remote/client.ts"; +import { cloudflareReadLink, cloudflareRunLink } from "../../src/cloudflare/client.ts"; + +let unique = 0; +const NEW_START = "2026-02-02T00:00:00.000Z"; +const NOW = 1_800_000_000; +let keys: TestKeys; + +beforeAll(async () => { + keys = await generateKeys(); +}); + +function executor() { + unique += 1; + return env.EXECUTOR.get(env.EXECUTOR.idFromName(`remote-${unique}-${Math.random()}`)); +} + +function on( + stub: ReturnType, + body: (instance: ExecutorObject) => T, +): Promise { + return runInDurableObject(stub, body); +} + +async function admission(stub: ReturnType): Promise { + await on(stub, (owner) => owner.configure([{ kid: keys.kid, jwk: keys.publicJwk }], NOW)); + const token = await signToken(keys, { + ...VALID_CLAIMS, + iat: NOW - 10, + nbf: NOW - 10, + exp: NOW + 600, + }); + return await on(stub, (owner) => owner.admitConnection({ token, release: POLICY.release })); +} + +async function admit(stub: ReturnType): Promise { + expect(await admission(stub)).toBe("admitted"); +} + +async function connect(stub: ReturnType): Promise { + await on(stub, (owner) => owner.configure([{ kid: keys.kid, jwk: keys.publicJwk }], NOW)); + const token = await signToken(keys, { + ...VALID_CLAIMS, + iat: NOW - 10, + nbf: NOW - 10, + exp: NOW + 600, + }); + const response = await stub.fetch("https://owner.invalid/executor", { + headers: { + authorization: `Bearer ${token}`, + upgrade: "websocket", + "x-release": POLICY.release, + "x-run-id": RUN_ID, + }, + }); + const socket = response.webSocket; + if (socket === null) { + throw new Error(`expected an executor WebSocket, received ${response.status}`); + } + socket.accept(); + return socket; +} + +function ask( + socket: WebSocket, + id: string, + command: Record, +): Promise> { + return askFrame(socket, JSON.stringify({ id, ...command })); +} + +function askFrame( + socket: WebSocket, + message: string | ArrayBuffer, +): Promise> { + return new Promise((resolve, reject) => { + const receive = (event: MessageEvent) => { + socket.removeEventListener("message", receive); + if (typeof event.data !== "string") { + reject(new Error("expected a text answer")); + return; + } + resolve(record(JSON.parse(event.data))); + }; + socket.addEventListener("message", receive); + socket.send(message); + }); +} + +/** + * The platform socket, as the runner's client needs it. + * + * A host binds its own socket to this interface; the runtime's event types are + * wider than the four members the client uses, so the binding is written out + * rather than asserted. + */ +function ownerSocket(socket: WebSocket, beforeSend?: (raw: string) => Promise | undefined) { + const listeners = new Map(); + const bound: OwnerSocket = { + send(data) { + // A frame may be held back before it reaches the owner, which is how a + // test puts a write between two pages of one read without reaching + // inside the client. + const waiting = beforeSend?.(data); + if (waiting === undefined) { + socket.send(data); + return; + } + void waiting.then(() => socket.send(data)); + }, + close: () => socket.close(), + addEventListener(type, listener) { + const forward: EventListener = (event) => listener(event as { data?: unknown }); + listeners.set(listener, forward); + socket.addEventListener(type, forward); + }, + removeEventListener(type, listener) { + const bound = listeners.get(listener); + const found = listeners.get(listener); + if (found !== undefined) { + socket.removeEventListener(type, found); + } + }, + }; + return bound; +} + +function record(value: unknown): Record { + if (value === null || typeof value !== "object" || Array.isArray(value)) { + throw new Error("expected an object answer"); + } + return Object.fromEntries(Object.entries(value)); +} + +function send( + stub: ReturnType, + id: string, + command: Record, +): Promise> { + return on(stub, (owner) => record(owner.send(1, JSON.stringify({ id, ...command })))); +} + +describe("the remote owner protocol", () => { + it("answers a binary frame once and closes the protocol", async () => { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + const socket = await connect(stub); + expect(await askFrame(socket, new Uint8Array([1]).buffer)).toEqual({ + id: "", + outcome: "refused", + refusal: "command:malformed-member", + }); + await new Promise((resolve) => setTimeout(resolve, 0)); + expect(socket.readyState).not.toBe(WebSocket.OPEN); + }); + + it("refuses pristine, foreign, unsupported, damaged, missing, and wrong-run storage", async () => { + const cases: readonly [string, (owner: ExecutorObject) => void, string][] = [ + ["pristine", () => undefined, "storage:foreign"], + ["foreign", (owner) => owner.makeForeign(), "storage:foreign"], + [ + "unsupported", + (owner) => { + owner.initialize(); + owner.rewriteMarker(0x584d4431, 2); + }, + "storage:unsupported-version-v2", + ], + [ + "damaged", + (owner) => { + owner.initialize(); + owner.dropTable("workflow_suspension_answers"); + }, + "storage:corrupt", + ], + [ + "missing", + (owner) => { + owner.initialize(); + owner.removeWorkspaceState(); + }, + "storage:corrupt", + ], + [ + "wrong-run", + (owner) => { + owner.initialize(); + owner.rewriteRunId("somebody-else"); + }, + "storage:corrupt", + ], + ]; + for (const [name, arrange, refusal] of cases) { + const stub = executor(); + await on(stub, arrange); + const admitted = await admission(stub); + const answer = + admitted === "admitted" ? await send(stub, name, { command: "frontier" }) : admitted; + expect([name, answer]).toEqual([ + name, + typeof answer === "string" ? refusal : { id: name, outcome: "refused", refusal }, + ]); + } + }); + + it("names a refusal category and repeats nothing it was given or holds", async () => { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + await on(stub, (owner) => owner.rewriteRunId("a-retained-secret-run")); + await admit(stub); + const damaged = await send(stub, "id-carrying-a-secret", { command: "frontier" }); + // The refusal names a category. It does not repeat the retained run + // identity it disagreed with, and it does not repeat the request beyond the + // correlation the runner needs to match its own question. + expect(damaged).toEqual({ + id: "id-carrying-a-secret", + outcome: "refused", + refusal: "storage:corrupt", + }); + const printed = JSON.stringify(damaged); + for (const retained of ["a-retained-secret-run", RUN_ID, ROOT_MANIFEST, "workflow_run"]) { + expect(printed).not.toContain(retained); + } + + const rejected = await send(stub, "unknown", { + command: "root", + workspaceRootId: "f".repeat(64), + somethingElse: "a value the request supplied", + }); + // Not even the correlation survives a request that never parsed: an id is + // echoed once the command has been read, and this one never was. + expect(rejected).toEqual({ id: "", outcome: "refused", refusal: "command:unknown-member" }); + expect(JSON.stringify(rejected)).not.toContain("a value the request supplied"); + }); + + it("anchors and reconstructs a journal larger than one page", async () => { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + for (let index = 0; index < 129; index += 1) { + await on(stub, (owner) => owner.appendJournal(`event-${index}`, `event ${index}`)); + } + // A real accepted connection: this reads across several object round trips + // and a pair socket does not survive the object being reset between them. + const socket = await connect(stub); + const frontier = record((await ask(socket, "frontier", { command: "frontier" }))["value"]); + expect(frontier["workspaceRootId"]).toBe(ROOT_ID); + expect(frontier["journalEventId"]).toBe("event-128"); + expect(record(frontier["record"])["runId"]).toBe(RUN_ID); + + await on(stub, (owner) => owner.appendJournal("event-later", "later")); + const first = record( + ( + await ask(socket, "journal-1", { + command: "journal", + anchorEventId: "event-128", + afterEventId: null, + }) + )["value"], + ); + expect(Array.isArray(first["entries"]) && first["entries"]).toHaveLength(128); + expect(first["done"]).toBe(false); + const second = record( + ( + await ask(socket, "journal-2", { + command: "journal", + anchorEventId: "event-128", + afterEventId: "event-127", + }) + )["value"], + ); + expect(second["entries"]).toEqual([ + expect.objectContaining({ eventId: "event-128", previousEventId: "event-127" }), + ]); + expect(second["done"]).toBe(true); + }); + + it("reads a whole retained history back through the runner's own client", async () => { + // The one test where the owner's answers and the runner's parser meet. Each + // half was already proven against a hand-built counterpart, which is + // exactly why a disagreement between them could survive: the owner may + // answer a shape no runner accepts and both halves still pass. This + // composes the real pages through the real client. + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + for (let index = 0; index < 129; index += 1) { + const id = `execution-${String(index).padStart(3, "0")}`; + await on(stub, (owner) => owner.beginExecution(id, `2026-01-01T00:00:0${index % 10}.000Z`)); + } + // Two stopped rows, so the optional members cross as well as the required + // ones. A record that only ever travelled in its shortest form would not + // prove the parser accepts the shape the owner actually builds. + await on(stub, (owner) => + owner.stopExecution("execution-001", "2026-01-01T01:00:00.000Z", "completed"), + ); + await on(stub, (owner) => + owner.stopExecution("execution-002", "2026-01-01T02:00:00.000Z", "failed", "it-stopped"), + ); + + const socket = await connect(stub); + let identifier = 0; + // 129 rows page at 128, so the read takes two requests. The later row is + // written between them: after the first page fixed the anchor, and before + // the owner is asked for the second. That is the moment the anchor exists + // to survive, and asserting it any later would prove nothing about paging. + let requests = 0; + let inserted = false; + const wire = ownerSocket(socket, (raw) => { + if (!raw.includes('"executions"')) { + return undefined; + } + requests += 1; + if (requests !== 2) { + return undefined; + } + inserted = true; + return on(stub, (owner) => owner.beginExecution("execution-later", NEW_START)); + }); + const outcome = await run(function* () { + const connection = yield* useOwnerConnection(wire); + const ids = () => `read-${(identifier += 1)}`; + const link = cloudflareRunLink(connection, ids, RUN_ID); + const first = yield* link.readExecutions(); + return { first, second: yield* link.readExecutions() }; + }); + // The write really did land between the two page requests. + expect([requests >= 2, inserted]).toEqual([true, true]); + + if (!outcome.first.ok) { + throw outcome.first.error; + } + const records = outcome.first.value; + expect(records).toHaveLength(129); + expect(records.map((held) => held.executionId)).toEqual( + Array.from({ length: 129 }, (_, index) => `execution-${String(index).padStart(3, "0")}`), + ); + expect(records[0]).toEqual({ + executionId: "execution-000", + startedAt: "2026-01-01T00:00:00.000Z", + }); + expect(records[1]).toEqual({ + executionId: "execution-001", + startedAt: "2026-01-01T00:00:01.000Z", + stoppedAt: "2026-01-01T01:00:00.000Z", + stopStatus: "completed", + }); + expect(records[2]).toEqual({ + executionId: "execution-002", + startedAt: "2026-01-01T00:00:02.000Z", + stoppedAt: "2026-01-01T02:00:00.000Z", + stopStatus: "failed", + stopReason: { kind: "host", code: "it-stopped" }, + }); + // Nothing physical crossed: the runner never sees a column name. + expect(Object.keys(records[0])).toEqual(["executionId", "startedAt"]); + + if (!outcome.second.ok) { + throw outcome.second.error; + } + // The later row is outside the first anchored snapshot and inside the next. + expect(outcome.second.value).toHaveLength(130); + expect(outcome.second.value[129]?.executionId).toBe("execution-later"); + }); + + it("ends a page on the byte bound, and refuses a record that can never fit", async () => { + // The entry bound is 128 rows; this one is reached by bytes first. Both + // ends measure the same serialized `rows` array, so what the owner decides + // fits is exactly what the runner accepts — and the whole history still + // arrives, in order, across however many pages that takes. + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + const padding = "p".repeat(64 * 1024); + for (let index = 0; index < 20; index += 1) { + const id = `${String(index).padStart(2, "0")}-${padding}`; + await on(stub, (owner) => owner.beginExecution(id, "2026-01-01T00:00:00.000Z")); + } + const socket = await connect(stub); + let identifier = 0; + let requests = 0; + const wire = ownerSocket(socket, (raw) => { + if (raw.includes('"executions"')) { + requests += 1; + } + return undefined; + }); + const outcome = await run(function* () { + const connection = yield* useOwnerConnection(wire); + const ids = () => `page-${(identifier += 1)}`; + return yield* cloudflareRunLink(connection, ids, RUN_ID).readExecutions(); + }); + if (!outcome.ok) { + throw outcome.error; + } + expect(outcome.value).toHaveLength(20); + expect(outcome.value.map((held) => held.executionId.slice(0, 2))).toEqual( + Array.from({ length: 20 }, (_, index) => String(index).padStart(2, "0")), + ); + // Well under 128 entries a page, so bytes ended these pages, not the count. + expect(requests).toBeGreaterThan(1); + + // One record larger than a whole page. There is no page that could carry + // it, so the owner refuses rather than answering with something the runner + // is required to reject. + const single = executor(); + await on(single, (owner) => owner.initialize()); + const huge = "h".repeat(600 * 1024); + await on(single, (owner) => owner.beginExecution(huge, "2026-01-01T00:00:00.000Z")); + const alone = await connect(single); + let count = 0; + const refused = await run(function* () { + const connection = yield* useOwnerConnection(ownerSocket(alone)); + const ids = () => `huge-${(count += 1)}`; + return yield* cloudflareRunLink(connection, ids, RUN_ID).readExecutions(); + }); + expect(refused.ok).toBe(false); + // Provider-neutral, with no private refusal spelling in it. + expect(String(refused.ok === false && refused.error)).not.toContain("command:"); + }); + + it("returns only content referenced by one validated root", async () => { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + // A real accepted connection: a `WebSocketPair` made inside the object does + // not survive the object being reset between calls, and this test reads + // across several of them. + const socket = await connect(stub); + expect(await ask(socket, "root", { command: "root", workspaceRootId: ROOT_ID })).toEqual({ + id: "root", + outcome: "performed", + value: { workspaceRootId: ROOT_ID, manifest: ROOT_MANIFEST }, + }); + expect( + await ask(socket, "manifest", { + command: "content", + workspaceRootId: ROOT_ID, + kind: "manifest", + digest: MANIFEST_ID, + sourceManifest: null, + }), + ).toEqual({ + id: "manifest", + outcome: "performed", + value: { + kind: "manifest", + digest: MANIFEST_ID, + size: new TextEncoder().encode(DOFS_MANIFEST).length, + bytes: encodeBase64(new TextEncoder().encode(DOFS_MANIFEST)), + }, + }); + expect( + await ask(socket, "blob", { + command: "content", + workspaceRootId: ROOT_ID, + kind: "blob", + digest: BLOB_ID, + sourceManifest: MANIFEST_ID, + }), + ).toMatchObject({ outcome: "performed", value: { digest: BLOB_ID, size: FILE_BYTES.length } }); + + const orphan = await on(stub, (owner) => + owner.addUnreferencedBlob(new TextEncoder().encode("orphan")), + ); + expect( + await ask(socket, "orphan", { + command: "content", + workspaceRootId: ROOT_ID, + kind: "blob", + digest: orphan, + sourceManifest: MANIFEST_ID, + }), + ).toEqual({ id: "orphan", outcome: "refused", refusal: "storage:corrupt" }); + }); + + it("refuses a root whose content graph is incomplete, before returning one", async () => { + // A root is a starting frontier: the runner materializes it and proposes + // against it. Discovering a piece is missing when the runner asks for it + // would mean the failure arrives after the run has been told where it + // stands, so the whole graph is proved before either read answers. + const damage: Record void> = { + "a missing manifest row": (owner) => owner.removeManifestRow(), + "a manifest payload that is not its identity": (owner) => owner.damageManifestPayload(), + "a manifest size that disagrees with its chunks": (owner) => owner.damageManifestSize(), + "a missing blob reached through a manifest": (owner) => owner.removeBlobRow(), + "blob bytes that are not their identity": (owner) => owner.damageRetainedBlob(), + "a blob size that disagrees with its bytes": (owner) => owner.damageBlobSize(), + "a blob reference the manifests still name": (owner) => owner.removeBlobReference(), + "a blob reference no manifest names": (owner) => + owner.addExtraBlobReference(new TextEncoder().encode("unaccounted for")), + }; + + for (const [description, arrange] of Object.entries(damage)) { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + await on(stub, arrange); + await admit(stub); + for (const command of [ + { command: "frontier" }, + { command: "root", workspaceRootId: ROOT_ID }, + ]) { + const answer = await send(stub, `${String(command.command)}`, command); + expect([description, command.command, answer]).toEqual([ + description, + command.command, + { id: command.command, outcome: "refused", refusal: "storage:corrupt" }, + ]); + } + } + }); + + it("says only that storage is damaged, and never what it read or was asked", async () => { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + const unaccounted = await on(stub, (owner) => + owner.addExtraBlobReference(new TextEncoder().encode("unaccounted for")), + ); + await admit(stub); + const answer = await send(stub, "root", { command: "root", workspaceRootId: ROOT_ID }); + expect(answer).toEqual({ id: "root", outcome: "refused", refusal: "storage:corrupt" }); + const printed = JSON.stringify(answer); + for (const withheld of [ + unaccounted, + BLOB_ID, + MANIFEST_ID, + ROOT_ID, + ROOT_MANIFEST, + "workspace_root_blob_refs", + "vfs_manifests", + "/README.md", + ]) { + expect(printed).not.toContain(withheld); + } + }); + + it("refuses retained bytes whose identity or recorded size is damaged", async () => { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + await on(stub, (owner) => owner.damageRetainedBlob()); + await admit(stub); + expect( + await send(stub, "blob", { + command: "content", + workspaceRootId: ROOT_ID, + kind: "blob", + digest: BLOB_ID, + sourceManifest: MANIFEST_ID, + }), + ).toEqual({ + id: "blob", + outcome: "refused", + refusal: "storage:corrupt", + }); + }); + + it("replays compatible commands and refuses conflicting reuse", async () => { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + await on(stub, (owner) => owner.appendJournal("before", "before")); + const socket = await connect(stub); + const first = await ask(socket, "same", { command: "frontier" }); + await on(stub, (owner) => owner.appendJournal("after", "after")); + // The same id and the same canonical request returns the anchored frontier + // it already decided, not the later one. + expect(await ask(socket, "same", { command: "frontier" })).toEqual(first); + expect(await ask(socket, "same", { command: "root", workspaceRootId: ROOT_ID })).toEqual({ + id: "same", + outcome: "refused", + refusal: "command:duplicate-conflict", + }); + }); + + it("keeps staged bytes private, durable across eviction, and scoped to one acquisition", async () => { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + const socket = await connect(stub); + const bytes = new TextEncoder().encode("proposed content"); + const digest = sha256Hex(bytes); + const command = { command: "stage", kind: "blob", digest, bytes: encodeBase64(bytes) }; + const first = await ask(socket, "stage", command); + expect(first).toMatchObject({ outcome: "performed", value: { digest, size: bytes.length } }); + expect(await on(stub, (owner) => owner.scratch())).toEqual({ commands: 1, staged: 1 }); + + await evictDurableObject(stub); + expect(await ask(socket, "stage", command)).toEqual(first); + expect(await on(stub, (owner) => owner.scratch())).toEqual({ commands: 1, staged: 1 }); + expect( + await ask(socket, "read-stage", { + command: "content", + workspaceRootId: ROOT_ID, + kind: "blob", + digest, + sourceManifest: MANIFEST_ID, + }), + ).toEqual({ id: "read-stage", outcome: "refused", refusal: "storage:corrupt" }); + await new Promise((resolve) => setTimeout(resolve, 0)); + expect( + await on(stub, (owner) => + owner.admitConnection({ token: "not-a-token", release: POLICY.release }), + ), + ).toBe("token:token-malformed"); + expect(await on(stub, (owner) => owner.scratch())).toEqual({ commands: 1, staged: 1 }); + const before = await on(stub, (owner) => owner.authoritative()); + const replaced = await on(stub, (owner) => owner.acquisitionId()); + // A real accepted connection for the replacement: the rest of this test + // reads across several object round trips. + const successor = await connect(stub); + // A second acquisition, and the first one's scratch is gone rather than + // inherited: it cannot be retried, adopted or read. + expect(await on(stub, (owner) => owner.acquisitionId())).not.toBe(replaced); + expect(await on(stub, (owner) => owner.scratch())).toEqual({ commands: 0, staged: 0 }); + expect(await on(stub, (owner) => owner.authoritative())).toBe(before); + // The predecessor's own command id is free again, and staging the same + // bytes writes a new row rather than finding the abandoned one. Nothing was + // inherited; it was discarded and done afresh. + expect( + await ask(successor, "stage", { + command: "stage", + kind: "blob", + digest, + bytes: encodeBase64(bytes), + }), + ).toMatchObject({ outcome: "performed", value: { digest } }); + expect(await on(stub, (owner) => owner.scratch())).toEqual({ commands: 1, staged: 1 }); + expect(await ask(successor, "frontier-new", { command: "frontier" })).toMatchObject({ + outcome: "performed", + value: { workspaceRootId: ROOT_ID }, + }); + expect(await on(stub, (owner) => owner.authoritative())).toBe(before); + }); + + it("grants a copied attachment or a foreign socket no read at all", async () => { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + await admit(stub); + const request = JSON.stringify({ id: "borrowed", command: "frontier" }); + expect(await on(stub, (owner) => record(owner.sendWithCopiedAttachment(request)))).toEqual({ + id: "", + outcome: "refused", + refusal: "acquisition:foreign-connection", + }); + expect(await on(stub, (owner) => record(owner.sendAsStranger(request)))).toEqual({ + id: "", + outcome: "refused", + refusal: "acquisition:foreign-connection", + }); + expect(await on(stub, (owner) => owner.scratch())).toEqual({ commands: 0, staged: 0 }); + }); + + it("leaves no staged row when decoding or digest validation fails", async () => { + const bytes = new TextEncoder().encode("piece"); + const oversized = new Uint8Array(MAX_CONTENT_BYTES + 1); + // Each of these is a broken channel rather than an answer, so each closes + // the connection it arrived on — which is why every case gets its own. + const cases: Record, string]> = { + "bad base64": [ + { kind: "blob", digest: sha256Hex(bytes), bytes: "not base64" }, + "command:malformed-member", + ], + "a digest that is not the bytes": [ + { kind: "blob", digest: "0".repeat(64), bytes: encodeBase64(bytes) }, + "command:malformed-member", + ], + "a piece past the bound": [ + { kind: "blob", digest: sha256Hex(oversized), bytes: encodeBase64(oversized) }, + "command:too-large", + ], + }; + for (const [description, [request, refusal]] of Object.entries(cases)) { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + const socket = await connect(stub); + const answer = await ask(socket, "staged", { command: "stage", ...request }); + expect([description, answer["refusal"]]).toEqual([description, refusal]); + expect([description, await on(stub, (owner) => owner.scratch())]).toEqual([ + description, + { commands: 0, staged: 0 }, + ]); + } + }); + + it("refuses aggregate staging overflow without a partial piece or decision", async () => { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + const socket = await connect(stub); + for (let index = 0; index < 2; index += 1) { + const bytes = new Uint8Array(MAX_CONTENT_BYTES); + bytes[0] = index; + expect( + await ask(socket, `piece-${index}`, { + command: "stage", + kind: "blob", + digest: sha256Hex(bytes), + bytes: encodeBase64(bytes), + }), + ).toMatchObject({ outcome: "performed", value: { size: MAX_CONTENT_BYTES } }); + } + const overflow = new Uint8Array([3]); + expect( + await ask(socket, "overflow", { + command: "stage", + kind: "blob", + digest: sha256Hex(overflow), + bytes: encodeBase64(overflow), + }), + ).toEqual({ id: "overflow", outcome: "refused", refusal: "command:capacity" }); + expect(await on(stub, (owner) => owner.scratch())).toEqual({ commands: 2, staged: 2 }); + }); + + it("bounds the retry ledger without evicting earlier decisions", async () => { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + const socket = await connect(stub); + const first = await ask(socket, "command-0", { command: "frontier" }); + for (let index = 1; index < MAX_COMMANDS; index += 1) { + expect(await ask(socket, `command-${index}`, { command: "frontier" })).toMatchObject({ + outcome: "performed", + }); + } + expect(await ask(socket, "overflow", { command: "frontier" })).toEqual({ + id: "overflow", + outcome: "refused", + refusal: "command:capacity", + }); + await new Promise((resolve) => setTimeout(resolve, 0)); + expect(socket.readyState).not.toBe(WebSocket.OPEN); + expect(await on(stub, (owner) => owner.scratch())).toEqual({ + commands: MAX_COMMANDS, + staged: 0, + }); + expect(first).toMatchObject({ outcome: "performed" }); + }); + + it("refuses a settlement rather than reporting placeholder success", async () => { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + await admit(stub); + expect( + await send(stub, "settle", { + command: "settle", + completion: { executionId: "execution", status: "completed" }, + expectedWorkspaceRootId: ROOT_ID, + }), + ).toEqual({ id: "settle", outcome: "refused", refusal: "command:unavailable" }); + }); +}); diff --git a/packages/workflow/tests/cloudflare/remote-publish.vitest.ts b/packages/workflow/tests/cloudflare/remote-publish.vitest.ts new file mode 100644 index 000000000..676880fd1 --- /dev/null +++ b/packages/workflow/tests/cloudflare/remote-publish.vitest.ts @@ -0,0 +1,923 @@ +/** + * Publishing one proposal on real workerd. + * + * This is the point where a remote run moves, and almost nothing about it is + * provable against a model. Whether content, roots, references, mappings, the + * current pointer, the journal and the retry decision commit together is a + * property of the Durable Object's own `transactionSync()`. Whether a lost + * response can be retried exactly once is a property of storage surviving + * eviction. Whether a stale socket can still write is a property of the + * runtime's socket list. + * + * So these run against a real namespace, real SQLite and real Hibernation + * WebSockets, and the assertions are about what was published, what was + * refused, and what was left exactly as it was. + */ + +import { env, evictDurableObject, runInDurableObject } from "cloudflare:test"; +import { beforeAll, describe, expect, it } from "vitest"; +import { serializeDurableEvent } from "@executablemd/durable-streams"; +import type { ExecutorObject } from "./support/executor-object.ts"; +import type { ExecutorObject as _ExecutorObject } from "./support/executor-object.ts"; +import { + NEXT_BLOB_ID, + NEXT_BYTES, + NEXT_ROOT_ID, + nextPublication, + POLICY, + ROOT_ID, + RUN_ID, + VALID_CLAIMS, +} from "./support/executor-object.ts"; +import { encodeBase64 } from "../../src/cloudflare/encoding.ts"; +import { sha256Hex } from "../../src/workspace/sha256.ts"; +import { locatorFingerprintOf } from "../../src/composition/locator.ts"; +import { generateKeys, signToken, type TestKeys } from "./support/tokens.ts"; + +let unique = 0; +const NOW = 1_800_000_000; +let keys: TestKeys; + +beforeAll(async () => { + keys = await generateKeys(); +}); + +function executor() { + unique += 1; + return env.EXECUTOR.get(env.EXECUTOR.idFromName(`publish-${unique}-${Math.random()}`)); +} + +function on(stub: ReturnType, body: (owner: ExecutorObject) => T): Promise { + return runInDurableObject(stub, body); +} + +function record(value: unknown): Record { + if (value === null || typeof value !== "object" || Array.isArray(value)) { + throw new Error("expected an object answer"); + } + return Object.fromEntries(Object.entries(value)); +} + +async function admit(stub: ReturnType): Promise { + await on(stub, (owner) => owner.configure([{ kid: keys.kid, jwk: keys.publicJwk }], NOW)); + const token = await signToken(keys, { + ...VALID_CLAIMS, + iat: NOW - 10, + nbf: NOW - 10, + exp: NOW + 600, + }); + expect(await on(stub, (owner) => owner.admitConnection({ token, release: POLICY.release }))).toBe( + "admitted", + ); +} + +/** + * A real accepted connection, which is what survives eviction. + * + * A `WebSocketPair` made inside the object is gone once the object is evicted; + * only a socket the runtime accepted through a request comes back with its + * attachment. The retry claim is about exactly that, so it has to use this. + */ +async function connect(stub: ReturnType): Promise { + await on(stub, (owner) => owner.configure([{ kid: keys.kid, jwk: keys.publicJwk }], NOW)); + const token = await signToken(keys, { + ...VALID_CLAIMS, + iat: NOW - 10, + nbf: NOW - 10, + exp: NOW + 600, + }); + const response = await stub.fetch("https://owner.invalid/executor", { + headers: { + authorization: `Bearer ${token}`, + upgrade: "websocket", + "x-release": POLICY.release, + "x-run-id": RUN_ID, + }, + }); + const socket = response.webSocket; + if (socket === null) { + throw new Error(`expected an executor WebSocket, received ${response.status}`); + } + socket.accept(); + return socket; +} + +function ask( + socket: WebSocket, + id: string, + command: Record, +): Promise> { + return new Promise((resolve, reject) => { + const receive = (message: MessageEvent) => { + socket.removeEventListener("message", receive); + if (typeof message.data !== "string") { + reject(new Error("expected a text answer")); + return; + } + resolve(record(JSON.parse(message.data))); + }; + socket.addEventListener("message", receive); + socket.send(JSON.stringify({ id, ...command })); + }); +} + +function event(name: string): string { + return serializeDurableEvent({ + type: "yield", + coroutineId: "root", + description: { type: "test", name }, + result: { status: "ok", value: name }, + }); +} + +/** The repository mapping one proposal carries alongside its bytes. */ +const LOCATOR = "https://git.example.invalid/octo/app.git"; + +const REPOSITORY = { + kind: "repository", + locator: LOCATOR, + record: { + name: "app", + locatorFingerprint: locatorFingerprintOf(LOCATOR), + requestedBase: null, + creationCommit: "9".repeat(40), + primaryBranch: "main", + objectFormat: "sha1", + checkoutPath: "/app", + }, +}; + +/** Stage the one missing piece over an accepted connection. */ +async function stageThrough(socket: WebSocket): Promise { + await ask(socket, "stage-blob", { + command: "stage", + kind: "blob", + digest: NEXT_BLOB_ID, + bytes: encodeBase64(NEXT_BYTES), + }); + const manifest = new TextEncoder().encode( + JSON.stringify({ version: 1, chunks: [{ hash: NEXT_BLOB_ID, size: NEXT_BYTES.length }] }), + ); + await ask(socket, "stage-manifest", { + command: "stage", + kind: "manifest", + digest: sha256Hex(manifest), + bytes: encodeBase64(manifest), + }); +} + +function commit(overrides: Record = {}): Record { + return { + command: "commit", + expectedWorkspaceRootId: ROOT_ID, + expectedJournalEventId: null, + publication: nextPublication(), + mappings: [REPOSITORY], + events: [event("published")], + ...overrides, + }; +} + +describe("publishing one proposal", () => { + it("adopts content, root, references, mapping, pointer and journal together", async () => { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + const socket = await connect(stub); + await stageThrough(socket); + + const before = await on(stub, (owner) => owner.published()); + expect(before).toMatchObject({ currentRootId: ROOT_ID, roots: 1, events: [] }); + + const answer = await ask(socket, "publish", commit()); + expect(answer).toEqual(expect.objectContaining({ outcome: "performed" })); + const value = record(answer["value"]); + expect(value["workspaceRootId"]).toBe(NEXT_ROOT_ID); + expect(Array.isArray(value["journalEventIds"]) && value["journalEventIds"]).toHaveLength(1); + + const after = await on(stub, (owner) => owner.published()); + expect(after["currentRootId"]).toBe(NEXT_ROOT_ID); + // The old root stays retained; publication moves only the pointer. + expect(after["roots"]).toBe(2); + expect(after["repositories"]).toEqual([{ name: "app", checkout_path: "/app" }]); + // The journal row names the root this commit selected, not the one it + // started from. + expect(after["events"]).toEqual([expect.objectContaining({ workspace_root_id: NEXT_ROOT_ID })]); + // Content the owner already held was reused by identity rather than + // resent: two blobs exist, and the proposal only staged one. + expect(after["blobs"]).toBe(2); + expect(after["blobRefs"]).toBe(3); + + // And the new frontier reads back whole. + const frontier = record(record(await ask(socket, "read", { command: "frontier" }))["value"]); + expect(frontier["workspaceRootId"]).toBe(NEXT_ROOT_ID); + expect( + record(await ask(socket, "root", { command: "root", workspaceRootId: NEXT_ROOT_ID })), + ).toMatchObject({ outcome: "performed" }); + }); + + it("admits root, journal anchor and every mapping as one state", async () => { + // The invocation snapshot D3c begins from. It is one read on the owner + // because it is one fact: mappings taken from one moment and a root from + // another would let an invocation start against a Workspace its retained + // rows do not describe, and nothing later could tell. + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + const socket = await connect(stub); + + const empty = record(record(await ask(socket, "empty", { command: "mappings" }))["value"]); + expect(empty).toEqual({ + workspaceRootId: ROOT_ID, + journalEventId: null, + repositories: [], + worktrees: [], + agentSessions: [], + }); + + await stageThrough(socket); + const published = await ask(socket, "publish", commit()); + expect(published).toEqual(expect.objectContaining({ outcome: "performed" })); + + // One commit moved the pointer, retained the mapping and wrote the row. + // The next snapshot observes all of it, and observes it together. + const after = record(record(await ask(socket, "after", { command: "mappings" }))["value"]); + expect(after["workspaceRootId"]).toBe(NEXT_ROOT_ID); + expect(typeof after["journalEventId"]).toBe("string"); + expect(after["repositories"]).toEqual([{ record: REPOSITORY.record, locator: LOCATOR }]); + expect(after["worktrees"]).toEqual([]); + expect(after["agentSessions"]).toEqual([]); + + // The same anchor the frontier reports, from the same owner state. + const frontier = record( + record(await ask(socket, "frontier", { command: "frontier" }))["value"], + ); + expect([frontier["workspaceRootId"], frontier["journalEventId"]]).toEqual([ + after["workspaceRootId"], + after["journalEventId"], + ]); + }); + + it("returns no partial snapshot when more is retained than one may carry", async () => { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + const socket = await connect(stub); + // Under the ceiling the snapshot answers whole. + await on(stub, (owner) => owner.fillRepositories(0, 200)); + const held = record(record(await ask(socket, "under", { command: "mappings" }))["value"]); + expect(Array.isArray(held["repositories"]) && held["repositories"]).toHaveLength(200); + + // Over it, the answer is a refusal rather than as much as would fit. A + // partial snapshot would describe a run holding fewer Repositories than it + // does, and every reconciliation against it would be decided wrongly. + await on(stub, (owner) => owner.fillRepositories(200, 200)); + const answer = await ask(socket, "over", { command: "mappings" }); + expect(answer["outcome"]).toBe("refused"); + expect(answer["value"]).toBe(undefined); + }); + + it("keeps the expected root current for a journal-only transaction", async () => { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + const socket = await connect(stub); + const answer = await ask( + socket, + "journal-only", + commit({ publication: null, mappings: [], events: [event("noted")] }), + ); + expect(answer).toMatchObject({ outcome: "performed" }); + const after = await on(stub, (owner) => owner.published()); + expect(after["currentRootId"]).toBe(ROOT_ID); + expect(after["roots"]).toBe(1); + expect(after["events"]).toEqual([expect.objectContaining({ workspace_root_id: ROOT_ID })]); + }); + + it("commits an empty transaction without inventing a Workspace change", async () => { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + const socket = await connect(stub); + expect( + await ask(socket, "empty", commit({ publication: null, mappings: [], events: [] })), + ).toMatchObject({ outcome: "performed", value: { workspaceRootId: ROOT_ID } }); + expect(await on(stub, (owner) => owner.published())).toMatchObject({ + currentRootId: ROOT_ID, + roots: 1, + events: [], + }); + }); + + it("rolls every category back when the transaction fails after applying", async () => { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + const socket = await connect(stub); + await stageThrough(socket); + const before = await on(stub, (owner) => owner.published()); + + expect( + await on(stub, (owner) => + owner.failAfterApply(JSON.stringify({ id: "doomed", ...commit() })), + ), + ).toBe("rolled-back"); + + // Content, root, references, mapping, pointer and journal are all back + // where they were — and so is the retry decision, so the same id is free. + expect(await on(stub, (owner) => owner.published())).toEqual(before); + expect(await on(stub, (owner) => owner.scratch())).toMatchObject({ commands: 2 }); + expect(await ask(socket, "doomed", commit())).toMatchObject({ outcome: "performed" }); + }); + + it("applies a lost-response retry exactly once, across eviction", async () => { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + const socket = await connect(stub); + await ask(socket, "stage-blob", { + command: "stage", + kind: "blob", + digest: NEXT_BLOB_ID, + bytes: encodeBase64(NEXT_BYTES), + }); + const manifestBytes = new TextEncoder().encode( + JSON.stringify({ version: 1, chunks: [{ hash: NEXT_BLOB_ID, size: NEXT_BYTES.length }] }), + ); + await ask(socket, "stage-manifest", { + command: "stage", + kind: "manifest", + digest: sha256Hex(manifestBytes), + bytes: encodeBase64(manifestBytes), + }); + + const first = await ask(socket, "once", commit()); + expect(first).toMatchObject({ outcome: "performed" }); + const published = await on(stub, (owner) => owner.published()); + + // The runner never saw that answer. The object is evicted, and the same + // healthy socket asks the same question again with the same id and the same + // canonical request. + await evictDurableObject(stub); + expect(await ask(socket, "once", commit())).toEqual(first); + + // One root, one journal row, one mapping — the retry returned the decision + // rather than doing the work a second time. + expect(await on(stub, (owner) => owner.published())).toEqual(published); + + // Reusing that id for a different request is not a retry. + expect(await ask(socket, "once", commit({ events: [event("something else")] }))).toMatchObject({ + outcome: "refused", + refusal: "command:duplicate-conflict", + }); + expect(await on(stub, (owner) => owner.published())).toEqual(published); + }); + + it("changes nothing when the frontier or the proposal is not what it claims", async () => { + const cases: Record> = { + "a root the run is not at": commit({ expectedWorkspaceRootId: `f${"0".repeat(63)}` }), + "an anchor the run is not at": commit({ expectedJournalEventId: "never-happened" }), + "an identity that is not the digest of its manifest": commit({ + publication: { ...nextPublication(), proposedWorkspaceRootId: `a${"1".repeat(63)}` }, + }), + "an inventory missing a piece the root names": commit({ + publication: { + ...nextPublication(), + content: (nextPublication()["content"] as unknown[]).slice(1), + }, + }), + "an inventory naming a piece the root does not": commit({ + publication: { + ...nextPublication(), + content: [ + ...(nextPublication()["content"] as Record[]), + { kind: "blob", digest: "e".repeat(64), size: 4 }, + ], + }, + }), + }; + + for (const [description, request] of Object.entries(cases)) { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + const socket = await connect(stub); + await stageThrough(socket); + const before = await on(stub, (owner) => owner.published()); + const answer = await ask(socket, "refused", request); + expect([description, answer["outcome"]]).toEqual([description, "refused"]); + expect([description, await on(stub, (owner) => owner.published())]).toEqual([ + description, + before, + ]); + } + }); + + it("refuses content this acquisition did not stage", async () => { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + const socket = await connect(stub); + // Nothing staged: the proposal names a piece the owner neither holds nor + // was given by this connection. + const before = await on(stub, (owner) => owner.published()); + expect(await ask(socket, "unstaged", commit())).toMatchObject({ outcome: "refused" }); + expect(await on(stub, (owner) => owner.published())).toEqual(before); + }); + + it("refuses a mapping that would rewrite an established identity", async () => { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + const socket = await connect(stub); + await stageThrough(socket); + expect(await ask(socket, "first", commit())).toMatchObject({ outcome: "performed" }); + const published = await on(stub, (owner) => owner.published()); + + // The same Repository name, a different creation commit. Creation identity + // is immutable, so this is refused rather than allowed to overwrite it. + expect( + await ask( + socket, + "second", + commit({ + expectedWorkspaceRootId: NEXT_ROOT_ID, + expectedJournalEventId: String( + (published["events"] as Record[])[0]?.["event_id"], + ), + publication: null, + mappings: [ + { ...REPOSITORY, record: { ...REPOSITORY.record, creationCommit: "1".repeat(40) } }, + ], + events: [], + }), + ), + ).toMatchObject({ outcome: "refused", refusal: "command:mapping-conflict" }); + expect(await on(stub, (owner) => owner.published())).toEqual(published); + }); + + it("retains only records that are exactly what the serializer produced", async () => { + // A record the database will accept as JSON is not a durable event. One + // retained here would be history a later read cannot parse, and the run + // would become unreplayable at the moment it was told it had committed. + const valid = event("real"); + const cases: Record = { + "JSON that is not an event": "{}", + "an event without its terminating newline": valid.trimEnd(), + "a noncanonical re-encoding": `${JSON.stringify(JSON.parse(valid.trimEnd()), null, 1)}\n`, + "not JSON at all": "event-1", + }; + for (const [description, record_] of Object.entries(cases)) { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + const socket = await connect(stub); + const before = await on(stub, (owner) => owner.published()); + const answer = await ask( + socket, + "bad-event", + commit({ publication: null, mappings: [], events: [record_] }), + ); + // The id is empty because the command never finished parsing: an id is + // echoed once the request has been read, and this one was not. + expect([description, answer]).toEqual([ + description, + { id: "", outcome: "refused", refusal: "command:malformed-member" }, + ]); + expect([description, await on(stub, (owner) => owner.published())]).toEqual([ + description, + before, + ]); + // Nothing recorded a decision for work that never happened. A malformed + // member is a broken channel rather than an answer, so the connection is + // gone too — which is why the ledger is read through the object. + expect([description, await on(stub, (owner) => owner.scratch())]).toEqual([ + description, + { commands: 0, staged: 0 }, + ]); + } + }); + + it("refuses a mapping that disagrees with retained identity in any field", async () => { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + // A real accepted connection: this walks several cases and a pair socket + // does not survive the object being reset between them. + const socket = await connect(stub); + await stageThrough(socket); + expect(await ask(socket, "first", commit())).toMatchObject({ outcome: "performed" }); + const published = await on(stub, (owner) => owner.published()); + const anchor = String((published["events"] as Record[])[0]?.["event_id"]); + + // Every field that establishes creation identity, one at a time. A partial + // comparison would report performed for a proposal that disagrees with what + // an earlier execution established. + const conflicts: Record> = { + "a different locator, with its own fingerprint": { + kind: "repository", + locator: "https://git.example.invalid/other.git", + record: { + ...REPOSITORY.record, + locatorFingerprint: locatorFingerprintOf("https://git.example.invalid/other.git"), + }, + }, + "a different requested base": { + ...REPOSITORY, + record: { ...REPOSITORY.record, requestedBase: "release" }, + }, + "a different creation commit": { + ...REPOSITORY, + record: { ...REPOSITORY.record, creationCommit: "1".repeat(40) }, + }, + "a different primary branch": { + ...REPOSITORY, + record: { ...REPOSITORY.record, primaryBranch: "trunk" }, + }, + "a different checkout path": { + ...REPOSITORY, + record: { ...REPOSITORY.record, checkoutPath: "/elsewhere" }, + }, + }; + for (const [description, mapping] of Object.entries(conflicts)) { + const answer = await ask( + socket, + `conflict-${description}`, + commit({ + expectedWorkspaceRootId: NEXT_ROOT_ID, + expectedJournalEventId: anchor, + publication: null, + mappings: [mapping], + events: [], + }), + ); + expect([description, answer["refusal"]]).toEqual([description, "command:mapping-conflict"]); + expect([description, await on(stub, (owner) => owner.published())]).toEqual([ + description, + published, + ]); + } + }); + + it("refuses a new checkout mapping that no publication creates", async () => { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + const socket = await connect(stub); + const before = await on(stub, (owner) => owner.published()); + + // A mapping-only commit would retain a claim about a directory nothing put + // there, and the next execution would find the claim and not the files. + expect( + await ask( + socket, + "no-publication", + commit({ publication: null, mappings: [REPOSITORY], events: [] }), + ), + ).toMatchObject({ refusal: "command:mapping-conflict" }); + expect(await on(stub, (owner) => owner.published())).toEqual(before); + + // A Worktree whose Repository is neither retained nor proposed belongs to + // nothing. + await stageThrough(socket); + expect( + await ask( + socket, + "orphan-worktree", + commit({ + mappings: [ + { + kind: "worktree", + record: { + repositoryName: "absent", + name: "feature", + requestedBranch: "feature", + requestedBase: null, + creationCommit: "2".repeat(40), + checkoutPath: "/app", + }, + }, + ], + }), + ), + ).toMatchObject({ refusal: "command:mapping-conflict" }); + expect(await on(stub, (owner) => owner.published())).toEqual(before); + }); + + it("refuses one proposal naming one mapping twice", async () => { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + const socket = await connect(stub); + await stageThrough(socket); + const before = await on(stub, (owner) => owner.published()); + expect( + await ask(socket, "duplicate", commit({ mappings: [REPOSITORY, REPOSITORY] })), + ).toMatchObject({ refusal: "command:mapping-conflict" }); + expect(await on(stub, (owner) => owner.published())).toEqual(before); + }); + + it("refuses to append history over a current root that is damaged", async () => { + // A commit accepts its starting root as the run's frontier. One whose graph + // cannot be materialized is not a frontier, and a proposal is not a licence + // to repair it. + const damage: Record void> = { + "a blob whose bytes are not its identity": (owner) => owner.damageRetainedBlob(), + "a blob whose recorded size is wrong": (owner) => owner.damageBlobSize(), + "a manifest whose recorded size is wrong": (owner) => owner.damageManifestSize(), + "a reference no manifest names": (owner) => + owner.addExtraBlobReference(new TextEncoder().encode("unaccounted for")), + }; + for (const [description, arrange] of Object.entries(damage)) { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + await on(stub, arrange); + const socket = await connect(stub); + const before = await on(stub, (owner) => owner.published()); + const answer = await ask( + socket, + "over-damage", + commit({ publication: null, mappings: [], events: [event("noted")] }), + ); + expect([description, answer["refusal"]]).toEqual([description, "storage:corrupt"]); + expect([description, await on(stub, (owner) => owner.published())]).toEqual([ + description, + before, + ]); + } + }); + + it("retains only a locator this system would hand to Git", async () => { + // A matching fingerprint says the two values agree with each other. It says + // nothing about whether the locator is one that may ever be used, and an + // authenticated proposal must not be able to retain a credential or an + // executable transport form. + const refused: Record = { + "a credential in the URL": "https://user:token@git.example.invalid/octo/app.git", + "an executable transport form": "ext::sh -c 'curl example.invalid'", + "a query that can carry a token": "https://git.example.invalid/app.git?access_token=abc", + "an unknown scheme": "javascript:alert(1)", + "a relative path": "../elsewhere", + }; + for (const [description, locator] of Object.entries(refused)) { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + const socket = await connect(stub); + await stageThrough(socket); + const before = await on(stub, (owner) => owner.published()); + const answer = await ask( + socket, + "bad-locator", + commit({ + mappings: [ + { + kind: "repository", + locator, + record: { ...REPOSITORY.record, locatorFingerprint: locatorFingerprintOf(locator) }, + }, + ], + }), + ); + expect([description, answer["refusal"]]).toEqual([description, "command:malformed-member"]); + expect([description, await on(stub, (owner) => owner.published())]).toEqual([ + description, + before, + ]); + } + }); + + it("retains the exact admitted locator, not its fingerprint", async () => { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + const socket = await connect(stub); + await stageThrough(socket); + expect(await ask(socket, "publish", commit())).toMatchObject({ outcome: "performed" }); + // The row a later restoration reads has to name the repository, not a + // digest of it. + expect(await on(stub, (owner) => owner.repositoryLocator("app"))).toBe(LOCATOR); + }); + + it("accepts a Repository and its Worktree in either order", async () => { + const worktree = { + kind: "worktree", + record: { + repositoryName: "app", + name: "feature", + requestedBranch: "feature", + requestedBase: null, + creationCommit: "2".repeat(40), + checkoutPath: "/app", + }, + }; + // Which of the two comes first in an array is not a difference between + // proposals, so both spellings of one transaction must be accepted. + for (const [description, mappings] of Object.entries({ + "parent first": [REPOSITORY, worktree], + "child first": [worktree, REPOSITORY], + })) { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + const socket = await connect(stub); + await stageThrough(socket); + const answer = await ask(socket, "both", commit({ mappings })); + expect([description, answer["outcome"]]).toEqual([description, "performed"]); + expect([description, await on(stub, (owner) => owner.published())]).toMatchObject([ + description, + { currentRootId: NEXT_ROOT_ID, repositories: [{ name: "app", checkout_path: "/app" }] }, + ]); + } + }); + + it("refuses a blob whose bytes were never retained beside its metadata", async () => { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + // A metadata row with no bytes is a half-written identity. Completing it + // from staging would repair authoritative damage as a side effect. + await on(stub, (owner) => owner.removeBlobBytesOnly(NEXT_BLOB_ID, NEXT_BYTES.length)); + const socket = await connect(stub); + await stageThrough(socket); + const before = await on(stub, (owner) => owner.published()); + expect(await ask(socket, "half", commit())).toMatchObject({ refusal: "storage:corrupt" }); + expect(await on(stub, (owner) => owner.published())).toEqual(before); + expect(await on(stub, (owner) => owner.scratch())).toMatchObject({ commands: 2 }); + }); + + it("returns the same decision to a connection that replaced the one that lost it", async () => { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + const socket = await connect(stub); + await stageThrough(socket); + + // The owner commits. The runner never sees the answer, and the connection + // that asked is gone — which is exactly the case the acquisition-scoped + // ledger cannot answer, because a replacement acquisition discards it. + const first = await ask(socket, "recovered", commit()); + expect(first).toMatchObject({ outcome: "performed" }); + const published = await on(stub, (owner) => owner.published()); + socket.close(1000, "lost"); + await new Promise((resolve) => setTimeout(resolve, 0)); + await evictDurableObject(stub); + + // A new connection, a new acquisition, the identical closed request. + const replacement = await connect(stub); + expect(await ask(replacement, "recovered", commit())).toEqual(first); + + // One root, one mapping, one set of journal rows. + expect(await on(stub, (owner) => owner.published())).toEqual(published); + + // And the identity still cannot be reused for something else. + expect( + await ask(replacement, "recovered", commit({ events: [event("different")] })), + ).toMatchObject({ outcome: "refused", refusal: "command:duplicate-conflict" }); + expect(await on(stub, (owner) => owner.published())).toEqual(published); + }); + + it("performs a proposal whose first attempt never reached the owner", async () => { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + const socket = await connect(stub); + await stageThrough(socket); + const before = await on(stub, (owner) => owner.published()); + + // The first attempt was lost on the way out, so the owner never saw it. + socket.close(1000, "lost before arriving"); + await new Promise((resolve) => setTimeout(resolve, 0)); + await evictDurableObject(stub); + + const replacement = await connect(stub); + await stageThrough(replacement); + expect(await ask(replacement, "never-arrived", commit())).toMatchObject({ + outcome: "performed", + }); + const after = await on(stub, (owner) => owner.published()); + expect(after["currentRootId"]).toBe(NEXT_ROOT_ID); + expect(after).not.toEqual(before); + }); + + it("grants a closed or foreign socket no publication", async () => { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + const socket = await connect(stub); + await stageThrough(socket); + const before = await on(stub, (owner) => owner.published()); + expect( + await on(stub, (owner) => + record(owner.sendAsStranger(JSON.stringify({ id: "foreign", ...commit() }))), + ), + ).toMatchObject({ outcome: "refused", refusal: "acquisition:foreign-connection" }); + expect(await on(stub, (owner) => owner.published())).toEqual(before); + }); +}); + +describe("the run's own records", () => { + it("counts retrieval revisions authoritatively, and clearing starts again", async () => { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + const socket = await connect(stub); + + // The run is created with a retrieval row, so the first replacement is the + // next revision rather than the first. + const before = await on(stub, (owner) => owner.retrieval()); + expect(before).not.toBe(null); + + const first = await ask(socket, "r1", { + command: "retrieval", + expectedWorkspaceRootId: ROOT_ID, + metadata: '{"locator":"https://example.invalid/a.git"}', + }); + expect(first).toMatchObject({ outcome: "performed" }); + + // Byte-identical metadata under a different id is a second replacement. + const second = await ask(socket, "r2", { + command: "retrieval", + expectedWorkspaceRootId: ROOT_ID, + metadata: '{"locator":"https://example.invalid/a.git"}', + }); + expect(second).toMatchObject({ outcome: "performed" }); + const revisions = [first, second].map((answer) => + Number(record(record(answer["value"])["retrieval"])["revision"]), + ); + expect(revisions[1]).toBe((revisions[0] ?? 0) + 1); + + // Clearing removes the row; the next replacement counts from one. + expect( + await ask(socket, "r3", { + command: "retrieval", + expectedWorkspaceRootId: ROOT_ID, + metadata: null, + }), + ).toEqual({ id: "r3", outcome: "performed", value: { retrieval: null } }); + expect(await on(stub, (owner) => owner.retrieval())).toBe(null); + const restarted = await ask(socket, "r4", { + command: "retrieval", + expectedWorkspaceRootId: ROOT_ID, + metadata: '{"locator":"https://example.invalid/b.git"}', + }); + expect(Number(record(record(restarted["value"])["retrieval"])["revision"])).toBe(1); + }); + + it("applies one retrieval replacement once across a lost answer and eviction", async () => { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + const socket = await connect(stub); + const request = { + command: "retrieval", + expectedWorkspaceRootId: ROOT_ID, + metadata: '{"locator":"https://example.invalid/once.git"}', + }; + const performed = await ask(socket, "once", request); + expect(performed).toMatchObject({ outcome: "performed" }); + const stored = await on(stub, (owner) => owner.retrieval()); + + socket.close(1000, "lost"); + await new Promise((resolve) => setTimeout(resolve, 0)); + await evictDurableObject(stub); + + const replacement = await connect(stub); + // The same invocation asked again: the retained decision answers, and the + // revision does not move. + expect(await ask(replacement, "once", request)).toEqual(performed); + expect(await on(stub, (owner) => owner.retrieval())).toEqual(stored); + + // The same identity for different content is a conflict, not a retry. + expect( + await ask(replacement, "once", { ...request, metadata: '{"locator":"other"}' }), + ).toMatchObject({ outcome: "refused", refusal: "command:duplicate-conflict" }); + expect(await on(stub, (owner) => owner.retrieval())).toEqual(stored); + }); + + it("refuses a replacement proposed against a root the run has left", async () => { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + const socket = await connect(stub); + const before = await on(stub, (owner) => owner.retrieval()); + expect( + await ask(socket, "stale", { + command: "retrieval", + expectedWorkspaceRootId: `f${"0".repeat(63)}`, + metadata: '{"locator":"x"}', + }), + ).toMatchObject({ outcome: "refused", refusal: "command:stale-root" }); + expect(await on(stub, (owner) => owner.retrieval())).toEqual(before); + }); + + it("anchors a multipage execution snapshot and excludes a later one", async () => { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + for (let index = 0; index < 129; index += 1) { + await on(stub, (owner) => + owner.beginExecution( + `execution-${index}`, + `2026-09-04T00:00:${String(index % 60).padStart(2, "0")}.000Z`, + ), + ); + } + const socket = await connect(stub); + + // The first request carries no anchor; the owner chooses the terminal row + // at this moment and answers with it. + const anchored = record( + (await ask(socket, "x1", { command: "executions", anchor: null, after: null }))["value"], + ); + expect(anchored["anchor"]).toBe(129); + expect(Array.isArray(anchored["rows"]) && anchored["rows"]).toHaveLength(128); + expect(anchored["done"]).toBe(false); + + // A later execution begins while the read is in flight. + await on(stub, (owner) => owner.beginExecution("execution-later", "2026-09-04T01:00:00.000Z")); + + const second = record( + (await ask(socket, "x2", { command: "executions", anchor: 129, after: 128 }))["value"], + ); + expect(Array.isArray(second["rows"]) && second["rows"]).toHaveLength(1); + expect(second["done"]).toBe(true); + // The one begun after the anchor is not in the snapshot. + expect(record((second["rows"] as Record[])[0] ?? {})["sequence"]).toBe(129); + }); +}); diff --git a/packages/workflow/tests/cloudflare/remote-workspace.vitest.ts b/packages/workflow/tests/cloudflare/remote-workspace.vitest.ts new file mode 100644 index 000000000..433108066 --- /dev/null +++ b/packages/workflow/tests/cloudflare/remote-workspace.vitest.ts @@ -0,0 +1,271 @@ +/** + * The runner's Workspace coordinator, against a real owner. + * + * Everything on the owner's side is real here: a real Durable Object, its own + * SQLite storage, a real accepted Hibernation WebSocket, and the production + * client, database handle, run binding and coordinator on the other end of it. + * What runs is `createRemoteWorkspaceEffect()` through + * `withRemoteWorkspaceEffects()`, so the admission read, the attempt, the + * anchor check, the mapping staging, the enlistment, the journal route and + * D3a's atomic commit are all the production path. + * + * The one stand-in is the runner's host filesystem: workerd has none, and the + * vendored DOFS cannot set a modification time, so it cannot reproduce a + * retained mtime — which is the thing materialization refuses a host for. The + * native adapter it stands in for is proved against real files in + * `packages/workflow/tests/remote-workspace-files.test.ts`. + */ + +import { env, runInDurableObject } from "cloudflare:test"; +import { beforeAll, describe, expect, it } from "vitest"; +import { InMemoryStream, type Workflow, type Json } from "@executablemd/durable-streams"; +import { run, type Operation } from "effection"; +import type { ExecutorObject } from "./support/executor-object.ts"; +import { POLICY, RUN_ID, VALID_CLAIMS } from "./support/executor-object.ts"; +import { generateKeys, signToken, type TestKeys } from "./support/tokens.ts"; +import { createWorkerFiles } from "./support/worker-files.ts"; +import { cloudflareReadLink, cloudflareRunLink } from "../../src/cloudflare/client.ts"; +import { + type OwnerSocket, + type SocketListener, + useOwnerConnection, +} from "../../src/remote/client.ts"; +import { + createRemoteWorkspaceEffect, + type RemoteRun, + useRemoteRun, + useRemoteWorkspaceEffects, + withRemoteWorkspaceEffects, +} from "../../src/remote/workspace.ts"; +import { durableRun } from "@executablemd/durable-streams"; +import { locatorFingerprintOf } from "../../src/composition/locator.ts"; +import { useMaterialization } from "../../src/remote/invocation.ts"; +import { JournaledEffectFailure } from "../../src/workspace/failure.ts"; + +let unique = 0; +const NOW = 1_800_000_000; +const LOCATOR = "https://git.example.invalid/octo/app.git"; +let keys: TestKeys; + +beforeAll(async () => { + keys = await generateKeys(); +}); + +function executor() { + unique += 1; + return env.EXECUTOR.get(env.EXECUTOR.idFromName(`coordinated-${unique}-${Math.random()}`)); +} + +function on( + stub: ReturnType, + body: (instance: ExecutorObject) => T, +): Promise { + return runInDurableObject(stub, body); +} + +async function connect(stub: ReturnType): Promise { + await on(stub, (owner) => owner.configure([{ kid: keys.kid, jwk: keys.publicJwk }], NOW)); + const token = await signToken(keys, { + ...VALID_CLAIMS, + iat: NOW - 10, + nbf: NOW - 10, + exp: NOW + 600, + }); + const response = await stub.fetch("https://owner.invalid/executor", { + headers: { + authorization: `Bearer ${token}`, + upgrade: "websocket", + "x-release": POLICY.release, + "x-run-id": RUN_ID, + }, + }); + const socket = response.webSocket; + if (socket === null) { + throw new Error(`expected an executor WebSocket, received ${response.status}`); + } + socket.accept(); + return socket; +} + +/** The platform socket, bound to the four members the client uses. */ +function ownerSocket(socket: WebSocket): OwnerSocket { + const listeners = new Map(); + return { + send: (data) => socket.send(data), + close: () => socket.close(), + addEventListener(type, listener) { + const forward: EventListener = (event) => listener(event as { data?: unknown }); + listeners.set(listener, forward); + socket.addEventListener(type, forward); + }, + removeEventListener(type, listener) { + const found = listeners.get(listener); + if (found !== undefined) { + socket.removeEventListener(type, found); + } + }, + }; +} + +function repository() { + return { + record: { + name: "app", + locatorFingerprint: locatorFingerprintOf(LOCATOR), + requestedBase: null, + creationCommit: "9".repeat(40), + primaryBranch: "main", + objectFormat: "sha1" as const, + checkoutPath: "/app", + }, + locator: LOCATOR, + }; +} + +/** + * Open one production run binding over a real accepted socket. + * + * Everything the coordinator will use comes from here, together: the client, + * the handle, the runtime and the routed journal. + */ +function* opened(socket: WebSocket): Operation { + const connection = yield* useOwnerConnection(ownerSocket(socket)); + let identifier = 0; + const next = () => `coordinated-${(identifier += 1)}`; + const host = createWorkerFiles(); + return yield* useRemoteRun({ + link: cloudflareRunLink(connection, next, RUN_ID), + files: host.files, + trees: host.trees, + createFilesystem: (at) => host.workspace(at("/")), + journal: new InMemoryStream(), + }); +} + +describe("the coordinator against a real owner", () => { + it("publishes Files, one mapping and the filtered result as one commit", async () => { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + const before = await on(stub, (owner) => owner.published()); + const socket = await connect(stub); + + const outcome = await run(function* () { + const opening = yield* opened(socket); + yield* useRemoteWorkspaceEffects(opening); + const effect = createRemoteWorkspaceEffect( + opening, + { type: "workspace", name: "write" }, + function* (filesystem, metadata): Operation { + yield* filesystem.writeFile("/NOTES.md", "written by the effect\n", 0o644); + // The checkout the Repository record names has to be in the Workspace + // this proposal publishes; the owner refuses a mapping to a place the + // root does not contain. + yield* filesystem.mkdir("/app", { mode: 0o755 }); + metadata.insertRepository(repository()); + return "published"; + }, + ); + function* workflow(): Workflow { + yield effect; + } + yield* withRemoteWorkspaceEffects(opening, durableRun(workflow, { stream: opening.journal })); + return yield* opening.journal.readAll(); + }); + // The result travelled inside the commit, so the ordinary journal never + // saw it. + expect(outcome.filter((event) => event.type === "yield")).toHaveLength(0); + + const after = await on(stub, (owner) => owner.published()); + // Content, root, references, mapping, pointer and the journal row moved + // together, and the pointer is no longer where it started. + expect(after["currentRootId"]).not.toBe(before["currentRootId"]); + expect(after["roots"]).toBe(2); + expect(after["repositories"]).toEqual([{ name: "app", checkout_path: "/app" }]); + expect(after["events"]).toEqual([ + expect.objectContaining({ workspace_root_id: after["currentRootId"] }), + ]); + + // A fresh admitted invocation, through the production read link: the owner + // answers with the new root, the new anchor and the retained mapping + // together, and that root materializes to the bytes the effect wrote. + const second = await connect(stub); + const observed = await run(function* () { + const connection = yield* useOwnerConnection(ownerSocket(second)); + let identifier = 0; + const next = () => `observe-${(identifier += 1)}`; + const reads = cloudflareReadLink(connection, next, RUN_ID); + const snapshot = yield* reads.invocationSnapshot(); + const host = createWorkerFiles(); + const materialization = yield* useMaterialization( + host.files, + host.trees, + reads, + snapshot.workspaceRootId, + (reason) => { + throw new Error(reason); + }, + ); + const workspace = host.workspace(materialization.at("/")); + return { + workspaceRootId: snapshot.workspaceRootId, + journalEventId: snapshot.journalEventId, + repositories: snapshot.repositories.map((stored) => stored.record.name), + notes: yield* workspace.readTextFile("/NOTES.md"), + }; + }); + expect(observed.workspaceRootId).toBe(after["currentRootId"]); + expect(typeof observed.journalEventId).toBe("string"); + expect(observed.repositories).toEqual(["app"]); + expect(observed.notes).toBe("written by the effect\n"); + }); + + it("commits only the filtered failed result, and moves nothing else", async () => { + const stub = executor(); + await on(stub, (owner) => owner.initialize()); + const before = await on(stub, (owner) => owner.published()); + const socket = await connect(stub); + + await run(function* () { + const opening = yield* opened(socket); + yield* useRemoteWorkspaceEffects(opening); + const effect = createRemoteWorkspaceEffect( + opening, + { type: "workspace", name: "refuse" }, + function* (filesystem, metadata): Operation { + yield* filesystem.writeFile("/SCRATCH.md", "discarded\n", 0o644); + yield* filesystem.mkdir("/app", { mode: 0o755 }); + metadata.insertRepository(repository()); + throw new DocumentedFailure("this Workspace effect refused"); + }, + ); + function* workflow(): Workflow { + yield effect; + } + try { + yield* withRemoteWorkspaceEffects( + opening, + durableRun(workflow, { stream: opening.journal }), + ); + } catch { + // The documented failure is the run's outcome; what it left behind is + // the claim being made. + } + }); + + const after = await on(stub, (owner) => owner.published()); + // The pointer did not move, no second root was retained, and the mapping + // the effect staged never became one. + expect(after["currentRootId"]).toBe(before["currentRootId"]); + expect(after["roots"]).toBe(before["roots"]); + expect(after["repositories"]).toEqual([]); + // One row, and it names the root the run is still on. + expect(after["events"]).toEqual([ + expect.objectContaining({ workspace_root_id: before["currentRootId"] }), + ]); + }); +}); + +/** A refusal the effect publishes rather than raises, as a document's would be. */ +class DocumentedFailure extends JournaledEffectFailure { + override name = "DocumentedFailure"; +} diff --git a/packages/workflow/tests/cloudflare/settle-parser.vitest.ts b/packages/workflow/tests/cloudflare/settle-parser.vitest.ts new file mode 100644 index 000000000..ba37f3a56 --- /dev/null +++ b/packages/workflow/tests/cloudflare/settle-parser.vitest.ts @@ -0,0 +1,168 @@ +/** + * The settle request, parsed inside a real Worker. + * + * The point of running this on workerd rather than portably is the import + * graph. `parseDocumentExecutionCompletion()` is shared code that reaches + * `canonicalize` and two spelling predicates in `@executablemd/core`, and until + * those were published from node-free subpaths that graph pulled `node:crypto` + * and `node:process` — which typechecks anywhere except the runtime that has to + * run it. A test that only proved the parser worked would have proved nothing + * about that; this one loads it where a Node builtin is genuinely absent. + * + * The owner's revalidation of acquisition, root and execution belongs to the + * checkpoint where a lifecycle transaction exists. What is asserted here is the + * private request contract: what a settle command has to be to be read at all. + */ + +import { describe, expect, it } from "vitest"; +import { CommandError, parseCommand } from "../../src/cloudflare/commands.ts"; + +const ROOT = "9f2c4b6a8d0e1f23456789abcdef0123456789abcdef0123456789abcdef0123"; + +function settle(overrides: Record = {}): string { + return JSON.stringify({ + id: "s1", + command: "settle", + completion: { executionId: "execution-1", status: "completed" }, + expectedWorkspaceRootId: ROOT, + ...overrides, + }); +} + +/** The refusal category, or the command name when it was read. */ +function read(raw: string): string { + try { + return parseCommand(raw).command; + } catch (error) { + return error instanceof CommandError ? error.refusal : "unexpected"; + } +} + +describe("a settle command", () => { + it("reads a complete completion through the shared parser", () => { + const command = parseCommand(settle()); + expect(command.command).toBe("settle"); + if (command.command !== "settle") { + throw new Error("expected a settle command"); + } + expect(command.completion).toEqual({ executionId: "execution-1", status: "completed" }); + expect(command.expectedWorkspaceRootId).toBe(ROOT); + }); + + it("reads a completion carrying a stop reason", () => { + const command = parseCommand( + settle({ + completion: { + executionId: "execution-1", + status: "failed", + reason: { kind: "host", code: "settlement-refused" }, + }, + }), + ); + if (command.command !== "settle") { + throw new Error("expected a settle command"); + } + expect(command.completion.reason).toEqual({ kind: "host", code: "settlement-refused" }); + }); + + it("refuses a completion the shared parser will not read", () => { + // Each of these is refused by the shared contract rather than by a private + // approximation of it, and each becomes this transport's own closed + // refusal rather than carrying the parser's message onto the wire. + expect(read(settle({ completion: { status: "completed" } }))).toBe("malformed-member"); + expect(read(settle({ completion: { executionId: "", status: "completed" } }))).toBe( + "malformed-member", + ); + expect(read(settle({ completion: { executionId: "e", status: "invented" } }))).toBe( + "malformed-member", + ); + expect(read(settle({ completion: { executionId: "e", status: "failed", reason: 7 } }))).toBe( + "malformed-member", + ); + expect(read(settle({ completion: "not an object" }))).toBe("malformed-member"); + expect(read(settle({ completion: undefined }))).toBe("malformed-member"); + }); + + it("refuses a missing or malformed expected root", () => { + expect(read(settle({ expectedWorkspaceRootId: undefined }))).toBe("malformed-member"); + expect(read(settle({ expectedWorkspaceRootId: "" }))).toBe("malformed-member"); + expect(read(settle({ expectedWorkspaceRootId: 1 }))).toBe("malformed-member"); + }); + it("refuses a member the command does not declare", () => { + expect(read(settle({ status: "completed" }))).toBe("unknown-member"); + expect(read(settle({ somethingElse: true }))).toBe("unknown-member"); + }); +}); + +/** + * A retained Workspace root is a content identity, and every command that names + * one names the same thing. A command shape that admitted "any non-empty text" + * would let a request select a root by a spelling the store can never hold, and + * would let two commands disagree about what a root is. + * + * Shape only. Whether a well-spelled root is the one this run is actually at is + * the owner's revalidation, in the checkpoint that has a lifecycle to check it + * against. + */ +describe("a root identity in a command", () => { + const wrong: Record = { + "one character short": ROOT.slice(1), + "one character long": `${ROOT}0`, + "uppercase hexadecimal": ROOT.toUpperCase(), + "hexadecimal with a non-hexadecimal letter": `${ROOT.slice(0, 63)}z`, + "a plausible-looking name": "root-a", + "not text at all": 7, + }; + + /** Every root field in the private command shapes, by the request it sits in. */ + const fields: Record string> = { + "root.workspaceRootId": (root) => + JSON.stringify({ id: "m1", command: "root", workspaceRootId: root }), + "content.workspaceRootId": (root) => + JSON.stringify({ + id: "r1", + command: "content", + workspaceRootId: root, + kind: "blob", + digest: ROOT, + sourceManifest: ROOT, + }), + "commit.expectedWorkspaceRootId": (root) => commit({ expectedWorkspaceRootId: root }), + "commit.publication.proposedWorkspaceRootId": (root) => + commit({ + publication: { proposedWorkspaceRootId: root, proposedManifest: "{}", content: [] }, + }), + "settle.expectedWorkspaceRootId": (root) => settle({ expectedWorkspaceRootId: root }), + }; + + function commit(overrides: Record): string { + return JSON.stringify({ + id: "c1", + command: "commit", + expectedWorkspaceRootId: ROOT, + expectedJournalEventId: null, + publication: { proposedWorkspaceRootId: ROOT, proposedManifest: "{}", content: [] }, + mappings: [], + events: [], + ...overrides, + }); + } + + it("reads the canonical spelling in every command that names one", () => { + for (const [field, request] of Object.entries(fields)) { + expect([field, read(request(ROOT))]).toEqual([field, field.split(".")[0]]); + } + }); + + it("refuses anything that is not the canonical spelling", () => { + for (const [field, request] of Object.entries(fields)) { + for (const [description, root] of Object.entries(wrong)) { + expect([field, description, read(request(root))]).toEqual([ + field, + description, + "malformed-member", + ]); + } + } + }); +}); diff --git a/packages/workflow/tests/cloudflare/storage-capabilities.vitest.ts b/packages/workflow/tests/cloudflare/storage-capabilities.vitest.ts new file mode 100644 index 000000000..ddbe22c9c --- /dev/null +++ b/packages/workflow/tests/cloudflare/storage-capabilities.vitest.ts @@ -0,0 +1,70 @@ +/** + * What a Durable Object's SQLite actually permits. + * + * The Cloudflare owner is built on these answers: the schema marker exists + * because the pragmas are refused, and the owner opens exactly one real + * transaction and enlists DOFS directly inside it because a reentrant + * transaction is refused. Both are properties of the runtime rather than of any + * model of it, so they are asserted here against real workerd — a platform + * change that moved either one should fail this suite rather than be discovered + * as a corrupted run. + * + * The assertions match categories, not the platform's wording: the exact + * sentence a runtime uses to refuse is not a contract, and pinning it would + * make this fail for a rephrasing. + */ + +import { env, runInDurableObject } from "cloudflare:test"; +import { describe, expect, it } from "vitest"; +import type { StorageProbeObject } from "./support/probe-object.ts"; + +function capabilities() { + const stub = env.STORAGE_PROBE.get(env.STORAGE_PROBE.idFromName("capabilities")); + return runInDurableObject(stub, (instance: StorageProbeObject) => instance.capabilities()); +} + +/** A refusal, whatever the runtime called it. */ +function refused(answer: string): boolean { + return answer.startsWith("refused:"); +} + +/** A refusal the runtime attributed to its authorization layer. */ +function unauthorized(answer: string): boolean { + return refused(answer) && answer.includes("SQLITE_AUTH"); +} + +/** A refusal directing the caller to the storage transaction API. */ +function transactionApiRequired(answer: string): boolean { + return refused(answer) && answer.includes("transactionSync"); +} + +describe("Durable Object SQLite storage", () => { + it("refuses the pragmas the Deno host carries its schema identity in", async () => { + const found = await capabilities(); + expect(unauthorized(found.applicationIdRead)).toBe(true); + expect(unauthorized(found.applicationIdWrite)).toBe(true); + expect(unauthorized(found.userVersionRead)).toBe(true); + expect(unauthorized(found.userVersionWrite)).toBe(true); + }); + + it("refuses SQL transaction statements, directly and through a nested wrapper", async () => { + const found = await capabilities(); + expect(transactionApiRequired(found.savepointDirect)).toBe(true); + expect(transactionApiRequired(found.nestedTransaction)).toBe(true); + // The one that decides the owner's commit shape: the vendored DOFS opens a + // transaction of its own for a filesystem write, so calling it inside an + // owner transaction is a reentrant call and is refused. + expect(transactionApiRequired(found.filesystemInsideTransaction)).toBe(true); + }); + + it("accepts what the owner is built on instead", async () => { + const found = await capabilities(); + expect(refused(found.schemaObjects)).toBe(false); + expect(refused(found.outerTransaction)).toBe(false); + expect(refused(found.xmdTableDdl)).toBe(false); + expect(refused(found.dofsSchema)).toBe(false); + expect(refused(found.dofsFilesystem)).toBe(false); + // A strict metadata table is what carries the identity the pragmas cannot. + expect(found.metadataTable).toContain("application_id"); + }); +}); diff --git a/packages/workflow/tests/cloudflare/support/executor-object.ts b/packages/workflow/tests/cloudflare/support/executor-object.ts new file mode 100644 index 000000000..adc2034ec --- /dev/null +++ b/packages/workflow/tests/cloudflare/support/executor-object.ts @@ -0,0 +1,614 @@ +/** + * A concrete owner, so admission and acquisition can be exercised end to end. + * + * It supplies the two things `WorkflowOwnerObject` leaves abstract — a policy + * and a `perform` — and nothing else. `perform` answers with the command it was + * given rather than doing durable work: what these tests are about is who is + * allowed to send one, not what each one means. + */ + +import { run } from "effection"; +import { serializeDurableEvent } from "@executablemd/durable-streams"; +import { acquisitionHolders } from "../../../src/cloudflare/acquisition.ts"; +import { WorkflowOwnerObject } from "../../../src/cloudflare/owner.ts"; +import type { AdmissionRequest, OwnerConfiguration } from "../../../src/cloudflare/owner.ts"; +import type { AdmissionPolicy } from "../../../src/cloudflare/admission.ts"; +import type { TokenVerification, VerificationKey } from "../../../src/cloudflare/token.ts"; +import { refusalOf } from "../../../src/cloudflare/owner.ts"; +import { sha256Hex } from "../../../src/workspace/sha256.ts"; +import { WORKSPACE_ROOT_DOMAIN } from "../../../src/workspace/root-manifest.ts"; +import { COMMAND_TABLE, STAGING_TABLE } from "../../../src/cloudflare/private-schema.ts"; +import { MARKER_TABLE } from "../../../src/cloudflare/marker.ts"; + +/** The identities this owner is configured to admit. */ +export const POLICY: AdmissionPolicy = { + issuer: "https://token.actions.githubusercontent.com", + audience: "https://factory.example", + repositoryId: "123456", + repositoryOwnerId: "654321", + eventName: "repository_dispatch", + workflowRef: "octo/repo/.github/workflows/factory.yml@refs/heads/main", + workflowSha: "0f2c9a1b3d4e5f60718293a4b5c6d7e8f9012345", + jobWorkflowRef: "octo/repo/.github/workflows/factory.yml@refs/heads/main", + release: "factory-2026.09.02-abcdef", +}; + +/** The claims a correctly issued token carries for the policy above. */ +export const VALID_CLAIMS: Record = { + iss: POLICY.issuer, + aud: POLICY.audience, + repository_id: POLICY.repositoryId, + repository_owner_id: POLICY.repositoryOwnerId, + event_name: POLICY.eventName, + workflow_ref: POLICY.workflowRef, + workflow_sha: POLICY.workflowSha, + job_workflow_ref: POLICY.jobWorkflowRef, +}; + +export const RUN_ID = "5cktgrv2zyutngh7bbddr2tyg2b5a567cg725hu5e7u42orerxaa"; +export const FILE_BYTES = new TextEncoder().encode("hello from the retained Workspace"); +export const BLOB_ID = sha256Hex(FILE_BYTES); +export const DOFS_MANIFEST = JSON.stringify({ + version: 1, + chunks: [{ hash: BLOB_ID, size: FILE_BYTES.length }], +}); +export const MANIFEST_ID = sha256Hex(new TextEncoder().encode(DOFS_MANIFEST)); +export const ROOT_MANIFEST = JSON.stringify({ + format: 1, + entries: [ + { path: "/", kind: "directory", mode: 493, mtime: 0 }, + { + path: "/README.md", + kind: "file", + mode: 420, + mtime: 0, + size: FILE_BYTES.length, + manifest: MANIFEST_ID, + hardlink: null, + }, + ], +}); +export const ROOT_ID = sha256Hex(`${WORKSPACE_ROOT_DOMAIN}${ROOT_MANIFEST}`); + +/** A second root: the same tree with one more file, as a proposal would be. */ +export const NEXT_BYTES = new TextEncoder().encode("published by the runner"); +export const NEXT_BLOB_ID = sha256Hex(NEXT_BYTES); +export const NEXT_DOFS_MANIFEST = JSON.stringify({ + version: 1, + chunks: [{ hash: NEXT_BLOB_ID, size: NEXT_BYTES.length }], +}); +export const NEXT_MANIFEST_ID = sha256Hex(new TextEncoder().encode(NEXT_DOFS_MANIFEST)); +export const NEXT_ROOT_MANIFEST = JSON.stringify({ + format: 1, + entries: [ + { path: "/", kind: "directory", mode: 493, mtime: 0 }, + { + path: "/NOTES.md", + kind: "file", + mode: 420, + mtime: 0, + size: NEXT_BYTES.length, + manifest: NEXT_MANIFEST_ID, + hardlink: null, + }, + { + path: "/README.md", + kind: "file", + mode: 420, + mtime: 0, + size: FILE_BYTES.length, + manifest: MANIFEST_ID, + hardlink: null, + }, + // The checkout a Repository mapping claims, in canonical byte order — a + // mapping whose directory the proposed root does not contain is a record + // about files nobody wrote. + { path: "/app", kind: "directory", mode: 493, mtime: 0 }, + ], +}); +export const NEXT_ROOT_ID = sha256Hex(`${WORKSPACE_ROOT_DOMAIN}${NEXT_ROOT_MANIFEST}`); + +/** + * The proposal that publishes `NEXT_ROOT_ID`. + * + * Its inventory is the exact closure of the proposed manifest: both file + * manifests and both blobs, once each, in canonical order. One of each is + * already authoritative, which is what proves the owner reuses retained content + * by identity rather than requiring it to be sent again. + */ +export function nextPublication(): Record { + const content: { kind: string; digest: string; size: number }[] = [ + { kind: "blob", digest: BLOB_ID, size: FILE_BYTES.length }, + { kind: "blob", digest: NEXT_BLOB_ID, size: NEXT_BYTES.length }, + { kind: "manifest", digest: MANIFEST_ID, size: DOFS_MANIFEST.length }, + { kind: "manifest", digest: NEXT_MANIFEST_ID, size: NEXT_DOFS_MANIFEST.length }, + ]; + content.sort((left, right) => + `${left.kind}:${left.digest}` < `${right.kind}:${right.digest}` ? -1 : 1, + ); + return { + proposedWorkspaceRootId: NEXT_ROOT_ID, + proposedManifest: NEXT_ROOT_MANIFEST, + content, + }; +} +const CREATED_AT = "2026-09-03T00:00:00.000Z"; + +export class ExecutorObject extends WorkflowOwnerObject { + /** + * The verification material this owner is configured with. + * + * Installed by a test before it connects, exactly as a deployment would + * install a fetched JWKS. It is closure state on the object, never something + * an admission request can name. + */ + #keys: VerificationKey[] = []; + #now = 1_800_000_000; + #skew = 0; + + configure(keys: VerificationKey[], now?: number, skew?: number): void { + this.#keys = keys; + if (now !== undefined) { + this.#now = now; + } + if (skew !== undefined) { + this.#skew = skew; + } + } + + protected configuration(): OwnerConfiguration { + const verification: TokenVerification = { + keys: this.#keys, + skewSeconds: this.#skew, + now: () => this.#now, + }; + return { policy: POLICY, verification }; + } + + initialize(): void { + this.open(RUN_ID, () => { + const blob = hexBytes(BLOB_ID); + const manifest = hexBytes(MANIFEST_ID); + this.ctx.storage.sql.exec( + "INSERT INTO vfs_blobs (hash, size, last_seen) VALUES (?, ?, 0)", + blob, + FILE_BYTES.length, + ); + this.ctx.storage.sql.exec( + "INSERT INTO vfs_blob_bytes (hash, bytes) VALUES (?, ?)", + blob, + new Uint8Array(FILE_BYTES), + ); + this.ctx.storage.sql.exec( + "INSERT INTO vfs_manifests (hash, size, encoded, last_seen) VALUES (?, ?, ?, 0)", + manifest, + FILE_BYTES.length, + new TextEncoder().encode(DOFS_MANIFEST), + ); + this.ctx.storage.sql.exec( + "INSERT INTO workspace_roots (root_id, format_version, manifest) VALUES (?, 1, ?)", + ROOT_ID, + ROOT_MANIFEST, + ); + this.ctx.storage.sql.exec( + "INSERT INTO workspace_root_manifest_refs (root_id, manifest_hash) VALUES (?, ?)", + ROOT_ID, + manifest, + ); + this.ctx.storage.sql.exec( + "INSERT INTO workspace_root_blob_refs (root_id, blob_hash) VALUES (?, ?)", + ROOT_ID, + blob, + ); + this.ctx.storage.sql.exec( + "INSERT INTO workspace_state (singleton_id, current_root_id) VALUES (1, ?)", + ROOT_ID, + ); + this.ctx.storage.sql.exec( + `INSERT INTO workflow_run + (id, run_id, definition, base, props, status, created_at, updated_at) + VALUES (1, ?, ?, ?, ?, 'running', ?, ?)`, + RUN_ID, + JSON.stringify({ + version: 1, + kind: "git", + objectFormat: "sha1", + objectId: "0".repeat(40), + rootDocumentPath: "README.md", + }), + "main", + "{}", + CREATED_AT, + CREATED_AT, + ); + this.ctx.storage.sql.exec( + "INSERT INTO definition_retrieval (id, metadata, revision, updated_at) VALUES (1, ?, 1, ?)", + JSON.stringify({ locator: "https://example.invalid/repository.git" }), + CREATED_AT, + ); + }); + } + + appendJournal(eventId: string, name: string): void { + this.ctx.storage.sql.exec( + "INSERT INTO journal_events (event_id, record, workspace_root_id) VALUES (?, ?, ?)", + eventId, + serializeDurableEvent({ + type: "yield", + coroutineId: "root", + description: { type: "test", name }, + result: { status: "ok", value: name }, + }), + ROOT_ID, + ); + } + + scratch(): { commands: number; staged: number } { + const commands = this.ctx.storage.sql + .exec(`SELECT count(*) AS count FROM ${COMMAND_TABLE}`) + .toArray()[0]?.["count"]; + const staged = this.ctx.storage.sql + .exec(`SELECT count(*) AS count FROM ${STAGING_TABLE}`) + .toArray()[0]?.["count"]; + return { + commands: typeof commands === "number" ? commands : -1, + staged: typeof staged === "number" ? staged : -1, + }; + } + + /** + * Everything this run authoritatively holds, as one comparable value. + * + * Row-for-row rather than a count: cleanup that deleted a journal event and + * inserted another would keep every count identical, and the claim being + * checked is that acquisition cleanup touched none of this. + */ + authoritative(): string { + const tables = [ + "SELECT id, run_id, definition, base, props, status, created_at, updated_at FROM workflow_run ORDER BY id", + "SELECT root_id, format_version, manifest FROM workspace_roots ORDER BY root_id", + "SELECT singleton_id, current_root_id FROM workspace_state ORDER BY singleton_id", + "SELECT sequence, event_id, record, workspace_root_id FROM journal_events ORDER BY sequence", + "SELECT root_id, lower(hex(manifest_hash)) AS h FROM workspace_root_manifest_refs ORDER BY root_id, h", + "SELECT root_id, lower(hex(blob_hash)) AS h FROM workspace_root_blob_refs ORDER BY root_id, h", + "SELECT lower(hex(hash)) AS h, lower(hex(bytes)) AS b FROM vfs_blob_bytes ORDER BY h", + "SELECT lower(hex(hash)) AS h, size, lower(hex(encoded)) AS e FROM vfs_manifests ORDER BY h", + ]; + return sha256Hex( + JSON.stringify(tables.map((query) => this.ctx.storage.sql.exec(query).toArray())), + ); + } + + /** + * Fail inside the owner transaction, after every category has been written. + * + * Injected rather than simulated: the claim is that the runtime's own + * transaction rolls content, roots, references, mappings, the pointer, the + * journal and the retry decision back together, and only a real failure + * inside a real `transactionSync()` can show that. + */ + failAfterApply(raw: string): string { + try { + return String( + this.transactions.run(this.ctx.storage, () => { + const socket = this.ctx.getWebSockets("executor")[0]; + if (socket === undefined) { + throw new Error("no live acquisition"); + } + const answer = this.onRunnerMessage(socket, RUN_ID, raw); + throw new Error(`forced failure after ${JSON.stringify(answer)}`); + }), + ); + } catch (error) { + return error instanceof Error && error.message.startsWith("forced failure") + ? "rolled-back" + : `threw:${String(error)}`; + } + } + + /** Everything a reader could observe about the published frontier. */ + published(): Record { + const state = this.ctx.storage.sql + .exec("SELECT current_root_id FROM workspace_state WHERE singleton_id = 1") + .toArray()[0]; + const roots = this.ctx.storage.sql + .exec("SELECT count(*) AS found FROM workspace_roots") + .toArray()[0]; + const events = this.ctx.storage.sql + .exec("SELECT event_id, workspace_root_id FROM journal_events ORDER BY sequence") + .toArray(); + const repositories = this.ctx.storage.sql + .exec("SELECT name, checkout_path FROM workspace_repositories ORDER BY name") + .toArray(); + const blobs = this.ctx.storage.sql + .exec("SELECT count(*) AS found FROM vfs_blob_bytes") + .toArray()[0]; + const refs = this.ctx.storage.sql + .exec("SELECT count(*) AS found FROM workspace_root_blob_refs") + .toArray()[0]; + return { + currentRootId: state?.["current_root_id"] ?? null, + roots: Number(roots?.["found"] ?? -1), + events, + repositories, + blobs: Number(blobs?.["found"] ?? -1), + blobRefs: Number(refs?.["found"] ?? -1), + }; + } + + /** The exact locator a retained Repository row holds. */ + repositoryLocator(name: string): string { + const row = this.ctx.storage.sql + .exec("SELECT locator FROM workspace_repositories WHERE name = ?", name) + .toArray()[0]; + return row === undefined ? "" : String(row["locator"]); + } + + /** A blob's metadata with no bytes beside it: a half-written identity. */ + removeBlobBytesOnly(digest: string, size: number): void { + this.ctx.storage.sql.exec( + "INSERT INTO vfs_blobs (hash, size, last_seen) VALUES (?, ?, 0) ON CONFLICT(hash) DO NOTHING", + hexBytes(digest), + size, + ); + } + + /** Begin one document execution, as a lifecycle transition would. */ + beginExecution(executionId: string, startedAt: string): void { + this.ctx.storage.sql.exec( + "INSERT INTO document_executions (execution_id, started_at) VALUES (?, ?)", + executionId, + startedAt, + ); + } + + /** Stop one document execution, as the matching transition would. */ + stopExecution(executionId: string, stoppedAt: string, status: string, code?: string): void { + this.ctx.storage.sql.exec( + "UPDATE document_executions SET stopped_at = ?, stop_status = ?, stop_reason_kind = ?, stop_reason_code = ? WHERE execution_id = ?", + stoppedAt, + status, + code === undefined ? null : "host", + code ?? null, + executionId, + ); + } + + /** What the retrieval row holds right now. */ + retrieval(): Record | null { + const row = this.ctx.storage.sql + .exec("SELECT metadata, revision, updated_at FROM definition_retrieval WHERE id = 1") + .toArray()[0]; + return row === undefined ? null : row; + } + + /** Retain more Repository rows than one admitted snapshot may carry. */ + fillRepositories(from: number, count: number): void { + for (let index = from; index < from + count; index += 1) { + const name = `repo-${String(index).padStart(4, "0")}`; + this.ctx.storage.sql.exec( + `INSERT INTO workspace_repositories (name, locator, locator_fingerprint, requested_base, + creation_commit, primary_branch, object_format, checkout_path) + VALUES (?, ?, ?, NULL, ?, 'main', 'sha1', ?)`, + name, + `https://git.example.invalid/${name}.git`, + "a".repeat(64), + "9".repeat(40), + `/${name}`, + ); + } + } + + damageRetainedBlob(): void { + this.ctx.storage.sql.exec( + "UPDATE vfs_blob_bytes SET bytes = ?", + new TextEncoder().encode("bad"), + ); + } + + /** + * Collect the DOFS manifest a retained file entry still names. + * + * The reference row goes first because the schema will not let it go second: + * `ON DELETE RESTRICT` is what stops content vanishing from under a root that + * references it. What this reproduces is the state that restriction cannot + * prevent — a root whose manifest still names content the store no longer + * keeps, with the reference collected alongside it. + */ + removeManifestRow(): void { + this.ctx.storage.sql.exec( + "DELETE FROM workspace_root_manifest_refs WHERE lower(hex(manifest_hash)) = ?", + MANIFEST_ID, + ); + this.ctx.storage.sql.exec("DELETE FROM vfs_manifests WHERE lower(hex(hash)) = ?", MANIFEST_ID); + } + + /** Keep the manifest row, change the bytes it is identified by. */ + damageManifestPayload(): void { + this.ctx.storage.sql.exec( + "UPDATE vfs_manifests SET encoded = ? WHERE lower(hex(hash)) = ?", + new TextEncoder().encode('{"version":1,"chunks":[]}'), + MANIFEST_ID, + ); + } + + /** Keep identity and payload, disagree about how many bytes they describe. */ + damageManifestSize(): void { + this.ctx.storage.sql.exec( + "UPDATE vfs_manifests SET size = size + 1 WHERE lower(hex(hash)) = ?", + MANIFEST_ID, + ); + } + + /** Collect the blob a referenced manifest chunk still names, reference first. */ + removeBlobRow(): void { + this.removeBlobReference(); + this.ctx.storage.sql.exec("DELETE FROM vfs_blob_bytes WHERE lower(hex(hash)) = ?", BLOB_ID); + this.ctx.storage.sql.exec("DELETE FROM vfs_blobs WHERE lower(hex(hash)) = ?", BLOB_ID); + } + + /** Keep the blob and its bytes, disagree about its recorded size. */ + damageBlobSize(): void { + this.ctx.storage.sql.exec( + "UPDATE vfs_blobs SET size = size + 1 WHERE lower(hex(hash)) = ?", + BLOB_ID, + ); + } + + /** Drop the root's reference to a blob its manifests still name. */ + removeBlobReference(): void { + this.ctx.storage.sql.exec( + "DELETE FROM workspace_root_blob_refs WHERE root_id = ? AND lower(hex(blob_hash)) = ?", + ROOT_ID, + BLOB_ID, + ); + } + + /** Reference content from the root that none of its manifests names. */ + addExtraBlobReference(bytes: Uint8Array): string { + const digest = this.addUnreferencedBlob(bytes); + this.ctx.storage.sql.exec( + "INSERT INTO workspace_root_blob_refs (root_id, blob_hash) VALUES (?, ?)", + ROOT_ID, + hexBytes(digest), + ); + return digest; + } + + makeForeign(): void { + this.ctx.storage.sql.exec("CREATE TABLE foreign_state (id INTEGER PRIMARY KEY)"); + } + + rewriteMarker(applicationId: number, schemaVersion: number): void { + this.ctx.storage.sql.exec( + `UPDATE ${MARKER_TABLE} SET application_id = ?, schema_version = ? WHERE id = 1`, + applicationId, + schemaVersion, + ); + } + + dropTable(name: string): void { + this.ctx.storage.sql.exec(`DROP TABLE ${name}`); + } + + rewriteRunId(runId: string): void { + this.ctx.storage.sql.exec("UPDATE workflow_run SET run_id = ? WHERE id = 1", runId); + } + + removeWorkspaceState(): void { + this.ctx.storage.sql.exec("DELETE FROM workspace_state"); + } + + addUnreferencedBlob(bytes: Uint8Array): string { + const digest = sha256Hex(bytes); + const hash = hexBytes(digest); + this.ctx.storage.sql.exec( + "INSERT INTO vfs_blobs (hash, size, last_seen) VALUES (?, ?, 0)", + hash, + bytes.length, + ); + this.ctx.storage.sql.exec( + "INSERT INTO vfs_blob_bytes (hash, bytes) VALUES (?, ?)", + hash, + new Uint8Array(bytes), + ); + return digest; + } + + /** + * Admit one connection, answering what happened rather than raising. + * + * The server half of the pair is what the object admitted. Verification is + * asynchronous, so this drives the admission operation through one Effection + * scope — the runtime callback boundary this host adapts at. + */ + async admitConnection(request: Partial): Promise { + const pair = new WebSocketPair(); + const server = pair[1]; + const presented: AdmissionRequest = { + runId: "runId" in request ? request.runId : RUN_ID, + release: "release" in request ? request.release : POLICY.release, + token: "token" in request ? request.token : undefined, + }; + try { + await run(() => this.admit(presented, server)); + return "admitted"; + } catch (error) { + return refusalOf(error); + } + } + + async fetch(request: Request): Promise { + const pair = new WebSocketPair(); + const client = pair[0]; + const server = pair[1]; + try { + await run(() => + this.admit( + { + runId: request.headers.get("x-run-id"), + release: request.headers.get("x-release"), + token: request.headers.get("authorization")?.replace(/^Bearer /, ""), + }, + server, + ), + ); + return new Response(null, { status: 101, webSocket: client }); + } catch (error) { + return new Response(refusalOf(error), { status: 403 }); + } + } + + /** The correlation the live acquisition is partitioned by. */ + acquisitionId(): string { + const held = acquisitionHolders(this.ctx)[0]; + return held === undefined ? "" : held.held.acquisitionId; + } + + /** How many live connections currently hold this run's executor. */ + holders(): number { + return acquisitionHolders(this.ctx).length; + } + + /** Send one message as the connection admitted at `index` (1-based). */ + send(index: number, raw: string): unknown { + const socket = this.ctx.getWebSockets("executor")[index - 1]; + if (socket === undefined) { + return { id: "", outcome: "refused", refusal: "no-such-connection" }; + } + return this.onRunnerMessage(socket, RUN_ID, raw); + } + + /** Send as a socket this object never admitted. */ + sendAsStranger(raw: string): unknown { + const pair = new WebSocketPair(); + return this.onRunnerMessage(pair[1], RUN_ID, raw); + } + + sendWithCopiedAttachment(raw: string): unknown { + const live = this.ctx.getWebSockets("executor")[0]; + if (live === undefined) { + return { id: "", outcome: "refused", refusal: "no-such-connection" }; + } + const pair = new WebSocketPair(); + pair[1].serializeAttachment(live.deserializeAttachment()); + return this.onRunnerMessage(pair[1], RUN_ID, raw); + } + + /** Close the connection admitted at `index`, releasing its acquisition. */ + closeConnection(index: number): void { + const socket = this.ctx.getWebSockets("executor")[index - 1]; + if (socket !== undefined) { + socket.close(1000, "done"); + this.webSocketClose(socket); + } + } +} + +function hexBytes(value: string): Uint8Array { + const bytes = new Uint8Array(value.length / 2); + for (let index = 0; index < bytes.length; index += 1) { + bytes[index] = Number.parseInt(value.slice(index * 2, index * 2 + 2), 16); + } + return bytes; +} diff --git a/packages/workflow/tests/cloudflare/support/owner-object.ts b/packages/workflow/tests/cloudflare/support/owner-object.ts new file mode 100644 index 000000000..b66641914 --- /dev/null +++ b/packages/workflow/tests/cloudflare/support/owner-object.ts @@ -0,0 +1,202 @@ +/** + * A Durable Object that exercises the owner's storage paths on real workerd. + * + * It is deliberately thin: each method does one thing the owner does — create + * the schema, recognize it again, commit a mixed change, or fail partway + * through one — so a test can assert the outcome rather than a model of it. + */ + +import { DurableObject } from "cloudflare:workers"; +import { mkdir as mkdirPath } from "../../../vendor/cloudflare-computer-dofs/generated/fs/mkdir.js"; +import { writeFileSync } from "../../../vendor/cloudflare-computer-dofs/generated/fs/writeFile.js"; +import { + initializeObject, + recognizeObject, + WorkflowObjectStorageError, +} from "../../../src/cloudflare/recognition.ts"; +import { MARKER_TABLE } from "../../../src/cloudflare/marker.ts"; +import { + OwnerTransactionNestedError, + OwnerTransactions, +} from "../../../src/cloudflare/owner-transaction.ts"; +import type { OwnerStorage } from "../../../src/cloudflare/storage.ts"; + +/** One run row, so initialization writes what a real run would. */ +const RUN_ID = "run-under-test"; + +export class OwnerObject extends DurableObject { + readonly #transactions = new OwnerTransactions(); + + /** Create the schema, DOFS schema, an empty root and the run row, then mark it. */ + initialize(): string { + try { + initializeObject(this.ctx.storage, this.#transactions, () => { + this.ctx.storage.sql.exec( + "INSERT INTO workflow_run (run_id, definition, base, props, status, created_at, updated_at) VALUES (?, ?, ?, ?, ?, ?, ?)", + RUN_ID, + JSON.stringify({ version: 1 }), + "main", + "{}", + "running", + 0, + 0, + ); + }); + return "initialized"; + } catch (error) { + return describe(error); + } + } + + /** Read the storage back as a version-1 workflow run. */ + recognize(): string { + try { + recognizeObject(this.ctx.storage); + return "recognized"; + } catch (error) { + return describe(error); + } + } + + /** What the marker holds right now. */ + marker(): Record[] { + return this.ctx.storage.sql + .exec(`SELECT application_id, schema_version FROM ${MARKER_TABLE}`) + .toArray(); + } + + /** Drop one declared object, so recognition sees a shape that disagrees. */ + damage(table: string): void { + this.ctx.storage.sql.exec(`DROP TABLE ${table}`); + } + + /** Write an unrelated object, so pristine detection sees a foreign store. */ + addForeignObject(): void { + this.ctx.storage.sql.exec("CREATE TABLE somebody_elses (id INTEGER PRIMARY KEY)"); + } + + /** Replace the marker's identity with another application's. */ + rewriteMarker(applicationId: number, schemaVersion: number): void { + this.ctx.storage.sql.exec( + `UPDATE ${MARKER_TABLE} SET application_id = ?, schema_version = ? WHERE id = 1`, + applicationId, + schemaVersion, + ); + } + + /** + * Change DOFS content and a WorkflowRun row in one transaction. + * + * `fail` throws after both have been changed, which is the case that decides + * whether the two categories really share a transaction. + */ + commitMixedChange(fail: boolean): string { + try { + this.#transactions.run(this.ctx.storage, ({ dofs }) => { + mkdirPath(dofs, "/published", { recursive: true }, () => 0); + // oxlint-disable-next-line local/no-sync-filesystem + writeFileSync( + dofs, + "/published/root.txt", + new TextEncoder().encode("frontier"), + {}, + () => 0, + ); + this.ctx.storage.sql.exec( + "UPDATE workflow_run SET status = ?, updated_at = ? WHERE run_id = ?", + "suspended", + 1, + RUN_ID, + ); + if (fail) { + throw new Error("forced failure after both categories changed"); + } + }); + return "committed"; + } catch (error) { + return describe(error); + } + } + + /** + * Open an owner transaction inside one, on this object's own storage. + * + * The runtime admits exactly one, so this must be refused before it reaches + * the transaction API rather than by the runtime rejecting a savepoint. + */ + nestOnSameStorage(): string { + try { + this.#transactions.run(this.ctx.storage, () => { + this.#transactions.run(this.ctx.storage, () => undefined); + }); + return "nested"; + } catch (error) { + return error instanceof OwnerTransactionNestedError ? "refused:nested" : describe(error); + } + } + + /** + * Hold a transaction on this object's real storage and open another on a + * different storage at the same time. + * + * The second storage is a local stand-in rather than another object's: the + * runtime forbids touching another Durable Object's I/O, which is exactly why + * the guard has to be keyed by storage instance rather than shared. What is + * being proved is that holding one does not block the other. + */ + transactOnADifferentStorage(): string { + const other = standInStorage(); + try { + // A second gate stands for a second Durable Object: what must not happen + // is one object's open transaction refusing another object's. + const otherObject = new OwnerTransactions(); + return this.#transactions.run(this.ctx.storage, () => + otherObject.run(other, () => "committed while another storage transacted"), + ); + } catch (error) { + return describe(error); + } + } + + /** What the run row and the DOFS filesystem hold, read outside any transaction. */ + frontier(): { status: string; publishedPaths: number } { + const runRows = this.ctx.storage.sql + .exec("SELECT status FROM workflow_run WHERE run_id = ?", RUN_ID) + .toArray(); + const first = runRows[0]; + const paths = this.ctx.storage.sql + .exec("SELECT count(*) AS found FROM vfs_dirents WHERE name = ?", "root.txt") + .toArray(); + const found = paths[0]; + return { + status: first === undefined ? "absent" : String(first["status"]), + publishedPaths: found === undefined ? -1 : Number(found["found"]), + }; + } +} + +function describe(error: unknown): string { + if (error instanceof WorkflowObjectStorageError) { + return `refused:${error.failure.kind}`; + } + return `threw:${error instanceof Error ? error.message : String(error)}`; +} + +/** + * A second storage that is not this object's. + * + * It answers nothing useful — the transaction opened on it does no SQL — so it + * is only ever asked whether it is a different key than the real one. + */ +function standInStorage(): OwnerStorage { + return { + sql: { + exec(): { toArray(): Record[] } { + return { toArray: () => [] }; + }, + }, + transactionSync(closure: () => T): T { + return closure(); + }, + }; +} diff --git a/packages/workflow/tests/cloudflare/support/probe-object.ts b/packages/workflow/tests/cloudflare/support/probe-object.ts new file mode 100644 index 000000000..1daf1072e --- /dev/null +++ b/packages/workflow/tests/cloudflare/support/probe-object.ts @@ -0,0 +1,144 @@ +/** + * A Durable Object that answers what its own SQLite storage can actually do. + * + * The version-1 schema is recognized through `PRAGMA application_id` and + * `PRAGMA user_version`, and the vendored DOFS `Database` opens reentrant + * transactions with `SAVEPOINT` through `sql.exec`. Cloudflare's own + * documentation says `sql.exec()` cannot execute transaction statements and + * says nothing about those two pragmas, so neither assumption can be settled + * from prose — the owner either has the same recognition contract the Deno host + * has, or it does not, and that decides how §4 is written rather than being a + * detail inside it. + * + * So this object exists to be asked, on real workerd. It lives in test support + * rather than in production source: it measures the runtime, and the answers it + * gives are asserted by `storage-capabilities.vitest.ts` so a platform change + * that moved any of them would fail rather than pass quietly. + */ + +import { DurableObject } from "cloudflare:workers"; +import { dofsStorage } from "../../../src/cloudflare/storage.ts"; +import { Database as DofsDatabase } from "../../../vendor/cloudflare-computer-dofs/generated/storage.js"; +import { initializeSchema as initializeDofsSchema } from "../../../vendor/cloudflare-computer-dofs/generated/schema/index.js"; +import { mkdir as mkdirPath } from "../../../vendor/cloudflare-computer-dofs/generated/fs/mkdir.js"; +import { writeFileSync } from "../../../vendor/cloudflare-computer-dofs/generated/fs/writeFile.js"; + +export interface StorageCapabilities { + readonly applicationIdRead: string; + readonly applicationIdWrite: string; + readonly userVersionRead: string; + readonly userVersionWrite: string; + readonly schemaObjects: string; + readonly outerTransaction: string; + readonly nestedTransaction: string; + readonly savepointDirect: string; + readonly dofsSchema: string; + readonly dofsFilesystem: string; + readonly xmdTableDdl: string; + readonly metadataTable: string; + readonly filesystemInsideTransaction: string; +} + +/** Run `body`, reporting what it answered or how it refused, never throwing. */ +function attempt(body: () => unknown): string { + try { + const value = body(); + return `ok:${JSON.stringify(value ?? null)}`; + } catch (error) { + return `refused:${error instanceof Error ? error.message : String(error)}`; + } +} + +export class StorageProbeObject extends DurableObject { + capabilities(): StorageCapabilities { + const sql = this.ctx.storage.sql; + const dofs = new DofsDatabase(dofsStorage(this.ctx.storage)); + return { + applicationIdWrite: attempt(() => { + sql.exec("PRAGMA application_id = 1701078349"); + return "written"; + }), + applicationIdRead: attempt(() => sql.exec("PRAGMA application_id").toArray()), + userVersionWrite: attempt(() => { + sql.exec("PRAGMA user_version = 1"); + return "written"; + }), + userVersionRead: attempt(() => sql.exec("PRAGMA user_version").toArray()), + schemaObjects: attempt(() => + sql.exec("SELECT type, name FROM sqlite_schema WHERE name NOT LIKE 'sqlite_%'").toArray(), + ), + outerTransaction: attempt(() => { + dofs.transactionSync(() => { + sql.exec("CREATE TABLE IF NOT EXISTS probe_outer (id INTEGER PRIMARY KEY)"); + }); + return "committed"; + }), + // The one the documentation forbids: a reentrant transactionSync issues + // SAVEPOINT through sql.exec while the outer transaction is open. + nestedTransaction: attempt(() => { + dofs.transactionSync(() => { + dofs.transactionSync(() => { + sql.exec("CREATE TABLE IF NOT EXISTS probe_nested (id INTEGER PRIMARY KEY)"); + }); + }); + return "committed"; + }), + savepointDirect: attempt(() => { + sql.exec("SAVEPOINT probe_sp"); + sql.exec("RELEASE probe_sp"); + return "accepted"; + }), + // Does the vendored DOFS install its own schema against real storage, + // and does doing so nest a transaction on the way? + dofsSchema: attempt(() => { + initializeDofsSchema(dofs, () => 0); + return "initialized"; + }), + // And does its filesystem work afterwards — the operation the owner would + // perform for every Workspace mutation. + dofsFilesystem: attempt(() => { + mkdirPath(dofs, "/probe", { recursive: true }, () => 0); + // The probe measures what this exact synchronous primitive does on real + // storage, so the asynchronous alternative would answer a different + // question than the one being asked. + // oxlint-disable-next-line local/no-sync-filesystem + writeFileSync(dofs, "/probe/one.txt", new TextEncoder().encode("hello"), {}, () => 0); + return "written"; + }), + // An ordinary XMD table, to show plain DDL is not what is refused. + xmdTableDdl: attempt(() => { + sql.exec("CREATE TABLE IF NOT EXISTS workflow_run (run_id TEXT PRIMARY KEY NOT NULL)"); + sql.exec("INSERT OR REPLACE INTO workflow_run (run_id) VALUES (?)", "probe"); + return sql.exec("SELECT run_id FROM workflow_run").toArray(); + }), + // The exact shape §4 mandates for an owner commit: DOFS filesystem work + // inside one `transactionSync`. If DOFS opens a transaction of its own on + // that path it becomes a reentrant call, which the runtime refuses. + filesystemInsideTransaction: attempt(() => { + dofs.transactionSync(() => { + mkdirPath(dofs, "/inside", { recursive: true }, () => 0); + // oxlint-disable-next-line local/no-sync-filesystem + writeFileSync( + dofs, + "/inside/two.txt", + new TextEncoder().encode("committed"), + {}, + () => 0, + ); + }); + return "committed"; + }), + // The shape a replacement for the pragmas would have to take. + metadataTable: attempt(() => { + sql.exec( + "CREATE TABLE IF NOT EXISTS xmd_schema (key TEXT PRIMARY KEY NOT NULL, value INTEGER NOT NULL)", + ); + sql.exec( + "INSERT OR REPLACE INTO xmd_schema (key, value) VALUES ('application_id', ?)", + 1701078349, + ); + return sql.exec("SELECT key, value FROM xmd_schema").toArray(); + }), + }; + } +} diff --git a/packages/workflow/tests/cloudflare/support/tokens.ts b/packages/workflow/tests/cloudflare/support/tokens.ts new file mode 100644 index 000000000..7436edc99 --- /dev/null +++ b/packages/workflow/tests/cloudflare/support/tokens.ts @@ -0,0 +1,68 @@ +/** + * Signing tokens for the admission tests, with keys generated here. + * + * Real signatures against a real key pair, so the assertions are about + * verification rather than about a stub that agreed to say yes. The key never + * leaves this process and is generated per run. + */ + +/** One generated key pair, and the JWK a verifier is configured with. */ +export interface TestKeys { + readonly signing: CryptoKey; + readonly publicJwk: JsonWebKey; + readonly kid: string; +} + +function base64url(bytes: Uint8Array): string { + let binary = ""; + for (const byte of bytes) { + binary += String.fromCharCode(byte); + } + return btoa(binary).replaceAll("+", "-").replaceAll("/", "_").replaceAll("=", ""); +} + +function encodeSegment(value: unknown): string { + return base64url(new TextEncoder().encode(JSON.stringify(value))); +} + +export async function generateKeys(kid = "test-key"): Promise { + const generated = await crypto.subtle.generateKey( + { + name: "RSASSA-PKCS1-v1_5", + modulusLength: 2048, + publicExponent: new Uint8Array([1, 0, 1]), + hash: "SHA-256", + }, + true, + ["sign", "verify"], + ); + // `generateKey` is typed as either a key or a pair; an RSA signing algorithm + // always answers with a pair, and reading it as one is what proves that here. + if (!("privateKey" in generated) || !("publicKey" in generated)) { + throw new Error("expected an RSA key pair"); + } + const exported = await crypto.subtle.exportKey("jwk", generated.publicKey); + if (exported instanceof ArrayBuffer) { + throw new Error("expected a JWK export"); + } + return { signing: generated.privateKey, publicJwk: exported, kid }; +} + +/** Sign one compact JWS over `claims`. */ +export async function signToken( + keys: TestKeys, + claims: Record, + header: Record = {}, +): Promise { + const encodedHeader = encodeSegment({ alg: "RS256", typ: "JWT", kid: keys.kid, ...header }); + const encodedPayload = encodeSegment(claims); + const signed = new TextEncoder().encode(`${encodedHeader}.${encodedPayload}`); + const signature = await crypto.subtle.sign("RSASSA-PKCS1-v1_5", keys.signing, signed); + return `${encodedHeader}.${encodedPayload}.${base64url(new Uint8Array(signature))}`; +} + +/** A token whose payload was edited after it was signed. */ +export function tamper(token: string, claims: Record): string { + const parts = token.split("."); + return `${parts[0]}.${encodeSegment(claims)}.${parts[2]}`; +} diff --git a/packages/workflow/tests/cloudflare/support/worker-files.ts b/packages/workflow/tests/cloudflare/support/worker-files.ts new file mode 100644 index 000000000..c694c34fa --- /dev/null +++ b/packages/workflow/tests/cloudflare/support/worker-files.ts @@ -0,0 +1,293 @@ +/** + * A filesystem for the runner half, inside the worker. + * + * The owner in these tests is real: a real Durable Object, real SQLite, a real + * accepted WebSocket, and the production client and coordinator on the other + * end of it. The runner's *host filesystem* cannot be. workerd has no native + * filesystem, and the vendored DOFS cannot set a modification time — so it + * cannot reproduce a retained mtime, which is exactly what materialization + * refuses a host for. + * + * So this stands in for the one thing the runtime cannot provide, and nothing + * else. It is not a model of the owner, and it is not a model of the + * coordinator: it stores modes, whole-second times, symbolic links and hardlink + * identity the way a filesystem does, and the production + * `materializeWorkspaceRoot`, `captureWorkspace` and coordinator run against it + * unchanged. The native adapter this stands in for is proved against real files + * in `packages/workflow/tests/remote-workspace-files.test.ts`. + */ + +import { type Operation } from "effection"; +import type { RunnerFiles, RunnerNode } from "../../../src/remote/materialize.ts"; +import type { TemporaryTrees } from "../../../src/remote/invocation.ts"; +import type { + WorkspaceEntry, + WorkspaceFilesystem, + WorkspaceStat, +} from "../../../src/workspace/filesystem.ts"; + +/** One file's bytes, shared by every path hardlinked to it. */ +interface Content { + bytes: Uint8Array; + readonly identity: string; +} + +interface Node { + kind: "directory" | "file" | "symlink"; + mode: number; + mtime: number; + content?: Content; + target?: string; +} + +function failure(code: string): Error { + const error = new Error(`the Workspace operation failed (${code})`); + error.name = "WorkspaceFsError"; + Reflect.set(error, "code", code); + return error; +} + +function parentOf(path: string): string { + const at = path.lastIndexOf("/"); + return at <= 0 ? "/" : path.slice(0, at); +} + +/** One tree, addressed by absolute path. */ +export function createWorkerFiles(): { + files: RunnerFiles; + trees: TemporaryTrees; + workspace(root: string): WorkspaceFilesystem; +} { + // The tree's own root exists from the start, the way a filesystem's does. + const nodes = new Map([["/", { kind: "directory", mode: 0o755, mtime: 0 }]]); + let identities = 0; + let roots = 0; + let clock = 1_700_000_000; + + function node(path: string): Node { + const found = nodes.get(path); + if (found === undefined) { + throw failure("ENOENT"); + } + return found; + } + + function requireParent(path: string): void { + const parent = nodes.get(parentOf(path)); + if (parent === undefined || parent.kind !== "directory") { + throw failure("ENOENT"); + } + } + + function children(path: string): string[] { + const prefix = path === "/" ? "/" : `${path}/`; + return [...nodes.keys()].filter( + (candidate) => + candidate !== path && + candidate.startsWith(prefix) && + !candidate.slice(prefix.length).includes("/"), + ); + } + + function describe(path: string, name: string): RunnerNode { + const held = node(path); + return { + name, + kind: held.kind, + mode: held.mode, + mtime: held.mtime, + size: held.content?.bytes.length ?? held.target?.length ?? 0, + identity: held.content?.identity, + target: held.target, + }; + } + + function nameOf(path: string): string { + return path.slice(path.lastIndexOf("/") + 1); + } + + const files: RunnerFiles = { + // deno-lint-ignore require-yield + *makeDirectory(path, mode): Operation { + if (nodes.has(path)) { + throw failure("EEXIST"); + } + if (path !== "/") { + requireParent(path); + } + nodes.set(path, { kind: "directory", mode, mtime: (clock += 1) }); + }, + + // deno-lint-ignore require-yield + *writeFile(path, bytes, mode): Operation { + requireParent(path); + identities += 1; + nodes.set(path, { + kind: "file", + mode, + mtime: (clock += 1), + content: { bytes: new Uint8Array(bytes), identity: `content-${identities}` }, + }); + }, + + // deno-lint-ignore require-yield + *makeSymlink(target, path): Operation { + requireParent(path); + nodes.set(path, { kind: "symlink", mode: 0o777, mtime: (clock += 1), target }); + }, + + // deno-lint-ignore require-yield + *makeHardlink(existing, path): Operation { + requireParent(path); + const source = node(existing); + if (source.content === undefined) { + throw failure("EPERM"); + } + // The same content, so both paths are one file and capture sees it. + nodes.set(path, { + kind: "file", + mode: source.mode, + mtime: source.mtime, + content: source.content, + }); + }, + + // deno-lint-ignore require-yield + *setMode(path, mode): Operation { + node(path).mode = mode; + }, + + // deno-lint-ignore require-yield + *setModifiedAt(path, mtime): Operation { + node(path).mtime = mtime; + }, + + setLinkModifiedAt: function* (path, mtime): Operation { + node(path).mtime = mtime; + }, + + setLinkMode: function* (path, mode): Operation { + node(path).mode = mode; + }, + + // deno-lint-ignore require-yield + *readFile(path): Operation { + const held = node(path); + if (held.content === undefined) { + throw failure("EISDIR"); + } + return new Uint8Array(held.content.bytes); + }, + + // deno-lint-ignore require-yield + *list(path): Operation { + const held = node(path); + if (held.kind !== "directory") { + throw failure("ENOTDIR"); + } + return children(path).map((child) => describe(child, nameOf(child))); + }, + + // deno-lint-ignore require-yield + *describe(path): Operation { + return describe(path, nameOf(path)); + }, + }; + + const trees: TemporaryTrees = { + *create(purpose): Operation { + roots += 1; + const root = `/${purpose}-${roots}`; + yield* files.makeDirectory(root, 0o755); + return root; + }, + + // deno-lint-ignore require-yield + *remove(path): Operation { + for (const candidate of [...nodes.keys()]) { + if (candidate === path || candidate.startsWith(`${path}/`)) { + nodes.delete(candidate); + } + } + }, + }; + + /** + * The Workspace filesystem over one tree in it. + * + * Containment is the native adapter's subject and is proved there; what this + * needs to be is a filesystem the coordinator can really change, so a commit + * carries bytes a document actually wrote. + */ + function workspace(root: string): WorkspaceFilesystem { + const at = (logical: string) => (logical === "/" ? root : `${root}${logical}`); + function stat(path: string): WorkspaceStat { + const held = node(path); + return { + kind: held.kind, + mode: held.mode, + mtime: held.mtime, + size: held.content?.bytes.length ?? 0, + }; + } + return { + *readFile(path): Operation { + return yield* files.readFile(at(path)); + }, + *readTextFile(path): Operation { + return new TextDecoder().decode(yield* files.readFile(at(path))); + }, + // deno-lint-ignore require-yield + *stat(path): Operation { + return stat(at(path)); + }, + // deno-lint-ignore require-yield + *lstat(path): Operation { + return stat(at(path)); + }, + // deno-lint-ignore require-yield + *readlink(path): Operation { + const target = node(at(path)).target; + if (target === undefined) { + throw failure("EINVAL"); + } + return target; + }, + // deno-lint-ignore require-yield + *readdir(path): Operation { + return children(at(path)).map((child) => ({ + name: nameOf(child), + kind: node(child).kind, + })); + }, + *writeFile(path, content, mode): Operation { + const bytes = typeof content === "string" ? new TextEncoder().encode(content) : content; + yield* files.writeFile(at(path), bytes, mode ?? 0o644); + }, + *mkdir(path, options = {}): Operation { + yield* files.makeDirectory(at(path), options.mode ?? 0o755); + }, + *remove(path): Operation { + yield* trees.remove(at(path)); + }, + // deno-lint-ignore require-yield + *rename(from, to): Operation { + const held = node(at(from)); + nodes.delete(at(from)); + nodes.set(at(to), held); + }, + // deno-lint-ignore require-yield + *chmod(path, mode): Operation { + node(at(path)).mode = mode; + }, + *symlink(target, path): Operation { + yield* files.makeSymlink(target, at(path)); + }, + *link(existing, path): Operation { + yield* files.makeHardlink(at(existing), at(path)); + }, + }; + } + + return { files, trees, workspace }; +} diff --git a/packages/workflow/tests/cloudflare/worker.ts b/packages/workflow/tests/cloudflare/worker.ts new file mode 100644 index 000000000..1da99ee72 --- /dev/null +++ b/packages/workflow/tests/cloudflare/worker.ts @@ -0,0 +1,17 @@ +/** + * The Worker the workerd suite runs against. + * + * It exists to publish the Durable Object classes under test and nothing else: + * the tests reach those objects through `runInDurableObject()` and their own + * stubs, so this handler answers no request a test depends on. + */ + +export { StorageProbeObject } from "./support/probe-object.ts"; +export { OwnerObject } from "./support/owner-object.ts"; +export { ExecutorObject } from "./support/executor-object.ts"; + +export default { + fetch(): Response { + return new Response("workflow owner test worker", { status: 200 }); + }, +}; diff --git a/packages/workflow/tests/cloudflare/wrangler.jsonc b/packages/workflow/tests/cloudflare/wrangler.jsonc new file mode 100644 index 000000000..9e7f7877d --- /dev/null +++ b/packages/workflow/tests/cloudflare/wrangler.jsonc @@ -0,0 +1,16 @@ +{ + "name": "workflow-owner-tests", + "main": "worker.ts", + "compatibility_date": "2026-08-01", + "compatibility_flags": ["nodejs_compat"], + "durable_objects": { + "bindings": [ + { "name": "STORAGE_PROBE", "class_name": "StorageProbeObject" }, + { "name": "OWNER", "class_name": "OwnerObject" }, + { "name": "EXECUTOR", "class_name": "ExecutorObject" }, + ], + }, + "migrations": [ + { "tag": "v1", "new_sqlite_classes": ["StorageProbeObject", "OwnerObject", "ExecutorObject"] }, + ], +} diff --git a/packages/workflow/tests/host-neutrality.test.ts b/packages/workflow/tests/host-neutrality.test.ts new file mode 100644 index 000000000..bc10781ba --- /dev/null +++ b/packages/workflow/tests/host-neutrality.test.ts @@ -0,0 +1,128 @@ +/** + * Tier WRH — what the shared package may know about a host. + * + * `@executablemd/workflow` names no provider. That claim is what lets a second + * host implement the same lifecycle without the Deno entrypoint being loaded at + * all, and it is worth exactly as much as the imports underneath it: one + * `node:sqlite` or `cloudflare:` specifier in a shared module, or one + * `typeof Deno` test, and every module that resolves through it inherits a host. + * + * So this reads the source rather than describing it. It walks the modules a + * consumer reaches through the package root and fails on anything that names a + * runtime — the runtime-named entrypoints and their own subtrees excepted, + * because installing host behavior is what those are for. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { readTextFile, walk } from "@effectionx/fs"; +import { each } from "effection"; +import type { Operation } from "effection"; +import { join, relative } from "node:path"; +import { fileURLToPath } from "node:url"; + +const PACKAGE = fileURLToPath(new URL("..", import.meta.url)); + +/** + * The subtrees that are allowed to know a host, because naming one is their job. + * + * The runtime-named entrypoints and their implementation subtrees, and nothing + * else. `software-factory.ts` is deliberately absent: it is product-specific + * rather than host-specific, uses the cross-runtime Web primitives, and is held + * to these rules like any shared module. `vendor` is pinned upstream source + * whose drift verifier owns its bytes. + */ +const RUNTIME_OWNED = [ + "deno.ts", + "cloudflare.ts", + "src/deno", + "src/cloudflare", + "tests/cloudflare", + "vitest.config.ts", + "vendor", +]; + +/** Specifiers only a host adapter may import. */ +const HOST_SPECIFIERS = [ + "node:sqlite", + "node:fs", + "node:os", + "node:child_process", + "cloudflare:workers", + "cloudflare:test", + "@cloudflare/", +]; + +/** Ways a module could ask which runtime it is running under. */ +const RUNTIME_DETECTION = [ + /\btypeof\s+Deno\b/, + /\btypeof\s+Bun\b/, + /\bnavigator\s*\.\s*userAgent\b/, + /\bprocess\s*\.\s*versions\s*\.\s*bun\b/, + /\bglobalThis\s*\.\s*Deno\b/, + /\bglobalThis\s*\.\s*Bun\b/, +]; + +function* sharedModules(): Operation { + const owned = RUNTIME_OWNED.map((entry) => join(PACKAGE, entry)); + const found: string[] = []; + for (const entry of yield* each(walk(PACKAGE, { includeDirs: false }))) { + const path = entry.path; + const exempt = owned.some((root) => path === root || path.startsWith(`${root}/`)); + const generated = ["/node_modules/", "/tests/", "/npm/"].some((part) => path.includes(part)); + if (!exempt && !generated && path.endsWith(".ts") && !path.endsWith(".d.ts")) { + found.push(path); + } + yield* each.next(); + } + return found.toSorted(); +} + +function* offenders(check: (source: string) => boolean): Operation { + const named: string[] = []; + for (const path of yield* sharedModules()) { + const source = yield* readTextFile(path); + if (check(source)) { + named.push(relative(PACKAGE, path)); + } + } + return named; +} + +describe("the shared workflow package", () => { + it("finds the modules it is making a claim about", function* () { + const modules = yield* sharedModules(); + expect(modules.length > 20).toEqual(true); + expect(modules.some((path) => path.endsWith("/src/lifecycle/execution.ts"))).toEqual(true); + expect(modules.some((path) => path.endsWith("/src/software-factory/run-id.ts"))).toEqual(true); + expect(modules.some((path) => path.endsWith("/src/sqlite/workflow-schema.ts"))).toEqual(true); + // The remote seam is ordinary shared code. It is the runner's half of a + // connection to a provider, which is exactly why it must name none: an + // exemption here would let the provider's vocabulary back in through the + // one module whose whole purpose is to keep it out. + expect(modules.some((path) => path.endsWith("/src/remote/read.ts"))).toEqual(true); + expect(modules.some((path) => path.endsWith("/src/remote/client.ts"))).toEqual(true); + expect(modules.some((path) => path.endsWith("/src/remote/records.ts"))).toEqual(true); + expect(modules.some((path) => path.endsWith("/src/workspace/root-manifest.ts"))).toEqual(true); + expect(modules.some((path) => path.endsWith("/src/workspace/sha256.ts"))).toEqual(true); + expect(modules.some((path) => path.includes("/src/deno/"))).toEqual(false); + expect(modules.some((path) => path.includes("/src/cloudflare/"))).toEqual(false); + }); + + it("imports no host-owned specifier outside a runtime-named entrypoint", function* () { + const named = yield* offenders((source) => + HOST_SPECIFIERS.some( + (specifier) => + source.includes(`from "${specifier}`) || source.includes(`import("${specifier}`), + ), + ); + expect(named).toEqual([]); + }); + + it("asks no module which runtime it is running under", function* () { + const named = yield* offenders((source) => + RUNTIME_DETECTION.some((pattern) => pattern.test(source)), + ); + expect(named).toEqual([]); + }); +}); diff --git a/packages/workflow/tests/public-entrypoint.test.ts b/packages/workflow/tests/public-entrypoint.test.ts index 4095eaa82..b4d0620a8 100644 --- a/packages/workflow/tests/public-entrypoint.test.ts +++ b/packages/workflow/tests/public-entrypoint.test.ts @@ -22,6 +22,7 @@ import { describe, it } from "@executablemd/test-support/bdd"; import { expect } from "@executablemd/test-support/expect"; import { until } from "effection"; +import { readTextFile } from "@effectionx/fs"; import { spawnSync } from "node:child_process"; import process from "node:process"; import { fileURLToPath } from "node:url"; @@ -52,6 +53,7 @@ const PROBE = fileURLToPath(new URL("./support/public-entrypoint-probe.ts", impo const HELPER_MODULE = fileURLToPath( new URL("./support/credential-helper-entry.ts", import.meta.url), ); +const CLOUDFLARE_ENTRYPOINT = fileURLToPath(new URL("../cloudflare.ts", import.meta.url)); describe("workflow published Deno entrypoint", () => { it("offers no route from the entrypoint to an authenticated invocation", function* () { @@ -153,10 +155,25 @@ describe("workflow published Deno entrypoint", () => { "useGitComposition", "denoGitAuthentication", "denoCredentialBroker", + "RemoteReadLink", + "cloudflareReadLink", + "stageCloudflareContent", ]) { expect(reachable).not.toContain(seam); } expect(COMPOSITION_IS_NOT_A_KEY).toBe(false); expect(yield* until(Promise.resolve(true))).toBe(true); }); + + it("keeps the Cloudflare private protocol out of its host entrypoint", function* () { + const source = yield* readTextFile(CLOUDFLARE_ENTRYPOINT); + for (const privateModule of [ + "commands.ts", + "acquisition.ts", + "dispatcher.ts", + "private-schema.ts", + ]) { + expect(source).not.toContain(privateModule); + } + }); }); diff --git a/packages/workflow/tests/remote-client.test.ts b/packages/workflow/tests/remote-client.test.ts new file mode 100644 index 000000000..6dcd3fb5f --- /dev/null +++ b/packages/workflow/tests/remote-client.test.ts @@ -0,0 +1,641 @@ +/** + * Tier WRH — carrying a request to a run's owner. + * + * Correlation and teardown are what this is about. A socket delivers what the + * owner sent whenever it sent it, so answers are matched by the id they name + * rather than by arrival order; and a connection that ends must fail the + * requests still waiting rather than leave a caller blocked on an answer that + * can never come. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { scoped, sleep, spawn } from "effection"; +import { + type OwnerSocket, + OwnerLinkError, + type SocketListener, + MAX_MESSAGE_BYTES, + useOwnerConnection, +} from "../src/remote/client.ts"; + +/** These tests are about correlation, so most of them read any value. */ +function readString(value: unknown): unknown { + return value; +} + +/** A parser that refuses anything but a string, so a bad value fails the link. */ +function requireString(value: unknown): string { + if (typeof value !== "string") { + throw new Error("expected a string"); + } + return value; +} + +/** + * What the connection refused with, having proved it refused at all. + * + * A caught value is `unknown`, and asserting it into `OwnerLinkError` would let + * an unrelated failure read as the transport category a test expected. + */ +function refusalOf(error: unknown): string { + if (!(error instanceof OwnerLinkError)) { + throw new Error(`expected an OwnerLinkError, got ${String(error)}`); + } + return error.refusal; +} + +/** + * A socket a test drives by hand, and can ask what happened to it. + * + * It counts closes and tracks the listeners still installed, because the claims + * under test are about teardown: that the connection closes its socket exactly + * once and stops listening. A fake that merely retained its callbacks would let + * a test assert cleanup that never happened — which is how the previous version + * of this suite passed while the connection leaked both. + */ +function fakeSocket(options: { failSend?: boolean } = {}) { + const sent: Record[] = []; + const listeners = new Map>(); + let closes = 0; + + const deliver = (type: string, event: { data?: unknown }) => { + for (const listener of listeners.get(type) ?? []) { + listener(event); + } + }; + + const socket: OwnerSocket = { + send(data: string): void { + if (options.failSend === true) { + throw new Error("the socket refused the write"); + } + sent.push(JSON.parse(data)); + }, + close(): void { + closes += 1; + }, + addEventListener(type, listener): void { + const existing = listeners.get(type) ?? new Set(); + existing.add(listener); + listeners.set(type, existing); + }, + removeEventListener(type, listener): void { + listeners.get(type)?.delete(listener); + }, + }; + + return { + socket, + sent, + get closes(): number { + return closes; + }, + /** How many listeners are still installed, of any type. */ + get listening(): number { + return [...listeners.values()].reduce((total, set) => total + set.size, 0); + }, + answer(value: unknown): void { + deliver("message", { + data: typeof value === "string" ? value : JSON.stringify(value), + }); + }, + end(): void { + deliver("close", {}); + }, + error(): void { + deliver("error", {}); + }, + }; +} + +describe("a connection to a run's owner", () => { + it("sends the command with its id and answers the caller that asked", function* () { + const wire = fakeSocket(); + yield* scoped(function* () { + const owner = yield* useOwnerConnection(wire.socket); + yield* sleep(0); + const asking = yield* spawn(() => owner.ask("a1", { command: "frontier" }, readString)); + yield* sleep(0); + // The request is on the wire before any answer exists. + expect(wire.sent).toEqual([{ command: "frontier", id: "a1" }]); + wire.answer({ id: "a1", outcome: "performed", value: { root: "root-a" } }); + expect(yield* asking).toEqual({ outcome: "performed", value: { root: "root-a" } }); + }); + yield* sleep(0); + }); + + it("matches answers by the id they name, not by arrival order", function* () { + const wire = fakeSocket(); + yield* scoped(function* () { + const owner = yield* useOwnerConnection(wire.socket); + yield* sleep(0); + const first = yield* spawn(() => owner.ask("a1", { command: "frontier" }, readString)); + const second = yield* spawn(() => owner.ask("a2", { command: "settle" }, readString)); + yield* sleep(0); + // Both requests are on the wire before either is answered. + expect(wire.sent.map((request) => request.id)).toEqual(["a1", "a2"]); + + // Answered in the opposite order to the asking. + wire.answer({ id: "a2", outcome: "performed", value: "second" }); + wire.answer({ id: "a1", outcome: "performed", value: "first" }); + + expect(yield* first).toEqual({ outcome: "performed", value: "first" }); + expect(yield* second).toEqual({ outcome: "performed", value: "second" }); + }); + yield* sleep(0); + }); + + it("hands back a refusal as an answer rather than a transport failure", function* () { + const wire = fakeSocket(); + yield* scoped(function* () { + const owner = yield* useOwnerConnection(wire.socket); + yield* sleep(0); + const asking = yield* spawn(() => owner.ask("a1", { command: "commit" }, readString)); + yield* sleep(0); + wire.answer({ id: "a1", outcome: "refused", refusal: "acquisition:already-running" }); + expect(yield* asking).toEqual({ + outcome: "refused", + refusal: "acquisition:already-running", + }); + }); + yield* sleep(0); + }); + + it("fails a request still waiting when the connection ends", function* () { + const wire = fakeSocket(); + let raised: unknown; + yield* scoped(function* () { + const owner = yield* useOwnerConnection(wire.socket); + yield* sleep(0); + const asking = yield* spawn(function* () { + try { + yield* owner.ask("a1", { command: "frontier" }, readString); + } catch (error) { + raised = error; + } + }); + wire.end(); + yield* asking; + }); + yield* sleep(0); + expect(raised).toBeInstanceOf(OwnerLinkError); + expect(refusalOf(raised)).toBe("closed"); + }); + + it("refuses to ask through a connection that already ended", function* () { + const wire = fakeSocket(); + let raised: unknown; + yield* scoped(function* () { + const owner = yield* useOwnerConnection(wire.socket); + yield* sleep(0); + wire.end(); + try { + yield* owner.ask("a1", { command: "frontier" }, readString); + } catch (error) { + raised = error; + } + }); + expect(refusalOf(raised)).toBe("closed"); + }); + + it("refuses a second request under an id already in flight", function* () { + const wire = fakeSocket(); + let raised: unknown; + yield* scoped(function* () { + const owner = yield* useOwnerConnection(wire.socket); + yield* sleep(0); + yield* spawn(() => owner.ask("a1", { command: "frontier" }, readString)); + yield* sleep(0); + try { + yield* owner.ask("a1", { command: "settle" }, readString); + } catch (error) { + raised = error; + } + wire.answer({ id: "a1", outcome: "performed", value: null }); + }); + expect(refusalOf(raised)).toBe("duplicate-answer"); + }); + + it("fails every waiter when it cannot read an answer", function* () { + const wire = fakeSocket(); + const raised: unknown[] = []; + yield* scoped(function* () { + const owner = yield* useOwnerConnection(wire.socket); + yield* sleep(0); + const first = yield* spawn(function* () { + try { + yield* owner.ask("a1", { command: "frontier" }, readString); + } catch (error) { + raised.push(error); + } + }); + const second = yield* spawn(function* () { + try { + yield* owner.ask("a2", { command: "settle" }, readString); + } catch (error) { + raised.push(error); + } + }); + yield* sleep(0); + // A commit may already have landed on the owner. Dropping this and + // leaving both callers waiting is the failure mode being refused. + wire.answer("not json at all"); + yield* first; + yield* second; + }); + expect(raised).toHaveLength(2); + for (const error of raised) { + expect(refusalOf(error)).toBe("malformed-answer"); + } + expect(wire.closes).toBe(1); + expect(wire.listening).toBe(0); + }); + + it("fails closed on an answer naming a request nobody made", function* () { + const wire = fakeSocket(); + let raised: unknown; + yield* scoped(function* () { + const owner = yield* useOwnerConnection(wire.socket); + yield* sleep(0); + const asking = yield* spawn(function* () { + try { + yield* owner.ask("a1", { command: "frontier" }, readString); + } catch (error) { + raised = error; + } + }); + yield* sleep(0); + wire.answer({ id: "somebody-else", outcome: "performed", value: 1 }); + yield* asking; + }); + expect(refusalOf(raised)).toBe("unknown-answer"); + }); + + it("tells the asker why its own answer was unreadable, and everyone else the channel ended", function* () { + const wire = fakeSocket(); + const raised: unknown[] = []; + yield* scoped(function* () { + const owner = yield* useOwnerConnection(wire.socket); + yield* sleep(0); + const first = yield* spawn(function* () { + try { + yield* owner.ask("a1", { command: "frontier" }, requireString); + } catch (error) { + raised.push(error); + } + }); + const second = yield* spawn(function* () { + try { + yield* owner.ask("a2", { command: "settle" }, requireString); + } catch (error) { + raised.push(error); + } + }); + yield* sleep(0); + // Performed, and the value is not what the command's parser reads. The + // caller must not receive it, and the other waiter must not be left. + wire.answer({ id: "a1", outcome: "performed", value: { not: "a string" } }); + yield* first; + yield* second; + }); + expect(raised).toHaveLength(2); + // The request whose answer failed keeps the parser's own failure: the + // boundary above it can only classify what a value meant if it still holds + // the failure that said so. Reporting an unreachable owner here would be + // untrue — the owner answered, and this build could not read it. + const asker = raised.find((error) => !(error instanceof OwnerLinkError)); + expect(String(asker)).toContain("expected a string"); + // Nothing else is true for the other waiter except that the channel ended. + const other = raised.filter((error) => error !== asker); + expect(other.map(refusalOf)).toEqual(["malformed-answer"]); + expect(wire.closes).toBe(1); + expect(wire.listening).toBe(0); + }); + + it("still delivers a refusal without consulting the success parser", function* () { + const wire = fakeSocket(); + yield* scoped(function* () { + const owner = yield* useOwnerConnection(wire.socket); + yield* sleep(0); + const asking = yield* spawn(() => + owner.ask("a1", { command: "commit" }, () => { + throw new Error("a refusal must not reach this"); + }), + ); + yield* sleep(0); + wire.answer({ id: "a1", outcome: "refused", refusal: "acquisition:already-running" }); + expect(yield* asking).toEqual({ + outcome: "refused", + refusal: "acquisition:already-running", + }); + }); + }); + + it("refuses a refusal that is not a category this side can branch on", function* () { + const wire = fakeSocket(); + let raised: unknown; + yield* scoped(function* () { + const owner = yield* useOwnerConnection(wire.socket); + yield* sleep(0); + const asking = yield* spawn(function* () { + try { + yield* owner.ask("a1", { command: "frontier" }, readString); + } catch (error) { + raised = error; + } + }); + yield* sleep(0); + // An arbitrary remote sentence must not become this side's public failure + // identity, so it is read as an answer this build cannot understand. + wire.answer({ id: "a1", outcome: "refused", refusal: "something went wrong!" }); + yield* asking; + }); + expect(refusalOf(raised)).toBe("malformed-answer"); + }); + + it("refuses an answer whose correlation id is not one", function* () { + const wire = fakeSocket(); + let raised: unknown; + yield* scoped(function* () { + const owner = yield* useOwnerConnection(wire.socket); + yield* sleep(0); + const asking = yield* spawn(function* () { + try { + yield* owner.ask("a1", { command: "frontier" }, readString); + } catch (error) { + raised = error; + } + }); + yield* sleep(0); + wire.answer({ id: "x".repeat(200), outcome: "performed", value: 1 }); + yield* asking; + }); + expect(refusalOf(raised)).toBe("malformed-answer"); + }); + + it("closes the socket once and stops listening when its scope ends", function* () { + const wire = fakeSocket(); + let answered = false; + yield* scoped(function* () { + const owner = yield* useOwnerConnection(wire.socket); + yield* sleep(0); + expect(wire.listening).toBeGreaterThan(0); + yield* spawn(function* () { + yield* owner.ask("a1", { command: "frontier" }, readString); + answered = true; + }); + yield* sleep(0); + // Leaving with a request in flight. The connection is the acquisition, so + // the owner only learns this runner is gone when the socket closes. + }); + + expect(wire.closes).toBe(1); + expect(wire.listening).toBe(0); + expect(answered).toBe(false); + + // A late message and a late close reach nothing and raise nothing. + wire.answer({ id: "a1", outcome: "performed", value: "too late" }); + wire.end(); + expect(answered).toBe(false); + expect(wire.closes).toBe(1); + }); + + it("ends the same way however the connection is lost", function* () { + // Each of these is one teardown with one owner: the waiters learn why, the + // listeners go, and the socket closes exactly once. + const cases: [string, (wire: ReturnType) => void][] = [ + ["closed", (wire) => wire.end()], + ["socket-error", (wire) => wire.error()], + ["malformed-answer", (wire) => wire.answer("not json at all")], + ["unknown-answer", (wire) => wire.answer({ id: "nobody", outcome: "performed", value: 1 })], + ]; + + for (const [expected, provoke] of cases) { + const wire = fakeSocket(); + let raised: unknown; + yield* scoped(function* () { + const owner = yield* useOwnerConnection(wire.socket); + yield* sleep(0); + const asking = yield* spawn(function* () { + try { + yield* owner.ask("a1", { command: "frontier" }, readString); + } catch (error) { + raised = error; + } + }); + yield* sleep(0); + provoke(wire); + yield* asking; + }); + expect(refusalOf(raised)).toBe(expected); + expect(wire.closes).toBe(1); + expect(wire.listening).toBe(0); + } + }); + + it("keeps the failure that caused teardown when a close follows it", function* () { + const wire = fakeSocket(); + let raised: unknown; + yield* scoped(function* () { + const owner = yield* useOwnerConnection(wire.socket); + yield* sleep(0); + const asking = yield* spawn(function* () { + try { + yield* owner.ask("a1", { command: "frontier" }, readString); + } catch (error) { + raised = error; + } + }); + yield* sleep(0); + wire.answer("not json at all"); + // The remote end closes right after. The caller should still learn what + // actually went wrong rather than a generic `closed`. + wire.end(); + yield* asking; + }); + expect(refusalOf(raised)).toBe("malformed-answer"); + expect(wire.closes).toBe(1); + }); + + it("tears down when the socket refuses the write, and sends nothing", function* () { + const wire = fakeSocket({ failSend: true }); + let raised: unknown; + yield* scoped(function* () { + const owner = yield* useOwnerConnection(wire.socket); + yield* sleep(0); + try { + yield* owner.ask("a1", { command: "frontier" }, readString); + } catch (error) { + raised = error; + } + }); + expect(refusalOf(raised)).toBe("send-failed"); + expect(wire.sent).toEqual([]); + expect(wire.closes).toBe(1); + expect(wire.listening).toBe(0); + }); + + it("refuses a request larger than one message, before it is outstanding", function* () { + const wire = fakeSocket(); + let raised: unknown; + let reused: unknown; + yield* scoped(function* () { + const owner = yield* useOwnerConnection(wire.socket); + yield* sleep(0); + try { + // Under the bound on its own; over it once the correlation id and the + // framing around it are counted. Measuring one member instead would + // let exactly this request through. + yield* owner.ask( + "over", + { command: "retrieval", metadata: "m".repeat(MAX_MESSAGE_BYTES - 40) }, + readString, + ); + } catch (error) { + raised = error; + } + // The id never became outstanding, so it is still usable. A request that + // was registered and then refused would fail here as a duplicate. + try { + yield* spawn(function* () { + yield* owner.ask("over", { command: "frontier" }, readString); + }); + yield* sleep(0); + } catch (error) { + reused = error; + } + }); + expect(refusalOf(raised)).toBe("too-large"); + expect(reused).toBe(undefined); + // Exactly one message left: the small one. + expect(wire.sent).toEqual([{ id: "over", command: "frontier" }]); + }); + + it("refuses to send a correlation id it would refuse to read", function* () { + const wire = fakeSocket(); + let raised: unknown; + yield* scoped(function* () { + const owner = yield* useOwnerConnection(wire.socket); + yield* sleep(0); + try { + yield* owner.ask("", { command: "frontier" }, readString); + } catch (error) { + raised = error; + } + expect(refusalOf(raised)).toBe("malformed-request"); + try { + yield* owner.ask("x".repeat(200), { command: "frontier" }, readString); + } catch (error) { + raised = error; + } + }); + expect(refusalOf(raised)).toBe("malformed-request"); + // Nothing left, so nothing to correlate an answer to. + expect(wire.sent).toEqual([]); + }); + + it("refuses an answer whose branch carries a member it does not declare", function* () { + const cases: unknown[] = [ + { id: "a1", outcome: "performed", value: 1, refusal: "acquisition:stale" }, + { id: "a1", outcome: "refused", refusal: "acquisition:stale", value: 1 }, + { id: "a1", outcome: "performed" }, + { id: "a1", outcome: "refused" }, + { id: "a1", outcome: "performed", value: 1, extra: true }, + ]; + for (const answer of cases) { + const wire = fakeSocket(); + let raised: unknown; + yield* scoped(function* () { + const owner = yield* useOwnerConnection(wire.socket); + yield* sleep(0); + const asking = yield* spawn(function* () { + try { + yield* owner.ask("a1", { command: "frontier" }, readString); + } catch (error) { + raised = error; + } + }); + yield* sleep(0); + wire.answer(answer); + yield* asking; + }); + expect(refusalOf(raised)).toBe("malformed-answer"); + } + }); + + it("releases the socket when the scope holding it is cancelled", function* () { + const wire = fakeSocket(); + let raised: unknown; + yield* scoped(function* () { + const holding = yield* spawn(function* () { + const owner = yield* useOwnerConnection(wire.socket); + yield* sleep(0); + try { + yield* owner.ask("a1", { command: "frontier" }, readString); + } catch (error) { + raised = error; + } + }); + yield* sleep(0); + expect(wire.listening).toBeGreaterThan(0); + // Cancellation, rather than the scope reaching its end. The connection is + // the acquisition either way, so the socket must still close. + yield* holding.halt(); + }); + expect(wire.closes).toBe(1); + expect(wire.listening).toBe(0); + // Halting the caller means it is never told anything; the socket closing is + // what the owner observes. + expect(raised).toBe(undefined); + }); + + it("carries a refusal category this build has never heard of", function* () { + const wire = fakeSocket(); + let answered: unknown; + yield* scoped(function* () { + const owner = yield* useOwnerConnection(wire.socket); + yield* sleep(0); + const asking = yield* spawn(() => owner.ask("a1", { command: "commit" }, readString)); + yield* sleep(0); + // Well-spelled and not a category this layer knows. Deciding which + // categories exist belongs to the adapter that declares the union, so the + // connection hands it through rather than guessing on the adapter's + // behalf and failing a run over a word. + wire.answer({ id: "a1", outcome: "refused", refusal: "workspace:root-unknown-here" }); + answered = yield* asking; + }); + expect(answered).toEqual({ outcome: "refused", refusal: "workspace:root-unknown-here" }); + expect(wire.closes).toBe(1); + }); + + it("fails closed on a second answer to a request already settled", function* () { + const wire = fakeSocket(); + let answered: unknown; + let refused: unknown; + yield* scoped(function* () { + const owner = yield* useOwnerConnection(wire.socket); + yield* sleep(0); + const first = yield* spawn(() => owner.ask("a1", { command: "frontier" }, readString)); + yield* sleep(0); + wire.answer({ id: "a1", outcome: "performed", value: "once" }); + answered = yield* first; + + const second = yield* spawn(function* () { + try { + yield* owner.ask("a2", { command: "settle" }, readString); + } catch (error) { + refused = error; + } + }); + yield* sleep(0); + // The owner answers `a1` again. Correlation has broken. + wire.answer({ id: "a1", outcome: "performed", value: "twice" }); + yield* second; + }); + expect(answered).toEqual({ outcome: "performed", value: "once" }); + expect(refusalOf(refused)).toBe("duplicate-answer"); + expect(wire.closes).toBe(1); + expect(wire.listening).toBe(0); + }); +}); diff --git a/packages/workflow/tests/remote-database.test.ts b/packages/workflow/tests/remote-database.test.ts new file mode 100644 index 000000000..18bcef0bf --- /dev/null +++ b/packages/workflow/tests/remote-database.test.ts @@ -0,0 +1,478 @@ +/** + * Tier WRH — one run's storage, owned somewhere else. + * + * The interface is the same one the local host answers, so what is under test + * is conformance rather than mechanism: a snapshot stays a snapshot, a nested + * transaction refuses while unrelated work waits its turn, a closed handle is + * closed, and every read is a fresh anchored one rather than a cache. + * + * The owner is a deterministic fake. What it is standing in for — atomic + * application, authoritative revisions, anchored SQLite ordering — is proved on + * real workerd; what is proved here is the handle's own behaviour, which is + * arithmetic over what the owner said. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import type { DurableEvent, Json } from "@executablemd/durable-streams"; +import { Err, Ok, type Operation, type Result, scoped, sleep, spawn } from "effection"; +import type { WorkflowRunDatabase, WorkflowRunTransaction } from "../src/storage/api.ts"; +import { WorkflowDatabaseClosedError, WorkflowTransactionError } from "../src/storage/errors.ts"; +import type { DefinitionRetrieval, DocumentExecutionRecord } from "../src/storage/record.ts"; +import type { CommitIntent, StartingFrontier } from "../src/remote/collector.ts"; +import type { CommitDecision } from "../src/remote/publication.ts"; +import { + activeWorkspaceRoute, + type RemoteRunLink, + useRemoteRunDatabase, +} from "../src/remote/database.ts"; +import type { RemoteFrontierSnapshot } from "../src/remote/read.ts"; + +const ROOT = "a".repeat(64); +const RUN_ID = "remote-run"; + +function event(name: string): DurableEvent { + return { + type: "yield", + coroutineId: "root", + description: { type: "test", name }, + result: { status: "ok", value: name }, + }; +} + +function frontierOf(entries: { eventId: string; event: DurableEvent }[]): RemoteFrontierSnapshot { + return { + record: { + runId: RUN_ID, + definition: { + version: 1, + kind: "git", + objectFormat: "sha1", + objectId: "0".repeat(40), + rootDocumentPath: "README.md", + }, + base: "main", + props: {}, + status: "running", + createdAt: "2026-09-04T00:00:00.000Z", + updatedAt: "2026-09-04T00:00:00.000Z", + }, + retrieval: undefined, + workspaceRootId: ROOT, + journalEventId: entries.at(-1)?.eventId ?? null, + entries: entries.map((entry) => ({ ...entry, workspaceRootId: ROOT })), + }; +} + +/** An owner that answers from what it has been told, and records what it was asked. */ +function owner( + options: { + retrieval?: (metadata: string | null) => Result; + executions?: () => Result; + commit?: (intent: CommitIntent) => Result; + } = {}, +) { + const retained: { eventId: string; event: DurableEvent }[] = []; + const commits: CommitIntent[] = []; + const retrievals: (string | null)[] = []; + let frontierReads = 0; + + const link: RemoteRunLink = { + *frontier(): Operation { + const snapshot = frontierOf(retained); + return { + workspaceRootId: snapshot.workspaceRootId, + journalEventId: snapshot.journalEventId, + events: snapshot.entries.map((entry) => entry.event), + }; + }, + // deno-lint-ignore require-yield + *frontierSnapshot(): Operation { + frontierReads += 1; + return frontierOf(retained); + }, + // deno-lint-ignore require-yield + *commit(intent: CommitIntent): Operation> { + commits.push(intent); + if (options.commit !== undefined) { + return options.commit(intent); + } + const ids = intent.events.map((_entry, index) => `event-${retained.length + index}`); + for (const [index, offered] of intent.events.entries()) { + retained.push({ eventId: ids[index] ?? "", event: offered }); + } + return Ok({ workspaceRootId: intent.expectedWorkspaceRootId, journalEventIds: ids }); + }, + // deno-lint-ignore require-yield + *replaceRetrieval( + _expected: string, + metadata: string | null, + ): Operation> { + retrievals.push(metadata); + return options.retrieval === undefined + ? Ok( + metadata === null + ? undefined + : { + metadata: JSON.parse(metadata) as Json, + revision: retrievals.filter((entry) => entry !== null).length, + updatedAt: "2026-09-04T00:00:01.000Z", + }, + ) + : options.retrieval(metadata); + }, + // deno-lint-ignore require-yield + *readExecutions(): Operation> { + return options.executions === undefined ? Ok([]) : options.executions(); + }, + }; + + return { + link, + commits, + retrievals, + retained, + get frontierReads(): number { + return frontierReads; + }, + /** An event the owner retained without this handle asking. */ + appendElsewhere(name: string): void { + retained.push({ eventId: `outside-${retained.length}`, event: event(name) }); + }, + }; +} + +function useDatabase(link: RemoteRunLink): Operation { + return useRemoteRunDatabase(link, frontierOf([])); +} + +function ok(result: Result): T { + if (!result.ok) { + throw result.error; + } + return result.value; +} + +describe("a run whose storage is somewhere else", () => { + it("initializes its snapshots from one frontier and does not refresh them", function* () { + const remote = owner(); + yield* scoped(function* () { + const database = yield* useDatabase(remote.link); + expect(database.record.runId).toBe(RUN_ID); + expect(database.retrieval).toBe(undefined); + + remote.appendElsewhere("written by somebody else"); + // A read consults the owner; the handle's own snapshots do not move. + expect(yield* database.journal.readAll()).toHaveLength(1); + expect(database.record.runId).toBe(RUN_ID); + expect(database.retrieval).toBe(undefined); + }); + }); + + it("reads a fresh journal every time and never serves a cache", function* () { + const remote = owner(); + yield* scoped(function* () { + const database = yield* useDatabase(remote.link); + expect(yield* database.journal.readAll()).toEqual([]); + remote.appendElsewhere("later"); + expect(yield* database.journal.readAll()).toHaveLength(1); + const entries = ok(yield* database.readJournalEntries()); + // The entry snapshot carries what the journal alone cannot: the owner's + // identity for the row and the root it was written against. + expect(entries[0]?.eventId).toBe("outside-0"); + expect(entries[0]?.workspaceRootId).toBe(ROOT); + expect(remote.frontierReads).toBe(3); + }); + }); + + it("appends through the one commit path, as a journal-only transaction", function* () { + const remote = owner(); + yield* scoped(function* () { + const database = yield* useDatabase(remote.link); + yield* database.journal.append(event("appended")); + expect(remote.commits).toHaveLength(1); + expect(remote.commits[0]?.publication).toBe(null); + expect(remote.commits[0]?.events).toHaveLength(1); + expect(yield* database.journal.readAll()).toHaveLength(1); + }); + }); + + it("shows a transaction its own writes, and commits them once", function* () { + const remote = owner(); + yield* scoped(function* () { + const database = yield* useDatabase(remote.link); + remote.appendElsewhere("already there"); + const outcome = ok( + yield* database.transact(function* (transaction) { + yield* transaction.journal.append(event("mine")); + // Read-your-writes: the admitted prefix, then this transaction's own. + const seen = yield* transaction.journal.readAll(); + expect(seen).toHaveLength(2); + return "body value"; + }), + ); + expect(outcome).toBe("body value"); + expect(remote.commits).toHaveLength(1); + }); + }); + + it("refuses a nested transaction and any same-handle operation inside a body", function* () { + const remote = owner(); + yield* scoped(function* () { + const database = yield* useDatabase(remote.link); + const refusals: unknown[] = []; + ok( + yield* database.transact(function* () { + const nested = yield* database.transact(function* () { + return "never"; + }); + refusals.push(nested.ok ? undefined : nested.error); + const read = yield* database.readJournalEntries(); + refusals.push(read.ok ? undefined : read.error); + try { + yield* database.journal.append(event("from inside")); + } catch (error) { + refusals.push(error); + } + return "done"; + }), + ); + expect(refusals).toHaveLength(3); + for (const refusal of refusals) { + expect(refusal).toBeInstanceOf(WorkflowTransactionError); + } + // None of them reached the owner. + expect(remote.commits).toHaveLength(1); + }); + }); + + it("lets unrelated work wait its turn rather than refusing it", function* () { + const remote = owner(); + yield* scoped(function* () { + const database = yield* useDatabase(remote.link); + const order: string[] = []; + const holding = yield* spawn(() => + database.transact(function* () { + order.push("transaction started"); + yield* sleep(5); + order.push("transaction finishing"); + return "held"; + }), + ); + yield* sleep(0); + // A different scope, not a descendant of the body. + const waiting = yield* spawn(function* () { + const entries = yield* database.readJournalEntries(); + order.push(entries.ok ? "read succeeded" : "read refused"); + }); + yield* holding; + yield* waiting; + expect(order).toEqual(["transaction started", "transaction finishing", "read succeeded"]); + }); + }); + + it("does not let another handle inherit this one's open transaction", function* () { + const remote = owner(); + yield* scoped(function* () { + const first = yield* useDatabase(remote.link); + const second = yield* useDatabase(remote.link); + const outcome = ok( + yield* first.transact(function* () { + // A transaction on one handle says nothing about another. + const other = yield* second.readJournalEntries(); + return other.ok ? "second read" : "second refused"; + }), + ); + expect(outcome).toBe("second read"); + }); + }); + + it("refuses every member once its scope has ended", function* () { + const remote = owner(); + let database: WorkflowRunDatabase | undefined; + yield* scoped(function* () { + database = yield* useDatabase(remote.link); + }); + if (database === undefined) { + throw new Error("expected a handle"); + } + const closed = database; + const results = [ + yield* closed.readJournalEntries(), + yield* closed.replaceRetrievalMetadata({ where: "later" }), + yield* closed.readDocumentExecutions(), + yield* closed.transact(function* () { + return "never"; + }), + ]; + for (const result of results) { + expect(result.ok).toBe(false); + if (!result.ok) { + expect(result.error).toBeInstanceOf(WorkflowDatabaseClosedError); + } + } + let raised: unknown; + try { + yield* closed.journal.append(event("after close")); + } catch (error) { + raised = error; + } + expect(raised).toBeInstanceOf(WorkflowDatabaseClosedError); + // Nothing reached the owner after the handle closed. + expect(remote.commits).toHaveLength(0); + expect(remote.retrievals).toHaveLength(0); + }); + + it("updates only the handle whose replacement succeeded", function* () { + const remote = owner(); + yield* scoped(function* () { + const first = yield* useDatabase(remote.link); + const second = yield* useDatabase(remote.link); + ok(yield* first.replaceRetrievalMetadata({ locator: "https://example.invalid/x.git" })); + expect(first.retrieval?.revision).toBe(1); + // Another handle's replacement is not this handle's snapshot. + expect(second.retrieval).toBe(undefined); + + // Two calls carrying identical metadata are two replacements. + ok(yield* first.replaceRetrievalMetadata({ locator: "https://example.invalid/x.git" })); + expect(first.retrieval?.revision).toBe(2); + + ok(yield* first.replaceRetrievalMetadata(undefined)); + expect(first.retrieval).toBe(undefined); + expect(remote.retrievals).toEqual([ + '{"locator":"https://example.invalid/x.git"}', + '{"locator":"https://example.invalid/x.git"}', + null, + ]); + }); + }); + + it("canonicalizes metadata before it is sent", function* () { + const remote = owner(); + yield* scoped(function* () { + const database = yield* useDatabase(remote.link); + ok(yield* database.replaceRetrievalMetadata({ b: 1, a: { d: 2, c: 3 } })); + // Sorted keys and no incidental whitespace, so two callers writing the + // same metadata write the same bytes. + expect(remote.retrievals[0]).toBe('{"a":{"c":3,"d":2},"b":1}'); + }); + }); + + it("refuses metadata that is not a JSON value, without asking the owner", function* () { + const remote = owner(); + yield* scoped(function* () { + const database = yield* useDatabase(remote.link); + const offered = { locator: () => "not json" } as unknown as Json; + const refused = yield* database.replaceRetrievalMetadata(offered); + expect(refused.ok).toBe(false); + // Nothing was sent: an inadmissible value is not a request. + expect(remote.retrievals).toHaveLength(0); + expect(database.retrieval).toBe(undefined); + }); + }); + + it("fails closed when the answer describes another replacement", function* () { + const remote = owner({ + retrieval: () => + Ok({ + metadata: { locator: "something else entirely" }, + revision: 1, + updatedAt: "2026-09-04T00:00:01.000Z", + }), + }); + yield* scoped(function* () { + const database = yield* useDatabase(remote.link); + const refused = yield* database.replaceRetrievalMetadata({ locator: "what was asked" }); + expect(refused.ok).toBe(false); + // The snapshot is what it was: an answer about another value installs + // nothing, because it would change where the definition is fetched from. + expect(database.retrieval).toBe(undefined); + }); + }); + + it("leaves its snapshot alone when a replacement is refused", function* () { + const remote = owner({ + retrieval: () => Err(new WorkflowTransactionError("this run has moved")), + }); + yield* scoped(function* () { + const database = yield* useDatabase(remote.link); + const refused = yield* database.replaceRetrievalMetadata({ locator: "x" }); + expect(refused.ok).toBe(false); + expect(database.retrieval).toBe(undefined); + }); + }); + + it("hands a Workspace route only to the exact database and transaction", function* () { + const remote = owner(); + yield* scoped(function* () { + const database = yield* useDatabase(remote.link); + const other = yield* useDatabase(remote.link); + let held: WorkflowRunTransaction | undefined; + ok( + yield* database.transact(function* (transaction) { + held = transaction; + expect(yield* activeWorkspaceRoute(database, transaction)).not.toBe(undefined); + // A foreign database, or a transaction object that is not this one. + expect(yield* activeWorkspaceRoute(other, transaction)).toBe(undefined); + expect(yield* activeWorkspaceRoute(database, { journal: transaction.journal })).toBe( + undefined, + ); + return "done"; + }), + ); + if (held === undefined) { + throw new Error("expected a transaction"); + } + // Outside the body the route is gone, so a retained object reaches nothing. + expect(yield* activeWorkspaceRoute(database, held)).toBe(undefined); + }); + }); + + it("sends no commit when the body fails, and answers with the refusal when the owner does", function* () { + const failing = owner({ commit: () => Err(new WorkflowTransactionError("owner refused")) }); + yield* scoped(function* () { + const database = yield* useDatabase(failing.link); + const refused = yield* database.transact(function* (transaction) { + yield* transaction.journal.append(event("attempted")); + return "never returned"; + }); + expect(refused.ok).toBe(false); + }); + + const raising = owner(); + yield* scoped(function* () { + const database = yield* useDatabase(raising.link); + // A body that raised is a failed transaction, not a raised one: the + // interface answers with a `Result`, and the same condition returns + // `Err` from the local provider. + const failed = yield* database.transact(function* (transaction) { + yield* transaction.journal.append(event("attempted")); + throw new Error("the body failed"); + }); + expect(failed.ok).toBe(false); + if (!failed.ok) { + expect(String(failed.error)).toContain("the body failed"); + } + expect(raising.commits).toHaveLength(0); + // And the handle is still usable afterwards. + expect(ok(yield* database.readJournalEntries())).toEqual([]); + }); + }); + + it("returns the executions the owner assembled, in order", function* () { + const executions: DocumentExecutionRecord[] = [ + { executionId: "one", startedAt: "2026-09-04T00:00:00.000Z" }, + { + executionId: "two", + startedAt: "2026-09-04T00:00:01.000Z", + stoppedAt: "2026-09-04T00:00:02.000Z", + stopStatus: "completed", + }, + ]; + const remote = owner({ executions: () => Ok(executions) }); + yield* scoped(function* () { + const database = yield* useDatabase(remote.link); + expect(ok(yield* database.readDocumentExecutions())).toEqual(executions); + }); + }); +}); diff --git a/packages/workflow/tests/remote-interoperability.test.ts b/packages/workflow/tests/remote-interoperability.test.ts new file mode 100644 index 000000000..6c1e48f9f --- /dev/null +++ b/packages/workflow/tests/remote-interoperability.test.ts @@ -0,0 +1,196 @@ +/** + * Tier WRH — the two capture implementations describe one Workspace. + * + * The local host walks the DOFS tables inside its own SQLite file. The runner + * walks a real directory. Neither walk can be shared, and a root identity is a + * digest of what the walk produced — so if the two ever disagreed, a run would + * change its Workspace by moving between hosts, and every no-op remote effect + * would propose a root the local host had never seen. + * + * Nothing here is produced by the code under test. The fixture is built through + * the authoritative Workspace transaction and captured by the local provider's + * own `capture()`, exactly as a real run retains a root. That root is then + * served over the remote read boundary, materialized by the production runner + * adapter, and captured again by the runner's implementation. The two + * identities have to be the same string. + * + * The tree is the discriminating one: two hardlink groups holding identical + * bytes, two independent files holding identical bytes, an empty file, a + * symbolic link, distinct modes and modification times, and a file large enough + * to cross more than one chunk. + */ + +import type { RemoteInvocationSnapshot } from "../src/remote/records.ts"; +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { DatabaseSync } from "node:sqlite"; +import { type Operation, scoped } from "effection"; +import type { WorkflowRunDatabase } from "../mod.ts"; +import { runnerFiles, useRunnerTrees } from "../src/deno/remote-files.ts"; +import { + type PrivateWorkspaceTransaction, + transactWorkspaceRoots, +} from "../src/deno/workspace/private.ts"; +import { captureWorkspace, materializeWorkspaceRoot } from "../src/remote/materialize.ts"; +import type { RemoteContent, RemoteContentRequest, RemoteReadLink } from "../src/remote/read.ts"; +import { parseWorkspaceRootManifest } from "../src/workspace/root-manifest.ts"; +import { createRun, runPath, useStorageRoot, withStorage } from "./support/storage.ts"; + +function reject(reason: string): never { + throw new Error(reason); +} + +function* transact( + database: WorkflowRunDatabase, + body: (workspace: PrivateWorkspaceTransaction) => Operation, +): Operation { + const result = yield* transactWorkspaceRoots(database, body); + if (!result.ok) { + throw result.error; + } + return result.value; +} + +/** + * The content one retained root closes over, read out of the run's own store. + * + * The owner would read these rows; here the test does, so what crosses the + * remote read boundary is exactly what the local host retained rather than + * anything the runner computed. + */ +function retainedContent(path: string): { + manifests: Map; + blobs: Map; +} { + const database = new DatabaseSync(path, { readOnly: true }); + try { + const manifests = new Map(); + for (const row of database.prepare("SELECT hash, encoded FROM vfs_manifests").all()) { + manifests.set(hex(row["hash"]), bytes(row["encoded"])); + } + const blobs = new Map(); + for (const row of database.prepare("SELECT hash, bytes FROM vfs_blob_bytes").all()) { + blobs.set(hex(row["hash"]), bytes(row["bytes"])); + } + return { manifests, blobs }; + } finally { + database.close(); + } +} + +function bytes(value: unknown): Uint8Array { + if (!(value instanceof Uint8Array)) { + throw new Error("expected retained content to be bytes"); + } + return value; +} + +function hex(value: unknown): string { + return Array.from(bytes(value), (byte) => byte.toString(16).padStart(2, "0")).join(""); +} + +/** The retained root and its content, served the way an owner serves them. */ +function servedBy( + manifest: string, + rootId: string, + content: { manifests: Map; blobs: Map }, +): RemoteReadLink { + return { + // Materialization never asks for this; a stub that answered would say this + // test proved something it did not. + *invocationSnapshot(): Operation { + throw new Error("this read link carries no invocation snapshot"); + }, + // deno-lint-ignore require-yield + *frontier(): Operation { + throw new Error("this owner serves only a root and its content"); + }, + // deno-lint-ignore require-yield + *root(workspaceRootId: string) { + if (workspaceRootId !== rootId) { + throw new Error("asked for a root this owner does not hold"); + } + return parseWorkspaceRootManifest(manifest, reject); + }, + // deno-lint-ignore require-yield + *content(_rootId: string, request: RemoteContentRequest): Operation { + const found = + request.kind === "manifest" + ? content.manifests.get(request.digest) + : content.blobs.get(request.digest); + if (found === undefined) { + throw new Error(`the run retains no ${request.kind} ${request.digest}`); + } + return { kind: request.kind, digest: request.digest, bytes: found }; + }, + }; +} + +/** Everything the format carries, written through the authoritative surface. */ +function* buildWorkspace(workspace: PrivateWorkspaceTransaction): Operation { + const files = workspace.filesystem; + yield* files.mkdir("/docs", { mode: 0o755 }); + yield* files.mkdir("/docs/deep", { mode: 0o700 }); + yield* files.writeFile("/README.md", "a workspace\n", 0o644); + yield* files.writeFile("/empty", new Uint8Array(0), 0o600); + yield* files.writeFile("/docs/guide.md", "# guide\n", 0o644); + // Larger than one chunk, so its manifest names more than one piece. + yield* files.writeFile("/docs/deep/large.bin", new Uint8Array(700 * 1024).fill(7), 0o644); + yield* files.symlink("../README.md", "/docs/link"); + + // Two hardlink groups holding identical bytes: one manifest, two files. + yield* files.writeFile("/shared-a", "shared bytes\n", 0o644); + yield* files.link("/shared-a", "/shared-b"); + yield* files.writeFile("/other-a", "shared bytes\n", 0o644); + yield* files.link("/other-a", "/other-b"); + + // Two independent files holding identical bytes, which stay independent. + yield* files.writeFile("/loose-a", "loose bytes\n", 0o644); + yield* files.writeFile("/loose-b", "loose bytes\n", 0o644); + + // A mode a umask would narrow if a creation mode were trusted. + yield* files.writeFile("/group-writable", "wide\n", 0o666); + yield* files.mkdir("/wide-dir", { mode: 0o777 }); +} + +describe("a root the local host retained", () => { + it("materializes and recaptures to the same identity on the runner", function* () { + const root = yield* useStorageRoot(); + yield* withStorage(root, function* () { + const database = yield* createRun(); + const retained = yield* transact(database, function* (workspace) { + yield* buildWorkspace(workspace); + return yield* workspace.capture({ publish: true }); + }); + + // The fixture is the local provider's own capture, not the runner's. + const entries = parseWorkspaceRootManifest(retained.manifest, reject).entries; + const linked = entries.filter((entry) => entry.kind === "file" && entry.hardlink !== null); + expect(linked).toHaveLength(4); + expect(new Set(linked.map((entry) => (entry.kind === "file" ? entry.hardlink : "")))).toEqual( + new Set(["h0", "h1"]), + ); + expect(entries.some((entry) => entry.kind === "symlink")).toBe(true); + expect(entries.some((entry) => entry.kind === "file" && entry.size === 0)).toBe(true); + + const content = retainedContent(runPath(root, database.record.runId)); + const reads = servedBy(retained.manifest, retained.rootId, content); + + yield* scoped(function* () { + const files = runnerFiles(); + const trees = yield* useRunnerTrees(); + const tree = yield* trees.create("interoperability"); + const at = (logical: string) => (logical === "/" ? tree : `${tree}${logical}`); + + yield* materializeWorkspaceRoot(files, reads, at, retained.rootId, reject); + const recaptured = yield* captureWorkspace(files, at, reject); + + // One Workspace, two implementations, one identity. + expect(recaptured.root.rootId).toBe(retained.rootId); + expect(recaptured.root.manifest).toBe(retained.manifest); + expect([...recaptured.root.manifests]).toEqual([...retained.manifestHashes]); + expect([...recaptured.root.blobs]).toEqual([...retained.blobHashes]); + }); + }); + }); +}); diff --git a/packages/workflow/tests/remote-materialization.test.ts b/packages/workflow/tests/remote-materialization.test.ts new file mode 100644 index 000000000..0ba0b8046 --- /dev/null +++ b/packages/workflow/tests/remote-materialization.test.ts @@ -0,0 +1,244 @@ +/** + * Tier WRH — putting a retained root on a runner and reading it back. + * + * The claim under test is one equality: an untouched materialization captures + * to the exact root it was materialized from. Everything else in the remote + * provider rests on it. If it did not hold, a Workspace operation that changed + * nothing would still propose a new root, every no-op would look like a + * mutation, and the owner could not tell a real change from an artefact of how + * the runner unpacked the tree. + * + * A real temporary filesystem, deliberately. Modes, modification times, an + * empty file, a symbolic link and a hardlink group are properties of a + * filesystem, and a fake that stored them in a map would prove only that the + * map kept what it was given. + */ + +import type { RemoteInvocationSnapshot } from "../src/remote/records.ts"; +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { ensure, type Operation, resource, scoped, until } from "effection"; +import { + chmod, + link, + lutimes, + mkdir, + mkdtemp, + rm, + symlink, + utimes, + writeFile, +} from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import process from "node:process"; +import { runnerFiles, useRunnerTrees } from "../src/deno/remote-files.ts"; +import { + captureWorkspace, + materializeWorkspaceRoot, + type RunnerFiles, +} from "../src/remote/materialize.ts"; +import type { RemoteContent, RemoteContentRequest, RemoteReadLink } from "../src/remote/read.ts"; +import type { WorkspaceRootManifest } from "../src/workspace/root-manifest.ts"; +import { parseWorkspaceRootManifest } from "../src/workspace/root-manifest.ts"; +import { encodeContentManifest } from "../src/workspace/content-manifest.ts"; + +function reject(reason: string): never { + throw new Error(reason); +} + +/** Where one logical Workspace path sits under `root`. */ +function at(root: string): (logical: string) => string { + return (logical) => (logical === "/" ? root : join(root, logical.slice(1))); +} + +/** + * An owner that serves exactly what a capture produced. + * + * It answers from the capture's own manifests and blobs, so what crosses is + * what the runner would have had to send. Nothing here validates: the point is + * that materialization rebuilds the tree, and the validation of pieces is the + * connection's, proved where the connection is. + */ +function servedBy(captured: { + root: { manifest: string; rootId: string }; + contents: ReadonlyMap; + blobs: ReadonlyMap; +}): RemoteReadLink { + return { + // Materialization never asks for this; a stub that answered would say this + // test proved something it did not. + *invocationSnapshot(): Operation { + throw new Error("this read link carries no invocation snapshot"); + }, + // deno-lint-ignore require-yield + *frontier(): Operation { + throw new Error("this owner serves only a root and its content"); + }, + // deno-lint-ignore require-yield + *root(workspaceRootId: string): Operation { + if (workspaceRootId !== captured.root.rootId) { + throw new Error("asked for a root this owner does not hold"); + } + return parseWorkspaceRootManifest(captured.root.manifest, reject); + }, + // deno-lint-ignore require-yield + *content(_rootId: string, request: RemoteContentRequest): Operation { + const bytes = + request.kind === "manifest" + ? captured.contents.get(request.digest)?.manifestBytes + : captured.blobs.get(request.digest); + if (bytes === undefined) { + throw new Error("asked for content this owner does not hold"); + } + return { kind: request.kind, digest: request.digest, bytes }; + }, + }; +} + +/** One tree with every entry kind the format carries. */ +function* buildTree(root: string): Operation { + yield* until(mkdir(join(root, "docs"), { mode: 0o755 })); + yield* until(mkdir(join(root, "docs", "deep"), { mode: 0o700 })); + yield* until(writeFile(join(root, "README.md"), "a workspace\n", { mode: 0o644 })); + yield* until(writeFile(join(root, "empty"), new Uint8Array(0), { mode: 0o600 })); + yield* until(writeFile(join(root, "docs", "guide.md"), "# guide\n", { mode: 0o644 })); + // Larger than one chunk, so the manifest names more than one piece. + yield* until( + writeFile(join(root, "docs", "deep", "large.bin"), new Uint8Array(700 * 1024).fill(7), { + mode: 0o644, + }), + ); + yield* until(symlink("../README.md", join(root, "docs", "link"))); + + // Two hardlink groups holding *identical* bytes. They share one DOFS + // manifest and are still two files, so a materializer that indexed by + // content would link the second group to the first and merge them. + yield* until(writeFile(join(root, "shared-a"), "shared bytes\n", { mode: 0o644 })); + yield* until(link(join(root, "shared-a"), join(root, "shared-b"))); + yield* until(writeFile(join(root, "other-a"), "shared bytes\n", { mode: 0o644 })); + yield* until(link(join(root, "other-a"), join(root, "other-b"))); + + // And two independent files with the same bytes, which must stay two files + // with no hardlink group at all. + yield* until(writeFile(join(root, "loose-a"), "loose bytes\n", { mode: 0o644 })); + yield* until(writeFile(join(root, "loose-b"), "loose bytes\n", { mode: 0o644 })); + + // Modes the usual 0022 umask narrows at creation, set explicitly so the + // retained root genuinely carries them. Materialization then has to restore + // them under the same umask, which is only possible by setting them. + yield* until(writeFile(join(root, "group-writable"), "wide\n")); + yield* until(chmod(join(root, "group-writable"), 0o666)); + yield* until(mkdir(join(root, "wide-dir"))); + yield* until(chmod(join(root, "wide-dir"), 0o777)); + + for (const [path, mtime] of [ + [join(root, "README.md"), 1_700_000_001], + [join(root, "empty"), 1_700_000_002], + [join(root, "docs", "guide.md"), 1_700_000_003], + [join(root, "docs", "deep", "large.bin"), 1_700_000_004], + [join(root, "shared-a"), 1_700_000_005], + [join(root, "other-a"), 1_700_000_009], + [join(root, "loose-a"), 1_700_000_010], + [join(root, "loose-b"), 1_700_000_011], + [join(root, "group-writable"), 1_700_000_012], + [join(root, "wide-dir"), 1_700_000_013], + [join(root, "docs", "deep"), 1_700_000_006], + [join(root, "docs"), 1_700_000_007], + [root, 1_700_000_008], + ] as const) { + yield* until(utimes(path, mtime, mtime)); + } + // The link's own time, well in the past and set without following it. + yield* until(lutimes(join(root, "docs", "link"), 1_600_000_000, 1_600_000_000)); +} + +describe("materializing a retained Workspace root", () => { + it("captures an untouched materialization back to the exact root it came from", function* () { + // A umask that would narrow a created mode, so the restoration has to be + // explicit rather than incidental. + const previous = process.umask(0o022); + const files: RunnerFiles = runnerFiles(); + const trees = yield* useRunnerTrees(); + const source = yield* trees.create("source"); + yield* buildTree(source); + + const captured = yield* captureWorkspace(files, at(source), reject); + const entries = captured.root.entries; + // The tree really does exercise what the format carries. + expect(entries.filter((entry) => entry.kind === "directory")).toHaveLength(4); + expect(entries.filter((entry) => entry.kind === "symlink")).toHaveLength(1); + // Two groups of two, holding identical bytes and still two groups. + const linked = entries.filter((entry) => entry.kind === "file" && entry.hardlink !== null); + expect(linked).toHaveLength(4); + expect(new Set(linked.map((entry) => (entry.kind === "file" ? entry.hardlink : "")))).toEqual( + new Set(["h0", "h1"]), + ); + // And they share one manifest, which is what makes this discriminating: + // a materializer indexing by content would merge them. + expect(new Set(linked.map((entry) => (entry.kind === "file" ? entry.manifest : ""))).size).toBe( + 1, + ); + // Equal bytes did not make the independent pair into a group. + for (const path of ["/loose-a", "/loose-b"]) { + const loose = entries.find((entry) => entry.path === path); + expect(loose?.kind === "file" && loose.hardlink).toBe(null); + } + // The wide modes survived the umask rather than being narrowed by it. + expect(entries.find((entry) => entry.path === "/group-writable")?.mode).toBe(0o666); + expect(entries.find((entry) => entry.path === "/wide-dir")?.mode).toBe(0o777); + expect(entries.find((entry) => entry.path === "/docs/link")?.mtime).toBe(1_600_000_000_000); + expect(entries.some((entry) => entry.kind === "file" && entry.size === 0)).toBe(true); + // 700 KiB is two chunks at the pinned chunk size, so a file crosses the + // transport as more than one piece. + const large = entries.find((entry) => entry.path === "/docs/deep/large.bin"); + if (large?.kind !== "file") { + throw new Error("expected the large file to be captured as a file"); + } + expect(captured.contents.get(large.manifest)?.chunks).toHaveLength(2); + + const destination = yield* trees.create("destination"); + yield* materializeWorkspaceRoot( + files, + servedBy(captured), + at(destination), + captured.root.rootId, + reject, + ); + + const again = yield* captureWorkspace(files, at(destination), reject); + process.umask(previous); + expect(again.root.rootId).toBe(captured.root.rootId); + expect(again.root.manifest).toBe(captured.root.manifest); + expect([...again.root.manifests]).toEqual([...captured.root.manifests]); + expect([...again.root.blobs]).toEqual([...captured.root.blobs]); + }); + + it("encodes a content manifest the way the store stores one", function* () { + // The runner and the owner must name identical bytes identically, and the + // encoding is what decides that. + expect( + new TextDecoder().decode(encodeContentManifest([{ hash: "a".repeat(64), size: 3 }])), + ).toBe(`{"version":1,"chunks":[{"hash":"${"a".repeat(64)}","size":3}]}`); + expect(new TextDecoder().decode(encodeContentManifest([]))).toBe('{"version":1,"chunks":[]}'); + }); + + it("removes the materialization when its scope ends, however it ends", function* () { + const files: RunnerFiles = runnerFiles(); + let path = ""; + yield* scoped(function* () { + const trees = yield* useRunnerTrees(); + path = yield* trees.create("scoped"); + yield* until(writeFile(join(path, "present"), "here\n")); + }); + // The scope that owned it has ended, so the tree is gone rather than left + // behind for a later invocation to find. + let listed: unknown; + try { + listed = yield* files.list(path); + } catch (error) { + listed = error; + } + expect(listed).toBeInstanceOf(Error); + }); +}); diff --git a/packages/workflow/tests/remote-publication.test.ts b/packages/workflow/tests/remote-publication.test.ts new file mode 100644 index 000000000..6b3464153 --- /dev/null +++ b/packages/workflow/tests/remote-publication.test.ts @@ -0,0 +1,978 @@ +/** + * Tier WRH — what the production runner sends, and what it keeps. + * + * The owner's half is proved on real workerd, where atomicity and hibernation + * are real. This is the other half: whether the runner can build the command + * the owner accepts, whether it sends one at all when the work did not finish, + * and whether anything survives on disk that should not. + * + * The connection is a deterministic fake because what crosses it is arithmetic + * over what the transaction decided. The filesystem is real, because a tree + * that was supposed to be removed is not a claim a fake can settle. + */ + +import type { RemoteInvocationSnapshot } from "../src/remote/records.ts"; +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { serializeDurableEvent } from "@executablemd/durable-streams"; +import { ensure, type Operation, scoped, sleep, spawn, until } from "effection"; +import { mkdir, readdir, readFile, writeFile } from "node:fs/promises"; +import { agentSessionKey } from "../src/storage/agent-session.ts"; +import { cloudflareOwnerLink } from "../src/cloudflare/client.ts"; +import { runnerFiles, useRunnerTrees } from "../src/deno/remote-files.ts"; +import { + type CommitIntent, + createTransactionGate, + transactRemotely, +} from "../src/remote/collector.ts"; +import { useAttempt, useMaterialization } from "../src/remote/invocation.ts"; +import { type OwnerSocket, type SocketListener, useOwnerConnection } from "../src/remote/client.ts"; +import type { + RemoteContent, + RemoteContentRequest, + RemoteFrontierSnapshot, + RemoteReadLink, +} from "../src/remote/read.ts"; +import { captureWorkspace, type CapturedWorkspace } from "../src/remote/materialize.ts"; +import { + parseWorkspaceRootManifest, + WORKSPACE_ROOT_DOMAIN, +} from "../src/workspace/root-manifest.ts"; +import { sha256Hex } from "../src/workspace/sha256.ts"; +import { locatorFingerprintOf } from "../src/composition/locator.ts"; +import type { RetainedMapping } from "../src/remote/publication.ts"; + +function reject(reason: string): never { + throw new Error(reason); +} + +function event(name: string) { + return { + type: "yield" as const, + coroutineId: "root", + description: { type: "test", name }, + result: { status: "ok" as const, value: name }, + }; +} + +const LOCATOR = "https://git.example.invalid/octo/app.git"; + +/** The Repository mapping these tests enlist. */ +function repositoryMapping(): RetainedMapping { + return { + kind: "repository", + locator: LOCATOR, + record: { + name: "app", + locatorFingerprint: locatorFingerprintOf(LOCATOR), + requestedBase: null, + creationCommit: "9".repeat(40), + primaryBranch: "main", + objectFormat: "sha1", + checkoutPath: "/docs", + }, + }; +} + +/** A recording connection: every request it was sent, and canned answers. */ +function wire(answer: (request: Record) => Record) { + const sent: Record[] = []; + const listeners = new Map>(); + let deliver = true; + const socket: OwnerSocket = { + send(data: string): void { + const request = JSON.parse(data) as Record; + sent.push(request); + if (!deliver) { + return; + } + const response = answer(request); + for (const listener of listeners.get("message") ?? []) { + listener({ data: JSON.stringify({ id: request["id"], ...response }) }); + } + }, + close(): void {}, + addEventListener(type, listener): void { + const found = listeners.get(type) ?? new Set(); + found.add(listener); + listeners.set(type, found); + }, + removeEventListener(type, listener): void { + listeners.get(type)?.delete(listener); + }, + }; + return { + socket, + sent, + /** Stop answering, as a connection lost mid-request would. */ + silence(): void { + deliver = false; + }, + end(): void { + for (const listener of listeners.get("close") ?? []) { + listener({}); + } + }, + }; +} + +/** + * What a correct owner answers each private command with. + * + * The commit answer is derived from the request the way the owner derives it: + * the root the proposal selected — proposed when there is a publication, the + * unchanged expected one when there is not — and one minted identity for each + * event. The runner checks the answer against what it asked, so an owner that + * answered with something else would not be believed. + * + * `sizes` is what the owner measured after decoding staged bytes. + */ +function ownerAnswers(_rootId = "", _sizes: ReadonlyMap = new Map()) { + return (request: Record): Record => { + if (request["command"] === "stage") { + // The length the owner would have measured after decoding, computed from + // the encoding itself so this answers about the bytes it was actually + // sent rather than about a number a test remembered to set. + const encoded = String(request["bytes"] ?? ""); + const padding = encoded.endsWith("==") ? 2 : encoded.endsWith("=") ? 1 : 0; + return { + outcome: "performed", + value: { + kind: request["kind"], + digest: request["digest"], + size: (encoded.length / 4) * 3 - padding, + }, + }; + } + const publication = request["publication"]; + const selected = + publication === null || publication === undefined + ? request["expectedWorkspaceRootId"] + : (publication as Record)["proposedWorkspaceRootId"]; + const events = Array.isArray(request["events"]) ? request["events"] : []; + return { + outcome: "performed", + value: { + workspaceRootId: selected, + journalEventIds: events.map((_entry, index) => `event-${index}`), + }, + }; + }; +} + +/** The final command a transaction sent, proved to be one. */ +function lastCommit(sent: readonly Record[]): Record { + const commit = sent.at(-1); + if (commit === undefined || commit["command"] !== "commit") { + throw new Error("expected the last request to be a commit"); + } + return commit; +} + +/** One object member, read rather than asserted into shape. */ +function member(value: unknown, name: string): Record { + const found = value === null || typeof value !== "object" ? undefined : Object.entries(value); + const entry = found?.find(([key]) => key === name)?.[1]; + if (entry === null || typeof entry !== "object" || Array.isArray(entry)) { + throw new Error(`expected ${name} to be an object`); + } + return Object.fromEntries(Object.entries(entry)); +} + +/** One text member, read rather than asserted. */ +function text(value: Record, name: string): string { + const found = value[name]; + if (typeof found !== "string") { + throw new Error(`expected ${name} to be text`); + } + return found; +} + +/** One list member, read the same way. */ +function memberList(value: Record, name: string): Record[] { + const entry = value[name]; + if (!Array.isArray(entry)) { + throw new Error(`expected ${name} to be a list`); + } + return entry.map((item) => { + if (item === null || typeof item !== "object" || Array.isArray(item)) { + throw new Error(`expected every ${name} entry to be an object`); + } + return Object.fromEntries(Object.entries(item)); + }); +} + +function ids(): () => string { + let id = 0; + return () => `request-${(id += 1)}`; +} + +/** An owner that answers frontier/root/content from one captured tree. */ +function readsOf(captured: { + root: { manifest: string; rootId: string }; + contents: ReadonlyMap; + blobs: ReadonlyMap; +}): RemoteReadLink { + return { + // Materialization never asks for this; a stub that answered would say this + // test proved something it did not. + *invocationSnapshot(): Operation { + throw new Error("this read link carries no invocation snapshot"); + }, + // deno-lint-ignore require-yield + *frontier(): Operation { + return { + record: { + runId: "remote-run", + definition: { + version: 1, + kind: "git", + objectFormat: "sha1", + objectId: "0".repeat(40), + rootDocumentPath: "README.md", + }, + base: "main", + props: {}, + status: "running", + createdAt: "2026-09-03T00:00:00.000Z", + updatedAt: "2026-09-03T00:00:00.000Z", + }, + retrieval: undefined, + workspaceRootId: captured.root.rootId, + journalEventId: null, + entries: [], + }; + }, + // deno-lint-ignore require-yield + *root(workspaceRootId: string) { + if (workspaceRootId !== captured.root.rootId) { + throw new Error("asked for a root this owner does not hold"); + } + return parseWorkspaceRootManifest(captured.root.manifest, reject); + }, + // deno-lint-ignore require-yield + *content(_rootId: string, request: RemoteContentRequest): Operation { + const bytes = + request.kind === "manifest" + ? captured.contents.get(request.digest)?.manifestBytes + : captured.blobs.get(request.digest); + if (bytes === undefined) { + throw new Error("asked for content this owner does not hold"); + } + return { kind: request.kind, digest: request.digest, bytes }; + }, + }; +} + +/** A small starting tree, captured so an owner can serve it. */ +function* startingTree(): Operation<{ captured: CapturedWorkspace; reads: RemoteReadLink }> { + const files = runnerFiles(); + const trees = yield* useRunnerTrees(); + const root = yield* trees.create("source"); + yield* until(writeFile(`${root}/README.md`, "starting\n", { mode: 0o644 })); + yield* until(mkdir(`${root}/docs`, { mode: 0o755 })); + const captured = yield* captureWorkspace( + files, + (logical) => (logical === "/" ? root : `${root}${logical}`), + reject, + ); + return { captured, reads: readsOf(captured) }; +} + +describe("what the production runner publishes", () => { + it("sends one closed commit describing everything the transaction decided", function* () { + const files = runnerFiles(); + const trees = yield* useRunnerTrees(); + const { captured, reads } = yield* startingTree(); + const sizes = new Map(); + const transport = wire(ownerAnswers(captured.root.rootId, sizes)); + const connection = yield* useOwnerConnection(transport.socket); + + const materialization = yield* useMaterialization( + files, + trees, + reads, + captured.root.rootId, + reject, + ); + const attempt = yield* useAttempt(files, trees, reads, materialization, reject); + yield* until(writeFile(attempt.at("/NOTES.md"), "written by the effect\n", { mode: 0o644 })); + const proposed = yield* attempt.capture(); + for (const [digest, content] of proposed.contents) { + sizes.set(digest, content.manifestBytes.length); + } + for (const [digest, bytes] of proposed.blobs) { + sizes.set(digest, bytes.length); + } + + const link = cloudflareOwnerLink(connection, reads, ids()); + const committed = yield* transactRemotely( + link, + createTransactionGate(), + function* (transaction, enlist) { + yield* transaction.journal.append(event("published")); + enlist(attempt, [repositoryMapping()]); + return "done"; + }, + ); + expect(committed).toMatchObject({ ok: true }); + + // The last request is one closed commit carrying the whole proposal. + const commit = lastCommit(transport.sent); + expect(commit["expectedWorkspaceRootId"]).toBe(captured.root.rootId); + expect(commit["events"]).toEqual([serializeDurableEvent(event("published"))]); + const publication = member(commit, "publication"); + expect(publication["proposedWorkspaceRootId"]).toBe(proposed.root.rootId); + expect(sha256Hex(`${WORKSPACE_ROOT_DOMAIN}${String(publication["proposedManifest"])}`)).toBe( + proposed.root.rootId, + ); + expect(text(memberList(commit, "mappings")[0] ?? {}, "locator")).toBe(LOCATOR); + // Everything the proposal names was staged before the commit went out. + const staged = transport.sent.filter((request) => request["command"] === "stage"); + expect(staged.length).toBe(proposed.root.manifests.length + proposed.root.blobs.length); + }); + + it("sends no commit and keeps no tree when the body does not finish", function* () { + const files = runnerFiles(); + const outcomes: Record Operation> = {}; + for (const description of ["raises", "is cancelled"]) { + let attemptPath = ""; + const transport = wire(ownerAnswers("")); + yield* scoped(function* () { + const trees = yield* useRunnerTrees(); + const { captured, reads } = yield* startingTree(); + const connection = yield* useOwnerConnection(transport.socket); + const link = cloudflareOwnerLink(connection, reads, ids()); + const materialization = yield* useMaterialization( + files, + trees, + reads, + captured.root.rootId, + reject, + ); + let raised: unknown; + yield* scoped(function* () { + const attempt = yield* useAttempt(files, trees, reads, materialization, reject); + attemptPath = attempt.at("/"); + if (description === "raises") { + try { + yield* transactRemotely(link, createTransactionGate(), function* () { + throw new Error("the effect failed"); + }); + } catch (error) { + raised = error; + } + return; + } + const running = yield* spawn(() => + transactRemotely(link, createTransactionGate(), function* () { + yield* sleep(10_000); + return "never"; + }), + ); + yield* sleep(0); + yield* running.halt(); + }); + expect([description, description === "raises" ? raised instanceof Error : true]).toEqual([ + description, + true, + ]); + }); + // No commit was sent, and the attempt tree is gone. + expect([ + description, + transport.sent.some((request) => request["command"] === "commit"), + ]).toEqual([description, false]); + let listed: unknown; + try { + listed = yield* until(readdir(attemptPath)); + } catch (error) { + listed = error; + } + expect([description, listed instanceof Error]).toEqual([description, true]); + } + void outcomes; + }); + + it("transfers the attempt inside the transaction that the owner performed", function* () { + const files = runnerFiles(); + const trees = yield* useRunnerTrees(); + const { captured, reads } = yield* startingTree(); + const sizes = new Map(); + const transport = wire(ownerAnswers("", sizes)); + const connection = yield* useOwnerConnection(transport.socket); + const link = cloudflareOwnerLink(connection, reads, ids()); + const materialization = yield* useMaterialization( + files, + trees, + reads, + captured.root.rootId, + reject, + ); + const acceptedBefore = materialization.at("/"); + + let attemptRoot = ""; + let acceptedDuringBody = ""; + const committed = yield* scoped(function* () { + const attempt = yield* useAttempt(files, trees, reads, materialization, reject); + attemptRoot = attempt.at("/"); + yield* until(writeFile(attempt.at("/NOTES.md"), "published\n", { mode: 0o644 })); + const proposed = yield* attempt.capture(); + for (const [digest, content] of proposed.contents) { + sizes.set(digest, content.manifestBytes.length); + } + for (const [digest, blob] of proposed.blobs) { + sizes.set(digest, blob.length); + } + return yield* transactRemotely(link, createTransactionGate(), function* (_tx, enlist) { + enlist(attempt); + // Still the old Workspace while the answer is unknown. + acceptedDuringBody = materialization.at("/"); + return "done"; + }); + }); + + expect(committed).toMatchObject({ ok: true }); + expect(acceptedDuringBody).toBe(acceptedBefore); + + // By the time the transaction reported success the transfer had happened: + // the accepted path is the attempt's tree and reads the attempted bytes. + expect(materialization.at("/")).toBe(attemptRoot); + expect(materialization.workspaceRootId).not.toBe(captured.root.rootId); + expect(yield* until(readFile(materialization.at("/NOTES.md"), "utf8"))).toBe("published\n"); + + // And the tree the run used to be at is gone. + let listed: unknown; + try { + listed = yield* until(readdir(acceptedBefore)); + } catch (error) { + listed = error; + } + expect(listed).toBeInstanceOf(Error); + }); + + it("offers no way to move the accepted Workspace without the owner", function* () { + const files = runnerFiles(); + const trees = yield* useRunnerTrees(); + const { captured, reads } = yield* startingTree(); + const materialization = yield* useMaterialization( + files, + trees, + reads, + captured.root.rootId, + reject, + ); + const accepted = materialization.at("/"); + + yield* scoped(function* () { + const attempt = yield* useAttempt(files, trees, reads, materialization, reject); + yield* until(writeFile(attempt.at("/NOTES.md"), "never published\n", { mode: 0o644 })); + + // Everything a caller can reach by name. Reading where the Workspace is, + // reading what an attempt holds — and nothing that moves either. + expect(Object.keys(materialization).toSorted()).toEqual(["at", "workspaceRootId"]); + expect(Object.keys(attempt).toSorted()).toEqual(["at", "capture"]); + const reachable = [ + ...Object.getOwnPropertyNames(attempt), + ...Object.getOwnPropertyNames(materialization), + ]; + for (const name of ["promote", "transfer", "replace", "accept", "propose", "seal"]) { + expect(reachable).not.toContain(name); + } + + // A caller can still capture. What it gets back is a description, and + // there is nothing to hand it to: `enlist` takes an attempt, so a + // publication that no live attempt owns cannot be expressed at all. + const described = yield* attempt.capture(); + expect(described.root.rootId).not.toBe(captured.root.rootId); + }); + + expect(materialization.at("/")).toBe(accepted); + expect(materialization.workspaceRootId).toBe(captured.root.rootId); + }); + + it("cannot be changed by a caller that kept its own copy", function* () { + const files = runnerFiles(); + const trees = yield* useRunnerTrees(); + const { captured, reads } = yield* startingTree(); + const transport = wire(ownerAnswers("")); + const connection = yield* useOwnerConnection(transport.socket); + const link = cloudflareOwnerLink(connection, reads, ids()); + const materialization = yield* useMaterialization( + files, + trees, + reads, + captured.root.rootId, + reject, + ); + + const mappings: RetainedMapping[] = [repositoryMapping()]; + yield* scoped(function* () { + const attempt = yield* useAttempt(files, trees, reads, materialization, reject); + yield* transactRemotely(link, createTransactionGate(), function* (_transaction, enlist) { + enlist(attempt, mappings); + // The caller still holds the array it passed and edits it afterwards. + const first = mappings[0]; + if (first?.kind === "repository") { + mappings[0] = { ...first, locator: "https://elsewhere.invalid/x.git" }; + } + return "done"; + }); + }); + expect(text(memberList(lastCommit(transport.sent), "mappings")[0] ?? {}, "locator")).toBe( + LOCATOR, + ); + }); + + it("sends a journal-only commit with no publication and stages nothing", function* () { + const files = runnerFiles(); + yield* scoped(function* () { + const trees = yield* useRunnerTrees(); + void trees; + const { captured, reads } = yield* startingTree(); + const transport = wire(ownerAnswers(captured.root.rootId)); + const connection = yield* useOwnerConnection(transport.socket); + const link = cloudflareOwnerLink(connection, reads, ids()); + const committed = yield* transactRemotely( + link, + createTransactionGate(), + function* (transaction) { + yield* transaction.journal.append(event("noted")); + return "done"; + }, + ); + expect(committed).toMatchObject({ ok: true }); + + const commit = lastCommit(transport.sent); + // A transaction that only appended proposes nothing. Inventing a + // Workspace change to make the shape uniform would publish a root nobody + // asked for, so `publication` is null and nothing was staged. + expect(commit["publication"]).toBe(null); + expect(commit["mappings"]).toEqual([]); + expect(transport.sent.some((request) => request["command"] === "stage")).toBe(false); + }); + void files; + }); + + it("encodes every kind of retained mapping the owner accepts", function* () { + const files = runnerFiles(); + const trees = yield* useRunnerTrees(); + const { captured, reads } = yield* startingTree(); + const transport = wire(ownerAnswers("")); + const connection = yield* useOwnerConnection(transport.socket); + const link = cloudflareOwnerLink(connection, reads, ids()); + const materialization = yield* useMaterialization( + files, + trees, + reads, + captured.root.rootId, + reject, + ); + yield* scoped(function* () { + const attempt = yield* useAttempt(files, trees, reads, materialization, reject); + yield* transactRemotely(link, createTransactionGate(), function* (_transaction, enlist) { + enlist(attempt, [ + repositoryMapping(), + { + kind: "worktree", + record: { + repositoryName: "app", + name: "feature", + requestedBranch: "feature", + requestedBase: null, + creationCommit: "2".repeat(40), + checkoutPath: "/docs", + }, + }, + { + kind: "agent-session", + record: { + provider: "acp", + agentCommand: "/usr/bin/agent", + sessionIdentity: "session-1", + sessionKey: agentSessionKey({ + provider: "acp", + agentCommand: "/usr/bin/agent", + sessionIdentity: "session-1", + }), + policy: "strict", + assertion: { kind: "acp-session", value: "abc" }, + createdAt: "2026-09-03T00:00:00.000Z", + }, + }, + ]); + return "done"; + }); + }); + const mappings = memberList(lastCommit(transport.sent), "mappings"); + expect(mappings.map((mapping) => mapping["kind"])).toEqual([ + "repository", + "worktree", + "agent-session", + ]); + // Only a Repository carries the locator; the other two are the record. + expect(mappings.filter((mapping) => "locator" in mapping)).toHaveLength(1); + }); + + it("retries a lost answer with the same identity and the same bytes", function* () { + // A retry happens on a new connection: the one that lost the answer is + // gone, and a connection refuses to reuse a correlation id of its own. What + // has to be stable is the identity across those two connections, because + // that is what the owner recognizes the retry by. + const sent: Record[][] = []; + let intent: CommitIntent | undefined; + for (const attempt of [0, 1]) { + yield* scoped(function* () { + const { captured, reads } = yield* startingTree(); + const transport = wire(ownerAnswers(captured.root.rootId)); + sent.push(transport.sent); + const connection = yield* useOwnerConnection(transport.socket); + const link = cloudflareOwnerLink(connection, reads, ids()); + intent ??= { + expectedWorkspaceRootId: captured.root.rootId, + expectedJournalEventId: null, + events: [], + publication: null, + mappings: [], + bytes: new Map(), + }; + const committed = yield* link.commit(intent); + expect([attempt, committed.ok]).toEqual([attempt, true]); + }); + } + + const first = sent[0]?.find((request) => request["command"] === "commit"); + const second = sent[1]?.find((request) => request["command"] === "commit"); + expect(first?.["id"]).toBe(second?.["id"]); + // Byte-equivalent, so the owner sees the request it already decided. + expect(JSON.stringify(first)).toBe(JSON.stringify(second)); + }); + + it("asks a different question for a different proposal", function* () { + const { captured, reads } = yield* startingTree(); + const transport = wire(ownerAnswers(captured.root.rootId)); + const connection = yield* useOwnerConnection(transport.socket); + const link = cloudflareOwnerLink(connection, reads, ids()); + const intent: CommitIntent = { + expectedWorkspaceRootId: captured.root.rootId, + expectedJournalEventId: null, + events: [], + publication: null, + mappings: [], + bytes: new Map(), + }; + yield* link.commit(intent); + yield* link.commit({ ...intent, events: [event("later")] }); + const commits = transport.sent.filter((request) => request["command"] === "commit"); + expect(commits).toHaveLength(2); + expect(commits[0]?.["id"]).not.toBe(commits[1]?.["id"]); + }); + + it("promotes nothing and keeps no tree when the owner refuses", function* () { + const files = runnerFiles(); + let attemptPath = ""; + yield* scoped(function* () { + const trees = yield* useRunnerTrees(); + const { captured, reads } = yield* startingTree(); + const transport = wire(() => ({ outcome: "refused", refusal: "command:stale-root" })); + const connection = yield* useOwnerConnection(transport.socket); + const link = cloudflareOwnerLink(connection, reads, ids()); + const materialization = yield* useMaterialization( + files, + trees, + reads, + captured.root.rootId, + reject, + ); + yield* scoped(function* () { + const attempt = yield* useAttempt(files, trees, reads, materialization, reject); + attemptPath = attempt.at("/"); + yield* until(writeFile(attempt.at("/NOTES.md"), "refused\n", { mode: 0o644 })); + const committed = yield* transactRemotely(link, createTransactionGate(), function* () { + return "done"; + }); + // A refusal is an answer, and the answer is no. + expect(committed.ok).toBe(false); + }); + expect(materialization.workspaceRootId).toBe(captured.root.rootId); + }); + let listed: unknown; + try { + listed = yield* until(readdir(attemptPath)); + } catch (error) { + listed = error; + } + expect(listed).toBeInstanceOf(Error); + }); + + it("promotes nothing when the answer is lost", function* () { + const files = runnerFiles(); + yield* scoped(function* () { + const trees = yield* useRunnerTrees(); + const { captured, reads } = yield* startingTree(); + const transport = wire(ownerAnswers(captured.root.rootId)); + const connection = yield* useOwnerConnection(transport.socket); + const link = cloudflareOwnerLink(connection, reads, ids()); + const materialization = yield* useMaterialization( + files, + trees, + reads, + captured.root.rootId, + reject, + ); + yield* scoped(function* () { + const attempt = yield* useAttempt(files, trees, reads, materialization, reject); + yield* until(writeFile(attempt.at("/NOTES.md"), "unanswered\n", { mode: 0o644 })); + // The connection goes while the answer is in flight. + transport.silence(); + const asking = yield* spawn(() => + transactRemotely(link, createTransactionGate(), function* () { + return "done"; + }), + ); + yield* sleep(0); + transport.end(); + const committed = yield* asking; + // Undecided, not failed — whether the owner committed cannot be known + // from here. Either way nothing is promoted locally. + expect(committed.ok).toBe(false); + }); + expect(materialization.workspaceRootId).toBe(captured.root.rootId); + }); + }); + + it("sends nothing when a resource the body started fails to tear down", function* () { + let sent: Record[] = []; + let raised: unknown; + try { + yield* scoped(function* () { + const { captured, reads } = yield* startingTree(); + const transport = wire(ownerAnswers(captured.root.rootId)); + sent = transport.sent; + const connection = yield* useOwnerConnection(transport.socket); + const link = cloudflareOwnerLink(connection, reads, ids()); + yield* transactRemotely(link, createTransactionGate(), function* (transaction) { + yield* transaction.journal.append(event("appended")); + // A resource whose teardown fails. The body finished, but everything + // it started did not, so the transaction has not finished either — + // and the failure surfaces as the scope unwinds rather than inside it. + yield* ensure(() => { + throw new Error("teardown failed"); + }); + return "done"; + }); + }); + } catch (error) { + raised = error; + } + expect(raised).toBeInstanceOf(Error); + expect(sent.some((request) => request["command"] === "commit")).toBe(false); + }); + + it("sends nothing when the transaction exceeds a local bound", function* () { + const files = runnerFiles(); + const trees = yield* useRunnerTrees(); + const { captured, reads } = yield* startingTree(); + const transport = wire(ownerAnswers("")); + const connection = yield* useOwnerConnection(transport.socket); + const link = cloudflareOwnerLink(connection, reads, ids()); + const materialization = yield* useMaterialization( + files, + trees, + reads, + captured.root.rootId, + reject, + ); + let raised: unknown; + yield* scoped(function* () { + const attempt = yield* useAttempt(files, trees, reads, materialization, reject); + try { + yield* transactRemotely(link, createTransactionGate(), function* (_transaction, enlist) { + // More retained mappings than one intent may carry. + enlist( + attempt, + Array.from({ length: 300 }, () => repositoryMapping()), + ); + return "done"; + }); + } catch (error) { + raised = error; + } + }); + expect(raised).toBeInstanceOf(Error); + expect(transport.sent.some((request) => request["command"] === "commit")).toBe(false); + expect(materialization.workspaceRootId).toBe(captured.root.rootId); + }); + + it("refuses a performed answer that names a root this proposal did not select", function* () { + const { captured, reads } = yield* startingTree(); + // An owner agreeing to something else is not an owner this runner can go + // on talking to: believing it would promote a Workspace nobody proposed. + const transport = wire(() => ({ + outcome: "performed", + value: { workspaceRootId: "f".repeat(64), journalEventIds: [] }, + })); + const connection = yield* useOwnerConnection(transport.socket); + const link = cloudflareOwnerLink(connection, reads, ids()); + const committed = yield* link.commit({ + expectedWorkspaceRootId: captured.root.rootId, + expectedJournalEventId: null, + events: [], + publication: null, + mappings: [], + bytes: new Map(), + }); + expect(committed.ok).toBe(false); + }); + + it("refuses a performed answer that loses an event it was given", function* () { + const { captured, reads } = yield* startingTree(); + const transport = wire((request) => ({ + outcome: "performed", + value: { workspaceRootId: request["expectedWorkspaceRootId"], journalEventIds: [] }, + })); + const connection = yield* useOwnerConnection(transport.socket); + const link = cloudflareOwnerLink(connection, reads, ids()); + const committed = yield* link.commit({ + expectedWorkspaceRootId: captured.root.rootId, + expectedJournalEventId: null, + events: [event("appended")], + publication: null, + mappings: [], + bytes: new Map(), + }); + // One identity per event, or the two sides disagree about what history + // this commit created. + expect(committed.ok).toBe(false); + }); + + it("seals a nested mapping value against later mutation", function* () { + const files = runnerFiles(); + const trees = yield* useRunnerTrees(); + const { captured, reads } = yield* startingTree(); + const transport = wire(ownerAnswers("")); + const connection = yield* useOwnerConnection(transport.socket); + const link = cloudflareOwnerLink(connection, reads, ids()); + const materialization = yield* useMaterialization( + files, + trees, + reads, + captured.root.rootId, + reject, + ); + + const assertion = { kind: "acp-session", value: "admitted" }; + const identity = { + provider: "acp", + agentCommand: "/usr/bin/agent", + sessionIdentity: "session-1", + }; + yield* scoped(function* () { + const attempt = yield* useAttempt(files, trees, reads, materialization, reject); + yield* transactRemotely(link, createTransactionGate(), function* (_transaction, enlist) { + enlist(attempt, [ + { + kind: "agent-session", + record: { + ...identity, + sessionKey: agentSessionKey(identity), + policy: "strict", + assertion, + createdAt: "2026-09-03T00:00:00.000Z", + }, + }, + ]); + // The caller still holds the nested assertion object and edits it. + assertion.value = "changed after admission"; + return "done"; + }); + }); + + const mapping = memberList(lastCommit(transport.sent), "mappings")[0] ?? {}; + expect(text(member(member(mapping, "record"), "assertion"), "value")).toBe("admitted"); + }); + + it("commits the tree as it finally is, not as it was when enlisted", function* () { + const files = runnerFiles(); + const trees = yield* useRunnerTrees(); + const { captured, reads } = yield* startingTree(); + const sizes = new Map(); + const transport = wire(ownerAnswers("", sizes)); + const connection = yield* useOwnerConnection(transport.socket); + const link = cloudflareOwnerLink(connection, reads, ids()); + const materialization = yield* useMaterialization( + files, + trees, + reads, + captured.root.rootId, + reject, + ); + + let atEnlistment = ""; + yield* scoped(function* () { + const attempt = yield* useAttempt(files, trees, reads, materialization, reject); + yield* until(writeFile(attempt.at("/NOTES.md"), "first\n", { mode: 0o644 })); + const committed = yield* transactRemotely( + link, + createTransactionGate(), + function* (_transaction, enlist) { + enlist(attempt); + atEnlistment = (yield* attempt.capture()).root.rootId; + // The body goes on working after designating the attempt. Sealing + // happens after teardown, so this is what gets proposed. + yield* until(writeFile(attempt.at("/NOTES.md"), "second\n", { mode: 0o644 })); + const staged = yield* attempt.capture(); + for (const [digest, content] of staged.contents) { + sizes.set(digest, content.manifestBytes.length); + } + for (const [digest, blob] of staged.blobs) { + sizes.set(digest, blob.length); + } + return "done"; + }, + ); + expect(committed).toMatchObject({ ok: true }); + }); + + // The root the owner was asked to publish is the final one, not the one the + // tree held when the body enlisted it. + const proposed = member(lastCommit(transport.sent), "publication"); + expect(proposed["proposedWorkspaceRootId"]).not.toBe(atEnlistment); + // And the accepted tree recaptures to exactly the root that was committed. + expect(materialization.workspaceRootId).toBe(proposed["proposedWorkspaceRootId"]); + expect(yield* until(readFile(materialization.at("/NOTES.md"), "utf8"))).toBe("second\n"); + }); + + it("refuses a second Workspace publication in one transaction", function* () { + const files = runnerFiles(); + const trees = yield* useRunnerTrees(); + const { captured, reads } = yield* startingTree(); + const transport = wire(ownerAnswers("")); + const connection = yield* useOwnerConnection(transport.socket); + const link = cloudflareOwnerLink(connection, reads, ids()); + const materialization = yield* useMaterialization( + files, + trees, + reads, + captured.root.rootId, + reject, + ); + let raised: unknown; + yield* scoped(function* () { + const attempt = yield* useAttempt(files, trees, reads, materialization, reject); + try { + yield* transactRemotely(link, createTransactionGate(), function* (_transaction, enlist) { + enlist(attempt); + enlist(attempt); + return "done"; + }); + } catch (error) { + raised = error; + } + }); + // Two Workspaces proposed for one commit is a choice nobody may make on the + // run's behalf, so the transaction fails and nothing is sent. + expect(raised).toBeInstanceOf(Error); + expect(transport.sent.some((request) => request["command"] === "commit")).toBe(false); + }); +}); diff --git a/packages/workflow/tests/remote-read.test.ts b/packages/workflow/tests/remote-read.test.ts new file mode 100644 index 000000000..bff7e4342 --- /dev/null +++ b/packages/workflow/tests/remote-read.test.ts @@ -0,0 +1,785 @@ +/** + * Tier WRH — reading a run from an owner somewhere else. + * + * What is under test here is the runner's half: whether a private answer + * becomes a semantic value only after it has been proved to be one, and whether + * a channel that has stopped making sense is stopped rather than followed. + * + * The owner is a deterministic fake, deliberately. Command-specific parsing, + * refusal narrowing and journal reassembly are arithmetic over what arrived, + * and a fake can produce the answers a correct owner never would — a page that + * skips an event, a refusal category from another release, content that is not + * what it is named. What the real owner does with a real request is proved on + * real workerd, where the runtime is the thing being relied on. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { serializeDurableEvent } from "@executablemd/durable-streams"; +import { scoped } from "effection"; +import { + cloudflareOwnerLink, + cloudflareReadLink, + cloudflareRunLink, + stageCloudflareContent, +} from "../src/cloudflare/client.ts"; +import { + WorkflowRecordMalformedError, + WorkflowRequestError, + WorkflowSchemaVersionError, + WorkflowStorageError, +} from "../src/storage/errors.ts"; +import { useRemoteRunDatabase } from "../src/remote/database.ts"; +import { MAX_MESSAGE_BYTES } from "../src/remote/client.ts"; +import type { Result } from "effection"; +import type { DefinitionRetrieval } from "../src/storage/record.ts"; +import { SCHEMA_VERSION } from "../src/sqlite/workflow-schema.ts"; +import { createTransactionGate, transactRemotely } from "../src/remote/collector.ts"; +import { encodeBase64 } from "../src/cloudflare/encoding.ts"; +import type { OwnerSocket, SocketListener } from "../src/remote/client.ts"; +import { OwnerLinkError, useOwnerConnection } from "../src/remote/client.ts"; +import { RemoteRecordError } from "../src/remote/records.ts"; +import { + EMPTY_WORKSPACE_MANIFEST, + EMPTY_WORKSPACE_ROOT_ID, + workspaceRootId, +} from "../src/deno/workspace/manifest.ts"; +import { EXECUTION_PAGE_BYTES, executionPageBytes } from "../src/cloudflare/commands.ts"; +import { WORKSPACE_ROOT_DOMAIN } from "../src/workspace/root-manifest.ts"; +import { sha256Hex } from "../src/workspace/sha256.ts"; + +const RUN_ID = "remote-run"; +const ROOT_MANIFEST = JSON.stringify({ + format: 1, + entries: [{ path: "/", kind: "directory", mode: 493, mtime: 0 }], +}); +const ROOT_ID = sha256Hex(`${WORKSPACE_ROOT_DOMAIN}${ROOT_MANIFEST}`); +const CONTENT = new TextEncoder().encode( + JSON.stringify({ version: 1, chunks: [{ hash: "0".repeat(64), size: 1 }] }), +); +const CONTENT_ID = sha256Hex(CONTENT); + +function event(name: string): string { + return serializeDurableEvent({ + type: "yield", + coroutineId: "root", + description: { type: "test", name }, + result: { status: "ok", value: name }, + }); +} + +function runRecord(): Record { + return { + runId: RUN_ID, + definition: { + version: 1, + kind: "git", + objectFormat: "sha1", + objectId: "0".repeat(40), + rootDocumentPath: "README.md", + }, + base: "main", + props: {}, + status: "running", + createdAt: "2026-09-03T00:00:00.000Z", + updatedAt: "2026-09-03T00:00:00.000Z", + }; +} + +function object(value: unknown): Record { + if (value === null || typeof value !== "object" || Array.isArray(value)) { + throw new Error("expected an object"); + } + return Object.fromEntries(Object.entries(value)); +} + +function wire(answer: (request: Record) => Record) { + const listeners = new Map>(); + let closes = 0; + const socket: OwnerSocket = { + send(data: string): void { + const request = object(JSON.parse(data)); + const response = answer(request); + for (const listener of listeners.get("message") ?? []) { + listener({ data: JSON.stringify({ id: request["id"], ...response }) }); + } + }, + close(): void { + closes += 1; + }, + addEventListener(type, listener): void { + const found = listeners.get(type) ?? new Set(); + found.add(listener); + listeners.set(type, found); + }, + removeEventListener(type, listener): void { + listeners.get(type)?.delete(listener); + }, + }; + return { + socket, + get closes(): number { + return closes; + }, + get listeners(): number { + return [...listeners.values()].reduce((sum, found) => sum + found.size, 0); + }, + }; +} + +/** The frontier a database handle opens from, as the owner would answer it. */ +function frontierValue(): Record { + return { + record: runRecord(), + retrieval: null, + workspaceRootId: ROOT_ID, + journalEventId: null, + }; +} + +function ids(): () => string { + let id = 0; + return () => `request-${(id += 1)}`; +} + +/** The name one retained test event carries, read rather than asserted. */ +function effectName(entry: unknown): string { + if (entry === null || typeof entry !== "object" || !("description" in entry)) { + return ""; + } + const description = entry.description; + if (description === null || typeof description !== "object" || !("name" in description)) { + return ""; + } + const name: unknown = description["name"]; + return typeof name === "string" ? name : ""; +} + +function failure(error: unknown): string { + if (!(error instanceof OwnerLinkError)) { + throw new Error(`expected an OwnerLinkError, received ${String(error)}`); + } + return error.refusal; +} + +/** + * The parser's own failure, having proved it is one. + * + * The request whose answer could not be read keeps that failure rather than + * the channel's, because only the boundary above it can say what the value was + * supposed to mean. Reading it as a string would let an unrelated error pass + * for the category a test expected. + */ +function unreadable(error: unknown): string { + if (!(error instanceof RemoteRecordError)) { + throw new Error(`expected a RemoteRecordError, received ${String(error)}`); + } + return "malformed-record"; +} + +describe("semantic reads from a Cloudflare owner", () => { + it("uses the standard SHA-256 identity rather than an adapter-local digest", function* () { + // The published answers, including the two-block case the padding rule is + // easiest to get wrong on. + expect(sha256Hex("")).toBe("e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"); + expect(sha256Hex("abc")).toBe( + "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad", + ); + expect(sha256Hex("abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq")).toBe( + "248d6a61d20638b8e5c026930c3e6039a33ce45964ff2167f6ecedd419db06c1", + ); + expect(sha256Hex(new Uint8Array(1000).fill(0x61))).toBe( + "41edece42d63e8d9bf515a9ba6932e1c20cbc9f5a5d134645adb5db1b9737ea3", + ); + }); + + it("computes the identity the local host computes for the same root", function* () { + // The two hosts retain the same roots and must name them identically. This + // one is arithmetic in the language; the Deno host uses `node:crypto`. A + // difference here would be two hosts disagreeing about history. + expect(sha256Hex(`${WORKSPACE_ROOT_DOMAIN}${ROOT_MANIFEST}`)).toBe( + workspaceRootId(ROOT_MANIFEST), + ); + expect(sha256Hex(`${WORKSPACE_ROOT_DOMAIN}${EMPTY_WORKSPACE_MANIFEST}`)).toBe( + EMPTY_WORKSPACE_ROOT_ID, + ); + }); + + it("strictly parses the private staging decision", function* () { + const bytes = new TextEncoder().encode("staged"); + const digest = sha256Hex(bytes); + const transport = wire((request) => ({ + outcome: "performed", + value: { kind: request["kind"], digest: request["digest"], size: bytes.length }, + })); + yield* scoped(function* () { + const connection = yield* useOwnerConnection(transport.socket); + expect(yield* stageCloudflareContent(connection, "stage", "blob", bytes)).toEqual({ + kind: "blob", + digest, + size: bytes.length, + }); + }); + }); + + it("parses an anchored frontier, a canonical root, and verified content", function* () { + const transport = wire((request) => { + if (request["command"] === "frontier") { + return { + outcome: "performed", + value: { + record: runRecord(), + retrieval: { + metadata: { locator: "somewhere" }, + revision: 1, + updatedAt: "2026-09-03T00:00:00.000Z", + }, + workspaceRootId: ROOT_ID, + journalEventId: "event-2", + }, + }; + } + if (request["command"] === "journal" && request["afterEventId"] === null) { + return { + outcome: "performed", + value: { + anchorEventId: "event-2", + afterEventId: null, + entries: [ + { + eventId: "event-1", + previousEventId: null, + record: event("one"), + workspaceRootId: ROOT_ID, + }, + ], + done: false, + }, + }; + } + if (request["command"] === "journal") { + return { + outcome: "performed", + value: { + anchorEventId: "event-2", + afterEventId: "event-1", + entries: [ + { + eventId: "event-2", + previousEventId: "event-1", + record: event("two"), + workspaceRootId: ROOT_ID, + }, + ], + done: true, + }, + }; + } + if (request["command"] === "root") { + return { + outcome: "performed", + value: { workspaceRootId: ROOT_ID, manifest: ROOT_MANIFEST }, + }; + } + return { + outcome: "performed", + value: { + kind: "manifest", + digest: CONTENT_ID, + size: CONTENT.length, + bytes: encodeBase64(CONTENT), + }, + }; + }); + + yield* scoped(function* () { + const connection = yield* useOwnerConnection(transport.socket); + const reads = cloudflareReadLink(connection, ids(), RUN_ID); + const frontier = yield* reads.frontier(); + expect(frontier.entries.map((entry) => entry.eventId)).toEqual(["event-1", "event-2"]); + expect(frontier.workspaceRootId).toBe(ROOT_ID); + expect((yield* reads.root(ROOT_ID)).entries).toHaveLength(1); + expect( + (yield* reads.content(ROOT_ID, { kind: "manifest", digest: CONTENT_ID })).bytes, + ).toEqual(CONTENT); + }); + expect(transport.closes).toBe(1); + expect(transport.listeners).toBe(0); + }); + + it("closes on a journal page that does not continue its snapshot", function* () { + const anchored = (entries: Record[], done = true) => ({ + anchorEventId: "event-2", + afterEventId: null, + entries, + done, + }); + const entry = (eventId: string, previousEventId: string | null) => ({ + eventId, + previousEventId, + record: event(eventId), + workspaceRootId: ROOT_ID, + }); + + // Four ways one page can fail to be the continuation it claims to be. The + // structural consequence is one: the events never reach a caller, because + // a journal that is missing an event looks exactly like a shorter journal. + const pages: Record> = { + skipped: anchored([entry("event-2", "event-1")]), + "out of order": anchored([entry("event-2", null), entry("event-1", "event-2")], false), + duplicated: anchored([entry("event-1", null), entry("event-1", "event-1")], false), + "not terminal": anchored([entry("event-1", null)]), + }; + + for (const [description, page] of Object.entries(pages)) { + const transport = wire((request) => + request["command"] === "frontier" + ? { + outcome: "performed", + value: { + record: runRecord(), + retrieval: null, + workspaceRootId: ROOT_ID, + journalEventId: "event-2", + }, + } + : { outcome: "performed", value: page }, + ); + let raised: unknown; + yield* scoped(function* () { + const connection = yield* useOwnerConnection(transport.socket); + try { + yield* cloudflareReadLink(connection, ids(), RUN_ID).frontier(); + } catch (error) { + raised = error; + } + }); + expect([description, unreadable(raised)]).toEqual([description, "malformed-record"]); + expect([description, transport.closes]).toEqual([description, 1]); + expect([description, transport.listeners]).toEqual([description, 0]); + } + }); + + it("closes on an unknown same-release refusal", function* () { + const transport = wire(() => ({ outcome: "refused", refusal: "command:newer-release" })); + let raised: unknown; + yield* scoped(function* () { + const connection = yield* useOwnerConnection(transport.socket); + try { + yield* cloudflareReadLink(connection, ids(), RUN_ID).frontier(); + } catch (error) { + raised = error; + } + }); + expect(unreadable(raised)).toBe("malformed-record"); + expect(transport.closes).toBe(1); + }); + + it("closes on a retrieval answer describing a replacement nobody asked for", function* () { + // The contradiction is settled where the answer arrives, not by a caller + // noticing afterwards. Two sides that disagree about which replacement was + // performed have no shared state left to continue from. + const transport = wire(() => ({ + outcome: "performed", + value: { + retrieval: { + metadata: { locator: "something else entirely" }, + revision: 1, + updatedAt: "2026-09-04T00:00:01.000Z", + }, + }, + })); + let outcome: unknown; + yield* scoped(function* () { + const connection = yield* useOwnerConnection(transport.socket); + const link = cloudflareRunLink(connection, ids(), RUN_ID); + outcome = yield* link.replaceRetrieval(ROOT_ID, '{"locator":"what was asked"}'); + }); + expect((outcome as { ok: boolean }).ok).toBe(false); + expect(transport.closes).toBe(1); + }); + + it("returns a malformed record to the caller, and leaves nothing usable behind", function* () { + // The whole point of carrying the parser's failure: the public boundary + // says the owner returned a record this build cannot read, which is what + // happened, rather than that the owner could not be reached. + const mutations: Record[] = []; + const transport = wire((request) => { + if (request["command"] === "frontier") { + return { outcome: "performed", value: frontierValue() }; + } + mutations.push(request); + return { + outcome: "performed", + value: { + retrieval: { + metadata: { locator: "something else entirely" }, + revision: 1, + updatedAt: "2026-09-04T00:00:01.000Z", + }, + }, + }; + }); + let refused: Result | undefined; + let after: Result | undefined; + let held: DefinitionRetrieval | undefined; + yield* scoped(function* () { + const connection = yield* useOwnerConnection(transport.socket); + // One generator for both halves: two would mint the same correlation id + // and the connection would fail closed on the duplicate. + const next = ids(); + const link = cloudflareRunLink(connection, next, RUN_ID); + const database = yield* useRemoteRunDatabase(link, yield* link.frontierSnapshot()); + refused = yield* database.replaceRetrievalMetadata({ locator: "what was asked" }); + held = database.retrieval; + expect(mutations).toHaveLength(1); + after = yield* database.replaceRetrievalMetadata({ locator: "later" }); + // The channel is gone, so the second call never reached the owner. + expect(mutations).toHaveLength(1); + }); + expect(refused?.ok).toBe(false); + expect(refused?.ok === false && refused.error).toEqual( + expect.any(WorkflowRecordMalformedError), + ); + // Nothing private crossed with it. + expect(String(refused?.ok === false && refused.error)).not.toContain("something else"); + // The snapshot is what the frontier established: an answer about another + // value installs nothing, because it decides where the definition is read. + expect(held).toEqual(undefined); + expect(after?.ok).toBe(false); + expect(after?.ok === false && after.error).toEqual(expect.any(WorkflowStorageError)); + expect(transport.closes).toBe(1); + }); + + it("returns a request failure when the whole request cannot be carried", function* () { + // Metadata that fits the bound on its own and does not once the command + // around it and its correlation id are counted. The public boundary has to + // say the request was too large, not that the owner was unreachable. + const mutations: Record[] = []; + const transport = wire((request) => { + if (request["command"] === "frontier") { + return { outcome: "performed", value: frontierValue() }; + } + mutations.push(request); + // An honest owner: it performed exactly the replacement it was asked for. + return { + outcome: "performed", + value: { + retrieval: { + metadata: JSON.parse(String(request["metadata"])), + revision: 1, + updatedAt: "2026-09-04T00:00:01.000Z", + }, + }, + }; + }); + let refused: Result | undefined; + let accepted: Result | undefined; + yield* scoped(function* () { + const connection = yield* useOwnerConnection(transport.socket); + // One generator for both halves: two would mint the same correlation id + // and the connection would fail closed on the duplicate. + const next = ids(); + const link = cloudflareRunLink(connection, next, RUN_ID); + const database = yield* useRemoteRunDatabase(link, yield* link.frontierSnapshot()); + refused = yield* database.replaceRetrievalMetadata({ + locator: "m".repeat(MAX_MESSAGE_BYTES - 64), + }); + // Never sent, so the owner has no idea this was asked. + expect(mutations).toEqual([]); + // The connection was not spent on it either: the next one goes through. + accepted = yield* database.replaceRetrievalMetadata({ locator: "small" }); + expect(mutations).toHaveLength(1); + }); + expect(refused?.ok).toBe(false); + expect(refused?.ok === false && refused.error).toEqual(expect.any(WorkflowRequestError)); + expect(String(refused?.ok === false && refused.error)).not.toContain("too-large"); + expect(accepted?.ok).toBe(true); + expect(transport.closes).toBe(1); + }); + + it("refuses a version spelling outside what a version can be", function* () { + // Zero is a partial initialization and anything past the carrier is + // damaged retained data. Neither is a version this build is behind, so a + // same-release owner never sends one and this client never reads one. + for (const refusal of [ + "storage:unsupported-version-v0", + "storage:unsupported-version-v99999999999", + ]) { + const transport = wire(() => ({ outcome: "refused", refusal })); + let outcome: unknown; + yield* scoped(function* () { + const connection = yield* useOwnerConnection(transport.socket); + const next = ids(); + const link = cloudflareRunLink(connection, next, RUN_ID); + outcome = yield* link.readExecutions(); + }); + const failed = outcome as { ok: boolean; error: Error }; + expect([refusal, failed.ok]).toEqual([refusal, false]); + expect([refusal, failed.error]).toEqual([refusal, expect.any(WorkflowStorageError)]); + // Not a version report, and nothing of the spelling crossed. + expect(failed.error).not.toEqual(expect.any(WorkflowSchemaVersionError)); + expect(String(failed.error)).not.toContain("storage:"); + expect([refusal, transport.closes]).toEqual([refusal, 1]); + } + }); + + it("reports the schema version the owner actually read", function* () { + // A version this build cannot open is the one fact the refusal exists to + // carry. Reporting a placeholder would state something the owner never + // said, and a host deciding whether to upgrade would act on it. + // Seven, and a value wider than the grammar this refusal once had: both + // are versions the owner can recognize, so both must arrive exactly. + for (const stored of [7, 1_000_000]) { + const transport = wire(() => ({ + outcome: "refused", + refusal: `storage:unsupported-version-v${stored}`, + })); + let outcome: unknown; + yield* scoped(function* () { + const connection = yield* useOwnerConnection(transport.socket); + const next = ids(); + const link = cloudflareRunLink(connection, next, RUN_ID); + outcome = yield* link.readExecutions(); + }); + const failed = outcome as { ok: boolean; error: Error }; + expect([stored, failed.ok]).toEqual([stored, false]); + expect([stored, failed.error]).toEqual([stored, expect.any(WorkflowSchemaVersionError)]); + const version = failed.error as WorkflowSchemaVersionError; + expect([version.stored, version.supported]).toEqual([stored, SCHEMA_VERSION]); + } + }); + + it("closes when content bytes disagree with the requested identity", function* () { + const transport = wire(() => ({ + outcome: "performed", + value: { + kind: "blob", + digest: CONTENT_ID, + size: 1, + bytes: encodeBase64(new Uint8Array([1])), + }, + })); + let raised: unknown; + yield* scoped(function* () { + const connection = yield* useOwnerConnection(transport.socket); + try { + yield* cloudflareReadLink(connection, ids(), RUN_ID).content(ROOT_ID, { + kind: "blob", + digest: CONTENT_ID, + manifestDigest: CONTENT_ID, + }); + } catch (error) { + raised = error; + } + }); + expect(unreadable(raised)).toBe("malformed-record"); + expect(transport.closes).toBe(1); + }); + it("hands the collector one assembled frontier and no page mechanics", function* () { + const pages: Record = { + null: { + anchorEventId: "event-2", + afterEventId: null, + entries: [ + { + eventId: "event-1", + previousEventId: null, + record: event("one"), + workspaceRootId: ROOT_ID, + }, + ], + done: false, + }, + "event-1": { + anchorEventId: "event-2", + afterEventId: "event-1", + entries: [ + { + eventId: "event-2", + previousEventId: "event-1", + record: event("two"), + workspaceRootId: ROOT_ID, + }, + ], + done: true, + }, + }; + const transport = wire((request) => + request["command"] === "frontier" + ? { + outcome: "performed", + value: { + record: runRecord(), + retrieval: null, + workspaceRootId: ROOT_ID, + journalEventId: "event-2", + }, + } + : { outcome: "performed", value: pages[String(request["afterEventId"])] }, + ); + + let seen: unknown[] = []; + let committed: unknown; + yield* scoped(function* () { + const connection = yield* useOwnerConnection(transport.socket); + const request = ids(); + const link = cloudflareOwnerLink( + connection, + cloudflareReadLink(connection, request, RUN_ID), + request, + ); + // Two pages went over the wire. What the body reads back is one journal: + // the collector is handed the assembled prefix and never learns that a + // page, a cursor or an anchor was involved. + const outcome = yield* transactRemotely(link, createTransactionGate(), function* (tx) { + seen = yield* tx.journal.readAll(); + return "done"; + }); + committed = outcome; + }); + + expect(seen).toHaveLength(2); + expect(seen.map(effectName)).toEqual(["one", "two"]); + // D2 has reads and no commit. The transaction returns the owner's refusal + // rather than a success nothing performed. + expect(committed).toMatchObject({ ok: false }); + }); + it("assembles an execution snapshot only from pages that describe it", function* () { + const record = (id: string) => ({ executionId: id, startedAt: "2026-09-04T00:00:00.000Z" }); + const page = (rows: unknown[], overrides: Record = {}) => ({ + outcome: "performed", + value: { runId: RUN_ID, anchor: 2, after: null, rows, done: true, ...overrides }, + }); + const row = (sequence: number, id: string) => ({ sequence, record: record(id) }); + + // Each of these is a page that does not describe the snapshot it claims. + // The structural consequence is one: no partial history is returned. + const refused: Record = { + "another run's history": page([row(1, "a"), row(2, "b")], { runId: "somebody-else" }), + "a terminal page short of its anchor": page([row(1, "a")]), + "a first row that is not the first": page([row(2, "b")]), + "a gap between rows": page([row(1, "a"), row(3, "c")]), + "a repeated row": page([row(1, "a"), row(1, "a")]), + "a row beyond the anchor": page([row(1, "a"), row(2, "b"), row(3, "c")]), + "an empty page of a non-empty snapshot": page([], { done: false }), + "an empty snapshot that carries rows": page([row(1, "a")], { anchor: null }), + "a cursor it was not asked to continue from": page([row(1, "a"), row(2, "b")], { after: 7 }), + "a record with a member the shape does not declare": page([ + { sequence: 1, record: { ...record("a"), note: "extra" } }, + ]), + "a record that stopped without saying how": page([ + { sequence: 1, record: { ...record("a"), stopStatus: "completed" } }, + ]), + // Measured the same way the owner measures it, over the same wrappers. + // A page past the bound is refused whole: no prefix of it is returned. + "a page past the byte bound": page([ + row(1, "a"), + { sequence: 2, record: record("b".repeat(EXECUTION_PAGE_BYTES)) }, + ]), + }; + + for (const [description, answer] of Object.entries(refused)) { + const transport = wire(() => answer as Record); + let outcome: unknown; + yield* scoped(function* () { + const connection = yield* useOwnerConnection(transport.socket); + const link = cloudflareRunLink(connection, ids(), RUN_ID); + outcome = yield* link.readExecutions(); + }); + expect([description, (outcome as { ok: boolean }).ok]).toEqual([description, false]); + if (!(outcome as { ok: boolean }).ok) { + const failed = outcome as { error: Error }; + // A provider-neutral failure, with nothing private in it. + expect([description, failed.error]).toEqual([ + description, + expect.any(WorkflowStorageError), + ]); + expect(String(failed.error)).not.toContain("command:"); + } + } + }); + + it("accepts a page filled to the byte bound", function* () { + // The boundary itself, from the runner's side: one page whose serialized + // rows land at or just under the bound is honest and is assembled. If the + // two ends measured different things, this is the page they would + // disagree about. + const fill = (size: number) => ({ + sequence: 1, + record: { executionId: "e".repeat(size), startedAt: "2026-09-04T00:00:00.000Z" }, + }); + // The identity is ASCII, so one byte of it is one byte of the page and the + // largest that fits follows from the wrapper's own size. + const overhead = executionPageBytes([fill(0)]); + const largest = fill(EXECUTION_PAGE_BYTES - overhead); + expect(executionPageBytes([largest])).toBe(EXECUTION_PAGE_BYTES); + expect(executionPageBytes([fill(EXECUTION_PAGE_BYTES - overhead + 1)])).toBeGreaterThan( + EXECUTION_PAGE_BYTES, + ); + + const transport = wire(() => ({ + outcome: "performed", + value: { runId: RUN_ID, anchor: 1, after: null, rows: [largest], done: true }, + })); + let outcome: unknown; + yield* scoped(function* () { + const connection = yield* useOwnerConnection(transport.socket); + const next = ids(); + outcome = yield* cloudflareRunLink(connection, next, RUN_ID).readExecutions(); + }); + const found = outcome as { ok: boolean; value: { executionId: string }[] }; + expect(found.ok).toBe(true); + expect(found.value.map((held) => held.executionId)).toEqual([largest.record.executionId]); + }); + + it("assembles an honest snapshot across pages, and an empty one", function* () { + const record = (id: string) => ({ executionId: id, startedAt: "2026-09-04T00:00:00.000Z" }); + const pages: Record> = { + null: { + outcome: "performed", + value: { + runId: RUN_ID, + anchor: 2, + after: null, + rows: [{ sequence: 1, record: record("first") }], + done: false, + }, + }, + "1": { + outcome: "performed", + value: { + runId: RUN_ID, + anchor: 2, + after: 1, + rows: [{ sequence: 2, record: record("second") }], + done: true, + }, + }, + }; + const transport = wire( + (request) => + pages[String(request["after"])] ?? { outcome: "refused", refusal: "storage:corrupt" }, + ); + yield* scoped(function* () { + const connection = yield* useOwnerConnection(transport.socket); + const link = cloudflareRunLink(connection, ids(), RUN_ID); + const read = yield* link.readExecutions(); + expect(read.ok).toBe(true); + if (read.ok) { + expect(read.value.map((entry) => entry.executionId)).toEqual(["first", "second"]); + } + }); + + const empty = wire(() => ({ + outcome: "performed", + value: { runId: RUN_ID, anchor: null, after: null, rows: [], done: true }, + })); + yield* scoped(function* () { + const connection = yield* useOwnerConnection(empty.socket); + const link = cloudflareRunLink(connection, ids(), RUN_ID); + const read = yield* link.readExecutions(); + expect(read.ok && read.value).toEqual([]); + }); + }); +}); diff --git a/packages/workflow/tests/remote-transaction.test.ts b/packages/workflow/tests/remote-transaction.test.ts new file mode 100644 index 000000000..1d19b4dd3 --- /dev/null +++ b/packages/workflow/tests/remote-transaction.test.ts @@ -0,0 +1,455 @@ +/** + * Tier WRH — `transact()` against an owner somewhere else. + * + * The contract is that arbitrary callback control flow stays legal while the + * commit stays atomic, and the way that is achieved is by never inferring what + * the body did: the body runs locally, and only what it enlisted is sent. So + * these tests are mostly about what does *not* travel — a body that suspends + * leaves no transaction open, a body that fails sends nothing, and a result the + * owner refused is not returned as a success. + * + * The link is a deterministic fake because Cloudflare mechanics are not the + * subject here. What the owner does with an intent is proven on real workerd; + * what the client sends is proven here. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { Err, Ok, sleep, spawn, withResolvers, type Operation, type Result } from "effection"; +import type { DurableEvent, DurableStream } from "@executablemd/durable-streams"; +import { + type CommitIntent, + createTransactionGate, + type OwnerLink, + RemoteTransactionError, + requireNoOpenTransaction, + type StartingFrontier, + transactRemotely, +} from "../src/remote/collector.ts"; +import type { CommitDecision } from "../src/remote/publication.ts"; + +/** + * What the transaction refused with, having proved it refused at all. + * + * A caught value is `unknown`; asserting it would let an unrelated failure read + * as the refusal a test expected. + */ +function refusalOf(error: unknown): string { + if (!(error instanceof RemoteTransactionError)) { + throw new Error(`expected a RemoteTransactionError, got ${String(error)}`); + } + return error.refusal; +} + +/** The name a test event carries, read rather than asserted. */ +function nameOf(entry: DurableEvent | undefined): string { + if (entry === undefined || !("description" in entry)) { + return ""; + } + const description = entry.description; + if (description === null || typeof description !== "object") { + return ""; + } + if (!("name" in description)) { + return ""; + } + const name: unknown = description["name"]; + return typeof name === "string" ? name : ""; +} + +/** + * The journal as an untrusted caller reaches it. + * + * The collector's job is to parse what it is handed, so a test that proves it + * refuses junk must be able to hand it junk. Widening the parameter is how that + * happens without manufacturing a value that claims to already be an event — + * asserting one would assert away the thing under test. + */ +function offering(journal: DurableStream): { append(event: unknown): Operation } { + return journal; +} + +/** Rename a test event in place, to prove the collector cloned it. */ +function rename(entry: DurableEvent, name: string): void { + if (!("description" in entry)) { + return; + } + const description = entry.description; + if (description !== null && typeof description === "object") { + Object.assign(description, { name }); + } +} + +function event(name: string): DurableEvent { + return { + type: "yield", + coroutineId: "root", + description: { type: "test", name }, + result: { status: "ok", value: name }, + }; +} + +/** What a correct owner would answer for this intent. */ +function decisionFor(intent: CommitIntent): CommitDecision { + return { + workspaceRootId: intent.publication?.proposedWorkspaceRootId ?? intent.expectedWorkspaceRootId, + journalEventIds: intent.events.map((_event, index) => `event-${index}`), + }; +} + +/** A test-supplied outcome, carried through with the decision it implies. */ +function mapDecision(result: Result, intent: CommitIntent): Result { + return result.ok ? Ok(decisionFor(intent)) : result; +} + +/** A link that records what it was asked, and answers how a test tells it to. */ +function link( + options: { + frontier?: StartingFrontier; + commit?: (intent: CommitIntent) => Result; + blockFrontier?: { operation: Operation }; + blockCommit?: { operation: Operation }; + } = {}, +) { + const sent: CommitIntent[] = []; + const starting: StartingFrontier = options.frontier ?? { + workspaceRootId: "root-a", + journalEventId: "event-0", + events: [event("already-there")], + }; + const owner: OwnerLink = { + *frontier(): Operation { + if (options.blockFrontier !== undefined) { + yield* options.blockFrontier.operation; + } + return starting; + }, + *commit(intent: CommitIntent): Operation> { + sent.push(intent); + if (options.blockCommit !== undefined) { + yield* options.blockCommit.operation; + } + return options.commit === undefined + ? Ok(decisionFor(intent)) + : mapDecision(options.commit(intent), intent); + }, + }; + return { owner, sent, starting }; +} + +describe("a remote transaction", () => { + it("sends one intent carrying what the body enlisted", function* () { + const { owner, sent } = link(); + const gate = createTransactionGate(); + + const result = yield* transactRemotely(owner, gate, function* (transaction) { + yield* transaction.journal.append(event("one")); + yield* transaction.journal.append(event("two")); + return "body value"; + }); + + expect(result.ok).toBe(true); + expect(result.ok && result.value).toBe("body value"); + expect(sent).toHaveLength(1); + expect(sent[0]?.expectedWorkspaceRootId).toBe("root-a"); + expect(sent[0]?.expectedJournalEventId).toBe("event-0"); + expect(sent[0]?.events).toHaveLength(2); + }); + + it("reads the starting prefix and its own appends, in order", function* () { + const { owner } = link(); + const gate = createTransactionGate(); + let seen: string[] = []; + + yield* transactRemotely(owner, gate, function* (transaction) { + yield* transaction.journal.append(event("mine")); + const all = yield* transaction.journal.readAll(); + seen = all.map(nameOf); + return undefined; + }); + + expect(seen).toEqual(["already-there", "mine"]); + }); + + it("lets the body cross a suspension point with no owner transaction open", function* () { + const { owner, sent } = link(); + const gate = createTransactionGate(); + + const result = yield* transactRemotely(owner, gate, function* (transaction) { + yield* transaction.journal.append(event("before")); + // Nothing is held on the owner while this waits, which is the whole + // reason the body runs here rather than inside a transaction. + yield* sleep(1); + yield* transaction.journal.append(event("after")); + return "crossed"; + }); + + expect(result.ok && result.value).toBe("crossed"); + expect(sent).toHaveLength(1); + expect(sent[0]?.events).toHaveLength(2); + }); + + it("sends nothing when the body fails", function* () { + const { owner, sent } = link(); + const gate = createTransactionGate(); + + let raised: unknown; + try { + yield* transactRemotely(owner, gate, function* (transaction) { + yield* transaction.journal.append(event("doomed")); + throw new Error("the body decided otherwise"); + }); + } catch (error) { + raised = error; + } + + expect(String(raised)).toContain("the body decided otherwise"); + expect(sent).toEqual([]); + expect(gate.open).toBe(false); + }); + + it("returns the owner's refusal rather than the body's value", function* () { + const { owner, sent } = link({ commit: () => Err(new Error("stale expected root")) }); + const gate = createTransactionGate(); + + const result = yield* transactRemotely(owner, gate, function* (transaction) { + yield* transaction.journal.append(event("hopeful")); + return "never returned"; + }); + + expect(result.ok).toBe(false); + expect(!result.ok && String(result.error)).toContain("stale expected root"); + expect(sent).toHaveLength(1); + }); + + it("refuses a transaction opened inside a transaction", function* () { + const { owner, sent } = link(); + const gate = createTransactionGate(); + + let raised: unknown; + try { + yield* transactRemotely(owner, gate, function* () { + yield* transactRemotely(owner, gate, function* () { + return undefined; + }); + return undefined; + }); + } catch (error) { + raised = error; + } + + expect(raised).toBeInstanceOf(RemoteTransactionError); + expect(refusalOf(raised)).toBe("nested-transaction"); + expect(sent).toEqual([]); + }); + + it("refuses an ordinary same-handle operation while a body is running", function* () { + const { owner } = link(); + const gate = createTransactionGate(); + let raised: unknown; + + yield* transactRemotely(owner, gate, function* () { + try { + requireNoOpenTransaction(gate); + } catch (error) { + raised = error; + } + return undefined; + }); + + expect(raised).toBeInstanceOf(RemoteTransactionError); + expect(refusalOf(raised)).toBe("operation-inside-body"); + // And the gate is closed again afterwards, so the next operation is fine. + requireNoOpenTransaction(gate); + }); + + it("refuses a transaction handle used after its body closed", function* () { + const { owner } = link(); + const gate = createTransactionGate(); + let escaped: { journal: { append(event: DurableEvent): Operation } } | undefined; + + yield* transactRemotely(owner, gate, function* (transaction) { + escaped = transaction; + return undefined; + }); + + let raised: unknown; + try { + yield* escaped!.journal.append(event("too late")); + } catch (error) { + raised = error; + } + expect(refusalOf(raised)).toBe("transaction-closed"); + }); + + it("owns the handle from before the first suspension until after the commit", function* () { + const held = withResolvers(); + const { owner, sent } = link({ blockFrontier: held }); + const gate = createTransactionGate(); + + const first = yield* spawn(() => + transactRemotely(owner, gate, function* () { + return "first"; + }), + ); + yield* sleep(0); + + // The first transaction is suspended inside `frontier()`. A second must not + // pass the gate and act from the same starting frontier. + let raised: unknown; + try { + yield* transactRemotely(owner, gate, function* () { + return "second"; + }); + } catch (error) { + raised = error; + } + expect(refusalOf(raised)).toBe("nested-transaction"); + + held.resolve(); + yield* first; + expect(sent).toHaveLength(1); + }); + + it("keeps the handle while the commit is still undecided", function* () { + const held = withResolvers(); + const { owner } = link({ blockCommit: held }); + const gate = createTransactionGate(); + + const first = yield* spawn(() => + transactRemotely(owner, gate, function* (transaction) { + yield* transaction.journal.append(event("one")); + return "first"; + }), + ); + yield* sleep(0); + + // The body has finished, but which state won is not yet established. + expect(gate.open).toBe(true); + let raised: unknown; + try { + requireNoOpenTransaction(gate); + } catch (error) { + raised = error; + } + expect(refusalOf(raised)).toBe("operation-inside-body"); + + held.resolve(); + yield* first; + expect(gate.open).toBe(false); + }); + + it("releases the handle however the transaction ends", function* () { + const gate = createTransactionGate(); + + const succeeded = link(); + yield* transactRemotely(succeeded.owner, gate, function* () { + return undefined; + }); + expect(gate.open).toBe(false); + + const refused = link({ commit: () => Err(new Error("refused")) }); + yield* transactRemotely(refused.owner, gate, function* () { + return undefined; + }); + expect(gate.open).toBe(false); + + const failed = link(); + try { + yield* transactRemotely(failed.owner, gate, function* () { + throw new Error("body failed"); + }); + } catch { + // The refusal is the subject of another test; this one is about the gate. + } + expect(gate.open).toBe(false); + + const broken: OwnerLink = { + *frontier(): Operation { + throw new Error("transport failed"); + }, + *commit(): Operation> { + return Ok({ workspaceRootId: "root-a", journalEventIds: [] }); + }, + }; + try { + yield* transactRemotely(broken, gate, function* () { + return undefined; + }); + } catch { + // Likewise. + } + expect(gate.open).toBe(false); + }); + + it("refuses an event it cannot admit, and sends nothing", function* () { + const { owner, sent } = link(); + const gate = createTransactionGate(); + + let raised: unknown; + try { + yield* transactRemotely(owner, gate, function* (transaction) { + yield* offering(transaction.journal).append({ nothing: true }); + return undefined; + }); + } catch (error) { + raised = error; + } + expect(refusalOf(raised)).toBe("malformed-event"); + expect(sent).toEqual([]); + }); + + it("refuses more bytes than one intent may carry", function* () { + const { owner, sent } = link(); + const gate = createTransactionGate(); + const wide = event("x".repeat(200_000)); + + let raised: unknown; + try { + yield* transactRemotely(owner, gate, function* (transaction) { + for (let index = 0; index < 40; index += 1) { + yield* transaction.journal.append(wide); + } + return undefined; + }); + } catch (error) { + raised = error; + } + expect(refusalOf(raised)).toBe("events-too-large"); + expect(sent).toEqual([]); + }); + + it("commits what it admitted, not what a reader mutated afterwards", function* () { + const { owner, sent } = link(); + const gate = createTransactionGate(); + + yield* transactRemotely(owner, gate, function* (transaction) { + yield* transaction.journal.append(event("admitted")); + // Read it back and edit what came out. The collector handed over a copy, + // so the intent still carries what `append()` admitted. + const read = yield* transaction.journal.readAll(); + const mine = read[read.length - 1]; + if (mine !== undefined) { + rename(mine, "changed by a reader"); + } + return undefined; + }); + + expect(nameOf(sent[0]?.events[0])).toBe("admitted"); + }); + + it("commits what it was handed, not what the caller mutated afterwards", function* () { + const { owner, sent } = link(); + const gate = createTransactionGate(); + const mutable = event("original"); + + yield* transactRemotely(owner, gate, function* (transaction) { + yield* transaction.journal.append(mutable); + rename(mutable, "changed"); + return undefined; + }); + + const committed = sent[0]?.events[0]; + expect(nameOf(committed)).toBe("original"); + }); +}); diff --git a/packages/workflow/tests/remote-workspace-files.test.ts b/packages/workflow/tests/remote-workspace-files.test.ts new file mode 100644 index 000000000..221280586 --- /dev/null +++ b/packages/workflow/tests/remote-workspace-files.test.ts @@ -0,0 +1,181 @@ +/** + * Tier WRH — what the runner's Workspace filesystem will act on. + * + * The attempt is a real directory on a host that has an outside, and a symbolic + * link is a path the kernel follows on its own. So these use real temporary + * files and the production adapter: a fake filesystem would follow whatever the + * fake decided to follow, which is the one thing under test. + * + * The rule is the host provider's, stated in `packages/runtime/host-files.ts`: + * a complete `..` segment leaves and `..notes.md` does not; an operation about + * a link does not follow it; and a link's target is a Workspace path, so an + * absolute one names the Workspace root rather than the machine's. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { type Operation, until } from "effection"; +import { mkdir, readFile, symlink, writeFile } from "node:fs/promises"; +import { useRunnerTrees } from "../src/deno/remote-files.ts"; +import { createRemoteWorkspaceFilesystem } from "../src/deno/remote-workspace-files.ts"; +import type { WorkspaceFilesystem } from "../src/workspace/filesystem.ts"; + +const SECRET = "the file outside\n"; + +interface Scene { + readonly files: WorkspaceFilesystem; + readonly attempt: string; + readonly outside: string; +} + +/** + * An attempt directory, and a separate directory it must never reach. + * + * Both are real, and the outside one holds a file whose bytes are recognizable: + * an escape that succeeded would return exactly them. + */ +function* scene(): Operation { + const trees = yield* useRunnerTrees(); + const attempt = yield* trees.create("attempt"); + const outside = yield* trees.create("outside"); + yield* until(writeFile(`${outside}/secret.txt`, SECRET, { mode: 0o644 })); + yield* until(mkdir(`${attempt}/docs`, { mode: 0o755 })); + yield* until(writeFile(`${attempt}/docs/inside.txt`, "inside\n", { mode: 0o644 })); + const files = createRemoteWorkspaceFilesystem( + (logical) => (logical === "/" ? attempt : `${attempt}${logical}`), + () => {}, + ); + return { files, attempt, outside }; +} + +/** What an operation refused with, having proved it refused at all. */ +function* refusal(operation: Operation): Operation { + try { + yield* operation; + return "it was allowed"; + } catch (error) { + return String(error); + } +} + +describe("the runner's Workspace filesystem", () => { + it("follows a link inside the attempt, and leaves it a link", function* () { + const { files, attempt } = yield* scene(); + yield* until(symlink("docs/inside.txt", `${attempt}/here`)); + + expect(yield* files.readTextFile("/here")).toBe("inside\n"); + // The contract of each: one is about the file, the other about the entry. + expect((yield* files.stat("/here")).kind).toBe("file"); + expect((yield* files.lstat("/here")).kind).toBe("symlink"); + expect(yield* files.readlink("/here")).toBe("docs/inside.txt"); + + yield* files.writeFile("/here", "through the link\n", 0o644); + // The file the link names changed; the link is still a link. + expect(yield* until(readFile(`${attempt}/docs/inside.txt`, "utf8"))).toBe("through the link\n"); + expect((yield* files.lstat("/here")).kind).toBe("symlink"); + }); + + it("reads a Workspace-absolute link target against the attempt, not the host", function* () { + const { files, attempt } = yield* scene(); + // The target is a Workspace path. Interpreted by the kernel it would be a + // machine path; interpreted here it is this attempt's own `/docs`. + yield* until(symlink("/docs/inside.txt", `${attempt}/logical`)); + expect(yield* files.readTextFile("/logical")).toBe("inside\n"); + }); + + it("exposes nothing through a final link that leaves the attempt", function* () { + const { files, attempt, outside } = yield* scene(); + yield* until(symlink(`${outside}/secret.txt`, `${attempt}/escape`)); + yield* until(symlink("../../../../etc/hosts", `${attempt}/relative`)); + + for (const path of ["/escape", "/relative"]) { + // The host-absolute target is a Workspace path here, so it names nothing; + // the relative one climbs out of the tree and is refused. Neither is a + // way to the bytes, which is the claim. + expect([path, yield* refusal(files.readTextFile(path))]).not.toEqual([ + path, + "it was allowed", + ]); + expect(yield* refusal(files.readTextFile(path))).not.toContain("the file outside"); + yield* refusal(files.writeFile(path, "overwritten\n")); + yield* refusal(files.chmod(path, 0o600)); + } + // Neither the outside file nor the link it went through changed. + expect(yield* until(readFile(`${outside}/secret.txt`, "utf8"))).toBe(SECRET); + expect(yield* files.readlink("/escape")).toBe(`${outside}/secret.txt`); + }); + + it("reaches nothing through an ancestor link that leaves the attempt", function* () { + const { files, attempt, outside } = yield* scene(); + yield* until(symlink(outside, `${attempt}/door`)); + yield* until(symlink("../..", `${attempt}/up`)); + + for (const path of ["/door/secret.txt", "/up/anything"]) { + expect(yield* refusal(files.readTextFile(path))).not.toContain("the file outside"); + yield* refusal(files.writeFile(path, "created\n")); + yield* refusal(files.mkdir(path, { recursive: true })); + yield* refusal(files.remove(path)); + yield* refusal(files.chmod(path, 0o600)); + yield* refusal(files.rename("/docs/inside.txt", path)); + yield* refusal(files.link("/docs/inside.txt", path)); + } + expect(yield* until(readFile(`${outside}/secret.txt`, "utf8"))).toBe(SECRET); + // And the file that was there to move is still where it was. + expect(yield* files.readTextFile("/docs/inside.txt")).toBe("inside\n"); + }); + + it("contains both ends of a rename and a hardlink", function* () { + const { files, attempt, outside } = yield* scene(); + yield* until(symlink(outside, `${attempt}/door`)); + + expect(yield* refusal(files.rename("/docs/inside.txt", "/../moved"))).toContain( + "outside the tree", + ); + expect(yield* refusal(files.link("/docs/inside.txt", "/../linked"))).toContain( + "outside the tree", + ); + yield* refusal(files.rename("/door/secret.txt", "/taken")); + yield* refusal(files.link("/door/secret.txt", "/taken")); + // Nothing arrived, and nothing left. + expect(yield* refusal(files.readTextFile("/taken"))).not.toContain("the file outside"); + expect(yield* files.readTextFile("/docs/inside.txt")).toBe("inside\n"); + }); + + it("does not turn a dangling outward link into a way to write outside", function* () { + const { files, attempt, outside } = yield* scene(); + yield* until(symlink(`${outside}/absent.txt`, `${attempt}/dangling`)); + yield* refusal(files.writeFile("/dangling", "created outside\n")); + // Nothing was created where the link pointed. + expect(yield* refusal(until(readFile(`${outside}/absent.txt`, "utf8")))).toContain("ENOENT"); + // The link is still exactly what it was. + expect(yield* files.readlink("/dangling")).toBe(`${outside}/absent.txt`); + }); + + it("admits an ordinary name beginning with two dots, and refuses a whole segment", function* () { + const { files } = yield* scene(); + yield* files.writeFile("/..notes.md", "two dots is a name\n", 0o644); + expect(yield* files.readTextFile("/..notes.md")).toBe("two dots is a name\n"); + expect(yield* files.readTextFile("/docs/../..notes.md")).toBe("two dots is a name\n"); + + for (const path of ["/..", "/../escaped", "/docs/../../escaped", ""]) { + expect([path, yield* refusal(files.writeFile(path, "no"))]).toEqual([ + path, + expect.stringContaining("outside the tree this invocation owns"), + ]); + } + }); + + it("says nothing about the host in what it refuses with", function* () { + const { files, attempt, outside } = yield* scene(); + yield* until(symlink(`${outside}/secret.txt`, `${attempt}/escape`)); + const reported = [ + yield* refusal(files.readTextFile("/escape")), + yield* refusal(files.readTextFile("/../escaped")), + yield* refusal(files.readTextFile("/docs/absent.txt")), + ].join("\n"); + // Not where this invocation put its tree, and not where a link pointed. + expect(reported).not.toContain(attempt); + expect(reported).not.toContain(outside); + expect(reported).not.toContain("secret.txt"); + }); +}); diff --git a/packages/workflow/tests/remote-workspace.test.ts b/packages/workflow/tests/remote-workspace.test.ts new file mode 100644 index 000000000..341dcb4b6 --- /dev/null +++ b/packages/workflow/tests/remote-workspace.test.ts @@ -0,0 +1,922 @@ +/** + * Tier WRH — the runner's Workspace coordinator, end to end. + * + * What this is about is ordering and authority, not arithmetic. The Files are + * real: a documented failure has to leave a directory that recaptures to the + * root it started from, and a fake cannot settle that. The owner is a scripted + * connection, because what crosses it here is what the coordinator decided — + * whether the atomic commit is really atomic is proved on real workerd, where + * atomicity is real. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import type { DurableStream } from "@executablemd/durable-streams"; +import { + durableRun, + establishJournalProvenance, + InMemoryStream, + type DurableEvent, + type Json, + type JournalProvenance, + type Workflow, +} from "@executablemd/durable-streams"; +import { type Operation, scoped, sleep, spawn, until } from "effection"; +import { mkdir, readFile, writeFile } from "node:fs/promises"; +import { runnerFiles, useRunnerTrees } from "../src/deno/remote-files.ts"; +import { encodeBase64 } from "../src/cloudflare/encoding.ts"; +import { createRemoteWorkspaceFilesystem } from "../src/deno/remote-workspace-files.ts"; +import { captureWorkspace, type CapturedWorkspace } from "../src/remote/materialize.ts"; +import type { RemoteContent, RemoteContentRequest, RemoteReadLink } from "../src/remote/read.ts"; +import type { RemoteFrontierSnapshot } from "../src/remote/read.ts"; +import type { RemoteInvocationSnapshot } from "../src/remote/records.ts"; +import { useRemoteRunDatabase } from "../src/remote/database.ts"; +import { cloudflareRunLink, cloudflareReadLink } from "../src/cloudflare/client.ts"; +import { type OwnerSocket, type SocketListener, useOwnerConnection } from "../src/remote/client.ts"; +import type { WorkspaceFilesystem } from "../src/workspace/filesystem.ts"; +import type { WorkspaceMetadata } from "../src/workspace/metadata.ts"; +import { + createRemoteWorkspaceEffect, + type RemoteRun, + type RemoteRunOptions, + type RemoteWorkspaceMutation, + useRemoteRun, + type RemoteWorkspaceRuntime, + useRemoteWorkspaceEffects, + withRemoteWorkspaceEffects, +} from "../src/remote/workspace.ts"; +import { routeRemoteRunJournal } from "../src/remote/journal-route.ts"; +import { createInvocationMappings } from "../src/remote/mappings.ts"; +import { JournaledEffectFailure } from "../src/workspace/failure.ts"; +import { parseWorkspaceRootManifest } from "../src/workspace/root-manifest.ts"; +import { locatorFingerprintOf } from "../src/composition/locator.ts"; +import { agentSessionKey, resolveAgentSession } from "../src/storage/agent-session.ts"; +import type { WorkflowRunDatabase } from "../src/storage/api.ts"; + +const RUN_ID = "remote-run"; +const LOCATOR = "https://git.example.invalid/octo/app.git"; + +function reject(reason: string): never { + throw new Error(reason); +} + +/** A refusal the effect publishes rather than raises, as a document's would be. */ +class DocumentedFailure extends JournaledEffectFailure { + override name = "DocumentedFailure"; +} + +function runRecord() { + return { + runId: RUN_ID, + definition: { + version: 1, + kind: "git", + objectFormat: "sha1", + objectId: "0".repeat(40), + rootDocumentPath: "README.md", + }, + base: "main", + props: {}, + status: "running", + createdAt: "2026-09-03T00:00:00.000Z", + updatedAt: "2026-09-03T00:00:00.000Z", + }; +} + +function repository(name = "app") { + return { + record: { + name, + locatorFingerprint: locatorFingerprintOf(LOCATOR), + requestedBase: null, + creationCommit: "9".repeat(40), + primaryBranch: "main", + objectFormat: "sha1" as const, + checkoutPath: `/${name}`, + }, + locator: LOCATOR, + }; +} + +function emptySnapshot(workspaceRootId: string): RemoteInvocationSnapshot { + return { + workspaceRootId, + journalEventId: null, + repositories: [], + worktrees: [], + agentSessions: [], + }; +} + +/** A connection whose answers a test writes, and which records what it was sent. */ +function wire(answer: (request: Record) => Record) { + const sent: Record[] = []; + const listeners = new Map>(); + const socket: OwnerSocket = { + send(data: string): void { + const request = JSON.parse(data) as Record; + sent.push(request); + const response = answer(request); + if (response["outcome"] === "lost") { + // The connection went while the answer was in flight. + for (const listener of listeners.get("close") ?? []) { + listener({}); + } + return; + } + for (const listener of listeners.get("message") ?? []) { + listener({ data: JSON.stringify({ id: request["id"], ...response }) }); + } + }, + close(): void {}, + addEventListener(type, listener): void { + const found = listeners.get(type) ?? new Set(); + found.add(listener); + listeners.set(type, found); + }, + removeEventListener(type, listener): void { + listeners.get(type)?.delete(listener); + }, + }; + return { socket, sent }; +} + +/** The owner's answers for one starting tree, and what it was asked to commit. */ +function ownerOf( + captured: CapturedWorkspace, + snapshot: () => RemoteInvocationSnapshot | { refused: string }, +) { + const commits: Record[] = []; + let refusal: string | undefined; + let lost = false; + return { + commits, + refuse(reason: string): void { + refusal = reason; + }, + lose(): void { + lost = true; + }, + get lost(): boolean { + return lost; + }, + answer(request: Record): Record { + const command = request["command"]; + if (command === "mappings") { + const value = snapshot(); + return "refused" in value + ? { outcome: "performed", value } + : { outcome: "performed", value }; + } + if (command === "frontier") { + return { + outcome: "performed", + value: { + record: runRecord(), + retrieval: null, + workspaceRootId: captured.root.rootId, + journalEventId: null, + }, + }; + } + if (command === "root") { + return { + outcome: "performed", + value: { + workspaceRootId: captured.root.rootId, + manifest: captured.root.manifest, + }, + }; + } + if (command === "content") { + const digest = String(request["digest"]); + const bytes = + request["kind"] === "manifest" + ? captured.contents.get(digest)?.manifestBytes + : captured.blobs.get(digest); + if (bytes === undefined) { + throw new Error("asked for content this owner does not hold"); + } + return { + outcome: "performed", + value: { + kind: request["kind"], + digest, + size: bytes.length, + bytes: encodeBase64(bytes), + }, + }; + } + if (command === "stage") { + const encoded = String(request["bytes"] ?? ""); + const padding = encoded.endsWith("==") ? 2 : encoded.endsWith("=") ? 1 : 0; + return { + outcome: "performed", + value: { + kind: request["kind"], + digest: request["digest"], + size: (encoded.length / 4) * 3 - padding, + }, + }; + } + commits.push(request); + if (lost) { + return { outcome: "lost" }; + } + if (refusal !== undefined) { + return { outcome: "refused", refusal }; + } + const publication = request["publication"]; + const events = Array.isArray(request["events"]) ? request["events"] : []; + return { + outcome: "performed", + value: { + workspaceRootId: + publication === null || publication === undefined + ? request["expectedWorkspaceRootId"] + : (publication as Record)["proposedWorkspaceRootId"], + journalEventIds: events.map((_entry, index) => `event-${index}`), + }, + }; + }, + }; +} + +/** A small starting tree, captured so a scripted owner can serve it. */ +function* startingTree(): Operation { + const files = runnerFiles(); + const trees = yield* useRunnerTrees(); + const root = yield* trees.create("source"); + yield* until(writeFile(`${root}/README.md`, "starting\n", { mode: 0o644 })); + yield* until(mkdir(`${root}/docs`, { mode: 0o755 })); + return yield* captureWorkspace( + files, + (logical) => (logical === "/" ? root : `${root}${logical}`), + reject, + ); +} + +/** + * The constructor takes one link and no separate read input. + * + * A type-level assertion rather than a runtime one, because that is where the + * property lives: adding a `reads` member back to `RemoteRunOptions` — the + * shape this correction removed — stops this file compiling. + */ +type NoSeparateReads = "reads" extends keyof RemoteRunOptions ? never : true; +const ONE_LINK: NoSeparateReads = true; + +/** + * One scripted owner and a live connection to it, with nothing built on top. + * + * The harness below opens a binding; this stops short of that, so a test can + * hold two owners and ask what each one was actually sent. + */ +function* owner(captured: CapturedWorkspace) { + const scripted = ownerOf(captured, () => emptySnapshot(captured.root.rootId)); + const requests: Record[] = []; + const transport = wire((request) => { + requests.push(request); + return scripted.answer(request); + }); + const connection = yield* useOwnerConnection(transport.socket); + let identifier = 0; + return { + connection, + requests, + next: () => `owner-${(identifier += 1)}`, + commits: scripted.commits, + journal: new InMemoryStream(), + }; +} + +interface Harness { + readonly run: RemoteRun; + readonly commits: Record[]; + refuse(reason: string): void; + lose(): void; + readonly sent: Record[]; + readonly captured: CapturedWorkspace; +} + +/** + * Everything one remote invocation needs, wired the way a host would wire it. + * + * Deliberately the production pieces: the real client over a scripted socket, + * the real database handle, the real coordinator and the real native adapters. + * A test that assembled a simpler stand-in would prove that the stand-in works. + */ +function* harness( + snapshot: (rootId: string) => RemoteInvocationSnapshot | { refused: string } = emptySnapshot, + shared?: CapturedWorkspace, +): Operation { + const captured = shared ?? (yield* startingTree()); + const owner = ownerOf(captured, () => snapshot(captured.root.rootId)); + const transport = wire((request) => owner.answer(request)); + const connection = yield* useOwnerConnection(transport.socket); + let identifier = 0; + const next = () => `request-${(identifier += 1)}`; + // The production constructor: one link, and the handle, the routed journal + // and the provenance made together from it. + const run = yield* useRemoteRun({ + link: cloudflareRunLink(connection, next, RUN_ID), + files: runnerFiles(), + trees: yield* useRunnerTrees(), + createFilesystem: (at, authorize) => createRemoteWorkspaceFilesystem(at, authorize), + journal: new InMemoryStream(), + }); + return { + run, + commits: owner.commits, + refuse: owner.refuse, + lose: owner.lose, + sent: transport.sent, + captured, + }; +} + +/** + * One invocation, with its own journal and its own provenance. + * + * Separate per call because that is what a host does: a run's journal is + * established once per live session, and an invocation that reused another + * one's would be publishing into a journal it does not belong to. It also lets + * a later invocation observe what an earlier one left — which is the only way + * to see the accepted Workspace, since the coordinator owns its trees and + * removes them when it is done. + */ +function* invocation( + held: Harness, + name: string, + mutate: RemoteWorkspaceMutation, +): Operation<{ raised: unknown; events: DurableEvent[] }> { + return yield* scoped(function* () { + yield* useRemoteWorkspaceEffects(held.run); + const effect = createRemoteWorkspaceEffect(held.run, { type: "workspace", name }, mutate); + function* workflow(): Workflow { + yield effect; + } + const raised = yield* trapped( + withRemoteWorkspaceEffects(held.run, durableRun(workflow, { stream: held.run.journal })), + ); + return { raised, events: yield* held.run.journal.readAll() }; + }); +} + +/** A mutation that touches nothing: these tests are about who may run one. */ +// deno-lint-ignore require-yield +function* own(): Operation { + return "ran"; +} + +function yielded(events: readonly DurableEvent[]): DurableEvent[] { + return events.filter((event) => event.type === "yield"); +} + +describe("the runner's Workspace coordinator", () => { + it("commits Files, one mapping and the filtered result as one intent", function* () { + yield* scoped(function* () { + const held = yield* harness(); + const { raised, events } = yield* invocation( + held, + "write", + function* (filesystem, metadata): Operation { + yield* filesystem.writeFile("/NOTES.md", "written by the effect\n", 0o644); + yield* filesystem.mkdir("/app", { mode: 0o755 }); + metadata.insertRepository(repository()); + // Read-your-writes: its own insert, before anything is committed. + return metadata.readRepository("app")?.record.checkoutPath ?? "missing"; + }, + ); + expect(raised).toBe(undefined); + + // Exactly one intent, carrying all three things together. + expect(held.commits).toHaveLength(1); + const intent = held.commits[0] ?? {}; + expect(intent["expectedWorkspaceRootId"]).toBe(held.captured.root.rootId); + const mappings = intent["mappings"]; + expect(Array.isArray(mappings) && mappings).toHaveLength(1); + expect((mappings as Record[])[0]?.["kind"]).toBe("repository"); + expect(intent["publication"]).not.toBe(null); + // The result travelled in this same intent rather than through the + // ordinary journal, so nothing was written before the owner agreed. + expect(Array.isArray(intent["events"]) && intent["events"]).toHaveLength(1); + expect(yielded(events)).toHaveLength(0); + }); + }); + + it("journals a documented failure against the unchanged root, and keeps nothing", function* () { + yield* scoped(function* () { + const held = yield* harness(); + const { raised } = yield* invocation( + held, + "refuse", + function* (filesystem, metadata): Operation { + yield* filesystem.writeFile("/SCRATCH.md", "discarded\n", 0o644); + yield* filesystem.remove("/README.md"); + metadata.insertRepository(repository()); + throw new DocumentedFailure("this Workspace effect refused"); + }, + ); + expect(String(raised)).toContain("this Workspace effect refused"); + + // One commit, and it proposes nothing about the Workspace. + expect(held.commits).toHaveLength(1); + const intent = held.commits[0] ?? {}; + expect(intent["publication"]).toBe(null); + expect(intent["mappings"]).toEqual([]); + expect(intent["expectedWorkspaceRootId"]).toBe(held.captured.root.rootId); + expect(Array.isArray(intent["events"]) && intent["events"]).toHaveLength(1); + + // That the owner still holds the starting root after this is a claim + // about storage, and it is made against real owner storage in + // `remote-workspace.vitest.ts`. What is settled here is that nothing was + // proposed: no publication, no mapping, and the root this commit expected + // is the one the invocation was admitted from. + }); + }); + + it("prevents the document from running when the admitted root is unreachable", function* () { + yield* scoped(function* () { + const held = yield* harness(() => emptySnapshot("f".repeat(64))); + let executed = 0; + // deno-lint-ignore require-yield + yield* invocation(held, "unreachable", function* (): Operation { + executed += 1; + return "ran"; + }); + // Materialization is before the transaction, so this never reaches the + // anchor check — and it must still leave the run exactly as it was. + expect(executed).toBe(0); + expect(held.commits).toEqual([]); + }); + }); + + it("refuses before the document runs when the run moved since admission", function* () { + yield* scoped(function* () { + // The root still materializes, so the invocation gets all the way to the + // transaction; the journal anchor is what has moved. Nothing later could + // notice on its own — both answers were true when they were given. + const held = yield* harness((rootId) => ({ + ...emptySnapshot(rootId), + journalEventId: "event-from-another-moment", + })); + let executed = 0; + // deno-lint-ignore require-yield + const { raised } = yield* invocation(held, "drifted", function* (): Operation { + executed += 1; + return "ran"; + }); + expect(String(raised)).toContain("moved past"); + expect(executed).toBe(0); + expect(held.commits).toEqual([]); + }); + }); + + it("leaves the accepted Workspace alone when the owner refuses the commit", function* () { + yield* scoped(function* () { + const held = yield* harness(); + held.refuse("command:stale-root"); + const { raised } = yield* invocation( + held, + "refused", + function* (filesystem): Operation { + yield* filesystem.writeFile("/NOTES.md", "written by the effect\n", 0o644); + return "ran"; + }, + ); + expect(raised).not.toBe(undefined); + // The owner said no, so nothing is promoted and nothing private crossed. + expect(String(raised)).not.toContain("command:"); + expect(held.commits).toHaveLength(1); + }); + }); + + it("cannot pair one run's handle with another run's link, journal or provenance", function* () { + yield* scoped(function* () { + // Two owners, deliberately begun from the same root and the same empty + // journal. Every structural value they hold is equal; only the objects + // differ, and only the objects decide. + const tree = yield* startingTree(); + const a = yield* harness(emptySnapshot, tree); + const b = yield* harness(emptySnapshot, tree); + expect(a.run.database.record.runId).toBe(b.run.database.record.runId); + + // An effect made against A, coordinated under B. + yield* scoped(function* () { + yield* useRemoteWorkspaceEffects(b.run); + const effect = createRemoteWorkspaceEffect(a.run, { type: "workspace", name: "a" }, own); + function* workflow(): Workflow { + yield effect; + } + const raised = yield* trapped( + withRemoteWorkspaceEffects(b.run, durableRun(workflow, { stream: b.run.journal })), + ); + expect(String(raised)).toContain("foreign"); + }); + + // A's coordinator installed, B's binding asked to use it. + yield* scoped(function* () { + yield* useRemoteWorkspaceEffects(a.run); + const effect = createRemoteWorkspaceEffect(b.run, { type: "workspace", name: "b" }, own); + function* workflow(): Workflow { + yield effect; + } + const raised = yield* trapped( + withRemoteWorkspaceEffects(b.run, durableRun(workflow, { stream: b.run.journal })), + ); + expect(String(raised)).toContain("no remote Workspace coordinator is installed"); + }); + + // B throughout, running over A's journal. The provenance is A's. + yield* scoped(function* () { + yield* useRemoteWorkspaceEffects(b.run); + const effect = createRemoteWorkspaceEffect(b.run, { type: "workspace", name: "c" }, own); + function* workflow(): Workflow { + yield effect; + } + const raised = yield* trapped( + withRemoteWorkspaceEffects(b.run, durableRun(workflow, { stream: a.run.journal })), + ); + expect(String(raised)).toContain("provenance"); + }); + + // A value shaped like a binding is not one. + const forged = { database: b.run.database, journal: b.run.journal }; + expect( + String(yield* trapped(useRemoteWorkspaceEffects(forged as unknown as RemoteRun))), + ).toContain("not a remote run this build opened"); + + // Neither owner was asked for anything, and neither journal moved. + expect([a.commits, b.commits]).toEqual([[], []]); + expect(yielded(yield* a.run.journal.readAll())).toEqual([]); + expect(yielded(yield* b.run.journal.readAll())).toEqual([]); + }); + }); + + it("cannot be opened from one owner's reads and another owner's commits", function* () { + yield* scoped(function* () { + // The construction the correction closes. Two owners, deliberately begun + // from one captured tree, so their root, anchor and run record are equal + // and only the objects differ. Before this, `useRemoteRun` took the + // database/commit link and the Workspace read link separately, and this + // combination produced a legitimate binding: the invocation would be + // admitted from A's mappings and content and commit its result to B. + const tree = yield* startingTree(); + const a = yield* owner(tree); + const b = yield* owner(tree); + + // Each link is built from one connection and carries its own reads, so + // reading through one reaches that owner and no other. + const linkA = cloudflareRunLink(a.connection, a.next, RUN_ID); + const linkB = cloudflareRunLink(b.connection, b.next, RUN_ID); + yield* linkA.invocationSnapshot(); + expect(a.requests.map((request) => request["command"])).toEqual(["mappings"]); + expect(b.requests).toEqual([]); + + expect(ONE_LINK).toBe(true); + const options: RemoteRunOptions = { + link: linkB, + files: runnerFiles(), + trees: yield* useRunnerTrees(), + createFilesystem: (at, authorize) => createRemoteWorkspaceFilesystem(at, authorize), + journal: new InMemoryStream(), + }; + + let executed = 0; + const run = yield* useRemoteRun(options); + yield* useRemoteWorkspaceEffects(run); + const effect = createRemoteWorkspaceEffect( + run, + { type: "workspace", name: "one-owner" }, + function* (filesystem): Operation { + executed += 1; + yield* filesystem.writeFile("/NOTES.md", "written by the effect\n", 0o644); + return "ran"; + }, + ); + function* workflow(): Workflow { + yield effect; + } + yield* withRemoteWorkspaceEffects(run, durableRun(workflow, { stream: run.journal })); + + // Everything the invocation read and everything it committed went to B. + // A answered the one snapshot this test asked it for directly, and + // nothing else: no root, no content, no staging, no commit. + expect(executed).toBe(1); + expect(a.requests.map((request) => request["command"])).toEqual(["mappings"]); + expect(b.commits).toHaveLength(1); + expect(yielded(yield* a.journal.readAll())).toEqual([]); + }); + }); + + it("refuses before the split, not after: the other run's journal stays empty", function* () { + yield* scoped(function* () { + // The discriminator. Before this correction, B's transaction would enlist + // the Workspace while the publication appended through A's journal, and a + // refusal from B would leave A holding an event for a commit that never + // happened. The refusal has to come first. + const tree = yield* startingTree(); + const a = yield* harness(emptySnapshot, tree); + const b = yield* harness(emptySnapshot, tree); + b.refuse("command:stale-root"); + + yield* scoped(function* () { + yield* useRemoteWorkspaceEffects(b.run); + const effect = createRemoteWorkspaceEffect( + b.run, + { type: "workspace", name: "split" }, + function* (filesystem): Operation { + yield* filesystem.writeFile("/NOTES.md", "written by the effect\n", 0o644); + return "ran"; + }, + ); + function* workflow(): Workflow { + yield effect; + } + const raised = yield* trapped( + withRemoteWorkspaceEffects(b.run, durableRun(workflow, { stream: a.run.journal })), + ); + expect(String(raised)).toContain("provenance"); + }); + + // No commit reached either owner, and A holds no event for work that + // happened somewhere else. + expect([a.commits, b.commits]).toEqual([[], []]); + expect(yielded(yield* a.run.journal.readAll())).toEqual([]); + }); + }); + + it("refuses a binding whose scope has closed", function* () { + let retained: RemoteRun | undefined; + yield* scoped(function* () { + retained = (yield* harness()).run; + }); + if (retained === undefined) { + throw new Error("expected a binding"); + } + // The value outlived the scope that opened it; what it names did not. + const held = retained; + expect((yield* held.database.replaceRetrievalMetadata({ a: 1 })).ok).toBe(false); + const raised = yield* trapped( + scoped(function* () { + yield* useRemoteWorkspaceEffects(held); + const effect = createRemoteWorkspaceEffect(held, { type: "workspace", name: "late" }, own); + function* workflow(): Workflow { + yield effect; + } + return yield* withRemoteWorkspaceEffects( + held, + durableRun(workflow, { stream: held.journal }), + ); + }), + ); + expect(raised).not.toBe(undefined); + }); + + it("refuses a Files capability kept past the invocation that owned it", function* () { + yield* scoped(function* () { + const held = yield* harness(); + let escaped: WorkspaceFilesystem | undefined; + let metadata: WorkspaceMetadata | undefined; + // deno-lint-ignore require-yield + yield* invocation(held, "captured", function* (filesystem, held): Operation { + escaped = filesystem; + metadata = held; + return "ran"; + }); + const wrote = yield* trapped(escaped?.writeFile("/LATE.md", "too late") ?? sleep(0)); + expect(String(wrote)).toContain("stale"); + expect(() => metadata?.insertRepository(repository())).toThrow(); + }); + }); + + it("authorizes only paths beneath the attempt this invocation owns", function* () { + yield* scoped(function* () { + const held = yield* harness(); + const refused: string[] = []; + // deno-lint-ignore require-yield + const { raised } = yield* invocation(held, "escape", function* (filesystem): Operation { + return yield* (function* (): Operation { + for (const path of ["/../escaped", "/docs/../../escaped"]) { + const failure = yield* trapped(filesystem.writeFile(path, "outside")); + refused.push(String(failure)); + } + // Both ends of a rename: checking one would let the other leave. + refused.push(String(yield* trapped(filesystem.rename("/README.md", "/../moved")))); + // The Workspace root itself is a directory this invocation owns. + const entries = yield* filesystem.readdir("/"); + return entries.map((entry) => entry.name).toSorted(); + })(); + }); + expect(raised).toBe(undefined); + expect(refused).toHaveLength(3); + for (const failure of refused) { + expect(failure).toContain("outside the tree this invocation owns"); + } + }); + }); + + it("claims nothing when the answer to its commit is lost", function* () { + yield* scoped(function* () { + const held = yield* harness(); + held.lose(); + const { raised } = yield* invocation(held, "lost", function* (filesystem): Operation { + yield* filesystem.writeFile("/NOTES.md", "written by the effect\n", 0o644); + return "ran"; + }); + // Whether the owner committed is exactly what cannot be known from here. + // What must not happen is claiming it did. + expect(raised).not.toBe(undefined); + expect(String(raised)).not.toContain("command:"); + expect(held.commits).toHaveLength(1); + }); + }); + + it("sends nothing and keeps nothing when the invocation is cancelled", function* () { + yield* scoped(function* () { + const held = yield* harness(); + yield* useRemoteWorkspaceEffects(held.run); + const effect = createRemoteWorkspaceEffect( + held.run, + { type: "workspace", name: "cancelled" }, + function* (filesystem): Operation { + yield* filesystem.writeFile("/SLOW.md", "in progress\n", 0o644); + yield* sleep(10_000); + return "never"; + }, + ); + function* workflow(): Workflow { + yield effect; + } + const task = yield* spawn(() => + withRemoteWorkspaceEffects(held.run, durableRun(workflow, { stream: held.run.journal })), + ); + yield* sleep(0); + yield* task.halt(); + // Cancellation is control flow: nothing was claimed, and nothing was sent. + expect(held.commits).toEqual([]); + expect(yielded(yield* held.run.journal.readAll())).toHaveLength(0); + }); + }); +}); + +describe("what one invocation retains", () => { + function view(snapshot: RemoteInvocationSnapshot) { + return createInvocationMappings(snapshot, () => {}); + } + + it("reconciles a compatible same-name Repository without staging it again", function* () { + const mappings = view({ ...emptySnapshot("a".repeat(64)), repositories: [repository()] }); + // The retained row is what a same-name read answers with. + expect(mappings.metadata.readRepository("app")?.locator).toBe(LOCATOR); + mappings.metadata.insertRepository(repository()); + expect(mappings.deltas()).toEqual([]); + yield* sleep(0); + }); + + it("refuses a same-name Repository that is not the same Repository", function* () { + const mappings = view({ ...emptySnapshot("a".repeat(64)), repositories: [repository()] }); + const conflicting = repository(); + expect(() => + mappings.metadata.insertRepository({ + ...conflicting, + record: { ...conflicting.record, creationCommit: "1".repeat(40) }, + }), + ).toThrow(); + // A conflict never replaces what is already there. + expect(mappings.metadata.readRepository("app")?.record.creationCommit).toBe("9".repeat(40)); + expect(mappings.deltas()).toEqual([]); + yield* sleep(0); + }); + + it("shows an invocation its own inserts and stages each exactly once", function* () { + const mappings = view(emptySnapshot("a".repeat(64))); + mappings.metadata.insertRepository(repository()); + mappings.metadata.insertRepository(repository()); + expect(mappings.metadata.readRepository("app")?.record.name).toBe("app"); + mappings.metadata.insertWorktree({ + repositoryName: "app", + name: "feature", + requestedBranch: "feature", + requestedBase: null, + creationCommit: "9".repeat(40), + checkoutPath: "/app-feature", + }); + const deltas = mappings.deltas(); + // Parents before children, whatever order they were staged in. + expect(deltas.map((delta) => delta.kind)).toEqual(["repository", "worktree"]); + yield* sleep(0); + }); + + it("keeps a Repository locator out of every value the record carries", function* () { + const mappings = view(emptySnapshot("a".repeat(64))); + mappings.metadata.insertRepository(repository()); + const [delta] = mappings.deltas(); + expect(delta?.kind).toBe("repository"); + if (delta?.kind === "repository") { + expect(delta.locator).toBe(LOCATOR); + // The record names the fingerprint and never the bytes. + expect(JSON.stringify(delta.record)).not.toContain("git.example.invalid"); + } + yield* sleep(0); + }); + + it("refuses a Worktree with no Repository and a path this build does not admit", function* () { + const mappings = view(emptySnapshot("a".repeat(64))); + expect(() => + mappings.metadata.insertWorktree({ + repositoryName: "missing", + name: "feature", + requestedBranch: "feature", + requestedBase: null, + creationCommit: "9".repeat(40), + checkoutPath: "/app-feature", + }), + ).toThrow(); + mappings.metadata.insertRepository(repository()); + expect(() => + mappings.metadata.insertWorktree({ + repositoryName: "app", + name: "feature", + requestedBranch: "feature", + requestedBase: null, + creationCommit: "9".repeat(40), + checkoutPath: "not-a-workspace-path", + }), + ).toThrow(); + yield* sleep(0); + }); + + it("resolves an Agent session by the shared rules and stages it once", function* () { + const identity = { + provider: "claude", + agentCommand: "claude", + sessionIdentity: "expansion-1", + }; + const mappings = view(emptySnapshot("a".repeat(64))); + const sessionKey = agentSessionKey(identity); + // Nothing retained and nothing asserted: this is a new conversation. + expect(resolveAgentSession(undefined, "reattach", [], identity).kind).toBe("create"); + + // The pre-commit window: exactly one canonical assertion reconciles it. + const reconciled = resolveAgentSession( + mappings.agentSessions.read(sessionKey), + "reattach", + [{ kind: "session-id", value: "abc" }], + identity, + ); + expect(reconciled.kind).toBe("reattach"); + if (reconciled.kind === "reattach") { + mappings.agentSessions.commit(reconciled.record); + mappings.agentSessions.commit(reconciled.record); + } + expect(mappings.deltas().map((delta) => delta.kind)).toEqual(["agent-session"]); + + // A retained mapping the provider now contradicts refuses rather than + // starting a replacement conversation. + const retained = mappings.agentSessions.read(sessionKey); + expect(retained).not.toBe(undefined); + expect(() => + resolveAgentSession(retained, "reattach", [{ kind: "session-id", value: "other" }], identity), + ).toThrow(); + expect(() => resolveAgentSession(retained, "reattach", [], identity)).toThrow(); + yield* sleep(0); + }); + + it("refuses more mappings, and more mapping bytes, than one commit may carry", function* () { + const byCount = view(emptySnapshot("a".repeat(64))); + expect(() => { + for (let index = 0; index < 512; index += 1) { + byCount.metadata.insertRepository(repository(`app-${String(index).padStart(4, "0")}`)); + } + }).toThrow(); + + // Few enough to pass the count, large enough that no message could carry + // them. Bounding one without the other would leave the other reachable. + const byBytes = view(emptySnapshot("a".repeat(64))); + expect(() => { + for (let index = 0; index < 64; index += 1) { + const wide = repository(`wide-${String(index).padStart(4, "0")}`); + byBytes.metadata.insertRepository({ + ...wide, + record: { + ...wide.record, + creationCommit: "9".repeat(40), + primaryBranch: "b".repeat(8192), + }, + }); + } + }).toThrow(); + yield* sleep(0); + }); +}); + +function* trapped(operation: Operation): Operation { + try { + yield* operation; + return undefined; + } catch (error) { + return error; + } +} diff --git a/packages/workflow/tests/software-factory-run-id.test.ts b/packages/workflow/tests/software-factory-run-id.test.ts new file mode 100644 index 000000000..2e259b567 --- /dev/null +++ b/packages/workflow/tests/software-factory-run-id.test.ts @@ -0,0 +1,148 @@ +/** + * Tier WRH — the run id one GitHub issue is addressed by. + * + * The derivation is the whole of "one issue, one run": every host that admits + * the same issue has to arrive at the same 52 characters without asking anybody, + * and no value that moves while the work is going on may take part. The fixed + * vectors below were computed independently of this implementation, which is + * what makes them evidence that a second implementation would agree rather than + * a restatement of what this code happens to do. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { + admitFactoryRunSubject, + deriveFactoryRunId, + FactoryRunSubjectError, +} from "@executablemd/workflow/software-factory"; +// The encoder is an internal algorithm rather than a public promise, so the +// RFC 4648 vectors below reach it directly. Everything else in this file goes +// through the published product API, which is what a host actually holds. +import { base32Unpadded, factoryRunIdPreimage } from "../src/software-factory/run-id.ts"; + +/** An opaque node id of the shape GitHub's GraphQL API returns for an issue. */ +const NODE = "I_kwDOABCD12M5abcdef"; + +/** + * Computed outside this implementation, from the specified bytes. + * + * `sha256("github-issue-v1" || 0x00 || authority || 0x00 || nodeId)`, then + * lowercase unpadded RFC 4648 Base32 over all 32 bytes. + */ +const VECTORS = [ + { + authority: "github.com", + issueNodeId: NODE, + runId: "5cktgrv2zyutngh7bbddr2tyg2b5a567cg725hu5e7u42orerxaa", + }, + { + authority: "github.example.com:8443", + issueNodeId: NODE, + runId: "h7dgqsvqzv4p5k2hp2zebemci65qhinpdkxwk5d4caglndiw2xya", + }, + { + authority: "github.com", + issueNodeId: "I_kwDOABCD12M5abcdeg", + runId: "unydmnzwowpjcoua2tyza2topyivbbnm65ddklfjgs5yvlqiwqvq", + }, +] as const; + +/** The canonical authority a subject is admitted under, through the public seam. */ +function authorityOf(value: string): string { + return admitFactoryRunSubject({ authority: value, issueNodeId: NODE }).authority; +} + +function reason(body: () => unknown): string { + try { + body(); + } catch (error) { + if (error instanceof FactoryRunSubjectError) { + return error.reason; + } + throw error; + } + throw new Error("expected a FactoryRunSubjectError"); +} + +describe("the factory run id", () => { + it("derives the specified bytes for known subjects", function* () { + for (const vector of VECTORS) { + const runId = yield* deriveFactoryRunId({ + authority: vector.authority, + issueNodeId: vector.issueNodeId, + }); + expect(runId).toEqual(vector.runId); + expect(runId.length).toEqual(52); + expect(/^[a-z2-7]{52}$/.test(runId)).toEqual(true); + } + }); + + it("derives the same id twice for one subject", function* () { + const once = yield* deriveFactoryRunId({ authority: "github.com", issueNodeId: NODE }); + const again = yield* deriveFactoryRunId({ authority: "GITHUB.COM", issueNodeId: NODE }); + expect(again).toEqual(once); + }); + + it("separates the authority from the node id", function* () { + // Without the NUL separators these two subjects would share a preimage. + const left = yield* deriveFactoryRunId({ authority: "github.com", issueNodeId: "ab" }); + const right = yield* deriveFactoryRunId({ authority: "github.co", issueNodeId: "mab" }); + expect(left).not.toEqual(right); + }); + + it("writes the scheme tag, both separators and both inputs", function* () { + const bytes = new Uint8Array( + factoryRunIdPreimage({ authority: "github.com", issueNodeId: "x" }), + ); + expect(new TextDecoder().decode(bytes)).toEqual("github-issue-v1\0github.com\0x"); + expect([...bytes].filter((byte) => byte === 0).length).toEqual(2); + }); + + it("folds case and keeps a non-default port", function* () { + expect(authorityOf("GitHub.Com")).toEqual("github.com"); + expect(authorityOf("GitHub.Example.COM:8443")).toEqual("github.example.com:8443"); + }); + + it("refuses every part an authority may not carry", function* () { + expect(reason(() => authorityOf(""))).toEqual("authority-empty"); + expect(reason(() => authorityOf("https://github.com"))).toEqual("authority-has-scheme"); + expect(reason(() => authorityOf("user@github.com"))).toEqual("authority-has-userinfo"); + expect(reason(() => authorityOf("github.com/octo"))).toEqual("authority-has-path"); + expect(reason(() => authorityOf("github.com/"))).toEqual("authority-has-path"); + expect(reason(() => authorityOf("github.com?a=b"))).toEqual("authority-has-query"); + expect(reason(() => authorityOf("github.com#top"))).toEqual("authority-has-fragment"); + expect(reason(() => authorityOf("git hub.com"))).toEqual("authority-has-whitespace"); + expect(reason(() => authorityOf("-github.com"))).toEqual("authority-malformed-host"); + expect(reason(() => authorityOf("github.com:https"))).toEqual("authority-malformed-port"); + expect(reason(() => authorityOf("github.com:0"))).toEqual("authority-malformed-port"); + expect(reason(() => authorityOf("github.com:70000"))).toEqual("authority-malformed-port"); + }); + + it("refuses a default port written out, so one deployment has one spelling", function* () { + expect(reason(() => authorityOf("github.com:443"))).toEqual("authority-default-port"); + }); + + it("compares a node id byte for byte", function* () { + const subject = admitFactoryRunSubject({ authority: "github.com", issueNodeId: "Ab_C" }); + expect(subject.issueNodeId).toEqual("Ab_C"); + expect( + reason(() => admitFactoryRunSubject({ authority: "github.com", issueNodeId: "" })), + ).toEqual("node-id-empty"); + expect( + reason(() => admitFactoryRunSubject({ authority: "github.com", issueNodeId: "a\0b" })), + ).toEqual("node-id-has-nul"); + }); + + it("encodes Base32 to the RFC 4648 alphabet without padding", function* () { + // RFC 4648 §10 test vectors, lowercased and unpadded. + const encode = (text: string) => base32Unpadded(new TextEncoder().encode(text)); + expect(encode("")).toEqual(""); + expect(encode("f")).toEqual("my"); + expect(encode("fo")).toEqual("mzxq"); + expect(encode("foo")).toEqual("mzxw6"); + expect(encode("foob")).toEqual("mzxw6yq"); + expect(encode("fooba")).toEqual("mzxw6ytb"); + expect(encode("foobar")).toEqual("mzxw6ytboi"); + }); +}); diff --git a/packages/workflow/tests/workflow-export.test.ts b/packages/workflow/tests/workflow-export.test.ts index e327b27e5..3f0552b5b 100644 --- a/packages/workflow/tests/workflow-export.test.ts +++ b/packages/workflow/tests/workflow-export.test.ts @@ -460,7 +460,7 @@ function base64(content: Uint8Array): string { function artifactWorkspace( artifact: VerifiedXmdArtifact, rootId: string, -): { nodes: Map; entries: WorkspaceRootEntry[] } { +): { nodes: Map; entries: readonly WorkspaceRootEntry[] } { const root = artifact.roots.find((candidate) => candidate.rootId === rootId); if (root === undefined) { throw new Error(`the artifact holds no Workspace root ${rootId}`); diff --git a/packages/workflow/tests/workspace-effect.test.ts b/packages/workflow/tests/workspace-effect.test.ts index 25a352056..0faeb90cc 100644 --- a/packages/workflow/tests/workspace-effect.test.ts +++ b/packages/workflow/tests/workspace-effect.test.ts @@ -686,10 +686,39 @@ describe("Tier DLC — Workspace coordination selection", () => { "packages/durable-streams/*.ts", ], // Whole packages rather than named modules, so a coordination module - // added later is covered without this list being remembered. The single - // exception carries its reason: the HTTP stream is a client for a remote - // durable stream and reaches the platform's own `fetch`. - exclude: ["packages/workflow/src/deno/**", "packages/durable-streams/http-stream.ts"], + // added later is covered without this list being remembered. Each + // exception carries its reason. + // + // The two implementation subtrees are runtime-owned: scanning an adapter + // for the vocabulary of the runtime it adapts is a category error, and + // Code Rule 12 puts host behavior behind exactly these names. The package + // root and every shared module stay covered, so a host name reaching the + // neutral surface is still a failure. + // + // `src/sqlite` is a private physical SQLite backend the two runtime + // adapters share so version 1 is declared once rather than twice. It + // names a database engine because that is its subject, and it is not the + // provider-neutral coordination or external-effect surface — it owns no + // connection, path, transaction or lifecycle authority and is published + // from no entrypoint. + // + // The software factory is the other kind of exception. It is not a + // runtime adapter and is still held to the host-import and + // runtime-detection rules by `host-neutrality.test.ts`; what it is + // allowed is the product vocabulary, because + // `specs/github-actions-software-factory-spec.md` §1.1 makes GitHub the + // subject matter of that contract rather than one provider capturing a + // neutral boundary. + // + // The HTTP stream is a client for a remote durable stream and reaches the + // platform's own `fetch`. + exclude: [ + "packages/workflow/src/deno/**", + "packages/workflow/src/cloudflare/**", + "packages/workflow/src/software-factory/**", + "packages/workflow/src/sqlite/**", + "packages/durable-streams/http-stream.ts", + ], })) .map((entry) => entry.path) .sort(); @@ -729,7 +758,13 @@ describe("Tier DLC — Workspace coordination selection", () => { "packages/workflow/src/workspace/effect.ts", ]), ); + // An exclusion that matched nothing would scan the adapter and fail on its + // own vocabulary; one that matched too little would scan part of it. Both + // subtrees are checked, so a malformed pattern cannot pass quietly. expect(found.some((path) => path.includes("/src/deno/"))).toBe(false); + expect(found.some((path) => path.includes("/src/cloudflare/"))).toBe(false); + expect(found.some((path) => path.includes("/src/software-factory/"))).toBe(false); + expect(found.some((path) => path.includes("/src/sqlite/"))).toBe(false); const crossings: Record = {}; const unread: string[] = []; diff --git a/packages/workflow/tsconfig.cloudflare.json b/packages/workflow/tsconfig.cloudflare.json new file mode 100644 index 000000000..ad86bd059 --- /dev/null +++ b/packages/workflow/tsconfig.cloudflare.json @@ -0,0 +1,17 @@ +{ + "compilerOptions": { + "target": "ES2022", + "module": "ESNext", + "moduleResolution": "bundler", + "lib": ["ES2023"], + "types": ["@cloudflare/workers-types", "@cloudflare/vitest-plugin/types"], + "strict": true, + "allowImportingTsExtensions": true, + "allowJs": true, + "checkJs": false, + "noEmit": true, + "skipLibCheck": true, + "verbatimModuleSyntax": true + }, + "include": ["cloudflare.ts", "src/cloudflare/**/*.ts", "tests/cloudflare/**/*.ts"] +} diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 140cced90..d4709f0e7 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -4,6 +4,10 @@ settings: autoInstallPeers: true excludeLinksFromLockfile: false +overrides: + tsx: 4.23.1 + '@cloudflare/workers-types': 5.20260831.1 + importers: .: @@ -81,6 +85,12 @@ importers: specifier: ^4.3.6 version: 4.3.6 devDependencies: + '@cloudflare/vitest-plugin': + specifier: 1.1.3 + version: 1.1.3(@cloudflare/workers-types@5.20260831.1)(@vitest/runner@4.1.11)(@vitest/snapshot@4.1.11)(vitest@4.1.11(@types/node@22.19.15)(vite@8.2.2(@types/node@22.19.15)(esbuild@0.28.1)(tsx@4.23.1))) + '@cloudflare/workers-types': + specifier: 5.20260831.1 + version: 5.20260831.1 '@durable-streams/server': specifier: ^0.3.8 version: 0.3.8 @@ -117,6 +127,12 @@ importers: '@types/node': specifier: ^22.0.0 version: 22.19.15 + '@vitest/runner': + specifier: 4.1.11 + version: 4.1.11 + '@vitest/snapshot': + specifier: 4.1.11 + version: 4.1.11 expect: specifier: ^30.0.0 version: 30.3.0 @@ -127,11 +143,14 @@ importers: specifier: 1.74.0 version: 1.74.0 tsx: - specifier: ^4.19.0 - version: 4.21.0 + specifier: 4.23.1 + version: 4.23.1 typescript: specifier: ^5.0.0 version: 5.9.3 + vitest: + specifier: 4.1.11 + version: 4.1.11(@types/node@22.19.15)(vite@8.2.2(@types/node@22.19.15)(esbuild@0.28.1)(tsx@4.23.1)) packages/acp: dependencies: @@ -498,10 +517,67 @@ packages: resolution: {integrity: sha512-y7/yvZ2TPAnR9+jnc00klvNNLkJiXFFrQA/hlLCcxA9a2A4zQIOimyFQ9XfwYKiGD1fb5GY8vbKIIgO8d5Tb2A==} engines: {node: '>= 20.12.0'} + '@cloudflare/kv-asset-handler@0.5.0': + resolution: {integrity: sha512-jxQYkj8dSIzc0cD6cMMNdOc1UVjqSqu8BZdor5s8cGjW2I8BjODt/kWPVdY+u9zj3ms75Q5qaZgnxUad83+eAg==} + engines: {node: '>=22.0.0'} + + '@cloudflare/unenv-preset@2.16.1': + resolution: {integrity: sha512-ECxObrMfyTl5bhQf/lZCXwo5G6xX9IAUo+nDMKK4SZ8m4Jvvxp52vilxyySSWh2YTZz8+HQ07qGH/2rEom1vDw==} + peerDependencies: + unenv: 2.0.0-rc.24 + workerd: '>1.20260305.0 <2.0.0-0' + peerDependenciesMeta: + workerd: + optional: true + + '@cloudflare/vitest-plugin@1.1.3': + resolution: {integrity: sha512-ED1Rkaq5Wr5rCeHXpLoDyV4WGJzD0Ju0clM8jS7Hj+wjj/CwaMHeb8DXzUfUQPiHW9rTgwcttPPQnzau2kp6Jg==} + peerDependencies: + '@vitest/runner': ^4.1.0 + '@vitest/snapshot': ^4.1.0 + vitest: ^4.1.0 + + '@cloudflare/workerd-darwin-64@1.20260831.1': + resolution: {integrity: sha512-oyZ8xhu+gYTvoxV/sn6NRmTHK95RhEO1Dk54/6oPb0Uu70w7ZeRoCjkJ5aNmfS8Vrkdu6+oL0HNg6EcC61uQ2Q==} + engines: {node: '>=16'} + cpu: [x64] + os: [darwin] + + '@cloudflare/workerd-darwin-arm64@1.20260831.1': + resolution: {integrity: sha512-s6Go53KPnoXZ1sTGBZ3en3otfHDuMPJhiwXMYWU21JkJQkpoeRt6HFUwM0GPhK3YhXWm+8baGMvCGZYS/KA9eA==} + engines: {node: '>=16'} + cpu: [arm64] + os: [darwin] + + '@cloudflare/workerd-linux-64@1.20260831.1': + resolution: {integrity: sha512-WxNKBgjKgeYTolW3yl1Lt3Lu67UlxdeyzWYi9MIqrKBdyQcz+UNG36RevSBf8rv1sTWapRW234VX2keZ+wXapA==} + engines: {node: '>=16'} + cpu: [x64] + os: [linux] + + '@cloudflare/workerd-linux-arm64@1.20260831.1': + resolution: {integrity: sha512-JTF9+9clUT3gaCq7Xnmd+Q/wEMaitpngSTOec/Ffb/r3xexA9XwNJVFSOKfk6q61flHGjAYJ4H9B7Mu5Qur49w==} + engines: {node: '>=16'} + cpu: [arm64] + os: [linux] + + '@cloudflare/workerd-windows-64@1.20260831.1': + resolution: {integrity: sha512-do+KDYw0PABwsrKUQIccWBZB70kqKcADoSnvzJ8pvMaWUVB4qaCspEZYfm97WNdtY1wt8mlKYqIJyYUNOkTvQg==} + engines: {node: '>=16'} + cpu: [x64] + os: [win32] + + '@cloudflare/workers-types@5.20260831.1': + resolution: {integrity: sha512-yXg4pwfYjhsDH9rYc3qZ3K+z62DCSvO/aj7GiZo6AyDeWGZpyFRpPMYcQ6LF/zfaf1x0Ngw2gSqL8JjuUtMGlA==} + '@colors/colors@1.5.0': resolution: {integrity: sha512-ooWCrlZP11i8GImSjTHYHLkvFDP48nS4+204nGb1RiX/WXYHmJA2III9/e2DWVabCESdW7hBAEzHRqUn9OUVvQ==} engines: {node: '>=0.1.90'} + '@cspotcode/source-map-support@0.8.1': + resolution: {integrity: sha512-IchNf6dN4tHoMFIn/7OE8LWZ19Y6q/67Bmf6vnGREv8RSbBVb9LPJxEcnwrcwX6ixSvaiGoomAUvu4YSxXrVgw==} + engines: {node: '>=12'} + '@durable-streams/client@0.2.2': resolution: {integrity: sha512-zmr9ErxJP1ORljnog4kclWmEJGoTpGN+Mu8FJLVEgcaR9PqTeyKtadq1l1H+DhrPsfAPeG6BF/mEJs4HyI+Eig==} engines: {node: '>=18.0.0'} @@ -584,11 +660,8 @@ packages: peerDependencies: effection: ^3 || ^4 - '@esbuild/aix-ppc64@0.27.4': - resolution: {integrity: sha512-cQPwL2mp2nSmHHJlCyoXgHGhbEPMrEEU5xhkcy3Hs/O7nGZqEpZ2sUtLaL9MORLtDfRvVl2/3PAuEkYZH0Ty8Q==} - engines: {node: '>=18'} - cpu: [ppc64] - os: [aix] + '@emnapi/runtime@1.11.3': + resolution: {integrity: sha512-Xz4Tpyki7XyrpbUK1jR1AhdAdaXyhhY4lZ3neLodmhpuWfy2PAQN5B46sAiU4liOXGLkHypn/qU+jvfWSCYYLA==} '@esbuild/aix-ppc64@0.28.1': resolution: {integrity: sha512-Svl7tq8k/08+p6CXPpRjQ1fKX+1odH/BQbb48fV6fj3CWHhsoIOoY87w1oHXm0qEpkIK3ZfVgp0hed3XBXzXMQ==} @@ -596,300 +669,150 @@ packages: cpu: [ppc64] os: [aix] - '@esbuild/android-arm64@0.27.4': - resolution: {integrity: sha512-gdLscB7v75wRfu7QSm/zg6Rx29VLdy9eTr2t44sfTW7CxwAtQghZ4ZnqHk3/ogz7xao0QAgrkradbBzcqFPasw==} - engines: {node: '>=18'} - cpu: [arm64] - os: [android] - '@esbuild/android-arm64@0.28.1': resolution: {integrity: sha512-34EGEbCIAgosYz6goLcopX6Mo7NyGv9tfwEM2/7Ce2VcVRk568iSvniGWcUXIy7wEDR1wzolcxcriFVrWYcwBg==} engines: {node: '>=18'} cpu: [arm64] os: [android] - '@esbuild/android-arm@0.27.4': - resolution: {integrity: sha512-X9bUgvxiC8CHAGKYufLIHGXPJWnr0OCdR0anD2e21vdvgCI8lIfqFbnoeOz7lBjdrAGUhqLZLcQo6MLhTO2DKQ==} - engines: {node: '>=18'} - cpu: [arm] - os: [android] - '@esbuild/android-arm@0.28.1': resolution: {integrity: sha512-0k2F129Xdio1TdJfzJ8sy1Q47vUD2NnwdhiAf7drUN1EBTfPf4hsFCtmMgu/6m8JSzsBrlmVjudMBQqOfG8usQ==} engines: {node: '>=18'} cpu: [arm] os: [android] - '@esbuild/android-x64@0.27.4': - resolution: {integrity: sha512-PzPFnBNVF292sfpfhiyiXCGSn9HZg5BcAz+ivBuSsl6Rk4ga1oEXAamhOXRFyMcjwr2DVtm40G65N3GLeH1Lvw==} - engines: {node: '>=18'} - cpu: [x64] - os: [android] - '@esbuild/android-x64@0.28.1': resolution: {integrity: sha512-dbwY7ltSMDWsRatcRpCnES4F+im88OCUgGZjy52shC7GqHRE/cYlxNbB4Z4UpJswpcc4Qxd2oE/ufM0p61IKng==} engines: {node: '>=18'} cpu: [x64] os: [android] - '@esbuild/darwin-arm64@0.27.4': - resolution: {integrity: sha512-b7xaGIwdJlht8ZFCvMkpDN6uiSmnxxK56N2GDTMYPr2/gzvfdQN8rTfBsvVKmIVY/X7EM+/hJKEIbbHs9oA4tQ==} - engines: {node: '>=18'} - cpu: [arm64] - os: [darwin] - '@esbuild/darwin-arm64@0.28.1': resolution: {integrity: sha512-TZbWkQY7kvTAXbXUT7uVACR5cMHsDiSz9z7ZKAX/RTq/WJEk3QyRr0wZpNhBDX+/0CtdqUIJlOiodQcta6tY3Q==} engines: {node: '>=18'} cpu: [arm64] os: [darwin] - '@esbuild/darwin-x64@0.27.4': - resolution: {integrity: sha512-sR+OiKLwd15nmCdqpXMnuJ9W2kpy0KigzqScqHI3Hqwr7IXxBp3Yva+yJwoqh7rE8V77tdoheRYataNKL4QrPw==} - engines: {node: '>=18'} - cpu: [x64] - os: [darwin] - '@esbuild/darwin-x64@0.28.1': resolution: {integrity: sha512-zfdzgK9ACBNZLI/CyHTOx81SyNbM6YXn7rxSgX97VjyiPl9W1i4Ka4fgKECEoFCKGpvBj5qArWIGgQjOwkgskQ==} engines: {node: '>=18'} cpu: [x64] os: [darwin] - '@esbuild/freebsd-arm64@0.27.4': - resolution: {integrity: sha512-jnfpKe+p79tCnm4GVav68A7tUFeKQwQyLgESwEAUzyxk/TJr4QdGog9sqWNcUbr/bZt/O/HXouspuQDd9JxFSw==} - engines: {node: '>=18'} - cpu: [arm64] - os: [freebsd] - '@esbuild/freebsd-arm64@0.28.1': resolution: {integrity: sha512-wG2EA8ENdEI0qhkSZMjfqrdY+ziCYCPMmtZjjIwOmXFjmyzEHn+UUxk5of+SYsjtfs3VpnlC7QLzSI5hY/rOAw==} engines: {node: '>=18'} cpu: [arm64] os: [freebsd] - '@esbuild/freebsd-x64@0.27.4': - resolution: {integrity: sha512-2kb4ceA/CpfUrIcTUl1wrP/9ad9Atrp5J94Lq69w7UwOMolPIGrfLSvAKJp0RTvkPPyn6CIWrNy13kyLikZRZQ==} - engines: {node: '>=18'} - cpu: [x64] - os: [freebsd] - '@esbuild/freebsd-x64@0.28.1': resolution: {integrity: sha512-i7dZ9vQgnvSCzi/rYCXNgtF/U+eKZNJBzu3eTQbRgHnM7tNSizLOkRFAl3qzVc/Op/u5YkHHa4pf/3DOYHthLQ==} engines: {node: '>=18'} cpu: [x64] os: [freebsd] - '@esbuild/linux-arm64@0.27.4': - resolution: {integrity: sha512-7nQOttdzVGth1iz57kxg9uCz57dxQLHWxopL6mYuYthohPKEK0vU0C3O21CcBK6KDlkYVcnDXY099HcCDXd9dA==} - engines: {node: '>=18'} - cpu: [arm64] - os: [linux] - '@esbuild/linux-arm64@0.28.1': resolution: {integrity: sha512-yHs+0uc8+nvEAfAfxrWQKK5peSNzBc4PegcMO0EJ2hT71uA7vB8Ihg2e77R2P7SG5uYjPbHlLLmve4LLLRCf0g==} engines: {node: '>=18'} cpu: [arm64] os: [linux] - '@esbuild/linux-arm@0.27.4': - resolution: {integrity: sha512-aBYgcIxX/wd5n2ys0yESGeYMGF+pv6g0DhZr3G1ZG4jMfruU9Tl1i2Z+Wnj9/KjGz1lTLCcorqE2viePZqj4Eg==} - engines: {node: '>=18'} - cpu: [arm] - os: [linux] - '@esbuild/linux-arm@0.28.1': resolution: {integrity: sha512-qVXBOHQS+d5Y722GwJzJUtOLlX7km3CraOaGormF1pDtPd2C/l1SHRPgjLunLGe51Sh5YYWKMFDyV4SxgMQYTQ==} engines: {node: '>=18'} cpu: [arm] os: [linux] - '@esbuild/linux-ia32@0.27.4': - resolution: {integrity: sha512-oPtixtAIzgvzYcKBQM/qZ3R+9TEUd1aNJQu0HhGyqtx6oS7qTpvjheIWBbes4+qu1bNlo2V4cbkISr8q6gRBFA==} - engines: {node: '>=18'} - cpu: [ia32] - os: [linux] - '@esbuild/linux-ia32@0.28.1': resolution: {integrity: sha512-d1z4ZuP0ajrfz/FhGT4vv278rX8KnPPJx8i5+AtK7TYbx9Le9F1hyzurZpkEyjkGa9dUGhQow4C1NmeGvqxN2w==} engines: {node: '>=18'} cpu: [ia32] os: [linux] - '@esbuild/linux-loong64@0.27.4': - resolution: {integrity: sha512-8mL/vh8qeCoRcFH2nM8wm5uJP+ZcVYGGayMavi8GmRJjuI3g1v6Z7Ni0JJKAJW+m0EtUuARb6Lmp4hMjzCBWzA==} - engines: {node: '>=18'} - cpu: [loong64] - os: [linux] - '@esbuild/linux-loong64@0.28.1': resolution: {integrity: sha512-M5sRjUVZrkm1OAPR3dlOYzNmN+loZKGVi1VUQGrwuqLcbR6qeAz+famMhjASeH3YVKvZz+zT1jlh/keC3Rj/lg==} engines: {node: '>=18'} cpu: [loong64] os: [linux] - '@esbuild/linux-mips64el@0.27.4': - resolution: {integrity: sha512-1RdrWFFiiLIW7LQq9Q2NES+HiD4NyT8Itj9AUeCl0IVCA459WnPhREKgwrpaIfTOe+/2rdntisegiPWn/r/aAw==} - engines: {node: '>=18'} - cpu: [mips64el] - os: [linux] - '@esbuild/linux-mips64el@0.28.1': resolution: {integrity: sha512-mRObBZeHh2OxcBFPWE/FjylkRgZdYuiTR3vaTozquCGOH14iP9oN4x4Ge81CoIDYQrXmIxpFumJBu5MtZpnQJQ==} engines: {node: '>=18'} cpu: [mips64el] os: [linux] - '@esbuild/linux-ppc64@0.27.4': - resolution: {integrity: sha512-tLCwNG47l3sd9lpfyx9LAGEGItCUeRCWeAx6x2Jmbav65nAwoPXfewtAdtbtit/pJFLUWOhpv0FpS6GQAmPrHA==} - engines: {node: '>=18'} - cpu: [ppc64] - os: [linux] - '@esbuild/linux-ppc64@0.28.1': resolution: {integrity: sha512-slScBsMAb3GFDcdrCgLwZtPYRoH2H/youv10QiZyRjmsP48fznoveWytSgCI/R0ZcUgpc0ZhIUEx6LHts8yrfQ==} engines: {node: '>=18'} cpu: [ppc64] os: [linux] - '@esbuild/linux-riscv64@0.27.4': - resolution: {integrity: sha512-BnASypppbUWyqjd1KIpU4AUBiIhVr6YlHx/cnPgqEkNoVOhHg+YiSVxM1RLfiy4t9cAulbRGTNCKOcqHrEQLIw==} - engines: {node: '>=18'} - cpu: [riscv64] - os: [linux] - '@esbuild/linux-riscv64@0.28.1': resolution: {integrity: sha512-kw0owk1o0GFETUJyW0jc0G4Yzs0BHZn0JDZ8JRT088vjJYX777BAs1fDGxAC+q831qOs2DTC96mNsG2opdfyyQ==} engines: {node: '>=18'} cpu: [riscv64] os: [linux] - '@esbuild/linux-s390x@0.27.4': - resolution: {integrity: sha512-+eUqgb/Z7vxVLezG8bVB9SfBie89gMueS+I0xYh2tJdw3vqA/0ImZJ2ROeWwVJN59ihBeZ7Tu92dF/5dy5FttA==} - engines: {node: '>=18'} - cpu: [s390x] - os: [linux] - '@esbuild/linux-s390x@0.28.1': resolution: {integrity: sha512-/lAIjX8aYFRByhh6L5rYtPEDRqa9de/4V/juOXcta5frjvzXO4/sqEtyytse0g3zZFuWu5cDN0MkLz2qRDD2Ag==} engines: {node: '>=18'} cpu: [s390x] os: [linux] - '@esbuild/linux-x64@0.27.4': - resolution: {integrity: sha512-S5qOXrKV8BQEzJPVxAwnryi2+Iq5pB40gTEIT69BQONqR7JH1EPIcQ/Uiv9mCnn05jff9umq/5nqzxlqTOg9NA==} - engines: {node: '>=18'} - cpu: [x64] - os: [linux] - '@esbuild/linux-x64@0.28.1': resolution: {integrity: sha512-u/anNYF2mmVOEDwLtnQ1wOr3EZ9sTNGLWrsYGYwHWzGA3Si84IOkHXlbWTD1NB+9/1lcnweYKO54uhxZydNzfA==} engines: {node: '>=18'} cpu: [x64] os: [linux] - '@esbuild/netbsd-arm64@0.27.4': - resolution: {integrity: sha512-xHT8X4sb0GS8qTqiwzHqpY00C95DPAq7nAwX35Ie/s+LO9830hrMd3oX0ZMKLvy7vsonee73x0lmcdOVXFzd6Q==} - engines: {node: '>=18'} - cpu: [arm64] - os: [netbsd] - '@esbuild/netbsd-arm64@0.28.1': resolution: {integrity: sha512-oks0DYbLwWMmaakTsCb+zL4E+aHRVLom9IJZOAthMQEPiQmydXHkziYEsGYRx0uNV/IjEKGAV941JzH02pflqw==} engines: {node: '>=18'} cpu: [arm64] os: [netbsd] - '@esbuild/netbsd-x64@0.27.4': - resolution: {integrity: sha512-RugOvOdXfdyi5Tyv40kgQnI0byv66BFgAqjdgtAKqHoZTbTF2QqfQrFwa7cHEORJf6X2ht+l9ABLMP0dnKYsgg==} - engines: {node: '>=18'} - cpu: [x64] - os: [netbsd] - '@esbuild/netbsd-x64@0.28.1': resolution: {integrity: sha512-aeL6lAnN89Hz43Mlh1G8ARasbuoYvSITDEx0tHh5b7jJnHcssqgjy9Yx430GDpmCa6OyrKoS0aNRjKundRizGg==} engines: {node: '>=18'} cpu: [x64] os: [netbsd] - '@esbuild/openbsd-arm64@0.27.4': - resolution: {integrity: sha512-2MyL3IAaTX+1/qP0O1SwskwcwCoOI4kV2IBX1xYnDDqthmq5ArrW94qSIKCAuRraMgPOmG0RDTA74mzYNQA9ow==} - engines: {node: '>=18'} - cpu: [arm64] - os: [openbsd] - '@esbuild/openbsd-arm64@0.28.1': resolution: {integrity: sha512-MEFJe5C3R8pwXdZ5Y21oo6m7ePiS0d9pWucn99O/wvyJZChoIQKrQDxKrGeW8F5+T0okTHesAmDeiHDTIq0V/Q==} engines: {node: '>=18'} cpu: [arm64] os: [openbsd] - '@esbuild/openbsd-x64@0.27.4': - resolution: {integrity: sha512-u8fg/jQ5aQDfsnIV6+KwLOf1CmJnfu1ShpwqdwC0uA7ZPwFws55Ngc12vBdeUdnuWoQYx/SOQLGDcdlfXhYmXQ==} - engines: {node: '>=18'} - cpu: [x64] - os: [openbsd] - '@esbuild/openbsd-x64@0.28.1': resolution: {integrity: sha512-i/ZLIOafE0Z8cI/XANJAixoJL/uRAoS2xOA3rb0xN+KK0K177cMAsQYkzHtBrtMXAKuAc7HGgcWiZ/sRC1Nxgw==} engines: {node: '>=18'} cpu: [x64] os: [openbsd] - '@esbuild/openharmony-arm64@0.27.4': - resolution: {integrity: sha512-JkTZrl6VbyO8lDQO3yv26nNr2RM2yZzNrNHEsj9bm6dOwwu9OYN28CjzZkH57bh4w0I2F7IodpQvUAEd1mbWXg==} - engines: {node: '>=18'} - cpu: [arm64] - os: [openharmony] - '@esbuild/openharmony-arm64@0.28.1': resolution: {integrity: sha512-ge+Z7EXFNt2BO1oAMsVpiQ8EwndV9i1xXerAeTIK7AtPs3bKFXQM7nlRxDSIUIMeueR1CNXxqztLzdNeReKBJg==} engines: {node: '>=18'} cpu: [arm64] os: [openharmony] - '@esbuild/sunos-x64@0.27.4': - resolution: {integrity: sha512-/gOzgaewZJfeJTlsWhvUEmUG4tWEY2Spp5M20INYRg2ZKl9QPO3QEEgPeRtLjEWSW8FilRNacPOg8R1uaYkA6g==} - engines: {node: '>=18'} - cpu: [x64] - os: [sunos] - '@esbuild/sunos-x64@0.28.1': resolution: {integrity: sha512-BEjgtECkL3vY+SaSQ6nzVfiALUeFxpawyp8Jmf5PtYhf1Ug40N1h/hxlhts+f1FvSvarEigdxS3BlSMI2PJLcQ==} engines: {node: '>=18'} cpu: [x64] os: [sunos] - '@esbuild/win32-arm64@0.27.4': - resolution: {integrity: sha512-Z9SExBg2y32smoDQdf1HRwHRt6vAHLXcxD2uGgO/v2jK7Y718Ix4ndsbNMU/+1Qiem9OiOdaqitioZwxivhXYg==} - engines: {node: '>=18'} - cpu: [arm64] - os: [win32] - '@esbuild/win32-arm64@0.28.1': resolution: {integrity: sha512-lCv9eK/H6ZJWbE7bh2nw54CZ9M2nupBxJcTsdk/QQnWkdSjKGuxmmH8/GWrlT1eMmZfn4dGcCjRte397WqfQXA==} engines: {node: '>=18'} cpu: [arm64] os: [win32] - '@esbuild/win32-ia32@0.27.4': - resolution: {integrity: sha512-DAyGLS0Jz5G5iixEbMHi5KdiApqHBWMGzTtMiJ72ZOLhbu/bzxgAe8Ue8CTS3n3HbIUHQz/L51yMdGMeoxXNJw==} - engines: {node: '>=18'} - cpu: [ia32] - os: [win32] - '@esbuild/win32-ia32@0.28.1': resolution: {integrity: sha512-zvb/mB2bSCoJOpoCBgYKKpX6YM6mJBlBUVUtVj41DlZJVEB6/0CKlRYxP5wWl1C1ILiCoAU5wZZ4q1P3qeS6Eg==} engines: {node: '>=18'} cpu: [ia32] os: [win32] - '@esbuild/win32-x64@0.27.4': - resolution: {integrity: sha512-+knoa0BDoeXgkNvvV1vvbZX4+hizelrkwmGJBdT17t8FNPwG2lKemmuMZlmaNQ3ws3DKKCxpb4zRZEIp3UxFCg==} - engines: {node: '>=18'} - cpu: [x64] - os: [win32] - '@esbuild/win32-x64@0.28.1': resolution: {integrity: sha512-bm4Mowrv+GXMlpWX++EcXw/iLyd1o3+bJkC2DkWXYVvgZCqD/bSj9ctZeAMC3cIxgjRVR2Dufaiu4YPxr5gW1A==} engines: {node: '>=18'} @@ -920,6 +843,152 @@ packages: '@harperfast/extended-iterable@1.0.3': resolution: {integrity: sha512-sSAYhQca3rDWtQUHSAPeO7axFIUJOI6hn1gjRC5APVE1a90tuyT8f5WIgRsFhhWA7htNkju2veB9eWL6YHi/Lw==} + '@img/colour@1.1.0': + resolution: {integrity: sha512-Td76q7j57o/tLVdgS746cYARfSyxk8iEfRxewL9h4OMzYhbW4TAcppl0mT4eyqXddh6L/jwoM75mo7ixa/pCeQ==} + engines: {node: '>=18'} + + '@img/sharp-darwin-arm64@0.35.2': + resolution: {integrity: sha512-eEieHsMksAW4IiO5NzauESRl2D2qz3J/kwUxUrSfV06A93eEaRfMpHXyUb1mAqrR7i8U9A0GRqE9pjn6u1Jjpg==} + engines: {node: '>=20.9.0'} + cpu: [arm64] + os: [darwin] + + '@img/sharp-darwin-x64@0.35.2': + resolution: {integrity: sha512-BaktuGPCeHJMARpodR8jK4uKiZrPAy9WrfQW0sdI37clracq8Bp01AYS3SZgi5FS/y5twa9t4+LIuuxQjqRrWw==} + engines: {node: '>=20.9.0'} + cpu: [x64] + os: [darwin] + + '@img/sharp-freebsd-wasm32@0.35.2': + resolution: {integrity: sha512-YoAxdnd8hPUkvLHd3bWY+YA8nw3xM/RyRopYucNsWHVSan8NLVM3X2volsfoRDcXdUJPg6tXahSd7HXPK7lRnw==} + engines: {node: '>=20.9.0'} + os: [freebsd] + + '@img/sharp-libvips-darwin-arm64@1.3.1': + resolution: {integrity: sha512-4V/M3roRMTYjiwZY9IOVQOE8OyeCxFAkYmyZDrZl51uOKjibm3oeEJ4WAmLxutAfzFbC9jqUiPs2gbnGflH+7g==} + cpu: [arm64] + os: [darwin] + + '@img/sharp-libvips-darwin-x64@1.3.1': + resolution: {integrity: sha512-c0/DxItpJv2+dGhgycJBBgotdqruGYDvA79drdh0MD1dFpy7JzJ/PlXwi1H4rFf0eTy8tgbI91aHDnZIceY3jQ==} + cpu: [x64] + os: [darwin] + + '@img/sharp-libvips-linux-arm64@1.3.1': + resolution: {integrity: sha512-JznefmcK9j1JKPz8AkQDh89kjojubyfOasWBPKfzMIhPwsgDy9evpE/naJTXXXmghS1iFwR8u/kTwh/I2/+GCw==} + cpu: [arm64] + os: [linux] + + '@img/sharp-libvips-linux-arm@1.3.1': + resolution: {integrity: sha512-aGGy9aWzXgHBG7HNyQPWorZthlp7+x6fDRoPAQbGO3ThcttuTyKIx3NuSHb6zb4gBNq6/yNn9f1cy9nFKS/Vmg==} + cpu: [arm] + os: [linux] + + '@img/sharp-libvips-linux-ppc64@1.3.1': + resolution: {integrity: sha512-1EkwGNCZk6iWNCMWqrvdJ+r1j0PT1zIz60CNPhYnJlK/zyeWqlsPZIe+ocBVqPF8k/Ssee/NCk+tE9Ryrko6ng==} + cpu: [ppc64] + os: [linux] + + '@img/sharp-libvips-linux-riscv64@1.3.1': + resolution: {integrity: sha512-Ilays+w2bXdnxzxtQdmXR62u8o8GYa3eL4+Gr+1KiE4xperMZUslRaVPJwwPkzlHEjGfXAfRVAa/7CYCtSqsBw==} + cpu: [riscv64] + os: [linux] + + '@img/sharp-libvips-linux-s390x@1.3.1': + resolution: {integrity: sha512-VfBwVHQTbRoj4XlpA/KLZ7ltgMpz+4WSejFzQ+GnoImjo1PtEJ59QB2qR1xQEeRPYIkNrPIm2L4cICMvz4C2ew==} + cpu: [s390x] + os: [linux] + + '@img/sharp-libvips-linux-x64@1.3.1': + resolution: {integrity: sha512-+c8ukgwU62DS54nCAjw7keOfHUkmr0B5QHEdcOqRnodF/MNXJbVI8Eopoj4B/0H8Asr65I+A4Amrn7a85/md6A==} + cpu: [x64] + os: [linux] + + '@img/sharp-libvips-linuxmusl-arm64@1.3.1': + resolution: {integrity: sha512-qlKb/pwbkAi1WMsJrYHk7CuDrd12s27U2QnRhFYUoJNrRCmkosMTttuRFat/DDB3IlDm5qE1TJgZ4JDnHX8Ldw==} + cpu: [arm64] + os: [linux] + + '@img/sharp-libvips-linuxmusl-x64@1.3.1': + resolution: {integrity: sha512-yO21HwoUVLN8Qa+/SBjQLMYwBWAVJjeGPNe+hc0OUeMeifEtJqu5a1c4HayE1nNpDih9y3/KkoltfkDodmKAlg==} + cpu: [x64] + os: [linux] + + '@img/sharp-linux-arm64@0.35.2': + resolution: {integrity: sha512-af12Pnd0ZGu2HfP8NayB0kk6eC/lrfbQE6HlR4jD+34wdJ1Vw9TF6TMn6ZvffT+WgqVsl0hRbmNvz2u/23VmwA==} + engines: {node: '>=20.9.0'} + cpu: [arm64] + os: [linux] + + '@img/sharp-linux-arm@0.35.2': + resolution: {integrity: sha512-SE4kzF2mepn6z+6E7L6lsV8FzuLL6IPQdyX8ZiwROAG/G8td+hP/m7FsFPwidtrF19gvajuC9l6TxAVcsA4S7A==} + engines: {node: '>=20.9.0'} + cpu: [arm] + os: [linux] + + '@img/sharp-linux-ppc64@0.35.2': + resolution: {integrity: sha512-hYSBm7zcNtDCozCxQHYZJiu63b/bXsgRZuOxCIBZsStMM9Vap47iFHdbX4kCvQsblPB/k+clhELpdQJHQLSHvg==} + engines: {node: '>=20.9.0'} + cpu: [ppc64] + os: [linux] + + '@img/sharp-linux-riscv64@0.35.2': + resolution: {integrity: sha512-qQt0Kc13+Hoan/Awq/qMSQw3L+RI1NCRPgD5cUJ/1WSSmIoysLOc72jlRM3E0OHN9Yr313jgeQ2T+zW+F03QFA==} + engines: {node: '>=20.9.0'} + cpu: [riscv64] + os: [linux] + + '@img/sharp-linux-s390x@0.35.2': + resolution: {integrity: sha512-E4fLLfRPzDLlEeDaTzI98OFLcv++WL5ChLLMwPoVd0CIoZQqupBSNbOisPL5am9XsbQ9T84+iiMpUvbFtkunbA==} + engines: {node: '>=20.9.0'} + cpu: [s390x] + os: [linux] + + '@img/sharp-linux-x64@0.35.2': + resolution: {integrity: sha512-gi0zFJJRLswfCZmHtJdikXPOc5u7qamSOS3NHedLqLd4W8Q0NqjdBr6TTRIgsfFjqfTsHFgdfvJ9LwqSgcHiAA==} + engines: {node: '>=20.9.0'} + cpu: [x64] + os: [linux] + + '@img/sharp-linuxmusl-arm64@0.35.2': + resolution: {integrity: sha512-siWbOW1u6HFnFLrp0waKyW7VEf7jYvcDWdrXEFa8AkdAQgEvuu5Fz8/Y70w9EeqAdwDtfU012BhEHHaDqvQNzg==} + engines: {node: '>=20.9.0'} + cpu: [arm64] + os: [linux] + + '@img/sharp-linuxmusl-x64@0.35.2': + resolution: {integrity: sha512-YBqMMcjDi4QGYiSn4vNOYBhmlC4z5AXqkOUUqI2e0AFA4urNv4ESgOgwNl3K+4etQhha0twXlzeF20bbULm9Yg==} + engines: {node: '>=20.9.0'} + cpu: [x64] + os: [linux] + + '@img/sharp-wasm32@0.35.2': + resolution: {integrity: sha512-Mrv4JQNYVQ94xH+jzZ9r+gowleN8mv2FTgKT+PI6bx5C0G8TdNYndu161pg2i7uoBwxy2ImPMHrJOM2LZef7Bw==} + engines: {node: '>=20.9.0'} + + '@img/sharp-webcontainers-wasm32@0.35.2': + resolution: {integrity: sha512-QNV27pxs9wpApEiCfvHM1RDoP1w1+2KrUWWDPEhEwg+latvOrfuhWrHWZKwdSFwU6jh3myjw/yOCRsUIuOft3g==} + engines: {node: '>=20.9.0'} + cpu: [wasm32] + + '@img/sharp-win32-arm64@0.35.2': + resolution: {integrity: sha512-BiVRYc/t6/Vl3e1hBx0hugG4oN9Pydf4fgMSpxTQJmwGUg/YoXTWHiFeRymHfCZzifxu4F4rpk/I67D0LQ20wQ==} + engines: {node: '>=20.9.0'} + cpu: [arm64] + os: [win32] + + '@img/sharp-win32-ia32@0.35.2': + resolution: {integrity: sha512-YYEhx9PImCC7T0tI8JDMi4DB9LwLCXCU5OWNYEXAxh5Q1ShKkyC6byxzoBJ3gEFDnH2lQckWuDe70G7mB2XJog==} + engines: {node: ^20.9.0} + cpu: [ia32] + os: [win32] + + '@img/sharp-win32-x64@0.35.2': + resolution: {integrity: sha512-imoOyBcoM/iiUr4J6VPpCNjPnjvP/Gks95898yB8YqoGGYmHYbOyCuNv9FMhFgtaiHFGbHW8bxKqRV6VjtXThQ==} + engines: {node: '>=20.9.0'} + cpu: [x64] + os: [win32] + '@jest/diff-sequences@30.3.0': resolution: {integrity: sha512-cG51MVnLq1ecVUaQ3fr6YuuAOitHK1S4WUJHnsPFE/quQr33ADUx1FfrTCpMCRxvy0Yr9BThKpDjSlcTi91tMA==} engines: {node: ^18.14.0 || ^20.0.0 || ^22.0.0 || >=24.0.0} @@ -944,9 +1013,16 @@ packages: resolution: {integrity: sha512-JHm87k7bA33hpBngtU8h6UBub/fqqA9uXfw+21j5Hmk7ooPHlboRNxHq0JcMtC+n8VJGP1mcfnD3Mk+XKe1oSw==} engines: {node: ^18.14.0 || ^20.0.0 || ^22.0.0 || >=24.0.0} + '@jridgewell/resolve-uri@3.1.2': + resolution: {integrity: sha512-bRISgCIjP20/tbWSPWMEi54QVPRZExkuD9lJL+UIxUKtwVJA8wW1Trb1jMs1RFXo1CBTNZ/5hpC9QvmKWdopKw==} + engines: {node: '>=6.0.0'} + '@jridgewell/sourcemap-codec@1.5.5': resolution: {integrity: sha512-cYQ9310grqxueWbl+WuIUIaiUaDcj7WOq5fVhEljNVgRfOUhY9fy2zTvfoqWsnebh8Sl70VScFbICvJnLKB0Og==} + '@jridgewell/trace-mapping@0.3.9': + resolution: {integrity: sha512-3Belt6tdc8bPgAtbcmdtNJlirVoTmEb5e2gC94PnkwEW9jI6CAHUeoG85tjWP5WquqfavoMtMwiG4P926ZKKuQ==} + '@lmdb/lmdb-darwin-arm64@3.5.6': resolution: {integrity: sha512-mY5FG4TjPAkY4P0w+OhHaUka5mDh2TX2WKYIwuKzJ1zeW3VvRgxdam/lGJTquI+bthTx5CSHDW+BAQCnNAzkEA==} cpu: [arm64] @@ -1019,6 +1095,9 @@ packages: resolution: {integrity: sha512-9T3nD5q51X1d4QYW6vouKW9hBSb2Tb/wB/2XoTr4oP5SCGtp3a7aTHHewQFylred1B21/Bhev6gy4x01FPBcbQ==} engines: {node: '>=18'} + '@oxc-project/types@0.148.0': + resolution: {integrity: sha512-Nm4s/jB+4FpFsPhWGEC4h7rzksesmtnMXomo6rCMcg/b8zLQuOziRgkCS1fxDCXOlJB/6Q8oABOZ/OP6RIPj9A==} + '@oxfmt/binding-android-arm-eabi@0.41.0': resolution: {integrity: sha512-REfrqeMKGkfMP+m/ScX4f5jJBSmVNYcpoDF8vP8f8eYPDuPGZmzp56NIUsYmx3h7f6NzC6cE3gqh8GDWrJHCKw==} engines: {node: ^20.19.0 || >=22.12.0} @@ -1247,6 +1326,15 @@ packages: cpu: [x64] os: [win32] + '@poppinss/colors@4.1.6': + resolution: {integrity: sha512-H9xkIdFswbS8n1d6vmRd8+c10t2Qe+rZITbbDHHkQixH5+2x1FDGmi/0K+WgWiqQFKPSlIYB7jlH6Kpfn6Fleg==} + + '@poppinss/dumper@0.6.5': + resolution: {integrity: sha512-NBdYIb90J7LfOI32dOewKI1r7wnkiH6m920puQ3qHUeZkxNkQiFnXVWoE6YtFSv6QOiPPf7ys6i+HWWecDz7sw==} + + '@poppinss/exception@1.2.3': + resolution: {integrity: sha512-dCED+QRChTVatE9ibtoaxc+WkdzOSjYTKi/+uacHWIsfodVfpsueo3+DKpgU5Px8qXjgmXkSvhXvSCz3fnP9lw==} + '@radix-ui/number@1.1.3': resolution: {integrity: sha512-Road2bidD0uu/1BGDOWNdPI06g0lIRy6IF9GZcIrDK2KGItfor8IQwQa+yM2ERgHM1MmHxaxpTzk0/Jp42lNfA==} @@ -1653,6 +1741,99 @@ packages: peerDependencies: '@rjsf/utils': ^6.7.1 + '@rolldown/binding-android-arm-eabi@1.2.7': + resolution: {integrity: sha512-EypzgnYCwyVY4NDHKzGmNJT5b+XaQEBniHxsMdeIQLB/tcCzZnhqrzHpZFbX9iaxx+5RiB8caATBtfvZP7zVxQ==} + engines: {node: ^20.19.0 || >=22.12.0} + cpu: [arm] + os: [android] + + '@rolldown/binding-android-arm64@1.2.7': + resolution: {integrity: sha512-l17HE9EweWaqJZhuUuNBN/FzM62xw+DECVnJyvMsxn8vJFAGLy5QfLDoYAcronkAN8VxKZHezDpulHDPx95vFw==} + engines: {node: ^20.19.0 || >=22.12.0} + cpu: [arm64] + os: [android] + + '@rolldown/binding-darwin-arm64@1.2.7': + resolution: {integrity: sha512-8ED8ELFvHXc6OCETIn4gXObPiaR6bckM/ipXtbzlPVDRMBfEGjCKgO90F9YtfdpDatVx/ZQw7aZ1vUMf/+T3Mw==} + engines: {node: ^20.19.0 || >=22.12.0} + cpu: [arm64] + os: [darwin] + + '@rolldown/binding-darwin-x64@1.2.7': + resolution: {integrity: sha512-/WPripjtiAIZ2tWY7ddijORT0Ujg87wxWW/qcoFVCKAWVDPhtY0xr7Dj0M3GyNGz60jGwTElhro/mkF9dT7dDQ==} + engines: {node: ^20.19.0 || >=22.12.0} + cpu: [x64] + os: [darwin] + + '@rolldown/binding-freebsd-x64@1.2.7': + resolution: {integrity: sha512-14DI4NcqpvbICxSnGLx3PmtDaWqRP/KGSGb6C+JLLVPeZRl6dKdHba3pGsqT3vpdTqhEYIPG0MMQ8c0xYqoJxA==} + engines: {node: ^20.19.0 || >=22.12.0} + cpu: [x64] + os: [freebsd] + + '@rolldown/binding-linux-arm-gnueabihf@1.2.7': + resolution: {integrity: sha512-bxrWIRvHWQvbJwi+VIie/kDJmQxcNE6xxWwZdqF/ExVAigtHkv54WTLQPb+QsZdnFy18fg7JPfWGL0RH6vwIlQ==} + engines: {node: ^20.19.0 || >=22.12.0} + cpu: [arm] + os: [linux] + + '@rolldown/binding-linux-arm64-gnu@1.2.7': + resolution: {integrity: sha512-toOY2BChBZyuxU7OYX6Tn389di4IzAqPTycVcci0O7FSfBqzRB3RZn+K5Is6ANf4tmgRd/K1yZTsNTXbkXsnLg==} + engines: {node: ^20.19.0 || >=22.12.0} + cpu: [arm64] + os: [linux] + + '@rolldown/binding-linux-arm64-musl@1.2.7': + resolution: {integrity: sha512-lAIXTH/aiLRLxsTgQvfhjo4K1ydWIp00+V0voOr9beb/9ZmkUFrSIb03dXNFRgMNvkE6oGsF10ioQ6UsI+vS5Q==} + engines: {node: ^20.19.0 || >=22.12.0} + cpu: [arm64] + os: [linux] + + '@rolldown/binding-linux-ppc64-gnu@1.2.7': + resolution: {integrity: sha512-kdnwS28Pkenp/mZMRwjXXXwxQ7pIsm+bF919LUK93BOyhcLsrVKdP2p9fxpiPNPAbNuch8ypQt0pm2P2LYCAGg==} + engines: {node: ^20.19.0 || >=22.12.0} + cpu: [ppc64] + os: [linux] + + '@rolldown/binding-linux-s390x-gnu@1.2.7': + resolution: {integrity: sha512-516OdsyLdr5E65paF3yBF55t8mfm9+gmtCsK3xI7XKXIT7EfRlHhxL8K/NR6Hu8BWSgF5+1w74lTL0+nxcc8Qw==} + engines: {node: ^20.19.0 || >=22.12.0} + cpu: [s390x] + os: [linux] + + '@rolldown/binding-linux-x64-gnu@1.2.7': + resolution: {integrity: sha512-r8/z8n7GFaYRln3xmP1Cxy0HH/HLM0uBUPkEuSVEfKGDA89M0FsZRZJRSwe/tJjRx+fpH/gjorfhB8tmEbSFLA==} + engines: {node: ^20.19.0 || >=22.12.0} + cpu: [x64] + os: [linux] + + '@rolldown/binding-linux-x64-musl@1.2.7': + resolution: {integrity: sha512-pAsE8iiDxUg1xBqdhrTfg45AVDVpirjz00sblEYClGNNcMnDb+e8beQgqIAw6LvauX/APvgxUnwrgun/YYGBhw==} + engines: {node: ^20.19.0 || >=22.12.0} + cpu: [x64] + os: [linux] + + '@rolldown/binding-openharmony-arm64@1.2.7': + resolution: {integrity: sha512-lTcIYmmnQQA8Or/2DatS6oSqcdLHvendjS+zLu+FwgToynWMRSmQdpM65fTANJgIS4mjbMOo5KT2lnT9SAb96w==} + engines: {node: ^20.19.0 || >=22.12.0} + cpu: [arm64] + os: [openharmony] + + '@rolldown/binding-win32-arm64-msvc@1.2.7': + resolution: {integrity: sha512-e3Gu3WxbNk/UqQhxqU7YIYO+9ZBvWNz3U+h/qRFosscMFzdRPbXYSaSWgSnklv2fz1TgzBTcti2z35c/7irsHw==} + engines: {node: ^20.19.0 || >=22.12.0} + cpu: [arm64] + os: [win32] + + '@rolldown/binding-win32-x64-msvc@1.2.7': + resolution: {integrity: sha512-W/jg5qoRSqjsEv0+dZi4e687mcHqmVuU0P4fK6qS/xjetW2Gmc1W8j//z5nAeNcC8Ttm0hV46IjcYeuVwYhuiw==} + engines: {node: ^20.19.0 || >=22.12.0} + cpu: [x64] + os: [win32] + + '@rolldown/pluginutils@1.0.1': + resolution: {integrity: sha512-2j9bGt5Jh8hj+vPtgzPtl72j0yRxHAyumoo6TNfAjsLB04UtpSvPbPcDcBMxz7n+9CYB0c1GxQFxYRg2jimqGw==} + '@secretlint/core@13.0.4': resolution: {integrity: sha512-Wv49KcI5XX6xjLR1wxyjORA15PtMb5ar/M27ShimVudaSi6iAM04QCA5Ozx+uEahfHNefUUKbjKGpy/9pxuW7g==} engines: {node: '>=22.0.0'} @@ -1675,12 +1856,28 @@ packages: resolution: {integrity: sha512-t09vSN3MdfsyCHoFcTRCH/iUtG7OJ0CsjzB8cjAmKc/va/kIgeDI/TxsigdncE/4be734m0cvIYwNaV4i2XqAw==} engines: {node: '>=10'} + '@sindresorhus/is@7.2.0': + resolution: {integrity: sha512-P1Cz1dWaFfR4IR+U13mqqiGsLFf1KbayybWwdd2vfctdV6hDpUkgCY0nKOLLTMSoRd/jJNjtbqzf13K8DCCXQw==} + engines: {node: '>=18'} + + '@speed-highlight/core@1.2.24': + resolution: {integrity: sha512-qeW2e1l78afw8VhRPfPQ1Gjj+KU5XFQ/OFV5ti6eTa9bruO7mJyZtA4vw0ofqmA3tKCkROE9xLk3VZoeRc98nw==} + '@standard-schema/spec@1.1.0': resolution: {integrity: sha512-l2aFy5jALhniG5HgqrD6jXLi/rUWrKvqN/qJx6yoJsgKhblVd+iqqU4RCXavm/jPityDo5TCvKMnpjKnOriy0w==} + '@types/chai@5.2.3': + resolution: {integrity: sha512-Mw558oeA9fFbv65/y4mHtXDs9bPnFMZAL/jxdPFUpOHHIXX91mcgEHbS5Lahr+pwZFR8A7GQleRWeI6cGFC2UA==} + '@types/debug@4.1.12': resolution: {integrity: sha512-vIChWdVG3LG1SMxEvI/AK+FWJthlrqlTu7fbrlywTkkaONwk/UAGaULXRlf8vkzFBLVm0zkMdCquhL5aOjhXPQ==} + '@types/deep-eql@4.0.2': + resolution: {integrity: sha512-c9h9dVVMigMPc4bwTvC5dxqtqJZwQPePsWjPlpSOnojbor6pGqdk541lfA7AqFQr5pB1BRdq0juY9db81BwyFw==} + + '@types/estree@1.0.9': + resolution: {integrity: sha512-GhdPgy1el4/ImP05X05Uw4cw2/M93BCUmnEvWZNStlCzEKME4Fkk+YpoA5OiHNQmoS7Cafb8Xa3Pya8m1Qrzeg==} + '@types/hast@3.0.5': resolution: {integrity: sha512-rp/ezSWaD1m44dPKICGhiskI13nVr7qTloFwDa/IYkhhf5nzwP+zIQcIJh3WIFSBOy/H1PzB40jPjMDksN4F+g==} @@ -1720,6 +1917,35 @@ packages: '@ungap/structured-clone@1.3.3': resolution: {integrity: sha512-60YRaenCQcVjYEKOcG824+DRGGIQ3VKErcBoAEDJZz5bKIs2ZG+X/H9Nk+Q6EVkwJk5QNApxbrc5QtBSwtrXAg==} + '@vitest/expect@4.1.11': + resolution: {integrity: sha512-VX2x5vNJXET47KAFzwERI+KRMtTTCSWTfSMKsW7JsUsXV4psq++e3DvZpuTDOpHcxytiDs6p2nhVb2tVDiiUYw==} + + '@vitest/mocker@4.1.11': + resolution: {integrity: sha512-2XJVD55d1o5AZous5CCGKS74g/riOj9odEt2bQpCVZeblHyHdnMeFl4jl0XjU21stf4mbjUkew2eXQZt65g5CQ==} + peerDependencies: + msw: ^2.4.9 + vite: ^6.0.0 || ^7.0.0 || ^8.0.0 + peerDependenciesMeta: + msw: + optional: true + vite: + optional: true + + '@vitest/pretty-format@4.1.11': + resolution: {integrity: sha512-yiZzPbGTS9Sr/JpFl8zHrcIkAofNbFV6k21vIgQN/cY/oxZeXhJv5sc/MBJ5jFKWmWs+oJHw0UXLZjmf931+Vw==} + + '@vitest/runner@4.1.11': + resolution: {integrity: sha512-LztvUgdwMNJMIkj3hQnnxiC2Xy1zNxq928W/xhjCLaNCzqTZOudjwbQf6v9IntZGPw132i2Lq2rgTRZHD3JHNw==} + + '@vitest/snapshot@4.1.11': + resolution: {integrity: sha512-pN7ikn1ON7h8ee4gIAp4AzyK+zBtJPzVbqOgu5LCEh4VaJVbPQcgYQYJIMGQPXVeJJq1fnfazis7a5pFNPahog==} + + '@vitest/spy@4.1.11': + resolution: {integrity: sha512-apNa/prQy2qCeywhnixOHPRCgGNhvg7T4Dapfl1GahLp/R+uhBm5cPyFoNVyqsNd2h1nJxL6BqqdIjiABL60YA==} + + '@vitest/utils@4.1.11': + resolution: {integrity: sha512-zTCVGpyFsGWBhllOyKlTw/vnr6D9qxsfSDyfbyZmTyjHw5N/VuvzHpHoQjm2ZJzn4RJgx5w4r7V0er69CmLgPQ==} + '@x0k/json-schema-merge@1.0.4': resolution: {integrity: sha512-KvmMgAftbVzATq4IRnkno/SKSu+gjaR2ZUPJG5JUlY4W3twRJo03sk2914u8scmosibBZ0m7s6euZlJuqpv8Ww==} @@ -1774,6 +2000,10 @@ packages: resolution: {integrity: sha512-ik3ZgC9dY/lYVVM++OISsaYDeg1tb0VtP5uL3ouh1koGOaUMDPpbFIei4JkFimWUFPn90sbMNMXQAIVOlnYKJA==} engines: {node: '>=10'} + assertion-error@2.0.1: + resolution: {integrity: sha512-Izi8RQcffqCeNVgFigKli1ssklIbpHnCYc6AknXGYoB6grJqyeby7jv12JUQgmTAnIDnbck1uxksT4dzN3PWBA==} + engines: {node: '>=12'} + b4a@1.8.1: resolution: {integrity: sha512-aiqre1Nr0B/6DgE2N5vwTc+2/oQZ4Wh1t4NznYY4E00y8LCt6NqdRv81so00oo27D8MVKTpUa/MwUUtBLXCoDw==} peerDependencies: @@ -1822,6 +2052,9 @@ packages: bare-url@2.4.6: resolution: {integrity: sha512-iQxPClE07hETVpbRoX7JXX3v/ZQViCxe/SYCxylRLzdEx1xJAufPptfiOqR8tqiCtmbtMDANKWszzjLu1PMAZQ==} + blake3-wasm@2.1.5: + resolution: {integrity: sha512-F1+K8EbfOZE49dtoPtmxUQrpXaBIl3ICvasLh+nJta0xkz+9kF/7uet9fLnwKqhDrmj6g+6K3Tw9yQPUg2ka5g==} + boolbase@1.0.0: resolution: {integrity: sha512-JZOSA7Mo9sNGB8+UjSgzdLtokWAky1zbztM3WRLCbZ70/3cTANmQmOdR7y2g+J0e2WXywy1yS468tY+IruqEww==} @@ -1831,6 +2064,10 @@ packages: ccount@2.0.1: resolution: {integrity: sha512-eyrF0jiFpY+3drT6383f1qhkbGsLSifNAjA61IUjZjmLCWjItY6LB9ft9YhoDgwfmclB2zhu51Lc7+95b8NRAg==} + chai@6.2.2: + resolution: {integrity: sha512-NUPRluOfOiTKBKvWPtSD4PhFvWCqOi0BGStNWs57X9js7XGTprSmFoz5F0tWhR4WPjNeR9jXqdC7/UpSJTnlRg==} + engines: {node: '>=18'} + chalk@4.1.2: resolution: {integrity: sha512-oKnbhFyRIXpUuez8iBMmyEa4nbj4IOQyuhc/wy9kY7/WVPcwIO9VA668Pu8RkO7+0G76SLROeyw9CpQ061i4mA==} engines: {node: '>=10'} @@ -1856,6 +2093,9 @@ packages: resolution: {integrity: sha512-77PSwercCZU2Fc4sX94eF8k8Pxte6JAwL4/ICZLFjJLqegs7kCuAsqqj/70NQF6TvDpgFjkubQB2FW2ZZddvQg==} engines: {node: '>=8'} + cjs-module-lexer@1.2.3: + resolution: {integrity: sha512-0TNiGstbQmCFwt4akjjBg5pLRTSyj/PkWQ1ZoO2zntmg9yLqSRxwEa4iCfQLGjqhiqBfOJa7W/E8wfGrTDmlZQ==} + class-variance-authority@0.7.1: resolution: {integrity: sha512-Ka+9Trutv7G8M6WT6SeiRWz792K5qEqIGEGzXKhAE6xOWAY6pPH8U+9IY3oCMv6kqTmLsv7Xh/2w2RigkePMsg==} @@ -1899,6 +2139,13 @@ packages: resolution: {integrity: sha512-4bxK3+L+FHr9Xm/d69Syvvpvkj7lj7a4zz3B+tchuohg5WKeudyBS+4Oob5Zdgoh8I7+n2lj0lfa8I6cUXfcEg==} engines: {node: '>= 16'} + convert-source-map@2.0.0: + resolution: {integrity: sha512-Kvp459HrV2FEJ1CAsi1Ku+MY3kasH19TFykTz2xWmMeq6bk2NU3XXvfJ+Q61m0xktWwt+1HSYf3JZsTms3aRJg==} + + cookie@1.1.1: + resolution: {integrity: sha512-ei8Aos7ja0weRpFzJnEA9UHJ/7XQmqglbRwnf2ATjcB9Wq874VKH9kfjjirM6UhU2/E5fFYadylyhFldcqSidQ==} + engines: {node: '>=18'} + cross-spawn@7.0.6: resolution: {integrity: sha512-uV2QOWP2nWzsy2aMp8aRibhi9dlzF5Hgh5SHaB9OiTGEyDTiJJyx0uy51QXdyWbtAHNua4XJzUKca3OzKUd3vA==} engines: {node: '>= 8'} @@ -1949,10 +2196,11 @@ packages: resolution: {integrity: sha512-xUtoPkMggbz0MPyPiIWr1Kp4aeWJjDZ6SMvURhimjdZgsRuDplF5/s9hcgGhyXMhs+6vpnuoiZ2kFiu3FMnS8Q==} engines: {node: '>=18'} - esbuild@0.27.4: - resolution: {integrity: sha512-Rq4vbHnYkK5fws5NF7MYTU68FPRE1ajX7heQ/8QXXWqNgqqJ/GkmmyxIzUnf2Sr/bakf8l54716CcMGHYhMrrQ==} - engines: {node: '>=18'} - hasBin: true + error-stack-parser-es@1.0.5: + resolution: {integrity: sha512-5qucVt2XcuGMcEGgWI7i+yZpmpByQ8J1lHhcL7PwqCwu9FPP3VUXzT4ltHe5i2z9dePwEHcDVOAfSnHsOlCXRA==} + + es-module-lexer@2.3.2: + resolution: {integrity: sha512-poHGpORABojJJucnV9KbOavETW8lBVnphkW77ER5/BQ5Fz7oXSoCNek7IH3vR5nRjdsEz926ibFYX8KtLQmdyw==} esbuild@0.28.1: resolution: {integrity: sha512-HrJrvZv5ayxBzPfwphOoNzkzOIIlifzk0KJrGK2c8R4+LKpMtpYLQeUdjnwjWv/LZlkH2laZk+4w78pi99D4Vw==} @@ -1972,9 +2220,16 @@ packages: engines: {node: '>=4'} hasBin: true + estree-walker@3.0.3: + resolution: {integrity: sha512-7RUKfXgSMMkzt6ZuXmqapOurLGPPfgj6l9uRZ7lRGolvk0y2yocc35LdcxKC5PQZdn2DMqioAQ2NoWcrTKmm6g==} + events-universal@1.0.1: resolution: {integrity: sha512-LUd5euvbMLpwOF8m6ivPCbhQeSiYVNb8Vs0fQ8QjXo0JTkEHpz8pxdQf0gStltaPpw0Cca8b39KxvK9cfKRiAw==} + expect-type@1.4.0: + resolution: {integrity: sha512-KfYbmpRm0VbLjEvVa9yGwCi9GI34xvi7A/HXYWQO65CSD2u3MczUJSuwXKFIxlGsgBQizV9q5J9NHj4VG0n+pA==} + engines: {node: '>=12.0.0'} + expect@30.3.0: resolution: {integrity: sha512-1zQrciTiQfRdo7qJM1uG4navm8DayFa2TgCSRlzUyNkhcJ6XUZF3hjnpkyr3VhAqPH7i/9GkG7Tv5abz6fqz0Q==} engines: {node: ^18.14.0 || ^20.0.0 || ^22.0.0 || >=24.0.0} @@ -2014,6 +2269,15 @@ packages: fastq@1.20.1: resolution: {integrity: sha512-GGToxJ/w1x32s/D2EKND7kTil4n8OVk/9mycTc4VDza13lOvpUZTGX3mFSCtV9ksdGBVzvsyAVLM6mHFThxXxw==} + fdir@6.5.0: + resolution: {integrity: sha512-tIbYtZbucOs0BRGqPJkshJUYdL+SDH7dVM8gjy+ERp3WAUjLEFJE+02kanyHtwjWOnwrKYBiwAmM0p4kLJAnXg==} + engines: {node: '>=12.0.0'} + peerDependencies: + picomatch: ^3 || ^4 + peerDependenciesMeta: + picomatch: + optional: true + fsevents@2.3.3: resolution: {integrity: sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==} engines: {node: ^8.16.0 || ^10.6.0 || >=11.0.0} @@ -2027,9 +2291,6 @@ packages: resolution: {integrity: sha512-FJhYRoDaiatfEkUK8HKlicmu/3SGFD51q3itKDGoSTysQJBnfOcxU5GxnhE1E6soB76MbT0MBtnKJuXyAx+96Q==} engines: {node: '>=6'} - get-tsconfig@4.13.6: - resolution: {integrity: sha512-shZT/QMiSHc/YBLxxOkMtgSid5HFoauqCE3/exfsEcwg1WkeqjG+V40yBbBrsD+jW2HDXcs28xOfcbm2jI8Ddw==} - graceful-fs@4.2.11: resolution: {integrity: sha512-RbJ5/jmFcNNCcDV5o9eTnBLJ/HszWV0P73bc+Ff4nS/rJj+YaS6IGyiOL0VoBYX+l1Wrl3k63h/KrH+nhJ0XvQ==} @@ -2116,6 +2377,80 @@ packages: resolution: {integrity: sha512-dcS1ul+9tmeD95T+x28/ehLgd9mENa3LsvDTtzm3vyBEO7RPptvAD+t44WVXaUjTBRcrpFeFlC8WCruUR456hw==} engines: {node: '>=0.10.0'} + kleur@4.1.5: + resolution: {integrity: sha512-o+NO+8WrRiQEE4/7nwRJhN1HWpVmJm511pBHUxPLtp0BUISzlBplORYSmTclCnJvQq2tKu/sgl3xVpkc7ZWuQQ==} + engines: {node: '>=6'} + + lightningcss-android-arm64@1.33.0: + resolution: {integrity: sha512-gEpRTalKdosp4Bb8qWtc2iOgE5SeIHlpS1up9bFq2wAyYhl1UdTObYiHe98zEM9SQvSoqQZ1IQD0JNpg3Ml5pg==} + engines: {node: '>= 12.0.0'} + cpu: [arm64] + os: [android] + + lightningcss-darwin-arm64@1.33.0: + resolution: {integrity: sha512-Sciaz8eenNTKn9b3t7+xr0ipTp9YxKQY4npwQ3mrRuL0BAVHBLyZxofhaKBAVtzmtRZ/zTyo0/to4B1uWG/Djg==} + engines: {node: '>= 12.0.0'} + cpu: [arm64] + os: [darwin] + + lightningcss-darwin-x64@1.33.0: + resolution: {integrity: sha512-Z5UPAxzrjlWNNyGy6i65cJzzvgJ5D3T6wMvs+gWpY9d7qRhANrxqAp6LhxIgZhWEw18RfJTGcRxjuLIBr+m8XQ==} + engines: {node: '>= 12.0.0'} + cpu: [x64] + os: [darwin] + + lightningcss-freebsd-x64@1.33.0: + resolution: {integrity: sha512-QQM/Ti/hQajJwCY+RiWuCZ9sdtI/XQk7nDK5vC8kkdwixezOlDgvDx7+RT+QjK6FcFT4MpsuoBnHIo/O3StRRg==} + engines: {node: '>= 12.0.0'} + cpu: [x64] + os: [freebsd] + + lightningcss-linux-arm-gnueabihf@1.33.0: + resolution: {integrity: sha512-N7FVBe6iS24MlM6R/4RBTxGhQheZGs7tiQ9U32UtF75NzP5Q7xWPRqLBCKxlRQRk3rY1jCIPLzx7WzOhuUIRLQ==} + engines: {node: '>= 12.0.0'} + cpu: [arm] + os: [linux] + + lightningcss-linux-arm64-gnu@1.33.0: + resolution: {integrity: sha512-j2v/itmy4HlNxlc6voKXYgBqNi0Ng2LShg4z7GufpEgs05P+2suBVyi9I6YHq5uoVFx9ETin3eCEhLVyXGQnKg==} + engines: {node: '>= 12.0.0'} + cpu: [arm64] + os: [linux] + + lightningcss-linux-arm64-musl@1.33.0: + resolution: {integrity: sha512-yiO5ROMuYQgXbC60yjZU5CYSFZGKXL0HFATXt9mHJn1+zW55oCtMI9NfcVhYLMFDL7gV7oBPon/EmMMGg2OvtQ==} + engines: {node: '>= 12.0.0'} + cpu: [arm64] + os: [linux] + + lightningcss-linux-x64-gnu@1.33.0: + resolution: {integrity: sha512-ar+Ju7LmcN0Jo4FpL4hpFybwNG9/3A/Br5KW2n2jyODg3MEZXaDYADdemoNS+BDNfMgKvylJLj4S5tyRActuAg==} + engines: {node: '>= 12.0.0'} + cpu: [x64] + os: [linux] + + lightningcss-linux-x64-musl@1.33.0: + resolution: {integrity: sha512-RYiYbkokw0trfKqqzfF55lginwEPrD3OJDfTuJzFs1MK6iFnDenaz1fqLLtX4ITG3OktJQXOeTaw1awrBAlZPw==} + engines: {node: '>= 12.0.0'} + cpu: [x64] + os: [linux] + + lightningcss-win32-arm64-msvc@1.33.0: + resolution: {integrity: sha512-1K+MPfLSFVpphzpdbfkhlWk6wBrTObBzS2T6db10PNOZgR9GoVsAWzwNyuhUYYbTp23j+4RrncfujZ4uAzXvwA==} + engines: {node: '>= 12.0.0'} + cpu: [arm64] + os: [win32] + + lightningcss-win32-x64-msvc@1.33.0: + resolution: {integrity: sha512-OlEICDx/Xl0FqSp4bry8zFnCvGpig3Gl4gCquvYwHuqJKEC1+n9NgDniFvqHGmMv1ZkqDJrDqKKSykTDX+ehuA==} + engines: {node: '>= 12.0.0'} + cpu: [x64] + os: [win32] + + lightningcss@1.33.0: + resolution: {integrity: sha512-WkUDrojuJs0xkgGf2udWxa3yGBRxPtxUkB79i6aCZLRgc7PM8fZe9TosfPDcvEpQZbuFASnHYmRLBLUbmLOIIA==} + engines: {node: '>= 12.0.0'} + lmdb@3.5.6: resolution: {integrity: sha512-j3uE8ReKNyUWDjhfEFSJqE/1DLtfTR5Z8yFzVHvBjAk37wNg7HdScjcv8ttPHRvrdgPQMPWxFFI0SsdBzI5lBw==} hasBin: true @@ -2248,6 +2583,10 @@ packages: micromark@4.0.2: resolution: {integrity: sha512-zpe98Q6kvavpCr1NPVSCMebCKfD7CA2NqZ+rykeNhONIJBpc1tFKt9hucLGwha3jNTNI8lHpctWJWoimVF4PfA==} + miniflare@5.20260831.0-alpha: + resolution: {integrity: sha512-Hwgh1VDUiPCPGQKODQfUmy7hRAje1D55icB+9png3ueiM64rlSM87nSrtqpxAD+DlLWI4ehnYBuECaXV43zGmQ==} + engines: {node: '>=22.0.0'} + ms@2.1.3: resolution: {integrity: sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==} @@ -2261,6 +2600,11 @@ packages: mz@2.7.0: resolution: {integrity: sha512-z81GNO7nnYMEhrGh9LeymoE4+Yr0Wn5McHIZMK5cfQCl+NDX08sCZgUc9/6MHni9IWuFLm1Z3HTCXu2z9fN62Q==} + nanoid@3.3.18: + resolution: {integrity: sha512-DTg4MJbGMWkfi6VZFdNt2/caMbQy4Ou+Op/hJQvGEWcnVfoA1QA+xzRKAzw9jD6+GVOOeYr/mIcuDSdug6F6+w==} + engines: {node: ^10 || ^12 || ^13.7 || ^14 || >=15.0.1} + hasBin: true + node-addon-api@6.1.0: resolution: {integrity: sha512-+eawOlIgy680F0kBzPUNFhMZGtJ1YmqM6l4+Crf4IkImjYrO/mqPwRMh352g23uIaQKFItcQ64I7KMaJxHgAVA==} @@ -2279,6 +2623,10 @@ packages: resolution: {integrity: sha512-rJgTQnkUnH1sFw8yT6VSU3zD3sWmu6sZhIseY8VX+GRu3P6F7Fu+JNDoXfklElbLJSnc3FUQHVe4cU5hj+BcUg==} engines: {node: '>=0.10.0'} + obug@2.1.4: + resolution: {integrity: sha512-4a+OsYv9UktOJKE+l1A4OufDgdRF9PifWj+tJnHURo/P+WOxpG4GzUFL9qCalmWauao6ogiG+QvnCovwPoyAWA==} + engines: {node: '>=12.20.0'} + ordered-binary@1.6.1: resolution: {integrity: sha512-QkCdPooczexPLiXIrbVOPYkR3VO3T6v2OyKRkR1Xbhpy7/LAVXwahnRCgRp78Oe/Ehf0C/HATAxfSr6eA1oX+w==} @@ -2313,6 +2661,12 @@ packages: resolution: {integrity: sha512-ojmeN0qd+y0jszEtoY48r0Peq5dwMEkIlCOu6Q5f41lfkswXuKtYrhgoTpLnyIcHm24Uhqx+5Tqm2InSwLhE6Q==} engines: {node: '>=8'} + path-to-regexp@6.3.0: + resolution: {integrity: sha512-Yhpw4T9C6hPpgPeA28us07OJeqZ5EzQTkbfwuhsUg0c237RomFoETJgmp2sa3F/41gfLE6G5cqcYwznmeEeOlQ==} + + pathe@2.0.3: + resolution: {integrity: sha512-WUjGcAqP1gQacoQe+OBJsFA7Ld4DyXuUIjZ5cc75cLHvJ7dtNsTugphxIADwspS+AraAUePCKrSVtPLFj/F88w==} + picocolors@1.1.1: resolution: {integrity: sha512-xceH2snhtb5M9liqDsmEw56le376mTZkEX/jEb/RxNFyegNul7eNslCXP9FDj/Lcu0X8KEyMceP2ntpaHrDEVA==} @@ -2320,6 +2674,14 @@ packages: resolution: {integrity: sha512-5gTmgEY/sqK6gFXLIsQNH19lWb4ebPDLA4SdLP7dsWkIXHWlG66oPuVvXSGFPppYZz8ZDZq0dYYrbHfBCVUb1Q==} engines: {node: '>=12'} + picomatch@4.0.7: + resolution: {integrity: sha512-qcJu88Q2IWqJsDD529JKMdwGm/dvInW4HvQnRwiH9JtihJvzGOscDtHE3x1pBKeUOTysQ8kVmLnJ2kJu7yhcGA==} + engines: {node: '>=12'} + + postcss@8.5.26: + resolution: {integrity: sha512-u82N74LFzG8ca+dD8puPnplTXoGH4fTPpVGuIbt36G3qvNlkvfD0lEAZSxaly3KX8TS/L1A1gsCEmvKmBcVbkQ==} + engines: {node: ^10 || ^12 || >=14} + pretty-format@30.3.0: resolution: {integrity: sha512-oG4T3wCbfeuvljnyAzhBvpN45E8iOTXCU/TD3zXW80HA3dQ4ahdqMkWGiPWZvjpQwlbyHrPTWUAqUzGzv4l1JQ==} engines: {node: ^18.14.0 || ^20.0.0 || ^22.0.0 || >=24.0.0} @@ -2407,13 +2769,15 @@ packages: resolution: {integrity: sha512-Xf0nWe6RseziFMu+Ap9biiUbmplq6S9/p+7w7YXP/JBHhrUDDUhwa+vANyubuqfZWTveU//DYVGsDG7RKL/vEw==} engines: {node: '>=0.10.0'} - resolve-pkg-maps@1.0.0: - resolution: {integrity: sha512-seS2Tj26TBVOC2NIc2rOe2y2ZO7efxITtLZcGSOnHHNOQ7CkiUBfw0Iw2ck6xkIhPwLhKNLS8BO+hEpngQlqzw==} - reusify@1.1.0: resolution: {integrity: sha512-g6QUff04oZpHs0eG5p83rFLhHeV00ug/Yf9nZM6fLeUrPguBTkTQOdpAWWspMh55TZfVQDPaN3NQJfbVRAxdIw==} engines: {iojs: '>=1.0.0', node: '>=0.10.0'} + rolldown@1.2.7: + resolution: {integrity: sha512-g0EtLvBjTUB7jhyV0S/TCup3v/XSVl45vUIGbOGU4QPiyjTenCe4mKuFvW9fEgYmS2Fo42AUssRmNuMziXdrig==} + engines: {node: ^20.19.0 || >=22.12.0} + hasBin: true + scheduler@0.27.0: resolution: {integrity: sha512-eNv+WrVbKu1f3vbYJT/xtiF5syA5HPIMtf9IgY/nKg0sWqzAUEvqY/xm7OcZc/qafLx/iO9FgOmeSAp4v5ti/Q==} @@ -2421,6 +2785,15 @@ packages: resolution: {integrity: sha512-vfD3pmTzGpufjScBh50YHKzEu2lxBWhVEHsNGoEXmCmn2hKGfeNLYMzCJpe8cD7gqX7TJluOVpBkAequ6dgMmA==} engines: {node: '>=4'} + semver@7.8.5: + resolution: {integrity: sha512-Y7/KDsb8LjooZpwaqGyulO6DQlksgCncchHGk+sZIY4SBvUocMBEFH5Ur1fI4dV+Jvl0w6cjvucaIi40puRioA==} + engines: {node: '>=10'} + hasBin: true + + sharp@0.35.2: + resolution: {integrity: sha512-FVtFjtBCMiJS6yb5CX7Sop45WFMpeGw6oRKuJnXYgf/f1ms/D7LE/ZUSNxnW7rZ/dbslQWYkoqFHGPaDBtaK4w==} + engines: {node: '>=20.9.0'} + shebang-command@2.0.0: resolution: {integrity: sha512-kHxr2zZpYtdmrN1qDjrrX/Z1rR1kG8Dx+gkpK1G4eXmvXswmcE1hTWBWYUzlraYw1/yZp6YuDY77YtvbN0dmDA==} engines: {node: '>=8'} @@ -2432,6 +2805,9 @@ packages: shellwords-ts@3.0.1: resolution: {integrity: sha512-GabK4ApLMqHFRGlpgNqg8dmtHTnYHt0WUUJkIeMd3QaDrUUBEDXHSSNi3I0PzMimg8W+I0EN4TshQxsnHv1cwg==} + siginfo@2.0.0: + resolution: {integrity: sha512-ybx0WO1/8bSBLEWXZvEd7gMW3Sn3JFlW3TvX1nREbDLRNQNaeNN8WK0meBwPdAaOI7TtRRRJn/Es1zhrrCHu7g==} + sisteransi@1.0.5: resolution: {integrity: sha512-bLGGlR1QxBcynn2d5YmDX4MGjlZvy2MRBDRNHLJ8VI6l6+9FUiyTFNJ0IveOSP0bcXgVDPRcfGqA0pjaqUpfVg==} @@ -2448,6 +2824,10 @@ packages: resolution: {integrity: sha512-g9Q1haeby36OSStwb4ntCGGGaKsaVSjQ68fBxoQcutl5fS1vuY18H3wSt3jFyFtrkx+Kz0V1G85A4MyAdDMi2Q==} engines: {node: '>=8'} + source-map-js@1.2.1: + resolution: {integrity: sha512-UXWMKhLOwVKb728IUtQPXxfYU+usdybtUrK/8uGE8CQMvrhOpwvzDBwj0QhSL7MQc7vIsISBG8VQ8+IDQxpfQA==} + engines: {node: '>=0.10.0'} + space-separated-tokens@2.0.2: resolution: {integrity: sha512-PEGlAwrG8yXGXRjW32fGbg66JAlOAwbObuqVoJpv/mRgoWDQfgH1wDPvtzWyUSNAXBGSk8h755YDbbcEy3SH2Q==} @@ -2458,6 +2838,12 @@ packages: resolution: {integrity: sha512-XlkWvfIm6RmsWtNJx+uqtKLS8eqFbxUg0ZzLXqY0caEy9l7hruX8IpiDnjsLavoBgqCCR71TqWO8MaXYheJ3RQ==} engines: {node: '>=10'} + stackback@0.0.2: + resolution: {integrity: sha512-1XMJE5fQo1jGH6Y/7ebnwPOBEkIEnT4QF32d5R1+VXdXveM0IBMJt8zfaxX1P3QhVwrYe+576+jkANtSS2mBbw==} + + std-env@4.2.0: + resolution: {integrity: sha512-oCUKSupKTHX53EyjDtuZQ64pjLJ6yYCtpmEw0goYxtjG9KpbRe8KAsl2tBUGU9DyMcJ0RwJ8GqJAFzMXcXW1Rw==} + streamx@2.28.0: resolution: {integrity: sha512-1Yowhzjf0ivGMrTIkY9hav5TxobO9qIVqUE41fiCGMGgc3CLlf4MY+9AHmZqBWgDTue0fY9zWjYFVyf6Diuobw==} @@ -2479,6 +2865,10 @@ packages: structured-source@4.0.0: resolution: {integrity: sha512-qGzRFNJDjFieQkl/sVOI2dUjHKRyL9dAJi2gCPGJLbJHBIkyOHxjuocpIEfbLioX+qSJpvbYdT49/YCdMznKxA==} + supports-color@10.2.2: + resolution: {integrity: sha512-SS+jx45GF1QjgEXQx4NJZV9ImqmO2NPz5FNsIHrsDjh2YsHnawpan7SNQ1o8NuhrbHZy9AZhIoCUiCeaW/C80g==} + engines: {node: '>=18'} + supports-color@7.2.0: resolution: {integrity: sha512-qpCAvRl9stuOHveKsn7HncJRvv501qIacKzQlO/+Lwxc9+0q2wLyv4Dfvt80/DPn2pqOBsJdDiogXGR9+OvwRw==} engines: {node: '>=8'} @@ -2511,10 +2901,25 @@ packages: thenify@3.3.1: resolution: {integrity: sha512-RVZSIV5IG10Hk3enotrhvz0T9em6cyHBLkH/YAZuKqd8hRkKhSfCGIcP2KUY0EPxndzANBmNllzWPwak+bheSw==} + tinybench@2.9.0: + resolution: {integrity: sha512-0+DUvqWMValLmha6lr4kD8iAMK1HzV0/aKnCtWb9v9641TnP/MFb7Pc2bxoxQjTXAErryXVgUOfv2YqNllqGeg==} + + tinyexec@1.3.0: + resolution: {integrity: sha512-QKAl9m8gWWGHV8jZcPeym6j+XULi6tOf1mT83WYJ4Lk2ytW/uwAWkrP0uFsdoYMdueVJ0qs26wZ+23xeB4ibNQ==} + engines: {node: '>=18'} + + tinyglobby@0.2.17: + resolution: {integrity: sha512-wXR/dYpcqKmfWpEdZjiKJOwCNFndD0DMnrW/cYjVGttEkBfVgcLFHoNrlj47mjOVic9yyNu65alsgF4NQyTa2g==} + engines: {node: '>=12.0.0'} + tinypool@2.1.0: resolution: {integrity: sha512-Pugqs6M0m7Lv1I7FtxN4aoyToKg1C4tu+/381vH35y8oENM/Ai7f7C4StcoK4/+BSw9ebcS8jRiVrORFKCALLw==} engines: {node: ^20.0.0 || >=22.0.0} + tinyrainbow@3.1.1: + resolution: {integrity: sha512-yau8yJdTt989Mm0Bd/236QnzEiPf2xLLTqUZRUJOo/3CB078LSwzei343DgtJVmfJKJE3TMINY1u42SQsP6mXw==} + engines: {node: '>=14.0.0'} + trim-lines@3.0.1: resolution: {integrity: sha512-kRj8B+YHZCc9kQYdWfJB2/oUl9rA99qbowYYBtr4ui4mZyAQ2JpvVBd/6U2YloATfqBhBTSMhTpgBHtU0Mf3Rg==} @@ -2524,11 +2929,6 @@ packages: tslib@2.8.1: resolution: {integrity: sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w==} - tsx@4.21.0: - resolution: {integrity: sha512-5C1sg4USs1lfG0GFb2RLXsdpXqBSEhAaA/0kPL01wxzpMqLILNxIxIOKiILz+cdg/pLnOUxFYOR5yhHU666wbw==} - engines: {node: '>=18.0.0'} - hasBin: true - tsx@4.23.1: resolution: {integrity: sha512-GQHnkIfxyx1wYCOS/wonik5MVRZU9hi1TEZmzGZSCJB1y9YgoZ8H6itNE/u4suE+yLmOzuE4E5S4TZ/ZX2wcWQ==} engines: {node: '>=18.0.0'} @@ -2542,6 +2942,13 @@ packages: undici-types@6.21.0: resolution: {integrity: sha512-iwDZqg0QAGrg9Rav5H4n0M64c3mkR59cJ6wQp+7C4nI0gsmExaedaYLNO44eT4AtBBwjbTiGPMlt2Md0T9H9JQ==} + undici@7.29.0: + resolution: {integrity: sha512-IDxfleLmmbSskfWSUATiN1nfn2rDuvnMOqb5CWR92iIfojA0Ud+ulOAAEQ57LPr9rWmsreUyf5lwyao+7GNNVw==} + engines: {node: '>=20.18.1'} + + unenv@2.0.0-rc.24: + resolution: {integrity: sha512-i7qRCmY42zmCwnYlh9H2SvLEypEFGye5iRmEMKjcGi7zk9UquigRjFtTLz0TYqr0ZGLZhaMHl/foy1bZR+Cwlw==} + unicode-emoji-modifier-base@1.0.0: resolution: {integrity: sha512-yLSH4py7oFH3oG/9K+XWrz1pSi3dfUrWEnInbxMfArOfc1+33BlGPQtLsOYwvdMy11AwUBetYuaRxSPqgkq+8g==} engines: {node: '>=4'} @@ -2597,6 +3004,90 @@ packages: vfile@6.0.3: resolution: {integrity: sha512-KzIbH/9tXat2u30jf+smMwFCsno4wHVdNmzFyL+T/L3UGqqk6JKfVqOFOZEpZSHADH1k40ab6NUIXZq422ov3Q==} + vite@8.2.2: + resolution: {integrity: sha512-cFKLV/PRgAUlIRm5WjMjJ86jrftzpqcgH+Us+DS8mI3CDNiH30Whrz8uHL3+MOLPAgqbMBAqWdAHAphOAM+z/Q==} + engines: {node: ^20.19.0 || >=22.12.0} + hasBin: true + peerDependencies: + '@types/node': ^20.19.0 || >=22.12.0 + '@vitejs/devtools': ^0.4.0 || ^0.5.0 + esbuild: ^0.27.0 || ^0.28.0 + jiti: '>=1.21.0' + less: ^4.0.0 + sass: ^1.70.0 + sass-embedded: ^1.70.0 + stylus: '>=0.54.8' + sugarss: ^5.0.0 + terser: ^5.16.0 + tsx: 4.23.1 + yaml: ^2.4.2 + peerDependenciesMeta: + '@types/node': + optional: true + '@vitejs/devtools': + optional: true + esbuild: + optional: true + jiti: + optional: true + less: + optional: true + sass: + optional: true + sass-embedded: + optional: true + stylus: + optional: true + sugarss: + optional: true + terser: + optional: true + tsx: + optional: true + yaml: + optional: true + + vitest@4.1.11: + resolution: {integrity: sha512-fhACrNXUidIbGSBr5FlbuBkO7VWC1ZyLl0DO4CU2DrQoAPxX84Ysxs+HeGQpii5lZWV1Q4gBZTTu49mF+A6Edw==} + engines: {node: ^20.0.0 || ^22.0.0 || >=24.0.0} + hasBin: true + peerDependencies: + '@edge-runtime/vm': '*' + '@opentelemetry/api': ^1.9.0 + '@types/node': ^20.0.0 || ^22.0.0 || >=24.0.0 + '@vitest/browser-playwright': 4.1.11 + '@vitest/browser-preview': 4.1.11 + '@vitest/browser-webdriverio': 4.1.11 + '@vitest/coverage-istanbul': 4.1.11 + '@vitest/coverage-v8': 4.1.11 + '@vitest/ui': 4.1.11 + happy-dom: '*' + jsdom: '*' + vite: ^6.0.0 || ^7.0.0 || ^8.0.0 + peerDependenciesMeta: + '@edge-runtime/vm': + optional: true + '@opentelemetry/api': + optional: true + '@types/node': + optional: true + '@vitest/browser-playwright': + optional: true + '@vitest/browser-preview': + optional: true + '@vitest/browser-webdriverio': + optional: true + '@vitest/coverage-istanbul': + optional: true + '@vitest/coverage-v8': + optional: true + '@vitest/ui': + optional: true + happy-dom: + optional: true + jsdom: + optional: true + weak-lru-cache@1.2.2: resolution: {integrity: sha512-DEAoo25RfSYMuTGc9vPJzZcZullwIqRDSI9LOy+fkCJPi6hykCnfKaXTuPBDuXAUcqHXyOgFtHNp/kB2FjYHbw==} @@ -2605,10 +3096,42 @@ packages: engines: {node: '>= 8'} hasBin: true + why-is-node-running@2.3.0: + resolution: {integrity: sha512-hUrmaWBdVDcxvYqnyh09zunKzROWjbZTiNy8dBEjkS7ehEDQibXJ7XvlmtbwuTclUiIyN+CyXQD4Vmko8fNm8w==} + engines: {node: '>=8'} + hasBin: true + + workerd@1.20260831.1: + resolution: {integrity: sha512-A2LwrkBel/FnKABPfeBAMiL6v70+rugnunqQRfWsWZjlhsTZoBScWUVunMy/xLCGLjWCQL2zp39AVR6aO0jurQ==} + engines: {node: '>=16'} + hasBin: true + + wrangler@4.128.0: + resolution: {integrity: sha512-jNXy9e8/pbx8iqTzXPiuflnitKJZoAfEUSUUDLW87bwyeMvJ7kb3yQMSbxEcfNdfHqJW38KRcKaLljOYV4N/4w==} + engines: {node: '>=22.0.0'} + hasBin: true + peerDependencies: + '@cloudflare/workers-types': 5.20260831.1 + peerDependenciesMeta: + '@cloudflare/workers-types': + optional: true + wrap-ansi@7.0.0: resolution: {integrity: sha512-YVGIj2kamLSTxw6NsZjoBxfSwsn0ycdesmc4p+Q21c5zPuZ1pl+NfxVdxPtdHvmNVOQ6XSYG4AUtyt/Fi7D16Q==} engines: {node: '>=10'} + ws@8.21.0: + resolution: {integrity: sha512-Vsp28b7DRcimFQvrqu2Wek3z1iYxDCWqHYB8Qsnk/S4RfaCQzPGPyBNuVjJV3cd6UiKtUtp6sNM77gWvzcCH+g==} + engines: {node: '>=10.0.0'} + peerDependencies: + bufferutil: ^4.0.1 + utf-8-validate: '>=5.0.2' + peerDependenciesMeta: + bufferutil: + optional: true + utf-8-validate: + optional: true + y18n@5.0.8: resolution: {integrity: sha512-0pfFzegeDWJHJIAmTLRP2DwHjdF5s7jo9tuztdQxAhINCdvS+3nGINqPd00AphqJR/0LhANUS6/+7SCb98YOfA==} engines: {node: '>=10'} @@ -2621,6 +3144,12 @@ packages: resolution: {integrity: sha512-D1mvvtDG0L5ft/jGWkLpG1+m0eQxOfaBvTNELraWj22wSVUMWxZUvYgJYcKh6jGGIkJFhH4IZPQhR4TKpc8mBw==} engines: {node: '>=10'} + youch-core@0.3.3: + resolution: {integrity: sha512-ho7XuGjLaJ2hWHoK8yFnsUGy2Y5uDpqSTq1FkHLK4/oqKtyUU1AFbOOxY4IpC9f0fTLjwYbslUz0Po5BpD1wrA==} + + youch@4.1.0-beta.10: + resolution: {integrity: sha512-rLfVLB4FgQneDr0dv1oddCVZmKjcJ6yX6mS4pU82Mq/Dt9a3cLZQ62pDBL4AUO+uVrCvtWz3ZFUL2HFAFJ/BXQ==} + zod@4.3.6: resolution: {integrity: sha512-rftlrkhHZOcjDwkGlnUtZZkvaPHCsDATp4pGpuOOMDaTdDDXF91wuVDJoWoPsKX/3YPQ5fHuF3STjcYyKr+Qhg==} @@ -2660,9 +3189,53 @@ snapshots: fast-wrap-ansi: 0.2.2 sisteransi: 1.0.5 + '@cloudflare/kv-asset-handler@0.5.0': {} + + '@cloudflare/unenv-preset@2.16.1(unenv@2.0.0-rc.24)(workerd@1.20260831.1)': + dependencies: + unenv: 2.0.0-rc.24 + optionalDependencies: + workerd: 1.20260831.1 + + '@cloudflare/vitest-plugin@1.1.3(@cloudflare/workers-types@5.20260831.1)(@vitest/runner@4.1.11)(@vitest/snapshot@4.1.11)(vitest@4.1.11(@types/node@22.19.15)(vite@8.2.2(@types/node@22.19.15)(esbuild@0.28.1)(tsx@4.23.1)))': + dependencies: + '@vitest/runner': 4.1.11 + '@vitest/snapshot': 4.1.11 + cjs-module-lexer: 1.2.3 + esbuild: 0.28.1 + miniflare: 5.20260831.0-alpha + vitest: 4.1.11(@types/node@22.19.15)(vite@8.2.2(@types/node@22.19.15)(esbuild@0.28.1)(tsx@4.23.1)) + wrangler: 4.128.0(@cloudflare/workers-types@5.20260831.1) + zod: 4.4.3 + transitivePeerDependencies: + - '@cloudflare/workers-types' + - bufferutil + - utf-8-validate + + '@cloudflare/workerd-darwin-64@1.20260831.1': + optional: true + + '@cloudflare/workerd-darwin-arm64@1.20260831.1': + optional: true + + '@cloudflare/workerd-linux-64@1.20260831.1': + optional: true + + '@cloudflare/workerd-linux-arm64@1.20260831.1': + optional: true + + '@cloudflare/workerd-windows-64@1.20260831.1': + optional: true + + '@cloudflare/workers-types@5.20260831.1': {} + '@colors/colors@1.5.0': optional: true + '@cspotcode/source-map-support@0.8.1': + dependencies: + '@jridgewell/trace-mapping': 0.3.9 + '@durable-streams/client@0.2.2': dependencies: '@microsoft/fetch-event-source': 2.0.1 @@ -2748,184 +3321,217 @@ snapshots: dependencies: effection: 4.1.0 - '@esbuild/aix-ppc64@0.27.4': + '@emnapi/runtime@1.11.3': + dependencies: + tslib: 2.8.1 optional: true '@esbuild/aix-ppc64@0.28.1': optional: true - '@esbuild/android-arm64@0.27.4': + '@esbuild/android-arm64@0.28.1': optional: true - '@esbuild/android-arm64@0.28.1': + '@esbuild/android-arm@0.28.1': optional: true - '@esbuild/android-arm@0.27.4': + '@esbuild/android-x64@0.28.1': optional: true - '@esbuild/android-arm@0.28.1': + '@esbuild/darwin-arm64@0.28.1': optional: true - '@esbuild/android-x64@0.27.4': + '@esbuild/darwin-x64@0.28.1': optional: true - '@esbuild/android-x64@0.28.1': + '@esbuild/freebsd-arm64@0.28.1': optional: true - '@esbuild/darwin-arm64@0.27.4': + '@esbuild/freebsd-x64@0.28.1': optional: true - '@esbuild/darwin-arm64@0.28.1': + '@esbuild/linux-arm64@0.28.1': optional: true - '@esbuild/darwin-x64@0.27.4': + '@esbuild/linux-arm@0.28.1': optional: true - '@esbuild/darwin-x64@0.28.1': + '@esbuild/linux-ia32@0.28.1': + optional: true + + '@esbuild/linux-loong64@0.28.1': + optional: true + + '@esbuild/linux-mips64el@0.28.1': optional: true - '@esbuild/freebsd-arm64@0.27.4': + '@esbuild/linux-ppc64@0.28.1': optional: true - '@esbuild/freebsd-arm64@0.28.1': + '@esbuild/linux-riscv64@0.28.1': optional: true - '@esbuild/freebsd-x64@0.27.4': + '@esbuild/linux-s390x@0.28.1': optional: true - '@esbuild/freebsd-x64@0.28.1': + '@esbuild/linux-x64@0.28.1': optional: true - '@esbuild/linux-arm64@0.27.4': + '@esbuild/netbsd-arm64@0.28.1': optional: true - '@esbuild/linux-arm64@0.28.1': + '@esbuild/netbsd-x64@0.28.1': optional: true - '@esbuild/linux-arm@0.27.4': + '@esbuild/openbsd-arm64@0.28.1': optional: true - '@esbuild/linux-arm@0.28.1': + '@esbuild/openbsd-x64@0.28.1': optional: true - '@esbuild/linux-ia32@0.27.4': + '@esbuild/openharmony-arm64@0.28.1': optional: true - '@esbuild/linux-ia32@0.28.1': + '@esbuild/sunos-x64@0.28.1': optional: true - '@esbuild/linux-loong64@0.27.4': + '@esbuild/win32-arm64@0.28.1': optional: true - '@esbuild/linux-loong64@0.28.1': + '@esbuild/win32-ia32@0.28.1': optional: true - '@esbuild/linux-mips64el@0.27.4': + '@esbuild/win32-x64@0.28.1': optional: true - '@esbuild/linux-mips64el@0.28.1': - optional: true + '@floating-ui/core@1.8.0': + dependencies: + '@floating-ui/utils': 0.2.12 - '@esbuild/linux-ppc64@0.27.4': - optional: true + '@floating-ui/dom@1.8.0': + dependencies: + '@floating-ui/core': 1.8.0 + '@floating-ui/utils': 0.2.12 - '@esbuild/linux-ppc64@0.28.1': - optional: true + '@floating-ui/react-dom@2.1.9(react-dom@19.2.0(react@19.2.0))(react@19.2.0)': + dependencies: + '@floating-ui/dom': 1.8.0 + react: 19.2.0 + react-dom: 19.2.0(react@19.2.0) - '@esbuild/linux-riscv64@0.27.4': - optional: true + '@floating-ui/utils@0.2.12': {} - '@esbuild/linux-riscv64@0.28.1': - optional: true + '@fontsource/montserrat@5.3.0': {} - '@esbuild/linux-s390x@0.27.4': - optional: true + '@fontsource/space-mono@5.3.0': {} - '@esbuild/linux-s390x@0.28.1': - optional: true + '@harperfast/extended-iterable@1.0.3': {} - '@esbuild/linux-x64@0.27.4': + '@img/colour@1.1.0': {} + + '@img/sharp-darwin-arm64@0.35.2': + optionalDependencies: + '@img/sharp-libvips-darwin-arm64': 1.3.1 optional: true - '@esbuild/linux-x64@0.28.1': + '@img/sharp-darwin-x64@0.35.2': + optionalDependencies: + '@img/sharp-libvips-darwin-x64': 1.3.1 optional: true - '@esbuild/netbsd-arm64@0.27.4': + '@img/sharp-freebsd-wasm32@0.35.2': + dependencies: + '@img/sharp-wasm32': 0.35.2 optional: true - '@esbuild/netbsd-arm64@0.28.1': + '@img/sharp-libvips-darwin-arm64@1.3.1': optional: true - '@esbuild/netbsd-x64@0.27.4': + '@img/sharp-libvips-darwin-x64@1.3.1': optional: true - '@esbuild/netbsd-x64@0.28.1': + '@img/sharp-libvips-linux-arm64@1.3.1': optional: true - '@esbuild/openbsd-arm64@0.27.4': + '@img/sharp-libvips-linux-arm@1.3.1': optional: true - '@esbuild/openbsd-arm64@0.28.1': + '@img/sharp-libvips-linux-ppc64@1.3.1': optional: true - '@esbuild/openbsd-x64@0.27.4': + '@img/sharp-libvips-linux-riscv64@1.3.1': optional: true - '@esbuild/openbsd-x64@0.28.1': + '@img/sharp-libvips-linux-s390x@1.3.1': optional: true - '@esbuild/openharmony-arm64@0.27.4': + '@img/sharp-libvips-linux-x64@1.3.1': optional: true - '@esbuild/openharmony-arm64@0.28.1': + '@img/sharp-libvips-linuxmusl-arm64@1.3.1': optional: true - '@esbuild/sunos-x64@0.27.4': + '@img/sharp-libvips-linuxmusl-x64@1.3.1': optional: true - '@esbuild/sunos-x64@0.28.1': + '@img/sharp-linux-arm64@0.35.2': + optionalDependencies: + '@img/sharp-libvips-linux-arm64': 1.3.1 optional: true - '@esbuild/win32-arm64@0.27.4': + '@img/sharp-linux-arm@0.35.2': + optionalDependencies: + '@img/sharp-libvips-linux-arm': 1.3.1 optional: true - '@esbuild/win32-arm64@0.28.1': + '@img/sharp-linux-ppc64@0.35.2': + optionalDependencies: + '@img/sharp-libvips-linux-ppc64': 1.3.1 optional: true - '@esbuild/win32-ia32@0.27.4': + '@img/sharp-linux-riscv64@0.35.2': + optionalDependencies: + '@img/sharp-libvips-linux-riscv64': 1.3.1 optional: true - '@esbuild/win32-ia32@0.28.1': + '@img/sharp-linux-s390x@0.35.2': + optionalDependencies: + '@img/sharp-libvips-linux-s390x': 1.3.1 optional: true - '@esbuild/win32-x64@0.27.4': + '@img/sharp-linux-x64@0.35.2': + optionalDependencies: + '@img/sharp-libvips-linux-x64': 1.3.1 optional: true - '@esbuild/win32-x64@0.28.1': + '@img/sharp-linuxmusl-arm64@0.35.2': + optionalDependencies: + '@img/sharp-libvips-linuxmusl-arm64': 1.3.1 optional: true - '@floating-ui/core@1.8.0': - dependencies: - '@floating-ui/utils': 0.2.12 + '@img/sharp-linuxmusl-x64@0.35.2': + optionalDependencies: + '@img/sharp-libvips-linuxmusl-x64': 1.3.1 + optional: true - '@floating-ui/dom@1.8.0': + '@img/sharp-wasm32@0.35.2': dependencies: - '@floating-ui/core': 1.8.0 - '@floating-ui/utils': 0.2.12 + '@emnapi/runtime': 1.11.3 + optional: true - '@floating-ui/react-dom@2.1.9(react-dom@19.2.0(react@19.2.0))(react@19.2.0)': + '@img/sharp-webcontainers-wasm32@0.35.2': dependencies: - '@floating-ui/dom': 1.8.0 - react: 19.2.0 - react-dom: 19.2.0(react@19.2.0) - - '@floating-ui/utils@0.2.12': {} + '@img/sharp-wasm32': 0.35.2 + optional: true - '@fontsource/montserrat@5.3.0': {} + '@img/sharp-win32-arm64@0.35.2': + optional: true - '@fontsource/space-mono@5.3.0': {} + '@img/sharp-win32-ia32@0.35.2': + optional: true - '@harperfast/extended-iterable@1.0.3': {} + '@img/sharp-win32-x64@0.35.2': + optional: true '@jest/diff-sequences@30.3.0': {} @@ -2954,8 +3560,15 @@ snapshots: '@types/yargs': 17.0.35 chalk: 4.1.2 + '@jridgewell/resolve-uri@3.1.2': {} + '@jridgewell/sourcemap-codec@1.5.5': {} + '@jridgewell/trace-mapping@0.3.9': + dependencies: + '@jridgewell/resolve-uri': 3.1.2 + '@jridgewell/sourcemap-codec': 1.5.5 + '@lmdb/lmdb-darwin-arm64@3.5.6': optional: true @@ -2999,6 +3612,8 @@ snapshots: '@neophi/sieve-cache@1.5.0': {} + '@oxc-project/types@0.148.0': {} + '@oxfmt/binding-android-arm-eabi@0.41.0': optional: true @@ -3113,6 +3728,18 @@ snapshots: '@oxlint/binding-win32-x64-msvc@1.74.0': optional: true + '@poppinss/colors@4.1.6': + dependencies: + kleur: 4.1.5 + + '@poppinss/dumper@0.6.5': + dependencies: + '@poppinss/colors': 4.1.6 + '@sindresorhus/is': 7.2.0 + supports-color: 10.2.2 + + '@poppinss/exception@1.2.3': {} + '@radix-ui/number@1.1.3': {} '@radix-ui/primitive@1.1.7': {} @@ -3464,6 +4091,53 @@ snapshots: lodash: 4.18.1 lodash-es: 4.18.1 + '@rolldown/binding-android-arm-eabi@1.2.7': + optional: true + + '@rolldown/binding-android-arm64@1.2.7': + optional: true + + '@rolldown/binding-darwin-arm64@1.2.7': + optional: true + + '@rolldown/binding-darwin-x64@1.2.7': + optional: true + + '@rolldown/binding-freebsd-x64@1.2.7': + optional: true + + '@rolldown/binding-linux-arm-gnueabihf@1.2.7': + optional: true + + '@rolldown/binding-linux-arm64-gnu@1.2.7': + optional: true + + '@rolldown/binding-linux-arm64-musl@1.2.7': + optional: true + + '@rolldown/binding-linux-ppc64-gnu@1.2.7': + optional: true + + '@rolldown/binding-linux-s390x-gnu@1.2.7': + optional: true + + '@rolldown/binding-linux-x64-gnu@1.2.7': + optional: true + + '@rolldown/binding-linux-x64-musl@1.2.7': + optional: true + + '@rolldown/binding-openharmony-arm64@1.2.7': + optional: true + + '@rolldown/binding-win32-arm64-msvc@1.2.7': + optional: true + + '@rolldown/binding-win32-x64-msvc@1.2.7': + optional: true + + '@rolldown/pluginutils@1.0.1': {} + '@secretlint/core@13.0.4': dependencies: '@secretlint/profiler': 13.0.4 @@ -3483,12 +4157,25 @@ snapshots: '@sindresorhus/is@4.6.0': {} + '@sindresorhus/is@7.2.0': {} + + '@speed-highlight/core@1.2.24': {} + '@standard-schema/spec@1.1.0': {} + '@types/chai@5.2.3': + dependencies: + '@types/deep-eql': 4.0.2 + assertion-error: 2.0.1 + '@types/debug@4.1.12': dependencies: '@types/ms': 2.1.0 + '@types/deep-eql@4.0.2': {} + + '@types/estree@1.0.9': {} + '@types/hast@3.0.5': dependencies: '@types/unist': 3.0.3 @@ -3527,6 +4214,47 @@ snapshots: '@ungap/structured-clone@1.3.3': {} + '@vitest/expect@4.1.11': + dependencies: + '@standard-schema/spec': 1.1.0 + '@types/chai': 5.2.3 + '@vitest/spy': 4.1.11 + '@vitest/utils': 4.1.11 + chai: 6.2.2 + tinyrainbow: 3.1.1 + + '@vitest/mocker@4.1.11(vite@8.2.2(@types/node@22.19.15)(esbuild@0.28.1)(tsx@4.23.1))': + dependencies: + '@vitest/spy': 4.1.11 + estree-walker: 3.0.3 + magic-string: 0.30.21 + optionalDependencies: + vite: 8.2.2(@types/node@22.19.15)(esbuild@0.28.1)(tsx@4.23.1) + + '@vitest/pretty-format@4.1.11': + dependencies: + tinyrainbow: 3.1.1 + + '@vitest/runner@4.1.11': + dependencies: + '@vitest/utils': 4.1.11 + pathe: 2.0.3 + + '@vitest/snapshot@4.1.11': + dependencies: + '@vitest/pretty-format': 4.1.11 + '@vitest/utils': 4.1.11 + magic-string: 0.30.21 + pathe: 2.0.3 + + '@vitest/spy@4.1.11': {} + + '@vitest/utils@4.1.11': + dependencies: + '@vitest/pretty-format': 4.1.11 + convert-source-map: 2.0.0 + tinyrainbow: 3.1.1 + '@x0k/json-schema-merge@1.0.4': dependencies: '@types/json-schema': 7.0.15 @@ -3580,6 +4308,8 @@ snapshots: dependencies: tslib: 2.8.1 + assertion-error@2.0.1: {} + b4a@1.8.1: {} bail@2.0.2: {} @@ -3613,12 +4343,16 @@ snapshots: dependencies: bare-path: 3.1.1 + blake3-wasm@2.1.5: {} + boolbase@1.0.0: {} boundary@2.0.0: {} ccount@2.0.1: {} + chai@6.2.2: {} + chalk@4.1.2: dependencies: ansi-styles: 4.3.0 @@ -3636,6 +4370,8 @@ snapshots: ci-info@4.4.0: {} + cjs-module-lexer@1.2.3: {} + class-variance-authority@0.7.1: dependencies: clsx: 2.1.1 @@ -3689,6 +4425,10 @@ snapshots: dependencies: '@standard-schema/spec': 1.1.0 + convert-source-map@2.0.0: {} + + cookie@1.1.1: {} + cross-spawn@7.0.6: dependencies: path-key: 3.1.1 @@ -3725,34 +4465,9 @@ snapshots: environment@1.1.0: {} - esbuild@0.27.4: - optionalDependencies: - '@esbuild/aix-ppc64': 0.27.4 - '@esbuild/android-arm': 0.27.4 - '@esbuild/android-arm64': 0.27.4 - '@esbuild/android-x64': 0.27.4 - '@esbuild/darwin-arm64': 0.27.4 - '@esbuild/darwin-x64': 0.27.4 - '@esbuild/freebsd-arm64': 0.27.4 - '@esbuild/freebsd-x64': 0.27.4 - '@esbuild/linux-arm': 0.27.4 - '@esbuild/linux-arm64': 0.27.4 - '@esbuild/linux-ia32': 0.27.4 - '@esbuild/linux-loong64': 0.27.4 - '@esbuild/linux-mips64el': 0.27.4 - '@esbuild/linux-ppc64': 0.27.4 - '@esbuild/linux-riscv64': 0.27.4 - '@esbuild/linux-s390x': 0.27.4 - '@esbuild/linux-x64': 0.27.4 - '@esbuild/netbsd-arm64': 0.27.4 - '@esbuild/netbsd-x64': 0.27.4 - '@esbuild/openbsd-arm64': 0.27.4 - '@esbuild/openbsd-x64': 0.27.4 - '@esbuild/openharmony-arm64': 0.27.4 - '@esbuild/sunos-x64': 0.27.4 - '@esbuild/win32-arm64': 0.27.4 - '@esbuild/win32-ia32': 0.27.4 - '@esbuild/win32-x64': 0.27.4 + error-stack-parser-es@1.0.5: {} + + es-module-lexer@2.3.2: {} esbuild@0.28.1: optionalDependencies: @@ -3789,12 +4504,18 @@ snapshots: esprima@4.0.1: {} + estree-walker@3.0.3: + dependencies: + '@types/estree': 1.0.9 + events-universal@1.0.1: dependencies: bare-events: 2.9.1 transitivePeerDependencies: - bare-abort-controller + expect-type@1.4.0: {} + expect@30.3.0: dependencies: '@jest/expect-utils': 30.3.0 @@ -3834,6 +4555,10 @@ snapshots: dependencies: reusify: 1.1.0 + fdir@6.5.0(picomatch@4.0.7): + optionalDependencies: + picomatch: 4.0.7 + fsevents@2.3.3: optional: true @@ -3841,10 +4566,6 @@ snapshots: get-nonce@1.0.1: {} - get-tsconfig@4.13.6: - dependencies: - resolve-pkg-maps: 1.0.0 - graceful-fs@4.2.11: {} gray-matter@4.0.3: @@ -3950,6 +4671,57 @@ snapshots: kind-of@6.0.3: {} + kleur@4.1.5: {} + + lightningcss-android-arm64@1.33.0: + optional: true + + lightningcss-darwin-arm64@1.33.0: + optional: true + + lightningcss-darwin-x64@1.33.0: + optional: true + + lightningcss-freebsd-x64@1.33.0: + optional: true + + lightningcss-linux-arm-gnueabihf@1.33.0: + optional: true + + lightningcss-linux-arm64-gnu@1.33.0: + optional: true + + lightningcss-linux-arm64-musl@1.33.0: + optional: true + + lightningcss-linux-x64-gnu@1.33.0: + optional: true + + lightningcss-linux-x64-musl@1.33.0: + optional: true + + lightningcss-win32-arm64-msvc@1.33.0: + optional: true + + lightningcss-win32-x64-msvc@1.33.0: + optional: true + + lightningcss@1.33.0: + dependencies: + detect-libc: 2.1.2 + optionalDependencies: + lightningcss-android-arm64: 1.33.0 + lightningcss-darwin-arm64: 1.33.0 + lightningcss-darwin-x64: 1.33.0 + lightningcss-freebsd-x64: 1.33.0 + lightningcss-linux-arm-gnueabihf: 1.33.0 + lightningcss-linux-arm64-gnu: 1.33.0 + lightningcss-linux-arm64-musl: 1.33.0 + lightningcss-linux-x64-gnu: 1.33.0 + lightningcss-linux-x64-musl: 1.33.0 + lightningcss-win32-arm64-msvc: 1.33.0 + lightningcss-win32-x64-msvc: 1.33.0 + lmdb@3.5.6: dependencies: '@harperfast/extended-iterable': 1.0.3 @@ -4185,6 +4957,18 @@ snapshots: transitivePeerDependencies: - supports-color + miniflare@5.20260831.0-alpha: + dependencies: + '@cspotcode/source-map-support': 0.8.1 + sharp: 0.35.2 + undici: 7.29.0 + workerd: 1.20260831.1 + ws: 8.21.0 + youch: 4.1.0-beta.10 + transitivePeerDependencies: + - bufferutil + - utf-8-validate + ms@2.1.3: {} msgpackr-extract@3.0.4: @@ -4209,6 +4993,8 @@ snapshots: object-assign: 4.1.1 thenify-all: 1.6.0 + nanoid@3.3.18: {} + node-addon-api@6.1.0: {} node-emoji@2.2.0: @@ -4228,6 +5014,8 @@ snapshots: object-assign@4.1.1: {} + obug@2.1.4: {} + ordered-binary@1.6.1: {} oxfmt@0.41.0: @@ -4286,10 +5074,22 @@ snapshots: path-key@3.1.1: {} + path-to-regexp@6.3.0: {} + + pathe@2.0.3: {} + picocolors@1.1.1: {} picomatch@4.0.3: {} + picomatch@4.0.7: {} + + postcss@8.5.26: + dependencies: + nanoid: 3.3.18 + picocolors: 1.1.1 + source-map-js: 1.2.1 + pretty-format@30.3.0: dependencies: '@jest/schemas': 30.0.5 @@ -4387,10 +5187,29 @@ snapshots: require-from-string@2.0.2: {} - resolve-pkg-maps@1.0.0: {} - reusify@1.1.0: {} + rolldown@1.2.7: + dependencies: + '@oxc-project/types': 0.148.0 + '@rolldown/pluginutils': 1.0.1 + optionalDependencies: + '@rolldown/binding-android-arm-eabi': 1.2.7 + '@rolldown/binding-android-arm64': 1.2.7 + '@rolldown/binding-darwin-arm64': 1.2.7 + '@rolldown/binding-darwin-x64': 1.2.7 + '@rolldown/binding-freebsd-x64': 1.2.7 + '@rolldown/binding-linux-arm-gnueabihf': 1.2.7 + '@rolldown/binding-linux-arm64-gnu': 1.2.7 + '@rolldown/binding-linux-arm64-musl': 1.2.7 + '@rolldown/binding-linux-ppc64-gnu': 1.2.7 + '@rolldown/binding-linux-s390x-gnu': 1.2.7 + '@rolldown/binding-linux-x64-gnu': 1.2.7 + '@rolldown/binding-linux-x64-musl': 1.2.7 + '@rolldown/binding-openharmony-arm64': 1.2.7 + '@rolldown/binding-win32-arm64-msvc': 1.2.7 + '@rolldown/binding-win32-x64-msvc': 1.2.7 + scheduler@0.27.0: {} section-matter@1.0.0: @@ -4398,6 +5217,40 @@ snapshots: extend-shallow: 2.0.1 kind-of: 6.0.3 + semver@7.8.5: {} + + sharp@0.35.2: + dependencies: + '@img/colour': 1.1.0 + detect-libc: 2.1.2 + semver: 7.8.5 + optionalDependencies: + '@img/sharp-darwin-arm64': 0.35.2 + '@img/sharp-darwin-x64': 0.35.2 + '@img/sharp-freebsd-wasm32': 0.35.2 + '@img/sharp-libvips-darwin-arm64': 1.3.1 + '@img/sharp-libvips-darwin-x64': 1.3.1 + '@img/sharp-libvips-linux-arm': 1.3.1 + '@img/sharp-libvips-linux-arm64': 1.3.1 + '@img/sharp-libvips-linux-ppc64': 1.3.1 + '@img/sharp-libvips-linux-riscv64': 1.3.1 + '@img/sharp-libvips-linux-s390x': 1.3.1 + '@img/sharp-libvips-linux-x64': 1.3.1 + '@img/sharp-libvips-linuxmusl-arm64': 1.3.1 + '@img/sharp-libvips-linuxmusl-x64': 1.3.1 + '@img/sharp-linux-arm': 0.35.2 + '@img/sharp-linux-arm64': 0.35.2 + '@img/sharp-linux-ppc64': 0.35.2 + '@img/sharp-linux-riscv64': 0.35.2 + '@img/sharp-linux-s390x': 0.35.2 + '@img/sharp-linux-x64': 0.35.2 + '@img/sharp-linuxmusl-arm64': 0.35.2 + '@img/sharp-linuxmusl-x64': 0.35.2 + '@img/sharp-webcontainers-wasm32': 0.35.2 + '@img/sharp-win32-arm64': 0.35.2 + '@img/sharp-win32-ia32': 0.35.2 + '@img/sharp-win32-x64': 0.35.2 + shebang-command@2.0.0: dependencies: shebang-regex: 3.0.0 @@ -4406,6 +5259,8 @@ snapshots: shellwords-ts@3.0.1: {} + siginfo@2.0.0: {} + sisteransi@1.0.5: {} skillflag@0.2.1: @@ -4423,6 +5278,8 @@ snapshots: slash@3.0.0: {} + source-map-js@1.2.1: {} + space-separated-tokens@2.0.2: {} sprintf-js@1.0.3: {} @@ -4431,6 +5288,10 @@ snapshots: dependencies: escape-string-regexp: 2.0.0 + stackback@0.0.2: {} + + std-env@4.2.0: {} + streamx@2.28.0: dependencies: events-universal: 1.0.1 @@ -4461,6 +5322,8 @@ snapshots: dependencies: boundary: 2.0.0 + supports-color@10.2.2: {} + supports-color@7.2.0: dependencies: has-flag: 4.0.0 @@ -4506,21 +5369,25 @@ snapshots: dependencies: any-promise: 1.3.0 + tinybench@2.9.0: {} + + tinyexec@1.3.0: {} + + tinyglobby@0.2.17: + dependencies: + fdir: 6.5.0(picomatch@4.0.7) + picomatch: 4.0.7 + tinypool@2.1.0: {} + tinyrainbow@3.1.1: {} + trim-lines@3.0.1: {} trough@2.2.0: {} tslib@2.8.1: {} - tsx@4.21.0: - dependencies: - esbuild: 0.27.4 - get-tsconfig: 4.13.6 - optionalDependencies: - fsevents: 2.3.3 - tsx@4.23.1: dependencies: esbuild: 0.28.1 @@ -4531,6 +5398,12 @@ snapshots: undici-types@6.21.0: {} + undici@7.29.0: {} + + unenv@2.0.0-rc.24: + dependencies: + pathe: 2.0.3 + unicode-emoji-modifier-base@1.0.0: {} unified@11.0.5: @@ -4597,18 +5470,90 @@ snapshots: '@types/unist': 3.0.3 vfile-message: 4.0.3 + vite@8.2.2(@types/node@22.19.15)(esbuild@0.28.1)(tsx@4.23.1): + dependencies: + lightningcss: 1.33.0 + picomatch: 4.0.7 + postcss: 8.5.26 + rolldown: 1.2.7 + tinyglobby: 0.2.17 + optionalDependencies: + '@types/node': 22.19.15 + esbuild: 0.28.1 + fsevents: 2.3.3 + tsx: 4.23.1 + + vitest@4.1.11(@types/node@22.19.15)(vite@8.2.2(@types/node@22.19.15)(esbuild@0.28.1)(tsx@4.23.1)): + dependencies: + '@vitest/expect': 4.1.11 + '@vitest/mocker': 4.1.11(vite@8.2.2(@types/node@22.19.15)(esbuild@0.28.1)(tsx@4.23.1)) + '@vitest/pretty-format': 4.1.11 + '@vitest/runner': 4.1.11 + '@vitest/snapshot': 4.1.11 + '@vitest/spy': 4.1.11 + '@vitest/utils': 4.1.11 + es-module-lexer: 2.3.2 + expect-type: 1.4.0 + magic-string: 0.30.21 + obug: 2.1.4 + pathe: 2.0.3 + picomatch: 4.0.7 + std-env: 4.2.0 + tinybench: 2.9.0 + tinyexec: 1.3.0 + tinyglobby: 0.2.17 + tinyrainbow: 3.1.1 + vite: 8.2.2(@types/node@22.19.15)(esbuild@0.28.1)(tsx@4.23.1) + why-is-node-running: 2.3.0 + optionalDependencies: + '@types/node': 22.19.15 + transitivePeerDependencies: + - msw + weak-lru-cache@1.2.2: {} which@2.0.2: dependencies: isexe: 2.0.0 + why-is-node-running@2.3.0: + dependencies: + siginfo: 2.0.0 + stackback: 0.0.2 + + workerd@1.20260831.1: + optionalDependencies: + '@cloudflare/workerd-darwin-64': 1.20260831.1 + '@cloudflare/workerd-darwin-arm64': 1.20260831.1 + '@cloudflare/workerd-linux-64': 1.20260831.1 + '@cloudflare/workerd-linux-arm64': 1.20260831.1 + '@cloudflare/workerd-windows-64': 1.20260831.1 + + wrangler@4.128.0(@cloudflare/workers-types@5.20260831.1): + dependencies: + '@cloudflare/kv-asset-handler': 0.5.0 + '@cloudflare/unenv-preset': 2.16.1(unenv@2.0.0-rc.24)(workerd@1.20260831.1) + blake3-wasm: 2.1.5 + esbuild: 0.28.1 + miniflare: 5.20260831.0-alpha + path-to-regexp: 6.3.0 + unenv: 2.0.0-rc.24 + workerd: 1.20260831.1 + optionalDependencies: + '@cloudflare/workers-types': 5.20260831.1 + fsevents: 2.3.3 + transitivePeerDependencies: + - bufferutil + - utf-8-validate + wrap-ansi@7.0.0: dependencies: ansi-styles: 4.3.0 string-width: 4.2.3 strip-ansi: 6.0.1 + ws@8.21.0: {} + y18n@5.0.8: {} yargs-parser@20.2.9: {} @@ -4623,6 +5568,19 @@ snapshots: y18n: 5.0.8 yargs-parser: 20.2.9 + youch-core@0.3.3: + dependencies: + '@poppinss/exception': 1.2.3 + error-stack-parser-es: 1.0.5 + + youch@4.1.0-beta.10: + dependencies: + '@poppinss/colors': 4.1.6 + '@poppinss/dumper': 0.6.5 + '@speed-highlight/core': 1.2.24 + cookie: 1.1.1 + youch-core: 0.3.3 + zod@4.3.6: {} zod@4.4.3: {} diff --git a/scripts/tests/ci-workflow.test.ts b/scripts/tests/ci-workflow.test.ts index 922d5b164..0f04745d4 100644 --- a/scripts/tests/ci-workflow.test.ts +++ b/scripts/tests/ci-workflow.test.ts @@ -472,6 +472,26 @@ describe("the conditional CI jobs", () => { return found.if; } + /** + * The workerd suite runs nowhere else. A Durable Object's acquisition + * lifetime, its eviction and its transaction atomicity are properties of that + * runtime, and the `.vitest.ts` files that prove them are invisible to the + * Deno, Node and Bun corpora by design — so if this job stopped running the + * evidence would go with it and every other job would still be green. + */ + it("owns the Cloudflare typecheck and the workerd suite", function* () { + const jobs = yield* workflow(); + const job = jobs["test-cloudflare"]; + expect(job).toBeDefined(); + const commands = (job?.steps ?? []).flatMap((step) => + step.run === undefined ? [] : [step.run], + ); + expect(commands).toContain("pnpm check:cloudflare"); + expect(commands).toContain("pnpm test:cloudflare"); + // It runs on every event, so `green` requires success from it unconditionally. + expect(job?.if).toBeUndefined(); + }); + it("runs main-green on a pull request and on nothing else", function* () { expect(conditional(yield* workflow(), "main-green")).toEqual( "github.event_name == 'pull_request'", diff --git a/specs/executable-mdx-spec.md b/specs/executable-mdx-spec.md index bfdd3b769..a6e89118d 100644 --- a/specs/executable-mdx-spec.md +++ b/specs/executable-mdx-spec.md @@ -7497,6 +7497,46 @@ component grants that admission, and a repository component that takes the name `Fetch` is not the pinned identity. +### 6.19 Software-factory constructs + +An issue-driven software factory adds ten authored constructs. They belong to +the workflow host rather than to core: nothing registers them under `xmd run`, +and a document executed without that host has none of them. Their exact forms +are listed here so the public surface is readable in one place; the contract +behind each form — closed props, request, natural key, compatible pre-state, +normalized result, refusal and unavailability behavior, cancellation, replay, +provider ownership and credential boundary — belongs to the section named +beside it. + +`as` is required on every one of them, because every one binds a result. The +form is validated before any context, provider, ceiling or credential is +reached, so a missing prop, an unknown prop, a value outside a closed enum and a +missing `as` each fail before the effect exists. Durable effect identity is +engine-derived from the run and the expansion and is never a prop. + +| Construct | Exact authored form | Contract | +| --- | --- | --- | +| `Issue.Comment` | `…` — paired; the content is the body; binds `{ url }` | [Workflow workspaces](./workflow-workspace-spec.md) §10.3 | +| `PullRequest.Comment` | `…` — paired; the content is the body; binds `{ url }` | [Workflow workspaces](./workflow-workspace-spec.md) §7.10 | +| `PullRequest.Ready` | `` — binds `{ url, state: "open", draft: false }` | [Workflow workspaces](./workflow-workspace-spec.md) §7.10 | +| `PullRequest.Close` | `` — binds `{ url, state: "closed", merged: false }` | [Workflow workspaces](./workflow-workspace-spec.md) §7.10 | +| `PullRequest.Merged` | `` — binds `{ subject, state: "closed", merged: true, mergeCommit, decision: "adopted" }` | [Workflow workspaces](./workflow-workspace-spec.md) §7.11 | +| `Issue.Close` | ``, or the same form with `reason="not_planned"` — binds `{ url, state: "closed", reason }` | [Workflow workspaces](./workflow-workspace-spec.md) §10.3 | +| `Project.Status` | `` — binds the normalized `{ item, field, option }` | [Workflow workspaces](./workflow-workspace-spec.md) §10.6 | +| `Git.Merge` | ``, or the same form with `purpose="publish"` — binds `{ outcome: "clean", purpose, firstParent, secondParent, mergeBase, commit, workspaceRoot }` or `{ outcome: "conflicted", purpose, firstParent, secondParent, mergeBase, workspaceRoot, conflicts }` | [Workflow workspaces](./workflow-workspace-spec.md) §7.8 | +| `Git.PublishTarget` | `` — binds `{ target, expectedRemoteCommit, reviewedHead, sourceCommit, observedCommit, decision }` | [Workflow workspaces](./workflow-workspace-spec.md) §7.9 | +| `Evidence.Run` | ``, where `commands` is an ordered non-empty list of non-empty argv vectors — binds `{ completion, authoredCommands, executed, runTimeout? }`, each executed row `{ argv, outcome, status?, signal?, limit?, stdout, stderr }` and each channel `{ text, retainedBytes, producedBytes, truncated }` | [Workflow workspaces](./workflow-workspace-spec.md) §10.5 | + +Every binding above is the exact closed record its contract section defines; a member outside those shapes, an unknown member and a value outside a closed enum each refuse rather than being carried. + +Two things a reader looking for a component will not find here. The remote `WorkflowHost` is a host assembly contract — the existing `useRunHost()`, `useLifecycle()`, `useDelivery()` and `attach()`, with a Cloudflare implementation beside the Deno one and no transitions type of its own — and not an element a document writes; its runner-to-owner messages are private to one release rather than a public wire contract ([Workflow workspaces](./workflow-workspace-spec.md) §13.2). The factory's protocol records — subject, stage, implementation revision, handoff, role outcome, invalidation, verdict, conflict suspension, Stage 7 decision, merged-observation wait, stage-to-option table, active frontier and terminal settlement — are the closed versioned schemas of [the software factory](./github-actions-software-factory-spec.md) §11.2, which owns them; nothing here or in the Workspace specification duplicates them, and they are neither components nor a lifecycle controller beside the journal. + +None of these constructs is available to a workflow Agent, and none appears in +any generated-XMD read or write table +([Workflow workspaces](./workflow-workspace-spec.md) §§8.3-8.4). The standard +write table remains exactly core's paired `File:write`, the composition +package's lexical `Dir` and core's self-closing `File.Delete`. + ## 7. Entry point ### 8.1 `execute` @@ -9998,6 +10038,191 @@ Defined in [Workflow workspaces](./workflow-workspace-spec.md) §8. |---|------|--------| | WFX1 | SIGKILL and resume | A real `SIGKILL` part-way through leaves the run `running` with the effects that committed; the resume replays those exact events by id, performs the rest once each with no duplicate and no gap, advances the current root, and completes | +### The software-factory tiers + +The seven tiers below are the frozen evidence names for the software factory +specified by +[the software factory](./github-actions-software-factory-spec.md) and by +[Workflow workspaces](./workflow-workspace-spec.md) §§3.8, 7.8-7.11, 10.3, 10.5-10.7 +and 13.2, and by [the software factory](./github-actions-software-factory-spec.md) §11.2. Every construct and host they name is **specified; implementation +unbuilt**, so these tiers name the scenarios an implementation is accepted +against rather than tests that exist. Each lists the finite structural +scenarios — success, refusal, stale authority, interruption and cancellation, +teardown, replay, and denied Agent or generated-XMD authority — and no +malformed-input permutation without a distinct structural consequence. + +### Tier WRH — Remote host, executor and delivery separation + +Defined in [Workflow workspaces](./workflow-workspace-spec.md) §3.8 and §13.2 +and [Workflow runs](./workflow-spec.md) §9.8. + +| # | Test | Verify | +|---|------|--------| +| WRH1 | One owner | A run ID selects one durable owner arithmetically; two admissions of one ID reach that owner and no second registry answers | +| WRH2 | Acquisition is a connection | Start, resume, stale recovery, document execution, Workspace mutation, provider attachment, lifecycle transition, accepted-outcome publication and terminal settlement each validate the exact live acquisition and the expected Workspace root inside their own mutating transaction | +| WRH3 | A second executor | A second connection for a live run follows or is refused, and advances nothing either way | +| WRH4 | Stale authority | A closed, foreign or superseded acquisition reaches no mutation; a run left `running` by a closed connection is recovered by the next acquisition from the exact committed frontier | +| WRH5 | Runner crash | A runner killed between materializing a root and submitting changes leaves a prior or a new complete transaction and never a partial one | +| WRH6 | Content-addressed transfer | The owner refuses a submission whose acquisition, expected root or content does not validate, and publishes the new root and the filtered journal result atomically when it does | +| WRH7 | Delivery is not execution | An answer and a terminal decision each retain against their exact subject while taking no acquisition, beginning no execution, attaching no provider, appending no journal event and changing no run status | +| WRH8 | Delivery correlation | A value for a subject the run is not holding, a duplicate delivery and a spent delivery are each refused with nothing written | +| WRH9 | Consumption | A later executor consumes the retained value inside the run's transaction and appends the accepted event exactly once | +| WRH10 | Inspection | Status and history read immutable snapshots, take no acquisition, and authorize no transition | +| WRH11 | Teardown | Closing the connection releases executor ownership and rolls back nothing already committed | +| WRH12 | Completed replay | A completed run replays by reading its durable owner — lifecycle storage access, not external-effect replay — while attaching no Workspace, Agent, process, Git, Git-host, Issue, Project, credential or other external-effect provider, performing no effect again and starting no native operation | +| WRH13 | Host neutrality | Shared WorkflowRun modules import nothing Cloudflare-specific and detect no runtime; the runtime-named entrypoint is the only place the topology appears | +| WRH14 | The host boundary is unchanged | The Cloudflare adapter satisfies the existing `useRunHost()`, `useLifecycle()`, `useDelivery()` and `attach()` with the same provider-neutral transition and request types; no fifth method and no adapter-specific transitions type appears, and the shared CLI asks the same four questions it asks the Deno host | +| WRH17 | A machine wait is not a typed answer | The wait publishes a `machine_wait` event identified by a `waitId`, never a `suspension_request` or a suspension id; it exposes no response schema, no `xmd workflow answer` route, no form and no bound value, and inspection reports a run waiting on provider state | +| WRH18 | Atomic wait settlement | The `machine_wait` event and the `suspended` status commit together, the executor acquisition is released only after that commit, and a refused settlement publishes neither | +| WRH19 | Wake delivery | An authenticated intake correlated to the exact wait subject retains one bounded wake notification with no executor acquisition, no lifecycle outcome, no run-status change and no answer, verdict, stage, transition or observation result | +| WRH20 | Wake refusals | A duplicate notification changes nothing; one naming another wait, a spent wait, an invalidated wait or a terminal run refuses and leaves the active wait unchanged | +| WRH21 | Wake consumption is one transaction | A later executor consumes one notification and appends one `machine_wake` for that exact `waitId` in a single transaction — `intakeId` present exactly for `provider-intake` and absent exactly for `operator-resume`; a crash before commit leaves wait and notification pending, and replay after commit restores the event without consuming or appending again | +| WRH22 | Resume without authority | An explicit resume with neither a pending wake nor operator-resume authority ends nothing: it reports the same machine wait and settles `suspended` again | +| WRH15 | Release identity | Connection admission validates an exact immutable runner and owner build or protocol fingerprint from trusted deployment configuration and refuses a mismatch closed, before request parsing, acquisition or state access; no message shape is adapted, downgraded or negotiated, and no transport record is journaled, exported or authored | +| WRH16 | Ownership split | Connection admission, request parsing, transaction lifetime and stale recovery belong to the owner; provider attachment and cancellation of its own execution belong to the runner; content is produced by the runner and validated by the owner | + +### Tier WGI — Authenticated GitHub ingress + +Defined in [the software factory](./github-actions-software-factory-spec.md) §5. + +| # | Test | Verify | +|---|------|--------| +| WGI1 | Order | The webhook signature is verified before the payload is parsed as anything but bytes, and the complete objects are reread through the API before authorization is decided | +| WGI2 | Bad signature | An unsigned or wrongly signed delivery is refused before parsing and retains no intake | +| WGI3 | Bounded intake | One intake is retained per delivery or submission identity, holding only typed bounded fields | +| WGI4 | Duplicates | A repeated delivery of one identity finds the retained intake and writes nothing | +| WGI5 | Unavailable is not absent | A missing result page, an unavailable field, an ambiguous object and a partial permission read are each unavailable, and none of them admits an item | +| WGI6 | Admission ceiling | Admission requires the configured organization-owned Project and the exact repository, Project, item, status field and allowed option IDs | +| WGI6b | Stage mapping | Admission maps a completely reread option ID to a stage through the configured bijection and projection maps a stage back through its inverse; neither parses a display string, an invalid table refuses before an intake is retained, a token is minted, a run starts or anything is projected, and only the configured `Backlog`-to-`User` movement admits a new item | +| WGI7 | Dispatch carries nothing | `repository_dispatch` carries only the retained intake identity; a payload naming a stage, outcome, answer, decision, transition, credential or definition is refused | +| WGI8 | OIDC claims | Admission validates issuer, audience, repository ID, repository-owner ID, event name, workflow ref and SHA, and the configured workflow identity; a valid token for another repository or workflow admits no session | +| WGI9 | Human floor | An actor without Project write and repository write-or-higher authorizes no admission, answer, change, merge or abandonment | +| WGI10 | Comments are not authority | A comment naming an answer, a merge, an abandonment or a resume changes nothing | +| WGI11 | Secrets | No private key, webhook secret, OIDC configuration, installation token, endpoint, raw payload, cursor or host path appears in props, context data, a durable record, a comment, output or a diagnostic | +| WGI12 | Run identity | One issue admitted twice derives one run ID; a reread returning a different node ID for the same subject refuses as drift and creates no second run | + +### Tier WGE — Reconciled GitHub and Project projections + +Defined in [Workflow workspaces](./workflow-workspace-spec.md) §7.10, §10.3 and +§10.6. + +| # | Test | Verify | +|---|------|--------| +| WGE1 | Comment identity | A comment's natural key is its subject plus the engine-derived effect identity; a re-rendered or edited body is the same comment and produces no second one | +| WGE2 | Ready and close | `PullRequest.Ready` and `PullRequest.Close` are keyed by their exact subject; an already-ready pull request is adopted, a merged one conflicts for both, and each binds its literal closed record rather than an observed one | +| WGE2a2 | Create effects differ | An effect with a provider-native client idempotency or correlation key — an Issue upsert, a pull-request upsert — reconciles on that key under its existing complete-observation contract and carries no attempt state; only an effect with neither a native key nor a pre-existing subject to read requires the marker and the unattempted/attempted distinction, and one with neither mechanism refuses before its first mutation | +| WGE2b | Comment correlation | A provider that cannot write, preserve and completely query a stable opaque marker refuses before its first mutation; the authored logical body is preserved byte for byte as the authored portion of the projection while the correlation representation lives outside it, so the provider payload is not the authored bytes; the binding and every replay expose the authored body and the provider comment identity, never the transport encoding | +| WGE2c | Attempt state decides absence | Unattempted with no marker is proven absence and creates once; exactly one marker adopts; more than one is ambiguity; **attempted with no committed completion and no marker is permanent ambiguity, not absence**, so a removed marker inside the interrupted window stalls rather than duplicating; a marker removed after a committed completion changes nothing, because replay contacts no provider | +| WGE2d | Merged observation | `PullRequest.Merged` mutates nothing and only adopts: merged at the exact published commit is the completion, merged at another commit conflicts, still open is temporary unavailability, and closed unmerged conflicts rather than being waited out | +| WGE3 | Issue closure | `Issue.Close` adopts an issue already closed with the same reason and conflicts with one closed under the other reason | +| WGE4 | Project pre-state | `Project.Status` is keyed by exact item plus field, adopts an item already at the requested option, and performs once from another allowed option | +| WGE5 | Project unavailability | An unreadable board, unavailable field, ambiguous item and partial permission read are unavailable rather than absent, and none of them mutates | +| WGE6 | Ceilings | A project, item, field or option outside the host ceiling is refused; no authored prop widens it | +| WGE7 | Boundaries stay separate | An Issue-provider effect, a Git-host effect and a Project effect journal their own types and no adapter answers another's request | +| WGE8 | Form before provider | A missing prop, an unknown prop, a value outside a closed enum and a missing `as` each fail before a provider, ceiling or credential is reached | +| WGE9 | Cancellation | A cancelled effect tears the provider call down and publishes no completion | +| WGE10 | Interruption | An interrupted remote completion is reobserved and adopted only when it matches the retained intent | +| WGE11 | Replay | A completed record replays without contacting a provider | +| WGE12 | Projection is not authority | A board, a comment or a draft state ahead of the journal is reconciled as drift and never accepted as proof that a stage passed | +| WGE13 | Denied to generated XMD | A fragment naming any of these constructs is refused in the preflight, before any generated effect, and the standard write table still holds exactly `File:write`, `Dir` and `File.Delete` | + +### Tier WGM — Ordered merge and target publication + +Defined in [Workflow workspaces](./workflow-workspace-spec.md) §7.8 and §7.9. + +| # | Test | Verify | +|---|------|--------| +| WGM1 | Clean merge | A clean `Git.Merge` publishes the commit, the new Workspace root and the filtered result in one transaction | +| WGM2 | Conflict | A conflicted merge restores the pre-merge root, publishes normalized conflict evidence against it, and offers no mutation under that evidence | +| WGM3 | Parent order | `purpose="synchronize"` carries `[implementationHead, targetBase]` and `purpose="publish"` carries `[reviewedBase, reviewedHead]`; neither purpose reorders what it was given | +| WGM3b | `purpose` authorizes | Each purpose's authored parents and merge base are validated against the provider-authenticated merge ceiling — the current implementation head and observed target base for a synchronization, the Stage 7 decision's reviewed `{ headSha, baseSha }` for a publication — and both correct orders pass | +| WGM3c | What refuses before mutation | A missing ceiling, a purpose the ceiling does not authorize, a swapped parent, a stale parent, a stale merge base, another revision, and a ceiling for another Repository or checkout each refuse before any Git mutation | +| WGM4 | Stale identities | A changed head, base, merge base, Repository identity, conflict set or Workspace root makes a retained conflict admission stale, and nothing mutates under it | +| WGM5 | Interruption | A host killed between the merge and the commit leaves the checkout, the current root and the effect history unchanged | +| WGM6 | Compare-and-swap | `Git.PublishTarget` updates the ref only after observing the target equal to the expected commit | +| WGM7 | Adoption | A target already equal to the exact source commit is adopted with nothing performed | +| WGM8 | Race | A target moved to a third commit refuses without mutating, and the exact-revision reviews invalidate rather than the publication proceeding | +| WGM9 | Ceilings | Remote, ref, credential and non-force policy are host-owned; no authored prop sets or widens them | +| WGM10 | Distinct operations | `Git.Push`, `Git.PublishTarget`, `Git.Merge`, pull-request upsert, ready, close and merged observation are seven operations with distinct subjects and records; no force, force-with-lease, rebase, reset or host squash appears in any command trace or retained configuration | +| WGM11 | Replay | Completed merge and publication records replay without running Git and without contacting a Git host | +| WGM13 | Exact merge records | A clean result names the merge commit and published root; a conflicted one names the restored pre-merge root and the complete conflict set sorted by path in UTF-8 byte order, with stage numbers and side presence agreeing exactly, duplicate paths refused, and an absent side absent rather than null | +| WGM14 | Restoration failure | A pre-merge root that cannot be restored publishes no conflicted result and no new root, and activates the durable fail-stop fence | +| WGM15 | Exhaustive pre-states | Expected base performs once, the exact source commit adopts, a third commit conflicts, and incomplete observation, ambiguity and temporary unavailability each refuse as themselves | +| WGM12 | Denied to the Agent | Neither construct is reachable by a workflow Agent or by an admitted generated fragment | + +### Tier WER — Trusted evidence execution + +Defined in [Workflow workspaces](./workflow-workspace-spec.md) §10.5. + +| # | Test | Verify | +|---|------|--------| +| WER1 | Structured argv | `commands` is an ordered list of non-empty argument vectors; no shell string, interpreter or quoting layer takes part, and the retained argv is what ran | +| WER2 | Root | The commands run against the exact retained Workspace root the host materialized | +| WER3 | Ceilings | Executable, environment, time and output ceilings are host-owned and refuse rather than truncating silently what they were not given | +| WER4 | Results | The binding is one ordered bounded result per command, carrying its argv, exit status and bounded output | +| WER5 | Location | The execution happens on the trusted runner; the durable owner runs no native process | +| WER6 | Cancellation and teardown | A cancelled run terminates its child before publishing, and no child outlives the effect | +| WER7 | Replay | A completed record replays running nothing | +| WER4b | Fail-fast prefix | A command exiting `0` starts its successor; a non-zero exit, a signal and a timeout each become the last row and start none. `completion` is `"passed"` only when `executed` holds `authoredCommands` rows that all exited `0`, and `"failed"` otherwise, so a complete pass and a stopped prefix are distinguishable without the authored list | +| WER4c | Channels and truncation | stdout and stderr are retained separately, each stating its retained bytes, the bytes the child produced, and whether it was truncated; truncation is stated rather than inferred | +| WER4d | Outcome discriminants | `status` is present exactly for `exited`, `signal` exactly for `signalled`, and `limit` exactly for `timeout`; any of the three beside the wrong outcome, an unknown outcome and an unknown `limit` each refuse the record | +| WER4e | Two ceilings | A per-command ceiling and a whole-run ceiling are both host-owned and neither is a prop; a running command's effective deadline is the earlier of the two, and the timeout row's `limit` names which fired | +| WER4f | Whole-run expiry between commands | The result ends `completion: "failed"` with a `runTimeout` record naming the index that did not start, no next command runs, and no fabricated argv row appears | +| WER6b | What binds and what fails | A zero exit, a non-zero exit, a signal, a per-command timeout and a whole-run timeout each belong to an ordinary bound result; a launch failure, an output-pump failure and a teardown failure each fail the effect and bind no `EvidenceRunResult`; cancellation terminates the complete process tree and commits neither a completion nor a failure | +| WER6c | Precedence | Cancellation outranks everything; otherwise the first infrastructure failure is authoritative and a teardown failure after it is retained as secondary evidence rather than replacing it; a teardown failure with nothing before it is authoritative even when every command produced an observed exit | +| WER6d | Failure evidence is retained | A failed effect's error carries the safely collected executed prefix, the separately bounded channels, the primary infrastructure category and any secondary teardown category; nothing binds it, it is not an `EvidenceRunResult`, replaying the failed effect starts no process, and cancellation retains neither | +| WER6e | Enforcement that fails | A termination, drain or reap that fails while the host is enforcing a ceiling is an infrastructure failure rather than a timeout row | +| WER8 | Denied authority | `Evidence.Run` appears in no Agent capability and in no generated-XMD read or write table; a fragment naming it is refused before any generated effect | + +### Tier WFP — Factory protocol and frontier + +Defined in [the software factory](./github-actions-software-factory-spec.md) +§§1-2 and §12. + +| # | Test | Verify | +|---|------|--------| +| WFP1 | Run identity | The run ID equals the §1.1 derivation for its issue: the same canonical GitHub authority and the same issue node ID produce the same 52-character id in two independent implementations, and a changed repository name, issue number, Project identity, comment, branch, revision, definition SHA, delivery ID or actor changes it not at all | +| WFP1b | Identity drift | A reread returning a different canonical GitHub authority or node ID for a subject the host already retains refuses as unsupported provider-identity drift and derives no second run | +| WFP2 | Adjacency | An outcome advancing more than one stage is rejected, and the current stage's handoff commits before the next role is invoked | +| WFP3 | Same-stage amendment | A later accepted same-stage output replaces the frontier while the superseded output stays readable in the journal | +| WFP4 | Backward destinations | Each of the six destinations in §2.2 deactivates exactly the downstream handoffs its row names and requires every later stage again | +| WFP5 | Head invalidation | A changed `headSha` places the frontier at Stage 4 and invalidates Stages 5-7 | +| WFP6 | Base invalidation | A base-only move places the frontier at Stage 5, and a base move requiring synchronization or implementation work places it at Stage 4 | +| WFP7 | Exact subjects | Stage 6 accepts only a Planner verdict naming the same revision, and Stage 7 only a review chain naming the current one | +| WFP8 | Ready authority | Only an accepted Stage 6 verdict for the current revision takes the pull request out of draft; observed ready state manufactures no verdict | +| WFP9 | Closed parsers | Every §11.2 record carries its schema discriminant and version; an unknown schema, an unknown version, an unknown member, a missing required member and a value outside a closed enum each refuse, and a refusal names the member path and never the value | +| WFP9b | Frontier reduction | Each reduction input of §11.2 — advance, amend, invalidate, head change, base-only change, base-needs-work, and a terminal decision on an already-terminal run — produces exactly the stated frontier, and a reduction leaving a stage out of range, two entries for one stage, or an advance past an unaccepted stage refuses | +| WFP9c | Decision shapes | `merge` carries revision, actor and delivery identity; `abandon` adds a required reason; `change` adds a reason and the earliest invalidated stage; a decision naming a revision that is not the current frontier revision refuses | +| WFP9d | Stage-to-option table | The retained table holds exactly nine entries ordered by stage, one option ID per stage `0`-`8`, every option ID distinct, and every display name equal to the settled status string for its stage; missing, partial, duplicated, cross-field, unavailable and renamed cases each refuse | +| WFP9e | Merged-observation wait and wake | The wait record names a `waitId`, the canonical pull-request URL, the expected merge commit, the terminal decision and the revision, carries `retriesExhausted: true`, and holds no stage, outcome, verdict or response schema; the wake record names the same `waitId` and a closed `source`, with `intakeId` required exactly for `provider-intake` and absent exactly for `operator-resume`, and carries no observation result | +| WFP9f | Terminal references | Every event a terminal names must belong to this run and intent, parse under its exact effect kind, be complete, and agree on revision, actor and decision; missing, foreign, wrong-kind, incomplete, invalidated, duplicated and cross-path events each refuse settlement, and provider identities stay in the referenced results rather than being copied into the terminal | +| WFP10 | Definition identity | Definition incompatibility is a workflow lifecycle refusal and never an implementation correction | +| WFP11 | Journal is authority | A Project status, comment or pull-request state ahead of the journal is drift and authorizes no stage | + +### Tier WFL — Authored factory lifecycle and terminal settlement + +Defined in [the software factory](./github-actions-software-factory-spec.md) +§§8-10. + +| # | Test | Verify | +|---|------|--------| +| WFL1 | Clean synchronization | Observe, merge, record the revision, remain at Stage 4, run evidence, push the descendant, offer the new pair to Stage 5 — as separate durable effects, resuming from the first uncommitted one | +| WFL2 | Conflict suspension | Every conflict returns the conflicted shape of the `GitMergeResult` union, restores the pre-merge root, retains the complete conflict set that shape defines, publishes a handoff and suspends | +| WFL3 | No automatic resolution | No generated fragment, structured text-conflict capability, rebase, force, force-with-lease or reset resolves a conflict | +| WFL4 | Manual resolution | A pushed resolution is observed as a new Stage 4 revision, receives new evidence, and inherits no Stage 5-7 conclusion | +| WFL5 | Merge decision first | A merged decision is retained before any effect is attempted | +| WFL6 | Merge ordering | Merge construction, target publication, `PullRequest.Merged`, issue completion as `completed`, the Project move and terminal settlement are separate reconciled steps in that order | +| WFL6b | Paths do not borrow steps | The merged path closes no pull request and the abandoned path constructs no merge, publishes no target and observes no merged state; a `FactoryTerminal` naming a step from the other path refuses | +| WFL6c | Waiting for the host | A pull request still open runs the bounded host-configured retry and then enters the merged-observation machine wait; the wait appends no lifecycle outcome and moves no stage, a wake only permits one further observation, and terminal settlement stays absent until the adoption succeeds | +| WFL6d | Reobservation after a wake | Authored control flow invokes `PullRequest.Merged` again after the wake event; an adoption advances the terminal sequence, a still-open observation may retry and wait again at a new durable position, and merged-at-another-commit or closed-unmerged ends the wait as a conflict | +| WFL6e | Cancellation during a wait | Cancelling a waiting run follows ordinary run cancellation and invents neither a wake nor a merged observation | +| WFL7 | Terminal is last | The run becomes terminal `merged` only after every required projection completes | +| WFL8 | Abandonment | An exact-revision authenticated abandonment carrying a reason is retained first, then pull-request close unmerged, issue close as `not_planned`, the Project move and settlement; terminal `abandoned` follows all of them | +| WFL9 | Retention | Both terminal kinds retain the actor, exact revision, resulting provider identities and required reason, and retain the Project item, branch, comments, journal, Workspace roots and Agent evidence | +| WFL10 | Interrupted terminal | An interruption between two projections resumes at the first uncommitted one and publishes no terminal status until the rest complete | +| WFL11 | Provider-free replay | A terminal run replays attaching no provider | +| WFL12 | Reopening | Reopening the issue does not reopen the completed run; continuing requires a new linked issue and therefore a new run | + ### Tier SL — Own-scope context updates | # | Test | Verify | diff --git a/specs/github-actions-software-factory-spec.md b/specs/github-actions-software-factory-spec.md index ff1a0ef41..539053123 100644 --- a/specs/github-actions-software-factory-spec.md +++ b/specs/github-actions-software-factory-spec.md @@ -1,47 +1,103 @@ # GitHub Actions-hosted AI Software Factory -This specification defines an issue-driven software factory whose durable -procedure is an XMD workflow and whose invocation host is GitHub Actions. One +You open one GitHub issue and the factory carries it to a merged pull request or +an explicit abandonment, without anybody having to remember where it got to. One GitHub Project item shows who owns the work now; the retained XMD run proves how -the item reached that owner. - -The factory has no independent controller. GitHub Actions starts or resumes the -XMD workflow and supplies an authorized GitHub environment. The XMD workflow -invokes roles, validates their structured outcomes, records handoffs, performs -authorized GitHub and Workspace effects, updates the Project status, and -journals those effects. - -## 1. One item, one durable run +the item reached that owner. Every stage the item passed through, every revision +that was reviewed, and every effect that reached GitHub are in one durable +journal addressed by the issue itself. + +The factory has no independent controller. A durable XMD workflow run is the +whole procedure. GitHub Actions supplies an ephemeral trusted runner; a +Cloudflare Durable Object supplies the run's durable state; a dedicated GitHub +App supplies authenticated ingress and the credential every GitHub effect is +performed with. The XMD workflow invokes roles, validates their structured +outcomes, records handoffs, performs authorized GitHub and Workspace effects, +updates the Project status, and journals those effects. + +Three planes are separate throughout, and keeping them separate is what the rest +of this specification spends its length on: + +- the **executor plane**, one authenticated WebSocket connection that advances + the run; +- the **delivery plane**, authenticated transactions that retain an intake, an + answer or a decision without executing anything; and +- the **inspection plane**, read-only reads that can never become transition + authority. + +## 1. One issue, one durable run One issue corresponds to one durable XMD factory run. The run may produce and review many implementation revisions before it closes; a new implementation commit never creates a new factory run. -Two SHA identities remain separate throughout that run: +### 1.1 The run ID is derived from the issue + +Admission rereads the GitHub issue from the API before it derives anything. The +**factory run ID** is then the lowercase unpadded RFC 4648 Base32 encoding of +the full SHA-256 digest of these UTF-8 bytes, concatenated in this order: + +```text +"github-issue-v1" || 0x00 || canonical GitHub authority || 0x00 || issue node ID +``` + +The digest is all 32 bytes, so the run ID is 52 Base32 characters with no +padding and no separators. It is a public run ID in the sense §9 of the workflow +specification already defines: non-empty, containing no NUL, opaque to +everything but equality and lifecycle addressing. + +The **canonical GitHub authority** is the lowercase DNS hostname of the GitHub +deployment, plus `:` and the port when the port is not the scheme's default. It +carries no scheme, path, query, fragment, user information or trailing +separator, so one deployment has exactly one spelling. The **issue node ID** is +the exact string GitHub's GraphQL API returns for that issue, compared byte for +byte with no case folding and no Unicode normalization: it is an opaque provider +identity, and normalizing it would be inventing a second one. + +Nothing mutable participates. Repository names, issue numbers, Project, Project +item and status identities, comments, branch names, implementation revisions, +the workflow definition SHA, webhook delivery IDs and actor identities all +change while the run stays the run it was, so none of them is an input to the +derivation. + +Two consequences follow directly. Duplicate admission for the same authenticated +subject derives the same run ID and therefore routes to the same run rather than +creating a second one — the compatible-reuse rule of the workflow contract does +the rest. And a reread that returns a different retained provider identity for +the same subject, including an issue transfer that changes the node ID, is +unsupported drift: the host refuses it, names it as drift, and creates no second +run. Silently starting another run would leave two frontiers claiming one piece +of work. + +### 1.2 Two SHA identities, and neither is the run + +Two SHA identities remain separate throughout the run, and neither is the run +ID: - The **workflow definition SHA** is the immutable Git commit in the XMD workflow definition. It fixes the procedure and its component bundle for the - lifetime of the run. A different definition is not a revision of the same - run; it requires a new run or an eligible history fork under the workflow - lifecycle contract. -- The **implementation revision** is the evolving pair - `{ headSha, baseSha }`. `headSha` is the exact commit at the draft pull - request's head, and `baseSha` is the exact target-branch commit against which - that head is evaluated. + lifetime of the run. A different definition is not a revision of the same run; + it requires a new run or an eligible history fork under the workflow lifecycle + contract. +- The **implementation revision** is the evolving pair `{ headSha, baseSha }`. + `headSha` is the exact commit at the draft pull request's head, and `baseSha` + is the exact target-branch commit against which that head is evaluated. The definition SHA authorizes which procedure executes. The implementation revision identifies what the procedure is currently producing or reviewing. -Neither substitutes for the other, and neither a Project field nor a comment -may rewrite either identity. +Neither substitutes for the other, and neither a Project field nor a comment may +rewrite either identity. ## 2. Lifecycle and validation frontier +![Ownership bands across the eight factory stages](./assets/github-actions-software-factory-ownership-bands.svg) + The Project status has these values: | Stage | Status | Owner | Question answered | | ---: | --- | --- | --- | | 0 | Backlog | User | Is this item admitted to the factory? | -| 1 | Product Owner | User | What product outcome and acceptance boundary are intended? | +| 1 | User | User | What product outcome and acceptance boundary are intended? | | 2 | Architect | Architect | Is the structural contract ready? | | 3 | Planner | Planner | Is there an implementation-ready plan and evidence matrix? | | 4 | Implementor | Implementor | Does an implementation revision satisfy the accepted plan? | @@ -50,14 +106,14 @@ The Project status has these values: | 7 | User Review | User | Is this exact validated result accepted? | | 8 | Closed | None | Was the item merged or abandoned? | -The left side progressively removes uncertainty: product intent, structural -contract, implementation plan, then code. The right side validates the result: -implementation evidence, structural correctness, user acceptance, then -completion. +Those nine strings are the exact status vocabulary. The diagram above shows the +same eight stages as two ownership bands: the left side progressively removes +uncertainty — product intent, structural contract, implementation plan, then +code — and the right side validates the result — implementation evidence, +structural correctness, user acceptance, then completion. -Moving an authorized Backlog item to Product Owner admits it and starts its -factory run. Once admitted, an ordinary successful outcome advances exactly one -stage: +Moving an authorized Backlog item to User admits it and starts its factory run. +Once admitted, an ordinary successful outcome advances exactly one stage: ```text 1 -> 2 -> 3 -> 4 -> 5 -> 6 -> 7 -> 8 @@ -83,11 +139,13 @@ Self-correction is iteration, not progress: passes. - An Implementor producing, synchronizing, or correcting an implementation remains at Stage 4 until the latest implementation revision passes. -- A reviewer replacing a malformed or incomplete verdict remains at that - review stage until a valid verdict identifies the exact revision reviewed. +- A reviewer replacing a malformed or incomplete verdict remains at that review + stage until a valid verdict identifies the exact revision reviewed. The role's latest accepted same-stage output supersedes its earlier output at -the active frontier without erasing history. +the active frontier without erasing history. The superseded output stays in the +journal and stays readable; what it loses is the authority to carry the item +forward. ### 2.2 Backward invalidation @@ -96,7 +154,7 @@ invalidates. Forward progress then traverses every later stage again: | Returned to | Still valid | Must run again | | --- | --- | --- | -| Product Owner (1) | Nothing downstream | 2-7 | +| User (1) | Nothing downstream | 2-7 | | Architect (2) | Product decision | 2-7 | | Planner (3) | Product decision and architecture | 3-7 | | Implementor (4) | Product decision, architecture, and plan | 4-7 | @@ -105,17 +163,16 @@ invalidates. Forward progress then traverses every later stage again: The workflow validates a backward outcome against this frontier. The role supplies the reason and earliest invalidated stage; the XMD procedure decides -which accepted downstream handoffs become inactive and records that decision. -A Project edit alone never proves invalidation or approval. +which accepted downstream handoffs become inactive and records that decision. A +Project edit alone never proves invalidation or approval. ### 2.3 Revision invalidation Every Stage 5, Stage 6, and Stage 7 conclusion identifies the exact `{ headSha, baseSha }` it evaluated. -- Any merge, rebase, manual edit, generated mutation, conflict resolution, or - other change to `headSha` remains at or returns to Stage 4 and invalidates - Stages 5-7. +- Any merge, manual edit, conflict resolution, or other change to `headSha` + remains at or returns to Stage 4 and invalidates Stages 5-7. - Movement of `baseSha` without a head change invalidates the review context at Stage 5. The item repeats Stages 5-7 against the new pair when no synchronization or implementation correction is required. @@ -124,61 +181,273 @@ Every Stage 5, Stage 6, and Stage 7 conclusion identifies the exact - An invalidated Planner verdict returns to Stage 5. An invalidated Architect verdict returns to Stage 6. -Stage 6 may accept a Planner verdict only when that verdict names the same -implementation revision. Stage 7 may accept the review chain only when both -review verdicts name the current revision. A later SHA never inherits a verdict -for an earlier pair. +Stage 6 accepts a Planner verdict only when that verdict names the same +implementation revision. Stage 7 accepts the review chain only when both review +verdicts name the current revision. A later SHA never inherits a verdict for an +earlier pair. + +Only an accepted Stage 6 verdict for the current revision is **ready +authority**: it is what authorizes XMD to take the pull request out of draft and +move the item to Stage 7. Ready state observed on GitHub without that accepted +verdict is drift to reconcile, never a substitute for it. The workflow definition SHA does not participate in this invalidation table. Definition incompatibility is a workflow lifecycle refusal, not an implementation correction. -## 3. Durable procedure and GitHub projection +### 2.4 Terminal kinds + +Stage 7 -> 8 is the adjacent terminal transition, and it has exactly two kinds. +A **merged** terminal records that the reviewed revision reached the target +branch. An **abandoned** terminal records that an authorized human ended the run +without publishing it. Both are described in §10, which owns their effect +ordering; what matters here is that they are the only two ways an admitted run +becomes terminal, and that each names the exact reviewed revision it settled on. + +## 3. Deployment topology + +Three hosts carry one run, and each owns something the others cannot reach. + +```text +GitHub App (ingress, credentials) + │ webhook / form / repository_dispatch + ▼ +Cloudflare Durable Object ── SQLite ── WorkflowRun, Workspace roots, journal + ▲ + │ authenticated WebSocket (executor acquisition) + ▼ +GitHub Actions ephemeral runner ── native Git, evidence processes, Agent clients +``` + +**One SQLite-backed Cloudflare Durable Object, selected from the run ID, owns +the run.** It holds the WorkflowRun record and its filtered journal, the +immutable Workspace roots and their content-addressed bytes, the Agent-session +mappings and checkpoints, pending answers and Stage 7 decisions, the retained +intake records, and executor ownership. There is one durable owner per run, +selected arithmetically from the run ID exactly as local discovery is, so no +second registry can disagree with it. + +**Exported `.xmd` artifacts are immutable evidence only.** They are never live +state, discovery, continuation, answer delivery, or lock authority, and no +Actions artifact is any of those either. An artifact says what a run had +committed at one frontier; it does not say what a run may do next. + +**One authenticated Durable Object WebSocket connection owns the executor +acquisition.** The acquisition is the connection's lifetime and nothing else. It +has no duration, expiry, renewal, heartbeat, PID, liveness poll or application +lease, exactly as the local executor lock has none. Closing the WebSocket +invalidates the acquisition and releases executor ownership; it does not roll +back state that already committed. A second healthy executor follows the active +one or is refused, and cannot advance the run either way. + +**The ephemeral Actions runner executes what cannot run inside a Durable +Object.** Native Git, plan evidence processes and Agent clients run there, +against bounded materialized state, and only there. The runner materializes one +selected retained Workspace root, submits content-addressed changes, and the +Durable Object validates the acquisition, the expected root and the content +before it atomically publishes the new root together with the filtered journal +result. A runner crash therefore exposes only a prior or a new complete +transaction; a later connection performs stale recovery and resumes from the +exact committed WorkflowRun and Workspace frontier. + +**The Cloudflare runtime-named host owns persistence and admission.** Durable +transactions, intake, the authorization gates, token minting and executor +admission are its. Shared production modules stay provider-neutral: they do not +detect Cloudflare, Deno, GitHub Actions or any other runtime, and they reach +every host-specific behavior through the contextual APIs that already exist. + +**A completed replay reads its own history and nothing else.** It may reach and read the run's durable owner, because that owner is where the retained result is and an ephemeral client holds nothing to replay from. It attaches no Workspace, Agent, process, Git, Git-host, Issue, Project, credential or other external-effect provider, performs no effect again and starts no native operation. Reading retained completion from its authoritative owner is lifecycle storage access, not external-effect replay, and it is that second thing §10's terminal ordering exists to keep unnecessary. + +## 4. Durable procedure and GitHub projection The XMD workflow is the only procedure that may change the factory stage. On a -start or resume it: +start or resume the executor: -1. acquires the durable run's executor lock; +1. acquires the run's executor connection; 2. restores the retained workflow definition, Workspace, handoff chain, and incomplete effects; 3. observes the issue, draft pull request, Project item, and exact Git revision identities required by the current stage; 4. reconciles interrupted GitHub effects under their retained identities; -5. renders the active handoff chain and authorized observations to the current +5. consumes any retained answer or Stage 7 decision inside the run's + transaction, appending its accepted durable event exactly once; +6. renders the active handoff chain and authorized observations to the current role; -6. validates the role's structured outcome against the current stage, artifact +7. validates the role's structured outcome against the current stage, artifact identities, and implementation revision; -7. records a same-stage iteration or invalidates the frontier, or performs the +8. records a same-stage iteration or invalidates the frontier, or performs the immediately adjacent successful transition; -8. performs the authorized issue, pull-request, Git, and Project effects; and -9. journals the accepted decision and every effect before yielding the next - durable boundary. +9. performs the authorized issue, pull-request, Git, and Project effects; and +10. journals the accepted decision and every effect before yielding the next + durable boundary. + +Every step above is executor-owned. Start and resume, stale-execution recovery, +document execution, Workspace mutation, Agent attachment, native Git and +evidence execution, lifecycle transition, accepted-outcome publication and +terminal settlement each validate the exact live acquisition and the expected +Workspace root inside every mutation transaction. An acquisition that is closed, +foreign or stale reaches no mutation. + +Two kinds of operation are deliberately outside that list. + +**Delivery is not execution.** Authenticated webhook and form intake, typed +answer delivery, and Stage 7 decision delivery are delivery-plane transactions. +Each takes no executor acquisition, starts no execution, attaches no Workspace, +Agent or process provider, appends no lifecycle outcome and changes no run +status. Each validates its exact delivery identity and its exact pending +suspension or decision subject, and retains only the typed bounded value that +subject describes. This is the answer-delivery contract the workflow +specification already states, applied unchanged to a remote host and extended to +Stage 7 decisions. + +**Inspection is not authority.** Status and history reads are read-only, take no +executor acquisition, and cannot become transition authority. They observe the +same immutable snapshot surface local inspection observes. + +A later executor is what turns a retained answer or decision into progress. It +consumes the retained value inside the run's own transaction, appends the +accepted durable event or outcome once, and only then may authored XMD choose +the next transition. Delivery stores; execution decides. GitHub Actions supplies the trusted executable, the definition reference, the run identity, and credentials or provider configuration within a fixed ceiling. Its YAML does not parse role conclusions, choose stages, construct handoffs, change draft state, merge, comment, or update the Project independently of the -XMD workflow. - -Actions concurrency may reduce duplicate invocations, but it is not the -executor lock and cannot authorize a transition. A second invocation of the -same item follows or is refused by the durable run lifecycle. +XMD workflow. Actions concurrency may reduce duplicate invocations, but it is +not the executor acquisition and cannot authorize a transition. -The Project status is a human-facing projection of the journaled current stage. -Issue and pull-request comments are human-readable transition records. The XMD -journal is the durable execution record of which source records were accepted, -which role conclusion won, and which external effects completed. A Project -status ahead of the journal is drift, not proof that omitted roles passed. +The XMD journal is lifecycle authority. The Project status is a human-facing +projection of the journaled current stage, and issue and pull-request comments +are human-readable transition records. A Project status, comment or pull-request +state ahead of the journal is drift to reconcile, never proof that a skipped +role passed. GitHub offers no transaction spanning a comment, draft state, Project status, -branch update, or merge and the retained XMD store. Each is therefore a durable -external effect with stable identity and reconciliation: observe, adopt an -already-compatible result, perform from proven absence or compatible pre-state, -or refuse conflict and ambiguity. Interruption may leave GitHub ahead of the -local result; resume reconciles the same intended effect rather than repeating -it blindly. +branch update, or merge and the retained XMD store, and none is claimed across +Durable Object state, native Git and processes, GitHub and Project V2. Each +GitHub mutation is therefore one durable external effect with a stable +engine-derived identity and an effect-specific natural key: it observes before +it mutates, adopts only a compatible completion, performs once from proven +absence or an exact compatible pre-state, and refuses conflict, permanent +ambiguity, incomplete observation and temporary unavailability. Cancellation +tears the provider call down and publishes no invented completion; an +interrupted remote completion is reobserved and adopted only when it matches the +retained intent; a completed replay contacts no provider at all. + +The natural keys are exact. A comment uses its subject plus the engine-derived +effect identity, so the body and title are presentation rather than identity. A +Project status uses the exact Project item plus field. Ready, close and the +merged observation use their exact issue or pull-request subject. Target +publication uses the retained repository plus the configured remote and target +ref. Accepted outcomes are +retained before their projections are attempted, and reconciliation completes +each intended projection. + +## 5. Authenticated ingress + +A dedicated GitHub App is the factory's only ingress. It receives Project and +admission webhooks and authenticated human form submissions, and it is the +principal every GitHub effect is performed as. + +### 5.1 Order of operations + +An intake is admitted in this order, and no step may be reordered: + +1. **Verify before parsing.** The webhook signature is verified against the + App's webhook secret before the payload is parsed as anything but bytes. +2. **Reread from the API.** The complete GitHub objects — issue, repository, + Project, Project item, status field — are reread through the API. The + payload's copy of them is a notification, not a source of truth. +3. **Authenticate the installation and the actor.** The installation is resolved + for the exact repository, and the human actor is resolved as an identity + rather than a display name. +4. **Retain one bounded intake.** The intake is keyed by the GitHub delivery + identity for a webhook, or the submission identity for a form, and retains + only bounded typed fields. + +Admission requires a configured organization-owned Project V2 and the exact +repository, Project, Project item, status field and allowed option IDs. A +missing page of results, an unavailable field, an ambiguous object and a partial +permission read are all **unavailable** — they are neither absence nor +authorization. Treating an unreadable Project as an empty one is how an +unauthorized item would be admitted. + +**A configured table maps status options to stages, in both directions.** It is a total bijection between the nine `FactoryStage` values of §11.2 and nine exact Project V2 status option IDs: one option ID for every stage `0` through `8`, and every configured option ID appearing exactly once. It is host configuration and part of the admission and projection ceiling — never an authored prop, and never something a provider payload can supply. + +The table is validated before it is used. Startup and admission refuse — before an intake is retained, a token is minted, a run is started, or anything is projected — when the table is missing, partial, holds a duplicate, names an option the Project does not currently offer, names an option belonging to another field or another Project, or names an option whose display name reread from GitHub is not the settled §2 status string for its stage. + +Admission maps the completely reread option ID to a stage through that table, and projection maps a retained stage back to its option ID through the inverse. Neither direction parses a display string: the strings are what a person reads, the option IDs are what the factory compares, and a status renamed on the board is a configuration refusal rather than a silent remapping. + +Only the configured `Backlog`-to-`User` movement admits a new item. A Project edit at any other point is projection drift or an authenticated intake to reconcile, and is never a role verdict. + +A duplicate delivery of the same identity finds the retained intake and changes +nothing. That is the same compatible-reuse rule run identity uses, applied to +intake. + +### 5.2 Waking Actions carries no decision + +`repository_dispatch` carries only the retained intake identity. The receiving +Actions workflow may be woken by it, but the payload never carries or derives a +stage, a role outcome, answer text, a decision, a transition, a credential, or a +mutable factory definition. Everything the run needs it reads from the Durable +Object after it has authenticated. + +### 5.3 The runner authenticates with OIDC + +The Actions job authenticates to the provider through GitHub OIDC. Before it +admits a session, the provider validates the issuer, the configured audience, +the repository ID, the repository-owner ID, the event name, the workflow ref and +SHA, and the configured immutable workflow identity. Repository *names* are +mutable and are not what is checked; IDs are. + +### 5.4 Human answers and decisions + +Human answers and Stage 7 decisions arrive through a GitHub-App-authenticated +web form bound to the exact retained suspension or decision subject. The form +submission is a delivery-plane transaction under §4. + +**Comment text is never authority.** A comment never answers a question, merges, +authorizes a change, abandons a run or resumes execution. Comments are +transition records a person reads. + +A human may authorize admission, an answer, a Stage 7 change, a merge or an +abandonment only while holding Project write and repository write-or-higher +access, checked at the moment of the submission. The short-lived App +installation token performs the GitHub effects that follow; the journal retains +the human actor separately from the token that acted, so the record says who +decided as well as what was done. + +## 6. Principal, permissions and host ceilings + +The GitHub App has exactly these repository permissions — Metadata read, +Contents write, Issues write, Pull requests write, Checks read, Commit statuses +read — and organization Projects write. It has no Administration, Actions write, +Workflows write, Secrets, Environments, Deployments or Members permission, and +no force-push authority anywhere. + +The App installation is limited to configured repositories. The host narrows +further, per operation, to the exact repository, implementation branch, target +branch, organization Project, status field, option IDs, issue or pull-request +subject, reviewed revision, parent pair, and non-force operation. A ceiling the +installation grants is not a ceiling the host uses. -## 4. Issue and draft pull-request boundary +Three more ceilings are host configuration on the same terms, none of them an authored prop: the stage-to-option bijection of §5.1; the merge authority `Git.Merge` validates its authored parents against, which supplies the current implementation head and observed target base for a synchronization and the reviewed `{ headSha, baseSha }` the Stage 7 decision authorized for a publication; and the bounded retry — count, total duration and backoff — the merged observation of §10.4 runs under. `Evidence.Run`'s executable, environment, working-root, per-command duration, whole-run duration, output and process-tree ceilings are the same kind of configuration, stated in [Workflow workspaces](./workflow-workspace-spec.md) §10.5. + +Mutation of `.github/workflows/**` is refused, even though Contents write could +otherwise reach it. A factory that can rewrite the workflow that runs it is a +factory that can rewrite its own authorization. + +GitHub App private keys, webhook secrets, OIDC verification configuration, +issued installation tokens, provider endpoints, raw payloads, cursors and host +paths are provider secrets and closure state. None of them enters props, context +composition data, durable requests or results, comments, output or diagnostics. + +The configured target ruleset admits only this factory's dedicated App for the +target-branch update of §10, and host validation independently enforces the +target and the reviewed parent pair. Neither substitutes for the other: the +ruleset says who may write, the host says what may be written. + +## 7. Issue and draft pull-request boundary The issue owns product intent, architecture, planning, and the transition into implementation. @@ -191,8 +460,8 @@ implementation. Stage 4 creates or updates a draft pull request before its first handoff to Planner Review. The Stage 4 -> 5 handoff identifies the draft pull request and -its exact `{ headSha, baseSha }`. The draft pull request then owns implementation -iterations and Stage 5-7 review handoffs. +its exact `{ headSha, baseSha }`. The draft pull request then owns +implementation iterations and Stage 5-7 review handoffs. When invalidation crosses from the pull request to Stages 1-3, the XMD workflow writes the full handoff on the issue and a short linking record on the pull @@ -200,62 +469,65 @@ request. When the accepted issue chain crosses back into Stage 4, the draft pull request links the accepted issue handoff before implementation continues. The pull request remains draft throughout Stage 4 corrections and Stage 5 -review. It also remains draft while Stage 6 requests a backward correction. -Only an accepted Stage 6 verdict for the current revision authorizes XMD to make -the pull request ready and move the item to Stage 7. If interruption separates +review. It also remains draft while Stage 6 requests a backward correction. Only +the ready authority of §2.3 takes it out of draft. If interruption separates those GitHub effects, the run remains at its last journaled frontier and resume -reconciles both; ready state alone does not manufacture an Architect verdict. +reconciles both. -Stage 7 -> 8 is the adjacent terminal transition. It records `merged` or -`abandoned`, the reviewed implementation revision, the actor and decision that -authorized closure, and the resulting merge identity when one exists. Closing -the Project item never erases the issue, pull request, handoff history, or XMD -journal. +Closing the Project item never erases the issue, pull request, handoff history +or XMD journal. -## 5. Stage 4 base synchronization +## 8. Stage 4 base synchronization -The initial factory synchronizes a draft implementation branch by merging the -latest observed target base into it. It does not rebase published work. +The factory synchronizes a draft implementation branch by merging the latest +observed target base into it. It does not rebase published work, and it never +force-pushes. -A merge preserves published commit identities, produces a descendant that the -normal non-force Push effect can publish, and allows the final pull request to -use a squash merge when the repository's delivery policy wants a compact target -history. A rebase changes published identities and requires a separately -specified, reconciled force-with-lease effect. No factory role, generated XMD, -or GitHub Actions step has that effect in the initial contract. Force pushes are -refused. +A merge preserves published commit identities and produces a descendant the +ordinary non-force Push effect can publish. Rebase, force push, force-with-lease +and reset-based replacement are absent from this contract; adding any of them +later takes a new external-effect contract with its own reconciliation semantics +and invalidation proof, and cannot be represented as another spelling of Push. A clean base synchronization follows this durable sequence: 1. XMD observes and checkpoints the exact implementation head, target base, and merge base. 2. XMD performs the trusted merge against those identities inside the retained - Workspace. + Workspace, with parents ordered `[implementationHead, targetBase]`. 3. XMD records the resulting merge commit and Workspace root as a new implementation revision. 4. The item remains at Stage 4 because its head changed. -5. XMD runs the implementation evidence selected by the accepted plan. +5. XMD runs the implementation evidence the accepted plan selected. 6. When that evidence passes, XMD publishes the exact descendant through the ordinary non-force Push effect and offers the latest `{ headSha, baseSha }` to Planner Review. The merge, evidence, push, and Stage 4 pass are separate durable effects. No -transaction is claimed across native Git, test processes, GitHub, and the +transaction is claimed across native Git, evidence processes, GitHub and the Project. Resume restores completed effects and continues from the first uncommitted one. A manually changed branch is not adopted as a passed implementation. XMD first -observes its new exact head and base, records a new implementation revision, -and re-enters Stage 4. The Implementor and selected evidence evaluate that +observes its new exact head and base, records a new implementation revision, and +re-enters Stage 4. The Implementor and the selected evidence evaluate that revision before it can return to Stage 5. -## 6. Conflict boundary +## 9. Conflict boundary + +Every Git conflict suspends. There is one profile, and this is it: a conflicted +merge returns a closed conflicted result, restores the pre-merge Workspace root, +retains complete normalized conflict evidence, publishes an actionable handoff, +and suspends for manual resolution. No conflict is resolved automatically, by +generated XMD, or by any structured text-conflict capability. -Tool-less conflict handling grants no tool or checkout to the Agent. The Agent -observes source evidence rendered by XMD and proposes desired file contents; -XMD owns every inspection and mutation. +Tool-less conflict handling grants no tool and no checkout to the Agent. The +Agent observes source evidence rendered by XMD; XMD owns every inspection and +mutation. The workflow Agent still receives no Git or GitHub operation, no +filesystem or shell operation, no Workspace, checkout, Repository or host path, +no native tool, and no MCP server carrying equivalent authority. -### 6.1 Durable conflict identity +### 9.1 Durable conflict identity Before attempting a merge, XMD checkpoints: @@ -271,171 +543,485 @@ conflict set. Every conflict entry identifies: - the exact repository-relative path; - the conflict classification; -- the base, ours, and theirs object identities and modes when Git supplies - them; and +- the base, ours, and theirs object identities and modes when Git supplies them; + and - the corresponding stage numbers for entries retained in the unmerged index. The journal retains structured identity and classification, not only rendered -conflict markers. The conflict profile decides what happens to the native merge -state: - -- The suspend-on-conflict profile rolls the conflicted checkout back, publishes - the structured evidence against the unchanged pre-merge Workspace root, and - enters a durable human wait. It never offers a file mutation under that - evidence. -- A structured-resolution profile retains the conflicted Git state and its - Workspace root with the conflict result, or retains provider-owned state that - reconstructs that exact state and verifies the same conflict identity before - mutation. A later resume cannot combine evidence from one merge attempt with - the index of another. +conflict markers. It retains them so that a later capability could be specified +against real evidence, and so that a stale resolution can be rejected without +inspecting Agent output or trusting current Project state — not because anything +in this contract mutates under them. A changed head, target base, merge base, Repository identity, conflict set, or Workspace root makes the conflict admission stale. XMD discards no remote -history and grants no mutation under that stale identity; it observes the new +history and grants no mutation under a stale identity; it observes the new revision and restarts Stage 4. -### 6.2 Agent observation and proposal +### 9.2 Manual resolution + +Human resolution occurs outside the workflow Agent's authority, by pushing to +the implementation branch. On resume, XMD observes that push as a new Stage 4 +`{ headSha, baseSha }`, records the manual intervention, runs new implementation +evidence against it, and re-enters Stage 4. It inherits no Stage 5-7 conclusion: +no human edit carries the conflicted attempt's review verdicts forward. + +Ordinary text conflicts, binary files, submodules, ambiguous renames, unsafe +symbolic links and unrecognized index forms are classified and retained +identically, because they all suspend. Classification is evidence for the human +reading the handoff, not a branch in the procedure. + +## 10. Stage 7 target publication and terminal settlement + +Stage 7 is where the run leaves the factory, and its ordering is what makes a +completed replay provider-free. + +### 10.1 The merge is constructed, not requested + +Stage 7 merge creates a trusted merge commit whose **first parent is the exact +reviewed `baseSha`** and whose **second parent is the exact reviewed +`headSha`**. That is the opposite order from Stage 4's synchronization merge, +and the difference is deliberate: Stage 4 brings the target into the +implementation, Stage 7 brings the implementation onto the target. + +It does not call an ordinary GitHub squash, rebase or merge endpoint. Final +delivery is this constructed merge commit and nothing else, so the reviewed +parent pair is preserved in the published history rather than replaced by a +commit no reviewer saw. + +### 10.2 Target publication is a compare-and-swap + +`Git.PublishTarget` receives the exact merge commit, the reviewed head, and the +expected remote `baseSha`. The repository, remote, target ref, credential and +non-force policy are host-owned and are not authored props. + +It performs one non-force ref update only after observing the target equal to +`baseSha`. It adopts a target already equal to the exact merge commit, with +nothing performed. Every other observation — a target at another commit, an +incomplete observation, a permanent ambiguity, a temporary unavailability — +refuses or reconciles without mutating anything. + +A race that moves the target before or during publication therefore cannot +publish over it. Because the reviews name exact revisions, that race invalidates +them: the item returns to Stage 5 when only rereview is required, and to Stage 4 +when synchronization or implementation work is required. + +`Git.Push`, `Git.PublishTarget`, `Git.Merge`, pull-request upsert, `PullRequest.Ready`, `PullRequest.Close` and `PullRequest.Merged` are seven distinct operations with distinct subjects, ceilings and reconciliation. None is a spelling of another, and in particular publishing a target and observing that a pull request merged are two facts a Git host can hold separately. + +### 10.3 Terminal ordering + +There are two terminal paths and they share no step list. Each begins by retaining its authenticated exact-revision decision **before** any effect is attempted, and each ends with terminal settlement after every step before it has completed. A `change` decision is on neither path: it is not terminal, and it returns the run to the earliest stage it names. + +**The merged path**, in this order: + +1. Retain the `merge` decision of §11.2, bound to the exact reviewed revision and the authenticated actor. +2. Construct the trusted merge commit — `` with parents `[reviewedBase, reviewedHead]`. +3. Publish the target — `` under §10.2. +4. Observe that the pull request merged — ``, the reconciled Git-host observation of [Workflow workspaces](./workflow-workspace-spec.md) §7.11, against the published merge commit. A Git host records a pull request as merged on its own schedule, so this is its own retained step and not something publication implies. §10.4 says what the run does while it has not caught up. +5. Close the issue — ``. +6. Move the Project item to `Closed` — ``. +7. Publish terminal kind `merged`. + +**The abandoned path**, in this order: + +1. Retain the `abandon` decision of §11.2, bound to the exact reviewed revision, the authenticated actor and its required reason. +2. Close the pull request unmerged — ``. +3. Close the issue — ``. +4. Move the Project item to `Closed` — ``. +5. Publish terminal kind `abandoned`. + +An abandonment constructs no merge, publishes no target and observes no merged state; there is nothing it reviewed that it is publishing. A merge closes no pull request; the Git host closes it when the target moves, which is what step 4 observes rather than performs. + +Every step on either path is a separate reconciled effect or a separate retained transition, and no distributed transaction is claimed across the Durable Object, native Git and processes, GitHub and Project V2. An interruption resumes at the first uncommitted or unreconciled step. + +Terminal settlement is last on both paths for one reason: a completed run replays without contacting a provider. If terminal completion preceded a projection, the replay that was supposed to repair GitHub would be exactly the replay that is forbidden to reach it. -XMD renders the required conflict evidence and only the related source -observations to the Implementor. The workflow Agent still receives: +Both terminal kinds retain the authorizing actor, the exact reviewed revision, the resulting provider identities, and the reason where one is required, in the `FactoryTerminal` record of §11.2 — whose two shapes differ exactly as these two paths do. Both retain the Project item, the implementation branch, the issue and pull-request comments, the journal, the Workspace roots and the Agent evidence. Reopening the issue afterwards does not reopen the completed run; continuing that work requires a new linked issue, and therefore a new run. -- no Git or GitHub operation; -- no filesystem or shell operation; -- no Workspace, checkout, Repository, or host path; -- no native tool; and -- no MCP server carrying equivalent authority. +### 10.4 Waiting for the host to notice -The Implementor returns generated XMD containing proposed file writes and -deletions. That source is untrusted data until the workflow admits it. +A pull request observed still open after a successful target publication is temporary unavailability, not absence and not refusal — the Git host has not yet recognized its own ref moving. The factory waits for it in two stages, and neither is a human decision. -When the structured-resolution profile is installed, conflict resolution adds -a conflict-scoped generated-XMD admission. It is narrower than the workflow -host's ordinary write table: +The authored factory first performs a **bounded retry** around `PullRequest.Merged`. The retry count, total duration and backoff are host ceilings configured for the deployment; they are not props on the component and no document widens them. -- every write or deletion must name an exact path in the authorized conflict - set; -- the recorded Repository, head, base, merge base, conflict identity, and - Workspace root must still match; -- no generated component may stage, commit, merge, push, invoke a process, read - a credential, update GitHub, or mutate a non-conflict path; and -- the admitted source and complete ceiling are retained before the first file - effect. +When that retry is exhausted while the pull request is still open, the run enters a **machine wait** at Stage 7. -The Agent decides desired file contents. XMD applies the admitted file effects, -stages the exact conflict scope, verifies that the index contains no unresolved -entries and no unauthorized mutation, creates the merge commit with the -checkpointed head and base as its parents, runs the selected implementation -evidence, and publishes the resulting descendant through the ordinary -non-force Push effect. +A machine wait is a second kind of durable wait, and it is deliberately not a suspension in the typed-answer sense. A typed suspension publishes a request and a response schema and ends when somebody delivers one value that satisfies it. This wait asks nobody anything: it ends because a later execution looked at a provider again. It therefore has no response schema, no `xmd workflow answer` route, no web form and no bound value, and nothing about the typed-answer protocol — `suspension_request`, `suspension_answer`, a suspension ID — takes part in it. It is a second wait *kind* inside the existing lifecycle, never a second lifecycle controller. -The item remains at Stage 4 throughout. A clean result is a new implementation -revision, not a review pass. +What it does share is the lifecycle boundary. The retained wait event and the `suspended` run status commit together, the executor acquisition is released only after that commit, and a settlement the host refuses publishes neither. Its retained event kind is `machine_wait`, distinct from `suspension_request`, and its stable identity is a `waitId` the trusted execution derives from the run and the authored expansion on the same engine-owned terms every other durable position uses. The run's stop reason references that filtered `machine_wait` event, so inspection reports that the run is waiting on provider state and offers no response schema and no answer command. -### 6.3 Conflict classes +The wait's subject is the canonical pull-request URL, the expected merge commit, the retained merge-decision event identity, the current implementation revision and `retriesExhausted: true`, retained as the `MergedObservationWait` record of §11.2. -The protocol classifies every conflict before asking the Implementor for a -proposal. Ordinary text conflicts may be admitted by a bounded text-conflict -capability. Binary files, submodules, ambiguous renames, unsafe symbolic links, -unrecognized index forms, and every other unsupported class suspend for human -resolution. A mixed set containing one unsupported entry is unsupported as a -whole; no partial Agent mutation is admitted. +**Waking is permission to look again, not an answer.** Two sources may wake it. An authenticated intake for a relevant pull-request state change, correlated to that exact wait subject, is retained as a delivery-plane transaction under §5: it takes no executor acquisition, appends no lifecycle outcome, changes no run status, and supplies no answer, verdict, stage, transition or observation result. It records a bounded wake notification and nothing else. An authorized operator may also resume the run, which is executor-side control rather than a delivery — a resume is not a forged intake. -Human resolution occurs outside the workflow Agent's authority. On resume, XMD -accepts the manually changed branch only by observing its new exact -`{ headSha, baseSha }`, recording the manual intervention and re-entering Stage -4. No human edit inherits the conflicted attempt's review conclusions. +A duplicate wake notification changes nothing. One naming another wait, a spent wait, an invalidated wait or a terminal run refuses and leaves the active wait exactly as it was. -## 7. Structural consequences +A later executor consumes one retained wake notification inside the run's transaction and appends one filtered `machine_wake` event for that exact `waitId`; an authorized operator resume appends the same event with `source: "operator-resume"`. Consuming the wake and ending the retained wait are one transaction, so a crash before it commits leaves both the wait and the notification pending, and a replay after it commits restores the wake event without consuming or appending anything again. -This factory adds the following structural contracts. +After the wake event, authored control flow invokes `PullRequest.Merged` again. Only the compatible adoption of §7.11's first row advances the terminal sequence. An observation that is still open may retry and wait again at a new durable position; a merge at another commit and a pull request closed unmerged remain conflicts and end the wait as one rather than continuing it. -### 7.1 XMD workflow +An explicit resume with neither a pending provider wake nor operator-resume authority ends nothing. It reports the same machine wait and settles `suspended` again. Cancellation follows ordinary run cancellation and invents neither a wake nor a merged observation. + +The wait appends no lifecycle outcome and moves no stage, and terminal settlement stays absent until the observation adopts. + +## 11. Exact public contracts + +### 11.1 Authored construct inventory + +These are the authored public forms. `as` is mandatory wherever a result is +bound, and form validation runs before any context, provider or credential +access. Durable effect identity is always engine-derived from the run and the +expansion; it is never a document prop. + +| Construct | Exact authored form and result | Owner and durable boundary | +| --- | --- | --- | +| `Issue.Comment` | Paired `…`; the content is the body; binds `{ url }` | Issue-provider effect; the natural key is the canonical issue URL plus the engine effect identity, so the body is not identity | +| `PullRequest.Comment` | Paired `…`; the content is the body; binds `{ url }` | Git-host effect; canonical pull-request URL plus engine effect identity | +| `Project.Status` | Self-closing ``; binds the normalized `{ item, field, option }` | Project-provider effect; the exact configured Project, item, field and option ceiling, against the current option as pre-state | +| `PullRequest.Ready` | Self-closing ``; binds normalized ready pull-request evidence | Git-host effect; only an accepted Stage 6 outcome authorizes invocation | +| `PullRequest.Close` | Self-closing ``; binds normalized `{ url, state: "closed", merged: false }` | Git-host effect; used only by a retained abandonment | +| `Issue.Close` | Self-closing ``, or the same form with `reason="not_planned"`; binds the normalized URL, state and reason | Issue-provider effect; `reason` is a closed enum and must match the retained terminal intent | +| `Git.Merge` | Self-closing ``, or the same form with `purpose="publish"`; binds the `GitMergeResult` union of [Workflow workspaces](./workflow-workspace-spec.md) §7.8 — `{ outcome: "clean", purpose, firstParent, secondParent, mergeBase, commit, workspaceRoot }` or `{ outcome: "conflicted", purpose, firstParent, secondParent, mergeBase, workspaceRoot, conflicts }` | Workspace-local Git effect; repository, checkout, root and acquisition are authenticated provider state; a clean publication is atomic and a conflict restores before the result is published | +| `Git.PublishTarget` | Self-closing ``; binds normalized target, expected and published evidence | Git-host effect; remote, ref, credential and non-force ceiling are host-owned; exact compare-and-swap reconciliation | +| `Evidence.Run` | Self-closing ``, where `commands` is an ordered non-empty list of non-empty argv vectors; binds the `EvidenceRunResult` of [Workflow workspaces](./workflow-workspace-spec.md) §10.5 — `{ completion, authoredCommands, executed, runTimeout? }`, where each executed row is `{ argv, outcome, status?, signal?, limit?, stdout, stderr }` and each channel is `{ text, retainedBytes, producedBytes, truncated }` | Trusted runner-host effect; the exact retained root and the executable, environment, working-root, per-command duration, whole-run duration, output and process-tree ceilings; a fail-fast pipeline binding the executed prefix, launch/output-pump/teardown failures binding no result and retaining bounded error evidence, cancellation committing nothing; absent from Agent and generated-XMD capabilities; a completed replay runs nothing | +| `PullRequest.Merged` | Self-closing ``; binds `{ subject, state: "closed", merged: true, mergeCommit, decision: "adopted" }` | Git-host reconciled observation, [Workflow workspaces](./workflow-workspace-spec.md) §7.11; it mutates nothing and adoption is its only completion; keyed by the canonical pull-request URL | +| Remote `WorkflowHost` | The existing four-method host boundary — `useRunHost()`, `useLifecycle()`, `useDelivery()`, `attach()` — with a Cloudflare runtime-named implementation beside the Deno one; start, lookup, execute, deliver and inspect are lifecycle operations reached through it rather than method names of their own, and a remote host receives no transitions type of its own. Its transition and request types are provider-neutral and become package-root public types; the runner-to-owner messages are private to one release, admitted by an exact build fingerprint ([Workflow workspaces](./workflow-workspace-spec.md) §13.2) | A host assembly contract rather than an XMD component; the execution, delivery and inspection planes stay distinct across it | +| Factory protocol records | The closed versioned schemas of §11.2 | A provider-neutral durable protocol; neither an XMD component nor a TypeScript lifecycle controller | + +Each construct's closed props, form, binding, request, natural key, compatible pre-state, normalized result, refusal and unavailability behavior, cancellation, replay, provider ownership, credential boundary, and whether it is Workspace-local or an external reconciled effect are defined normatively in [Workflow workspaces](./workflow-workspace-spec.md): §7.8 `Git.Merge`, §7.9 `Git.PublishTarget`, §7.10 `PullRequest.Comment`, `PullRequest.Ready` and `PullRequest.Close`, §7.11 `PullRequest.Merged`, §10.3 `Issue.Comment` and `Issue.Close`, §10.5 `Evidence.Run`, §10.6 `Project.Status`, §10.7 the credential boundary they share, and §13.2 the remote host and its transport. A later implementation may choose ordinary private function and module names; it may not change these public forms, their records or their ownership. + +### 11.2 Factory protocol records + +The factory's lifecycle is journaled as closed immutable records, not held in a controller. These are the schemas the journal retains and every role outcome is parsed into. They are provider-neutral data and parsers; nothing here becomes a TypeScript state machine beside the journal, and nothing here is an XMD component. + +Every record carries `schema`, its discriminant, and `version`, which is `1` for all of them. Parsing is strict in both directions: an unknown `schema`, an unknown `version`, an unknown member, a missing required member, and a value outside a closed enum each refuse the record rather than being ignored or defaulted. A refusal names the member path and never the value behind it, on the same terms retained props and journal payloads are described. + +#### Identities and subjects + +```ts +type FactoryStage = 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8; + +type FactoryRole = "user" | "architect" | "planner" | "implementor"; + +interface FactorySubject { + readonly schema: "factory-subject"; + readonly version: 1; + readonly runId: string; + readonly authority: string; + readonly issueNodeId: string; + readonly issueUrl: string; + readonly repositoryId: string; + readonly projectItemId: string; + readonly statusFieldId: string; +} + +interface ImplementationRevision { + readonly schema: "implementation-revision"; + readonly version: 1; + readonly headSha: string; + readonly baseSha: string; +} +``` + +`FactoryStage` is the numeric stage; the nine status strings of §2 are its projection and are never parsed back into it. `runId` is the §1.1 derivation, and `authority` and `issueNodeId` are the exact bytes it was derived from, retained so drift is detectable without re-deriving. `repositoryId`, `projectItemId` and `statusFieldId` are provider identities compared byte for byte. `headSha` and `baseSha` are lowercase hexadecimal commit IDs. A revision is compared as the whole pair: two revisions are equal only when both halves are. + +#### Evidence and role outcomes + +```ts +interface EvidenceReference { + readonly schema: "evidence-reference"; + readonly version: 1; + readonly effectId: string; + readonly revision: ImplementationRevision; + readonly passed: boolean; +} + +interface FactoryHandoff { + readonly schema: "factory-handoff"; + readonly version: 1; + readonly stage: FactoryStage; + readonly role: FactoryRole; + readonly actor: FactoryActor; + readonly summary: string; + readonly revision?: ImplementationRevision; + readonly evidence?: readonly EvidenceReference[]; +} + +interface FactoryActor { + readonly schema: "factory-actor"; + readonly version: 1; + readonly kind: "human" | "agent"; + readonly id: string; +} + +type FactoryOutcome = + | { readonly schema: "factory-outcome"; readonly version: 1; readonly kind: "advance"; readonly from: FactoryStage; readonly to: FactoryStage; readonly handoff: FactoryHandoff } + | { readonly schema: "factory-outcome"; readonly version: 1; readonly kind: "amend"; readonly stage: FactoryStage; readonly handoff: FactoryHandoff; readonly supersedes: string } + | { readonly schema: "factory-outcome"; readonly version: 1; readonly kind: "invalidate"; readonly invalidation: FactoryInvalidation } + | { readonly schema: "factory-outcome"; readonly version: 1; readonly kind: "verdict"; readonly verdict: FactoryVerdict } + | { readonly schema: "factory-outcome"; readonly version: 1; readonly kind: "suspend"; readonly suspension: ConflictSuspension }; + +interface FactoryInvalidation { + readonly schema: "factory-invalidation"; + readonly version: 1; + readonly from: FactoryStage; + readonly earliestInvalidated: FactoryStage; + readonly reason: string; + readonly actor: FactoryActor; +} + +type FactoryVerdict = + | { readonly schema: "factory-verdict"; readonly version: 1; readonly stage: 5; readonly role: "planner"; readonly revision: ImplementationRevision; readonly decision: "pass" | "changes"; readonly reason: string; readonly evidence: readonly EvidenceReference[] } + | { readonly schema: "factory-verdict"; readonly version: 1; readonly stage: 6; readonly role: "architect"; readonly revision: ImplementationRevision; readonly decision: "pass" | "changes"; readonly reason: string; readonly plannerVerdict: string }; +``` + +`summary` and `reason` are the only presentation fields, and both are bounded: they are what a comment renders, never what identity compares. `supersedes`, `effectId` and `plannerVerdict` are journal event identities, so a record points at the history it replaces or depends on instead of copying it. An `advance` whose `to` is not `from + 1` refuses, and so does an `amend` whose `stage` is not the current frontier stage. A Stage 6 verdict whose `plannerVerdict` names a verdict for another revision refuses. + +#### Conflict suspension + +```ts +interface ConflictSuspension { + readonly schema: "conflict-suspension"; + readonly version: 1; + readonly revision: ImplementationRevision; + readonly mergeBase: string; + readonly workspaceRoot: string; + readonly conflictIdentity: string; + readonly mergeEffectId: string; + readonly suspensionId: string; +} +``` + +`workspaceRoot` is the *restored* pre-merge root, `mergeEffectId` names the `Git.Merge` event whose conflicted result holds the normalized conflict set of [Workflow workspaces](./workflow-workspace-spec.md) §7.8, and `conflictIdentity` is derived from the checkpointed identities and that normalized set. The conflict set is not copied here: one account of it, in the effect that produced it. + +#### Stage 7 decisions + +```ts +type Stage7Decision = + | { readonly schema: "stage-7-decision"; readonly version: 1; readonly kind: "merge"; readonly revision: ImplementationRevision; readonly actor: FactoryActor; readonly deliveryId: string } + | { readonly schema: "stage-7-decision"; readonly version: 1; readonly kind: "abandon"; readonly revision: ImplementationRevision; readonly actor: FactoryActor; readonly deliveryId: string; readonly reason: string } + | { readonly schema: "stage-7-decision"; readonly version: 1; readonly kind: "change"; readonly revision: ImplementationRevision; readonly actor: FactoryActor; readonly deliveryId: string; readonly earliestInvalidated: FactoryStage; readonly reason: string }; +``` + +All three bind the exact revision they were made against and the authenticated actor who made them, and all three name the delivery identity they arrived under. `abandon` requires a reason and `change` requires both a reason and the earliest stage it invalidates; `merge` takes neither, because approving what two reviews already passed adds no new claim. A decision whose `revision` is not the current frontier revision refuses. `merge` and `abandon` are terminal intents that §10.3 orders; `change` is not terminal and reduces to an invalidation. + +#### Waiting for the merged observation + +```ts +interface MergedObservationWait { + readonly schema: "merged-observation-wait"; + readonly version: 1; + readonly waitId: string; + readonly subject: string; + readonly expectedMergeCommit: string; + readonly decisionId: string; + readonly revision: ImplementationRevision; + readonly retriesExhausted: true; +} + +interface MergedObservationWake { + readonly schema: "merged-observation-wake"; + readonly version: 1; + readonly waitId: string; + readonly source: "provider-intake" | "operator-resume"; + readonly intakeId?: string; +} +``` + +`waitId` is the machine wait's own identity, derived from the run and the authored expansion; it is not a suspension ID, and no typed-answer record names it. `subject` is the canonical pull-request URL and `decisionId` names the retained `merge` decision this wait belongs to. `retriesExhausted` is literal: the record exists only after §10.4's bounded retry has run out, so a wait retained before that would be a run skipping the cheap path. The wait carries no stage, no outcome, no verdict and no response schema. + +`MergedObservationWake` is what a later executor appends for that exact `waitId`. `intakeId` is required exactly when `source` is `"provider-intake"` and absent exactly when it is `"operator-resume"`, because the operator path is authenticated executor-side control rather than a delivery, and a wake that claimed an intake it does not have would be a forged one. A wake says only that another observation attempt may occur; it carries no observation result, and consuming it authorizes exactly one reobservation. + +#### The configured stage-to-option table + +```ts +interface StageOptionTable { + readonly schema: "stage-option-table"; + readonly version: 1; + readonly projectId: string; + readonly statusFieldId: string; + readonly options: readonly StageOption[]; +} + +interface StageOption { + readonly stage: FactoryStage; + readonly optionId: string; + readonly displayName: string; +} +``` + +`options` holds exactly nine entries, one per stage `0` through `8`, ordered by `stage`. Every `optionId` is distinct, and every `displayName` equals the §2 status string for its stage. This is the retained form of §5.1's configuration: it is validated against a complete reread of the Project before it is used, and a table that does not satisfy every one of those conditions refuses rather than being partially applied. + +#### The active frontier and its reduction + +```ts +interface FactoryFrontier { + readonly schema: "factory-frontier"; + readonly version: 1; + readonly stage: FactoryStage; + readonly revision?: ImplementationRevision; + readonly accepted: readonly string[]; + readonly terminal?: FactoryTerminal; +} +``` + +`accepted` is the ordered list of journal event identities forming the active chain, oldest first, one per stage that has been passed. `revision` is absent before Stage 4 produces one. `terminal` is present only on a settled run. + +The frontier is a reduction over the retained outcomes, and its inputs and outputs are exactly these: + +| Input | Resulting frontier | +| --- | --- | +| `advance` from stage *n* to *n + 1* | `stage` becomes *n + 1*; the handoff's event is appended to `accepted` | +| `amend` at the current stage | `stage` is unchanged; the superseded event is replaced in `accepted` by the amending one, and the superseded event stays in the journal | +| `invalidate` naming earliest stage *e* | `stage` becomes *e*; every entry in `accepted` for a stage at or after *e* is dropped from the active chain and kept in the journal | +| a head change: a new `headSha` | `stage` becomes 4, `revision` becomes the new pair, and the Stage 5, 6 and 7 entries drop | +| a base-only change requiring no work | `stage` becomes 5, `revision` becomes the new pair, and the Stage 5, 6 and 7 entries drop | +| a base change requiring synchronization or implementation work | `stage` becomes 4 on the same terms as a head change | +| a `merge` or `abandon` decision on a run whose `terminal` is already present | refused; the frontier is unchanged | + +A reduction that would leave `stage` outside `0`-`8`, leave `accepted` holding two entries for one stage, or advance past a stage with no accepted entry refuses rather than producing a frontier. + +#### Terminal settlement + +```ts +type FactoryTerminal = + | { + readonly schema: "factory-terminal"; + readonly version: 1; + readonly kind: "merged"; + readonly revision: ImplementationRevision; + readonly actor: FactoryActor; + readonly decisionId: string; + readonly mergeCommit: string; + readonly publication: string; + readonly mergedObservation: string; + readonly issueClosure: string; + readonly projectClosure: string; + } + | { + readonly schema: "factory-terminal"; + readonly version: 1; + readonly kind: "abandoned"; + readonly revision: ImplementationRevision; + readonly actor: FactoryActor; + readonly decisionId: string; + readonly reason: string; + readonly pullRequestClosure: string; + readonly issueClosure: string; + readonly projectClosure: string; + }; +``` + +The two kinds do not share a member list, and that asymmetry is the contract: a `merged` terminal names a merge commit, a target publication and a merged observation, and an `abandoned` terminal names a pull-request closure and a reason and can name none of the first three. + +Every member ending in `Id` or naming a step is a **journal event identity**, and it stays one. The terminal record points at the reconciled effects that completed rather than restating their results, so each provider identity keeps one durable source — the effect result that observed it — and settlement ordering stays checkable against the journal instead of against a copy that could disagree with it. + +Validation follows those references rather than copying through them. Every referenced event must belong to this run and this active terminal intent, parse under its exact effect kind, be complete, and agree with the terminal record's revision, actor and decision where each applies; the referenced results carry the normalized provider identities, and terminal validation checks compatibility without duplicating them. A missing, foreign, wrong-kind, incomplete, invalidated, duplicated or cross-path event refuses settlement — and cross-path is exact: a `merged` terminal cannot name a pull-request close-unmerged or any abandonment effect, and an `abandoned` terminal cannot name a merge construction, a target publication or a merged observation. That is what makes §10.3's two orders checkable from the record alone. + +## 12. Structural consequences + +### 12.1 XMD workflow - The workflow holds one issue lifecycle in one durable run while Stage 4 emits zero or more implementation revisions. -- Every role outcome is validated against the current stage and active handoff - frontier. Review outcomes additionally carry the exact implementation +- Every role outcome is validated against the current stage and the active + handoff frontier. Review outcomes additionally carry the exact implementation revision. - Stage transition, revision observation, synchronization, conflict handling, - review, and closure remain authored XMD control flow. GitHub Actions contains - no parallel decision procedure. + review, and closure are authored XMD control flow. GitHub Actions contains no + parallel decision procedure, and neither does the Durable Object: it owns + state and admission, not stage choice. - The trusted definition comes from the run's immutable definition SHA. Draft pull-request content never becomes the workflow definition executed with GitHub credentials. -### 7.2 Journal +### 12.2 Journal - The immutable workflow definition continues to identify the run. -- The journal additionally retains implementation-revision observations, - active and invalidated handoffs, exact review subjects, merge checkpoints, - conflict classifications, conflict-scoped admission ceilings, evidence - outcomes, pushes, Project updates, and terminal reason. -- A conflict record must be sufficient to reject stale resolution without +- The journal additionally retains implementation-revision observations, active + and invalidated handoffs, exact review subjects, merge checkpoints, conflict + classifications, evidence outcomes, pushes, target publications, Project + updates, retained decisions and terminal reason. +- A conflict record is sufficient to reject a stale resolution without inspecting Agent output or trusting current Project state. - Exported `.xmd` artifacts remain immutable evidence. They are not the live run - store, executor lock, or continuation authority used by later Actions jobs. + store, the executor acquisition, or continuation authority for a later Actions + job. -### 7.3 Workspace and Git effects +### 12.3 Workspace and Git effects -- The suspend-on-conflict profile restores the pre-merge Workspace root and - retains conflict evidence only. A structured-resolution profile must instead - retain the conflicted Git index and working tree with the journal result that - identifies them, or retain equivalent provider-owned state from which that - exact conflict can be reconstructed and reverified. +- A conflicted merge restores the pre-merge Workspace root and retains conflict + evidence only. - A trusted merge effect observes and fixes head, base, and merge base before - mutation. Clean and conflicted results are distinct closed outcomes. -- Conflict-scoped file writes and deletions use the existing Workspace-local - transaction boundary but add exact path and conflict-identity ceilings. -- Merge commit creation verifies both parents and the empty conflict set before - publication. Push remains the existing reconciled non-force effect. -- Rebase and force-with-lease are absent. Adding either later requires a new - external-effect contract, reconciliation semantics, and invalidation proof; - it cannot be represented as another spelling of Push. + mutation. Clean and conflicted results are distinct closed outcomes, and the + parent order differs by purpose. +- Push remains the existing reconciled non-force effect. Target publication is a + separate reconciled effect with its own subject and compare-and-swap + pre-state. +- Rebase, force-with-lease and reset-based replacement are absent. -### 7.4 Invalidation frontier +### 12.4 Invalidation frontier - A changed implementation head always places the frontier at Stage 4. - A base-only change places it at Stage 5 unless Stage 4 work is required. -- A merge attempt, conflict proposal, clean merge, manual resolution, evidence - correction, or push does not advance the item by itself. +- A merge attempt, a clean merge, a manual resolution, an evidence correction, + or a push does not advance the item by itself. - Review approval is keyed by the complete SHA pair, so neither half can drift while Stages 5-7 remain accepted. -## 8. Remaining material decisions - -### 8.1 Product decision: first-release conflict scope - -The remaining product decision for conflict handling is whether the first -factory release implements conflict-scoped generated-XMD resolution for -ordinary text conflicts, or suspends for human resolution on every conflict. - -The recommended first release is the smaller contract: - -- perform clean merges automatically; -- suspend on every conflict; -- prohibit rebases and force pushes; and -- retain the structured conflict evidence needed to add ordinary text-conflict - resolution later as a bounded capability. - -This release proves the revision, invalidation, merge, non-force publication, -and human-resumption boundaries without making conflicted Workspace state and -conflict-scoped mutation admission prerequisites for the first useful factory. - -### 8.2 Deployment architecture decisions - -The lifecycle still requires three deployment choices before it can run across -ephemeral GitHub Actions runners: - -1. Select a durable WorkflowRun, Workspace, Agent-session, and executor-lock - provider reachable by every invocation. An Actions artifact is immutable - evidence and does not satisfy live continuation or locking. -2. Select the authorized ingress for Project admission, human answers, and - resume requests, including how the GitHub actor is authenticated. That - ingress may wake Actions but may not interpret role outcomes or own stage - transitions. -3. Select the GitHub principal and exact repository, pull-request, issue, and - Project permission ceilings, including who authorizes the Stage 7 merge or - abandonment and which target-branch merge method is permitted. - -These choices configure the host boundary. They do not create a second state -machine and do not transfer procedure authority out of XMD. +## 13. Structural acceptance checklist + +A factory implementation satisfies this specification when every item holds: + +1. The run ID equals the §1.1 derivation for its issue, and no mutable value + takes part in it. +2. Duplicate admission for one authenticated subject routes to one run; a + changed retained provider identity refuses as drift. +3. The nine Project statuses are exactly §2's, `User` included, and forward + progress is strictly adjacent. +4. Same-stage correction replaces the frontier without erasing history, and + backward invalidation reruns every later stage. +5. Stages 5-7 name an exact `{ headSha, baseSha }`, and no verdict is inherited + across a changed pair. +6. Only an accepted Stage 6 verdict takes the pull request out of draft. +7. Every Git conflict suspends, restores the pre-merge root, and retains + normalized conflict evidence; nothing resolves a conflict automatically. +8. A manual resolution is observed as a new Stage 4 revision and inherits no + Stage 5-7 conclusion. +9. Start, resume, recovery, mutation, transition, publication and settlement + validate the exact live executor acquisition and the expected Workspace root. +10. Intake, answer delivery and decision delivery take no acquisition, append no + lifecycle outcome and change no run status. +11. Inspection is read-only and authorizes no transition. +12. Native Git, evidence processes and Agent clients run only on the ephemeral + runner; the Durable Object runs none of them. +13. Webhook signature verification precedes parsing, and reread precedes + authorization. +14. `repository_dispatch` carries only a retained intake identity. +15. The OIDC admission validates issuer, audience, repository ID, owner ID, + event name, workflow ref and SHA, and the configured workflow identity. +16. Answers and Stage 7 decisions come only from the authenticated form bound to + the exact subject; no comment carries authority. +17. The App holds exactly §6's permissions, `.github/workflows/**` mutation is + refused, and no secret reaches props, context, durable records, comments, + output or diagnostics. +18. Every GitHub mutation observes before mutating, adopts only a compatible + completion, performs once, and refuses conflict, ambiguity, incomplete + observation and temporary unavailability. +19. The Stage 4 merge parent order is `[implementationHead, targetBase]` and the + Stage 7 order is `[reviewedBase, reviewedHead]`. +20. Target publication updates the ref only from an observed `baseSha`, adopts + only the exact merge commit, and never force-updates. +21. The merged state of the pull request is observed as its own retained step + after publication and before issue closure, and is adopted only at the exact + published merge commit. A pull request still open is a bounded retry and + then a durable machine wait; one closed unmerged is a conflict. +22. The merged and abandoned paths of §10.3 run in their stated orders, and + neither borrows a step from the other. +23. A terminal `merged` or `abandoned` state is published only after every + required projection completes, and a completed replay attaches no external + provider. +24. Every authored construct binds the exact record + [Workflow workspaces](./workflow-workspace-spec.md) defines for it, and + every factory protocol record parses under §11.2 with strict refusal of an + unknown schema, version or member. diff --git a/specs/workflow-spec.md b/specs/workflow-spec.md index a84a29315..409d12ab9 100644 --- a/specs/workflow-spec.md +++ b/specs/workflow-spec.md @@ -114,6 +114,15 @@ base is any revision expression, so both are external text on the same terms as retained props. A value installed without a run id, a base or a pinned commit identifies no run and is refused before any document executes. +A host that keeps its runs remotely has decided the same things in the same +order, and installs the same value. `retainedWorkflowInstallation()` names a run +its host already created and a commit its host already pinned; whether the +record behind that name lives in a local file or in a remote owner is host +arrangement the execution never learns. What the execution requires is +unchanged: the run id, base and pinned commit it was handed must be exactly what +the journal it reads records, and a journal recording a different run is +`StaleInputError` wherever the journal is kept. + ### 3.2 Where workflow-run identity is decided **Workflow-run identity is execution-owned, and it is not middleware of any @@ -328,6 +337,16 @@ after an interruption. The Deno host installs its own with entrypoint is the only place SQLite, run-id hashing, filesystem paths and host behavior appear. Shared modules import none of them and detect no runtime. +A second host installs its own the same way. Cloudflare is a runtime-named +adapter beside the Deno one, not a second contract: it answers `create()` and +`lookup()` with a `WorkflowRunDatabase` of its own, and every shared +WorkflowRun surface above it stays host-neutral. The lifecycle transition and request types a host assembly speaks — `WorkflowExecutionTransitions`, `WorkflowBeginRequest`, `WorkflowExecutionBegun`, `WorkflowForkRequest`, `WorkflowForkSelection` and `WorkflowRunCreation` — are part of that neutral surface and are published from the package root; a runtime-named entrypoint may re-export them for source compatibility, but what belongs behind one is the implementation and its retained encoding, not the shape of the request. Nothing below changes for it — +immutable run identity is compared the same way, recognition stays strict, +events reach storage already filtered, a caller still owns the transaction it +opened, and a completed run still replays without attaching a provider. What a +remote adapter adds is where the bytes live and how an executor reaches them +(§9.8), which identity and recognition already treat as host arrangement. + A handle is a lease belonging to the scope that asked for it. Lease teardown makes that handle unusable, and every later call answers with a closed-handle failure rather than reopening the file. It does not close the run's physical @@ -401,6 +420,23 @@ and a local checkout path are **retrieval metadata** — replaceable, excluded from the comparison, never containing credentials, and reauthorized by the host before use. A run that moves between hosts is the same run. +#### A host may derive the id it selects + +A run id is opaque, and §3.3 of the Workspace specification already lets an +authorized caller select one. A host may equally derive one from the subject the +run is about, and a derived id is an ordinary selected id: it has to be a +non-empty string containing no NUL, and it has to be the same string every time +the same subject is admitted. Nothing else about it is constrained, and nothing +here narrows the ids an ordinary caller may choose. + +The software factory derives its ids that way. A factory run id is the lowercase unpadded RFC 4648 Base32 encoding of the full SHA-256 digest of the UTF-8 bytes `github-issue-v1`, a NUL, the canonical GitHub authority, a NUL, and the exact GitHub issue GraphQL node ID — 52 characters of `a`-`z` and `2`-`7`, so the storage rule above is satisfied by construction. + +Those two inputs are the ones [the software factory](./github-actions-software-factory-spec.md) §1.1 defines, byte for byte, and this paragraph restates rather than generalizes them: the authority is the lowercase DNS hostname plus a non-default port, with no scheme, path, query, fragment, user information or trailing separator, and the node ID is GitHub's exact returned string with no case folding and no Unicode normalization. There is no broader Issue-provider authority in this hash — a hash whose inputs two documents spell differently is two hashes. + +That specification also owns every other factory protocol record: its §11.2 holds the closed versioned schemas, and no other document restates them. + +Because every input is immutable, admitting one issue twice derives one id and reaches one run through ordinary compatible reuse, and no separate idempotency concept appears. Two independent implementations given the same authority and node ID therefore produce the same id. A changed authority or node ID for a subject the host already retains is unsupported provider-identity drift, refused under §1.1 of that specification rather than derived into a second run. + ### 9.2 Creating a run is also how it is found `create()` answers with the stored run when the request describes it, and @@ -833,6 +869,52 @@ Version 1 reads and writes version 1. Unsupported versions are refused without the file being touched; partial version-1 initialization is corruption and is also left unchanged. +### 9.8 A remote host owns the same run + +Serialization decides who uses one connection next; it has never decided who may +advance a run. A remote host keeps both answers and gives each a different +mechanism. + +The owner of one run is selected from the public run id by the same arithmetic +§9.3 uses, so a remote run has exactly one durable owner and no second registry +can disagree with it. Inside that owner, operations on the run's storage are +serialized and each runs in a transaction, exactly as §9.6 states: the +connection queue, the transaction identities and the savepoint allocator are +provider-private and say nothing about lifecycle authority. + +**Executor acquisition is an authenticated connection.** The acquisition is that +connection's lifetime: the owner registers the exact acquisition when the +connection is admitted and invalidates it when the connection closes, which is +the staleness proof a remote host has in place of an operating system releasing +a file lock. Like the local lock it is not a time lease — no duration, expiry, +renewal, heartbeat, generation record or liveness poll — and closing it releases +executor ownership without rolling back anything already committed. A second +healthy executor follows or is refused, and cannot advance the run either way. + +**These requests require the acquisition**, and each validates the exact live +acquisition and the expected Workspace root inside its own mutating +transaction: start and resume, stale-execution recovery, document execution, +Workspace mutation, provider attachment, native execution performed against a +materialized root, lifecycle transition, accepted-outcome publication, and +terminal settlement. + +**These requests take no acquisition.** Delivery retains one externally supplied +value for one exact retained subject and does what typed answer delivery already +does: it begins no document execution, attaches no Workspace, inserts no +document-execution record, appends no journal event and changes no run status. +What authorizes it is the subject, not the caller's position in the lifecycle — +a suspension id the run's retained `suspension_request` names for an answer, and +the exact retained decision subject for a terminal decision. A value for a +subject the run is not holding is refused with nothing written. + +A **wake notification** is delivered on the same terms and is the one delivery that carries no value at all. A run may wait on a fact about a provider rather than on an answer, and that machine wait is a distinct event kind identified by a `waitId` rather than by a suspension id. An authenticated intake correlated to the exact wait subject retains a bounded notification saying that another observation may occur — no answer, verdict, stage, transition or observation result — and a later executor consumes it and appends the wake event in the run's own transaction. Delivery still stores and execution still decides; what changes is that here there is nothing stored for the execution to read except permission to look again. Read-only +inspection takes no acquisition either, and returns the immutable snapshot +surface of the lifecycle contract rather than a writable handle. + +A later executor is what turns retained delivery into progress: it consumes the +value inside the run's own transaction, appends the accepted event once, and +only then may execution continue past the wait. + ## 10. The document filesystem of a run A host attaches one run's Workspace to a document execution with diff --git a/specs/workflow-workspace-spec.md b/specs/workflow-workspace-spec.md index 5bcec74f9..78b24cfc5 100644 --- a/specs/workflow-workspace-spec.md +++ b/specs/workflow-workspace-spec.md @@ -359,6 +359,14 @@ resume, watchers, unattended iteration and remote host selection — is #300's. Nothing above waits on it: a suspended run continues through `xmd workflow answer` followed by an explicit `xmd workflow resume`. +**A machine wait is a second wait kind, not a second protocol.** Everything above describes a wait that ends when somebody delivers one typed value. A run can also wait on a fact about a provider — a remote object whose state a later observation will read — and that wait asks nobody anything. It has no request to answer, no response schema, no `xmd workflow answer` route and no bound value, so it publishes no `suspension_request`, consumes no retained answer and appends no `suspension_answer`. + +What it shares is this section's boundary and nothing else. Its retained event kind is `machine_wait`, distinct from `suspension_request`; its stable identity is a `waitId` the trusted execution derives from the run and the authored expansion, on the same terms every other durable position uses, and it is never a suspension ID. The wait event and the `suspended` run status commit together, the executor acquisition is released only after that commit, and a settlement the host refuses publishes neither. The stop reason references the filtered `machine_wait` event, so inspection reports a run waiting on provider state and offers no response schema and no answer command. + +A machine wait ends by being woken and then looking again. An authenticated intake correlated to the exact wait subject retains one bounded **wake notification** as an ordinary delivery-plane transaction — no executor acquisition, no lifecycle outcome, no run-status change — and it carries no answer, verdict, stage, transition or observation result. A duplicate notification changes nothing, and one naming another, spent, invalidated or terminal wait refuses without touching the active wait. An authorized operator resume is executor-side control rather than a delivery and reaches the same place. A later executor consumes one wake and appends one filtered `machine_wake` event for that exact `waitId` in one transaction, so a crash before the commit leaves both pending and a replay after it restores the event without consuming or appending again. The wake permits one further observation and decides nothing about its outcome; a resume with neither a pending wake nor operator authority reports the same wait and settles `suspended` again. + +The software factory's merged-observation wait ([the software factory](./github-actions-software-factory-spec.md) §10.4) is the first machine wait, and its closed records are §11.2 there. + ### 3.6 Interruption and cancellation differ Interrupting foreground execution, including with Ctrl-C, releases the current @@ -472,6 +480,45 @@ owning the same run lifecycle remotely. `export` produces the immutable portable evidence contract in `specs/xmd-artifact-spec.md`; it does not expose the live run database. +A remote host owns that surface unchanged. One durable owner is selected from +the public run ID by the same arithmetic local discovery uses, and it holds the +run record, the filtered journal, the immutable Workspace roots and their +content-addressed bytes, the Agent-session mappings, the retained delivery state +and executor ownership. + +**Executor ownership is the lifetime of one authenticated connection.** A remote +start or resume opens that connection, and the acquisition lives and dies with +it: the owner registers the exact acquisition on admission and invalidates it on +close, which is the staleness proof that replaces an operating system releasing +a file lock. It is not a time lease, and closing it rolls back nothing already +committed. Every request that advances the run — start, resume, stale recovery, +document execution, Workspace mutation, provider attachment, native execution +against a materialized root, lifecycle transition, accepted-outcome publication +and terminal settlement — validates that exact live acquisition and the expected +Workspace root inside its own mutating transaction. + +**Delivery and inspection are the exceptions, and they stay exceptions.** A +delivery retains one externally supplied value for one exact retained subject +under §3.5's rules — no acquisition, no execution, no attachment, no journal +event, no status change — whether the subject is a suspension request, a +terminal decision, or a machine wait whose notification carries no value at all. +Inspection is read-only and returns immutable snapshots. A +remote host that let either one advance a lifecycle would have built a second +state machine beside the journal. + +**A remote host admits its executor before it trusts it.** Where the connection +comes from an ephemeral CI runner, admission validates that runner's OIDC claims +— issuer, configured audience, repository ID, repository-owner ID, event name, +workflow ref and SHA, and the configured immutable workflow identity — before an +acquisition exists. Repository names are mutable and are not what is checked. + +**An ephemeral runner recovers like any interrupted executor.** It materializes +one selected retained root, works in it, and submits content-addressed changes +that the owner validates and publishes atomically with the filtered journal +result, so a runner that dies mid-flight exposes only a prior or a new complete +transaction. The next acquisition performs the ordinary stale-execution recovery +of §3.3 and resumes from the exact committed run and Workspace frontier. + ### 3.9 What is shipped The lifecycle above is the whole design, including §3.7's rule that a status @@ -1603,6 +1650,305 @@ not name its own number and repository are all refused. A well-formed answer to another question is still the wrong answer. +### 7.8 Ordered merge: `Git.Merge` + +A merge is Workspace-local. It runs inside the retained checkout, against exact +commits the document names, and it publishes through the ordinary effect +transaction of §10.1 rather than through a Git host. + +```md + +``` + +The props are closed and all five are required: two exact parent commits, the +exact merge base, a `purpose` of `"synchronize"` or `"publish"`, and `as`. Which +Repository and checkout the merge runs in is decided the way §7.1 decides it, +and the repository, checkout, Workspace root and executor acquisition are +authenticated provider state rather than props. Form validation runs first: a +missing prop, an unknown prop, a `purpose` outside the enum and a missing `as` +each fail before a Repository is observed or a provider is reached. + +**The parent order is the caller's, and `purpose` authorizes it.** `purpose="synchronize"` brings a target into an implementation and is authored `[implementationHead, targetBase]`. `purpose="publish"` brings a reviewed implementation onto a target and is authored `[reviewedBase, reviewedHead]`. The component neither infers the order from the purpose nor reorders what it was given — but it does not merely record the purpose either. It validates that what was authored equals the retained authority for the purpose declared. + +The provider-authenticated merge ceiling supplies that authority. For `purpose="synchronize"` it supplies the exact current implementation head and the observed target base, and `firstParent` must equal that head while `secondParent` must equal that base. For `purpose="publish"` it supplies the exact reviewed `{ headSha, baseSha }` the retained Stage 7 decision authorized, and `firstParent` must equal `baseSha` while `secondParent` must equal `headSha`. In both cases `mergeBase` must equal the completely observed merge base for that same authenticated pair. + +A missing ceiling, a purpose the ceiling does not authorize, a swapped parent, a stale parent, a stale merge base, a revision other than the authorized one, and a ceiling belonging to another Repository or checkout each refuse before any Git mutation. Retaining the purpose without checking it would leave the one mistake this contract most needs to catch — a Stage 7 publication authored in Stage 4's order — detectable only by reading the history afterwards. + +**The request is the four authored inputs plus what the provider authenticates.** The durable request carries `firstParent`, `secondParent`, `mergeBase` and `purpose` exactly as authored, together with the Repository identity, the checkout, the pre-merge Workspace root, the executor acquisition and the merge ceiling the provider validated. Those are provider-authenticated state and never authored props, and the effect is named by the run and the expansion like every other one. + +**The result is a closed discriminated union of exactly two shapes**, keyed on `outcome`: + +```ts +type GitMergeResult = + | { + readonly outcome: "clean"; + readonly purpose: "synchronize" | "publish"; + readonly firstParent: string; + readonly secondParent: string; + readonly mergeBase: string; + readonly commit: string; + readonly workspaceRoot: string; + } + | { + readonly outcome: "conflicted"; + readonly purpose: "synchronize" | "publish"; + readonly firstParent: string; + readonly secondParent: string; + readonly mergeBase: string; + readonly workspaceRoot: string; + readonly conflicts: readonly GitMergeConflict[]; + }; + +interface GitMergeConflict { + readonly path: string; + readonly classification: + | "content" + | "add/add" + | "modify/delete" + | "delete/modify" + | "rename" + | "mode" + | "binary" + | "submodule" + | "symlink" + | "unrecognized"; + readonly stages: readonly (1 | 2 | 3)[]; + readonly base?: GitMergeSide; + readonly ours?: GitMergeSide; + readonly theirs?: GitMergeSide; +} + +interface GitMergeSide { + readonly objectId: string; + readonly mode: string; +} +``` + +Every member is required unless the declaration marks it optional, every commit and object identity is a lowercase hexadecimal object ID of the repository's own object format, `mode` is the six-digit octal Git records, and `path` is an already-normalized repository-relative POSIX path under the same rules §9.1 of [Workflow runs](./workflow-spec.md) states for a root document path. `classification` is the closed enum above and `unrecognized` is its own value rather than an absent one, because a class this build cannot name is a fact about the merge and not a gap in the record. An unknown member and an unknown classification each refuse the record rather than being ignored. + +A **clean** result names the exact merge commit and the Workspace root published with it. Its mutation, root publication and filtered result commit together, so a crash before the commit leaves the checkout, the current root and the effect history the ones the run had. A **conflicted** result names the *unchanged* pre-merge root it restored and the complete conflict set; it offers no file mutation under that evidence and adopts no partial merge state. Both are successful effects with different outcomes, not a success and a failure. + +The conflict set is complete and ordered. Entries sort by `path` in UTF-8 byte order, and two entries for one path refuse the record rather than being merged or deduplicated. `stages` is the ascending list of unmerged-index stage numbers Git retained for that path, and side presence agrees with it exactly: stage 1 is `base`, 2 is `ours`, 3 is `theirs`, a stage the index does not hold has its member absent rather than null, empty or zeroed, and a side present without its stage — or a stage without its side — refuses. A set mixing classifications is retained whole and unaltered: every conflict suspends, so there is no partial handling for a mixed set to select. Rendered conflict markers alone are never the record, because a later reader has to tell a stale conflict from the one it is looking at without reparsing text. + +**Restoration is part of the conflicted outcome, not cleanup after it.** If the pre-merge root cannot be restored, the effect publishes no conflicted result and no new root: it is an infrastructure failure that activates the durable fail-stop fence, because a conflicted result naming a root the Workspace is not actually at would be evidence of a state nothing holds. + +Cancellation between the merge and the commit rolls the outer transaction back and publishes no completion at all. A completed record of either outcome replays without running Git. + +A merge never contacts a Git host, never pushes, and never rewrites a published identity. Rebase, force, force-with-lease and reset-based replacement are absent from this component and from every other one in this specification. + +### 7.9 Publishing a reviewed merge: `Git.PublishTarget` + +Publishing to a protected target branch is a Git-host effect and a different +question from advancing a branch this run owns. + +```md + +``` + +All four props are required and the set is closed. The remote, the target ref, +the credential and the non-force policy are host-owned: they are not props, and +no authored value widens them. Form validation runs before the host's ceiling is +read and before any credential exists. + +**The request names the target the host chose and the commits the document did.** Its natural key is the target identity alone — the retained Repository, the configured remote and the configured target ref — because one ref has one publication at a time whoever is asking: + +```ts +interface GitPublishTargetRequest { + readonly kind: "git-publish-target"; + readonly target: GitPublishTarget; + readonly expectedRemoteCommit: string; + readonly sourceCommit: string; + readonly reviewedHead: string; +} + +interface GitPublishTarget { + readonly repository: string; + readonly remote: string; + readonly ref: string; +} +``` + +`repository` is the Workspace-local Repository name, `remote` is the configured remote's name, and `ref` is the fully qualified destination ref. The credential, the locator behind the remote and the non-force policy are provider closure state and appear in neither the request nor the result. The three commits are lowercase hexadecimal object IDs of the repository's object format. + +**It is a compare-and-swap**, and its five observed pre-states are exhaustive: + +| Observation | Decision | +| --- | --- | +| the target equals `expectedRemoteCommit` | perform one non-force update, once | +| the target equals `sourceCommit` | adopt; nothing is performed | +| the target equals some third commit | conflict; refuse without mutating | +| the observation did not complete | incomplete observation; refuse as itself, adopt nothing, perform nothing | +| the observation cannot be decided, or the host is temporarily unreachable | permanent ambiguity and temporary unavailability respectively; each refuses as itself and never performs | + +A race that moves the target before or during publication therefore cannot publish over it, and an interrupted attempt is reobserved rather than repeated: a target that now equals `sourceCommit` is the adoption, and one that does not is not silently published over. + +**The result is one closed record**, and the binding is that record: + +```ts +interface GitPublishTargetResult { + readonly target: GitPublishTarget; + readonly expectedRemoteCommit: string; + readonly reviewedHead: string; + readonly sourceCommit: string; + readonly observedCommit: string; + readonly decision: "performed" | "adopted"; +} +``` + +`observedCommit` is what the target held when this attempt looked, so the record says what the publication moved from or found already done; `decision` is the closed pair above and no third value exists. `reviewedHead` is carried so the record says what the publication was authorized against, which is what lets an exact-revision review be invalidated by a target that moved. It is stable evidence of what the effect settled on, not a live branch snapshot. Every member is required, and an unknown member or an unknown `decision` refuses the record. + +Cancellation tears the provider call down and publishes no completion. A completed record replays without contacting a Git host. + +**Three operations stay distinct.** `Git.Push` (§7.4) advances a branch this run published, from an ancestry relation proved inside the authenticated object source. `Git.PublishTarget` updates a ref it does not own, from an exact expected pre-state. A Git host's own pull-request merge endpoint is neither, and this specification defines no component for one: a squash or a rebase performed by the host would publish a commit no reviewer saw, under parents the review never named. Whether the host has *noticed* that its pull request is now merged is a fourth question, and §7.11 owns it. + +### 7.10 Pull-request comments, readiness and closure + +Three more Git-host effects act on a pull request a canonical URL names. Each requires `as`, validates its form before any provider, ceiling or credential is reached, and reconciles under §10.2: observe, adopt a compatible completion, perform a proven absence once, refuse conflict, permanent ambiguity, incomplete observation and temporary unavailability. Every identity below is a canonical URL or a lowercase hexadecimal object ID; no credential, endpoint, raw payload, cursor or host path appears in any request, natural key or result. + +```md + +The Architect accepted {revision.headSha} against {revision.baseSha}. + + + + +``` + +#### `PullRequest.Comment` + +The component is paired, takes one required `url` and one required `as`, and its rendered content is the body verbatim. `url` is the **canonical pull-request URL** — the normalized single spelling of one pull request, on the terms §10.3 already states for a canonical target URL — and the durable request is exactly that URL, the engine-derived effect identity and the rendered body: + +```ts +interface PullRequestCommentRequest { + readonly kind: "pull-request-comment"; + readonly subject: string; + readonly effect: string; + readonly body: string; +} + +interface PullRequestCommentResult { + readonly subject: string; + readonly url: string; + readonly decision: "performed" | "adopted"; +} +``` + +`subject` is the canonical pull-request URL, `effect` is the engine-derived effect identity of [Workflow runs](./workflow-spec.md) §8, and `url` is the canonical URL of the comment this effect settled on. The binding is `{ url }`: that comment's own URL, which is the only fact the effect produces — the subject was already in hand at the call site. + +**The natural key is `subject` plus `effect`, and the body is never part of it.** A Git host issues no client-supplied idempotency key for a comment, so the effect identity has to be observable on the host for an interrupted creation to be found again. A comment provider therefore has to support one **stable opaque correlation marker**: a value it can write with a comment, preserve unchanged, and query completely. A provider that cannot do all three refuses the effect before its first mutation, the way a plain Git server refuses pull requests today — there is no fallback that searches prose. + +The marker is provider transport metadata. The **authored logical body is preserved byte for byte as the authored portion of the projection**, and the correlation representation lives outside that logical body rather than inside it; GitHub's adapter encodes it as a non-rendered HTML comment in its provider payload, so the payload the provider sends is not claimed to equal the authored bytes. It is not authored prose, not a credential and not lifecycle authority — publishing an engine-derived effect identity as an opaque non-secret correlation value is what it is for. The public binding and every replay expose the authored body and the provider's comment identity, never the transport encoding. + +**Observation is judged against the attempt state, not against the host alone.** Before any provider mutation, the durable effect retains that this exact request is prepared and unattempted; a live attempt is what moves it past that. What a complete observation means then depends on which side of that line the effect is on: + +| Attempt state and observation | Decision | +| --- | --- | +| unattempted, and no marker | proven absence; create once | +| unattempted or attempted, and exactly one marker | compatible completion; adopt with nothing performed | +| any state, and more than one marker | permanent ambiguity; refuse | +| **attempted with no committed local completion, and no marker** | **permanent ambiguity; refuse** | +| an observation that did not complete | incomplete observation; refuse, adopt nothing, perform nothing | +| the host is temporarily unreachable | temporary unavailability; refuse as itself | + +The fourth row is the one that matters. A marker that is absent *after* an attempt does not prove the comment was never created — it equally describes a person having edited or deleted it inside the interrupted window — so treating that as absence is how a duplicate gets published. Refusing it as ambiguity costs a stall and buys the guarantee. Once a local completion has committed, the marker no longer decides anything: a completed replay reads its own record and contacts no provider, so removing the marker afterwards changes nothing. + +An incomplete observation is never absence. A comment list the adapter could not finish reading is a search that did not answer, and an unfinished search reported as absence is the same duplicate by another route. + +#### `PullRequest.Ready` and `PullRequest.Close` + +Both are self-closing, take one required `url` and one required `as`, and are keyed by that exact canonical pull-request URL — one readiness and one closure per pull request, so neither carries an effect identity in its key. Their bindings are the closed records below, and each durable result is its binding plus the observation the attempt made: + +```ts +interface PullRequestReadyBinding { + readonly url: string; + readonly state: "open"; + readonly draft: false; +} + +interface PullRequestCloseBinding { + readonly url: string; + readonly state: "closed"; + readonly merged: false; +} + +interface PullRequestReadyResult extends PullRequestReadyBinding { + readonly observed: PullRequestObservation; + readonly decision: "performed" | "adopted"; +} + +interface PullRequestCloseResult extends PullRequestCloseBinding { + readonly observed: PullRequestObservation; + readonly decision: "performed" | "adopted"; +} + +interface PullRequestObservation { + readonly state: "open" | "closed"; + readonly draft: boolean; + readonly merged: boolean; +} +``` + +`state`, `draft` and `merged` are literal in each binding rather than observed values copied through, because a binding that could say `draft: true` would be a component reporting that it did not do what it is for. + +`PullRequest.Ready` performs once from an observed `{ state: "open", draft: true, merged: false }`, adopts an observed `{ state: "open", draft: false, merged: false }` with nothing performed, and refuses every other observation as a conflict — a merged or closed pull request among them, since readiness is not a thing to restore. `PullRequest.Close` performs once from an observed `{ state: "open", merged: false }` at either draft state, adopts an observed `{ state: "closed", merged: false }`, and conflicts with an observed `merged: true`, which is a completion of a different kind that closing must not overwrite. A pull request belonging to another repository, or one the URL names but the host does not hold, is a conflict for both. An incomplete observation, a permanent ambiguity and a temporary unavailability each refuse as themselves and perform nothing. + +Neither reopens, merges, comments on or pushes anything. Cancellation tears the provider call down and publishes no completion; a completed record of any of the three replays without contacting a Git host. Which of them a document may invoke, and what authorizes the invocation, is authored control flow above them. + +### 7.11 Observing that a pull request merged: `PullRequest.Merged` + +Publishing a merge commit to a target ref and a Git host recording that pull request as merged are two different facts, and the second one is not implied by the first. A host observes its own ref moving and closes the pull request on its own schedule, so a run that needs the merged state in its history has to observe it — and has to observe it as its own retained step rather than as a side effect of something else. + +```md + +``` + +The component is self-closing, its three props are required and the set is closed, and it is a Git-host effect of its own. It is not `PullRequest.Ready`, not `PullRequest.Close`, not a pull-request upsert and not `Git.PublishTarget`: overloading any of them would make one record answer two questions, and the two can disagree. + +**It mutates nothing.** It is a reconciled observation: adoption is its only completion, and there is no `performed` decision for it to reach. What it reconciles is *when* the fact becomes true, because a host that has not yet noticed the ref move is not a host that refused. + +```ts +interface PullRequestMergedRequest { + readonly kind: "pull-request-merged"; + readonly subject: string; + readonly expectedMergeCommit: string; +} + +interface PullRequestMergedResult { + readonly subject: string; + readonly state: "closed"; + readonly merged: true; + readonly mergeCommit: string; + readonly decision: "adopted"; +} +``` + +`subject` is the canonical pull-request URL and is the whole natural key. The binding is the result record. + +The observation is complete or it is nothing, and its five outcomes are distinct: + +| Observation | Decision | +| --- | --- | +| `merged: true` at exactly `expectedMergeCommit` | compatible completion; adopt | +| `merged: true` at another commit | conflict; the target carries somebody else's merge, and the exact-revision reviews invalidate rather than this step succeeding | +| `state: "open"`, `merged: false` | temporary unavailability; the host has not yet recognized the merge, and a later attempt starts again at observation | +| `state: "closed"`, `merged: false` | conflict; a pull request somebody closed by hand is a state incompatible with the merged path, not lag | +| an incomplete read, or an undecidable one | incomplete observation and permanent ambiguity respectively, each refusing as itself | + +The third and fourth rows are deliberately not one row. Still open after a publication is eventual consistency and is worth waiting for; closed unmerged is a person having intervened, and waiting for that to resolve itself would wait forever. + +Cancellation publishes no completion, and a completed record replays without contacting a Git host. What the run does while the third row persists — a bounded host-configured retry, then a machine wait that ends on reobservation rather than on a delivered answer — and where this step sits in the terminal sequence belong to [the software factory](./github-actions-software-factory-spec.md) §10.3 and §10.4. + ## 8. Agents inspect; XMD mutates ### 8.1 No directory registration @@ -1654,6 +2000,15 @@ This is what the host asks for and what it refuses. It is not a claim that every ACP adapter exposes no tool when asked for none; that portable proof is tracked by #496 and does not widen this ceiling. +Constructs added for a trusted host do not reach the Agent either, and they do +not reach it for a different reason than the tool set: an Agent never expands a +document. Merging, publishing a target, observing that a pull request merged, +commenting, changing draft state, closing an issue or a pull request, moving a +Project item and running evidence +are authored XMD the trusted host expands under its own acquisition. What an +Agent may return is text, and a fragment it returns is admitted only against the +tables §8.4 states — which name none of them. + `Session.Launch` is unsupported by this profile. The trusted workflow host states both ordinary-run native capability sets empty and installs no native foreground launcher, so a launch is refused before provider preparation, @@ -1973,8 +2328,16 @@ The write table is authority, not prompting guidance. Generated source cannot grant itself Push, PullRequest, an issue upsert, a repository, a process, an eval or exec block, a native command, a credential or an arbitrary network write merely by naming a component; the table excludes local Git even though those -effects are also Workspace-local. Trusted reusable Markdown components may be -admitted explicitly; generated XMD admits none of them. +effects are also Workspace-local. The constructs §§7.8-7.10, §10.5 and §10.6 add +change nothing about that: `Git.Merge`, `Git.PublishTarget`, +`PullRequest.Comment`, `PullRequest.Ready`, `PullRequest.Close`, +`PullRequest.Merged`, `Issue.Comment`, `Issue.Close`, `Project.Status` and +`Evidence.Run` appear in no +table this specification states, so a fragment naming one is refused in the +preflight before any generated effect, exactly as `` is. Adding a +construct to a table is a host act, and a factory host adds none of them. +Trusted reusable Markdown components may be admitted explicitly; generated XMD +admits none of them. **Approval is authored, and it is ordinary.** `` neither prompts nor approves. A workflow that requires approval reaches a branch, an elicitation, a @@ -2288,6 +2651,16 @@ performance retains is that observation, which is how the record says what the external resource held before this attempt moved it. Temporary unavailability is neither absence nor conflict, and never authorizes a mutation: a later explicit attempt starts again at observation. +**Whether absence can be proved at all depends on where the completion is visible**, and effects divide into two kinds. Most of them mutate a subject that already exists and whose own state answers the question: a Push reads the destination ref, a numbered pull-request update reads that pull request, `PullRequest.Ready` and `PullRequest.Close` read its state, `Issue.Close` reads the issue, `Project.Status` reads the field's current option. For those, a complete observation of the subject is decisive whether or not this effect has attempted anything, because what the observation reports is the resource itself. + +The other kind **creates a new object the host names**. Creating one is safely reconcilable through either of two mechanisms, and which one an effect has decides whether attempt state takes part. An effect the provider gives a native client idempotency or correlation key — the key an Issue upsert derives from the canonical target and this run's own effect identity is one — reconciles on that key under its already-stated natural-key and complete-observation contract, and carries no attempt state: creating an object is not by itself what makes an effect attempt-stateful. A pull-request upsert is the same, reconciling on its explicit head-and-base or numbered identity. + +A comment is the one construct here with neither. Nothing pre-exists to read, and the host issues no client-supplied idempotency key, so the completion is observable only through a correlation value the effect itself wrote. Absence then means "that value is not there", which is a different claim before and after a mutation has been attempted, and the effect therefore retains its **attempt state**: the exact request is retained as prepared and unattempted before any provider mutation, and a live attempt moves it past that. + +For such an effect the decision above narrows in exactly one place. Unattempted with nothing found is proven absence and performs once. **Attempted with no committed local completion and nothing found is permanent ambiguity, not absence** — the correlation value is equally missing because the object was never created and because somebody removed it inside the interrupted window, and performing on that reading is how a duplicate gets published. Exactly one correlation match is compatible completion in either state; more than one is permanent ambiguity in either state; and an incomplete observation stays incomplete rather than becoming absence, since an unfinished search reported as absence is the same duplicate by another route. Once a local completion has committed, none of it decides anything further: replay reads the record and contacts no provider. + +A provider that cannot write, preserve and completely query such a correlation value cannot supply this kind of effect at all, and refuses it from observation before any mutation — the same refusal a plain Git server gives for pull requests. A future host-named create effect that has neither a provider-native client key nor a preservable marker refuses on the same terms. That refusal is the contract rather than a gap in it: an effect that can prove neither absence nor completion has no safe way to run once. + **The record.** A decision publishes one journal result holding the request, the normalized pre-state, the normalized observations, the decision — `adopted` or `performed` — and the normalized result. Replaying it contacts no provider and @@ -2653,6 +3026,71 @@ used. With no configuration there is no Issue provider, so every request reaches `NoIssueProvider` — absence of configuration is fail-closed, never an open default. +#### Commenting on and closing an issue + +Two more Issue-provider effects act on an issue a canonical URL names. Both +require `as`, both validate their form before any provider, ceiling or +credential is reached, and both reconcile the way an upsert does — observe, +adopt a compatible completion, perform a proven absence once, refuse conflict +and ambiguity — inside the provider rather than through the Git host's shared +state machine. + +```md + +The Planner accepted the plan at {plan.revision}. + + + +``` + +`Issue.Comment` is paired, takes one required `url` and one required `as`, and its rendered content is the body verbatim. Its records mirror the pull-request comment of §7.10 exactly, under this boundary instead of the Git host's: + +```ts +interface IssueCommentRequest { + readonly kind: "issue-comment"; + readonly subject: string; + readonly effect: string; + readonly body: string; +} + +interface IssueCommentResult { + readonly subject: string; + readonly url: string; + readonly decision: "performed" | "adopted"; +} +``` + +`subject` is the canonical issue URL — the normalized single spelling this section already requires of a target — `effect` is the engine-derived effect identity, and `url` is the canonical URL of the comment the effect settled on. The binding is `{ url }`, that comment's own URL. + +**The natural key is `subject` plus `effect`, and the body is never part of it.** An Issue provider carries the same requirement §7.10 states for a pull-request comment: it supports one stable opaque correlation marker it can write, preserve and completely query, or it refuses the effect before its first mutation. The authored logical body is preserved byte for byte as the authored portion of the projection, and the correlation representation lives outside it. The attempt-state table of §7.10 governs the decision unchanged, including its fourth row — an absent marker after an attempted-but-uncommitted creation is permanent ambiguity rather than proven absence. + +`Issue.Close` is self-closing and takes one required `url`, one required `reason` from the closed enum `"completed" | "not_planned"`, and one required `as`. Its natural key is the canonical issue URL alone: + +```ts +interface IssueCloseRequest { + readonly kind: "issue-close"; + readonly subject: string; + readonly reason: "completed" | "not_planned"; +} + +interface IssueCloseBinding { + readonly url: string; + readonly state: "closed"; + readonly reason: "completed" | "not_planned"; +} + +interface IssueCloseResult extends IssueCloseBinding { + readonly observed: { readonly state: "open" | "closed"; readonly reason?: "completed" | "not_planned" }; + readonly decision: "performed" | "adopted"; +} +``` + +`state` is literal in the binding: a component for closing an issue does not report that the issue is open. The observed `reason` is absent rather than null when the issue is open or when the host records no reason for a closure it holds. + +The reason is part of what the effect means rather than a label on it, so a host that retained one terminal intent refuses a close naming the other. An observed open issue is performed once. An observed issue closed with the same reason is adopted with nothing performed. An observed issue closed with the other reason is a conflict, and so is one closed with no reason the host will state, because adopting it would let a `not_planned` closure stand as a `completed` one. An incomplete observation, a permanent ambiguity and a temporary unavailability each refuse as themselves. + +Neither reopens an issue, and neither derives authority from what it observes. Cancellation publishes no completion, and a completed record of either replays without contacting a provider. + ### 10.4 Worker Shell Worker Shell means Cloudflare's Workspace Shell capability implemented by @@ -2682,6 +3120,154 @@ Network is denied unless explicitly authorized. A committed result restores without starting a Worker; an effect interrupted before commit executes again against its pre-effect Workspace root. +### 10.5 Trusted evidence execution + +Some evidence can only be produced by running the project's own commands with +the project's own tools. That is not Worker Shell, and it is not a capability a +document may reach for by itself. + +```md + +``` + +`commands` is an authored **structured argv list**: an ordered, non-empty list whose every member is a non-empty list of strings. There is no interpreter, no quoting layer and no string to mis-split, which is what makes the record of what ran the same thing as what ran. `as` is required, the prop set is closed, and the form is validated before the host's ceilings are read — an empty list, an empty vector, a member that is not a list of strings, an unknown prop and a missing `as` each fail before any child exists. + +**The pipeline is fail-fast.** Commands run in authored order, and the first command that does not complete successfully is the last one that runs. A command is successful only when it exits normally with status `0`; a non-zero exit, a signal termination and a timeout are each retained as the final row and start no successor. A plan's evidence list is a pipeline — build, then test, then lint — and continuing past a failed build produces later rows evaluated against missing or stale prerequisites, which is evidence that is confidently wrong rather than absent. + +Breadth belongs inside one command whose own contract runs a corpus to completion, such as this repository's runtime-test shards, or in several separately authored `Evidence.Run` elements where the plan says the groups are independent. That a shard runs its files to the end says nothing about whether one arbitrary pipeline should continue after a failure. + +The host owns everything else. Which executables may run, what environment they see, the logical working root, how long each command and the whole list may take, how much output is retained, and what happens to a process tree are host ceilings rather than props. No host path, shell string, ambient environment or command authority enters an authored prop. The commands run against one exact retained Workspace root the host materialized, on the trusted runner where the native toolchain lives — never inside the run's durable owner, which has no toolchain and must not acquire one. + +**The result is the executed prefix, not one row per authored command:** + +```ts +interface EvidenceRunResult { + readonly completion: "passed" | "failed"; + readonly authoredCommands: number; + readonly executed: readonly EvidenceCommandResult[]; + readonly runTimeout?: EvidenceRunTimeout; +} + +interface EvidenceCommandResult { + readonly argv: readonly string[]; + readonly outcome: "exited" | "signalled" | "timeout"; + readonly status?: number; + readonly signal?: string; + readonly limit?: "command" | "run"; + readonly stdout: EvidenceChannel; + readonly stderr: EvidenceChannel; +} + +interface EvidenceRunTimeout { + readonly limit: "run"; + readonly notStartedAt: number; +} + +interface EvidenceChannel { + readonly text: string; + readonly retainedBytes: number; + readonly producedBytes: number; + readonly truncated: boolean; +} +``` + +`completion` is `"passed"` exactly when `executed` holds `authoredCommands` rows and every one of them is an `"exited"` row with `status: 0`; it is `"failed"` in every other case. `authoredCommands` is retained beside `executed` so a reader can tell a complete pass from a deliberately stopped prefix without knowing the authored list, which is the whole reason a prefix is safe to publish. + +`executed` holds the commands that ran, in authored order, and `argv` repeats the vector that ran. `status` is present exactly when `outcome` is `"exited"` and is the numeric exit status; `signal` is present exactly when `outcome` is `"signalled"`; `limit` is present exactly when `outcome` is `"timeout"` and names which ceiling fired. An unknown member, an unknown `outcome`, an unknown `limit`, and any of those three members beside the wrong outcome each refuse the record. + +`runTimeout` is present exactly when the whole-run ceiling expired **between** commands, with no child running. `notStartedAt` is the zero-based index into the authored list of the command that did not start. It is a member of its own rather than a row in `executed`, because a row would have to invent an argv that never ran and a channel that captured nothing. + +Both channels are retained separately and neither is folded into the other: `text` is the retained UTF-8 prefix, `retainedBytes` is its length in bytes, `producedBytes` is what the child actually produced, and `truncated` is `producedBytes > retainedBytes`. Truncation is stated rather than inferred from a length, so a reader never has to guess whether a command was quiet or cut off. + +**Two ceilings bound the work, and both are host-owned.** A per-command ceiling stops one command monopolizing the run; a whole-`Evidence.Run` ceiling bounds total wall-clock cost across the list, including process startup, output draining and teardown. Neither is an authored prop. Before starting each command the host requires positive remaining whole-run time, and while a command runs its effective deadline is the earlier of its own deadline and the whole-run deadline — so a timeout row's `limit` says which of the two fired. A whole-run ceiling that expires between commands ends the result with `completion: "failed"` and the `runTimeout` record above, and starts no successor. + +**A timeout is an ordinary unsuccessful outcome, not an infrastructure failure.** It records that the host enforced its ceiling successfully: it terminated and reaped the process tree and captured bounded channels. It becomes the last row and stops the pipeline. If termination, output draining or reaping *fails* while the host is enforcing that ceiling, the case is an infrastructure failure below rather than a timeout result — the difference is whether the host is reporting what it did or reporting that it could not. + +**Which cases bind, which fail, and which commit nothing.** + +| Case | Outcome | +| --- | --- | +| a command exits with status `0` | an ordinary row; the next command starts | +| a command exits non-zero, is terminated by a signal, or hits either duration ceiling | an ordinary final row; `completion` is `"failed"` and no successor starts | +| the whole-run ceiling expires between commands | `completion: "failed"` with a `runTimeout` record and no successor | +| the executable or environment ceiling refuses a command, or the child cannot be created | **launch failure**: the effect fails and produces no `EvidenceRunResult` | +| the host cannot read a channel it promised to bound | **output-pump failure**: the effect fails and produces no `EvidenceRunResult` | +| the host cannot terminate or reap a child or its process tree | **teardown failure**: the effect fails and produces no `EvidenceRunResult` | +| the effect is cancelled | the complete process tree is terminated and no completion and no failure record is committed | + +A non-zero status is evidence, not an infrastructure failure — it is the answer the evidence exists to obtain. An infrastructure failure is the host being unable to say what happened, which is why it publishes no result: half an answer read as a whole one is worse than no answer. + +**Precedence is fixed.** Cancellation wins over every other outcome, terminates the complete process tree, and commits neither a completion nor a failure record. Otherwise the first infrastructure failure is authoritative, and a teardown failure that follows it is retained as secondary evidence rather than replacing it. With no earlier infrastructure failure, a teardown failure is itself authoritative even when every command produced an observed exit: a host that cannot prove its process ownership settled cannot publish a successful binding. This is the rule the workflow lifecycle already applies to settlement, where teardown is part of the evidence rather than work performed after the outcome. + +**No successful binding is not the same as no retained evidence.** A failed effect retains bounded diagnostic evidence on its `Error`: the safely collected executed-command prefix, the separate bounded stdout and stderr channels, the primary infrastructure-failure category, and the secondary teardown category when there is one. That is filtered diagnostic failure evidence and it is deliberately not an `EvidenceRunResult` — nothing binds it, and no document reads it as a pass or a fail of the commands. Replaying a failed effect starts no process. Cancellation retains neither, because no completion won. + +The operation returns Effection's `Result` at the implementation boundary and puts its failure data on an `Error`, like every other outcome in this repository; the authored binding exists only for a successful `EvidenceRunResult`. There is no local success-or-failure union. + +`Evidence.Run` is not Worker Shell (§10.4) and does not replace it: Worker Shell is a contained interpreter over the Workspace filesystem, while this is native execution of an authored list under a trusted host's ceiling. It is absent from the workflow Agent's capabilities (§8.3) and from every generated-XMD table (§8.4), so neither an Agent nor a fragment it wrote can reach it. + +### 10.6 Project effects + +A **Project provider** is an external service that owns project boards and the +status of the items on them. GitHub Projects V2 is one adapter. + +This is a boundary of its own for the reason §10.3 gives about issues: a project +board need own neither a Git repository nor an issue collection, so a Project +status cannot truthfully execute or persist as a `git_host_effect` or as an +`issue_effect`. `Project.Status` therefore reaches its own contextual operation +and journals its own durable effect type, and it reuses the shape of the +reconciliation rather than the Git host's state machine. + +```md + +``` + +The component is self-closing, its four props are required and the set is closed, and it binds the normalized `{ item, field, option }`. Its natural key is the exact item plus the exact field — one item has one value of one field, whoever is asking — and its request and result are these: + +```ts +interface ProjectStatusRequest { + readonly kind: "project-status"; + readonly item: string; + readonly field: string; + readonly option: string; +} + +interface ProjectStatusBinding { + readonly item: string; + readonly field: string; + readonly option: string; +} + +interface ProjectStatusResult extends ProjectStatusBinding { + readonly observedOption?: string; + readonly decision: "performed" | "adopted"; +} +``` + +`item`, `field` and `option` are the provider's own opaque identities, compared byte for byte and never normalized, decoded or repaired — they are provider identities on the same terms an issue node ID is. `option` in the binding is the requested one, which after a successful effect is the one the item holds. `observedOption` is what the field held when this attempt looked, and it is absent rather than null when the field held no option at all. An unknown member and an unknown `decision` refuse the record. + +Its compatible pre-state is the option that item currently holds. An item already at the requested option is adopted with nothing performed; an item at another option the host's ceiling allows is performed once; an item at an option outside that ceiling is a conflict, because moving it would be publishing through a status this factory does not own. An unreadable board, an unavailable field, an ambiguous item and a partial permission read are **unavailable** rather than absent — reading an unreadable board as an empty one is how an unauthorized item would be moved — and a temporary unreachability refuses as itself. Cancellation publishes no completion, and a completed record replays without contacting a provider. + +Which project, item, field and options may be reached at all is a host ceiling +installed beside the credential. An authored prop selects within that ceiling +and can never widen it, which is the same rule §10.3's tracker follows. + +**A board is a projection.** The status it shows is published from the run's own +journaled lifecycle, never read as it. A board ahead of the journal is drift the +next execution reconciles, and it is not evidence that a transition happened. + +### 10.7 What never crosses these boundaries + +Every effect in §10.2, §10.3, §10.5 and §10.6 reaches its provider the same way, +and the same things stay out of the record. Credentials are not inputs: an +application private key, a webhook secret, an issued installation token, an +OIDC verification configuration, a provider endpoint, a raw provider payload, a +pagination cursor and a host path stay in the selected provider's own closure. +None of them enters a component prop, context composition data, a durable +request, a natural key, a retained result, a comment body, document output or a +diagnostic. What a durable record holds is the normalized request, the natural +key, the observed pre-state and the normalized result — enough to reconcile the +effect, and nothing that would make the journal a place to read a secret from. + ## 11. History forks Normal resume always uses the same immutable definition, normalized props, @@ -3146,6 +3732,65 @@ There is no public `Git.Fetch` here. The shipped Git scope is Repository clone and its remote reads, plus `Git.Push` observation and mutation; a future public fetch operation requires its own language and durability contract. +### 13.2 Remote topology + +A remote host runs the same contracts with the durable state and the native +tools in two different places. + +One SQLite-backed Cloudflare Durable Object per run is the durable owner, +selected from the public run ID by the same arithmetic §9.3 of the workflow +specification uses. It holds the run record and filtered journal, the immutable +Workspace roots and their content-addressed bytes, the Repository and Worktree +records, the Agent-session mappings, the retained delivery state and the +authenticated intake records, and it owns executor admission. It is a +runtime-named adapter beside the Deno one: shared modules reach it through the +same contextual storage, lifecycle and Workspace APIs, detect no runtime, and +import nothing Cloudflare-specific. + +**The host assembly contract does not change.** `WorkflowHost` keeps its four methods — `useRunHost()`, `useLifecycle()`, `useDelivery()` and `attach()` — and the Cloudflare adapter is one more implementation of them beside the Deno one. Starting, looking up, executing, delivering into and inspecting a run are lifecycle operations reached *through* that boundary, exactly as they are locally; they are not replacement method names, and no fifth method appears. A remote host receives no transitions type of its own either. What a remote adapter changes is where each of those four reaches, not what the shared CLI asks for. + +**The transition types those methods speak are provider-neutral.** `WorkflowExecutionTransitions`, `WorkflowBeginRequest`, `WorkflowExecutionBegun`, `WorkflowForkRequest`, `WorkflowForkSelection` and `WorkflowRunCreation` describe what any host's lifecycle does, not what one adapter retains, and they are already defined in the provider-neutral lifecycle module. They become package-root public types, and the Deno entrypoint may keep re-exporting them for source compatibility without owning their meaning. Runtime-specific implementations and retained encodings — SQLite, DOFS, run-id hashing, filesystem paths — stay behind their runtime-named entrypoints, which is the boundary that rationale was always about. That export move is the first implementation story's work; #710 settles that the types are neutral, and performs no production change. + +Executor ownership is one authenticated WebSocket connection whose lifetime is the acquisition. The owner registers the exact acquisition on admission and invalidates it on close; there is no duration, expiry, renewal, heartbeat, generation record or liveness poll, and a close rolls back nothing already committed. Every mutating transaction validates that exact acquisition and the expected Workspace root together. + +Native Git, evidence processes and Agent clients run on the ephemeral runner and nowhere else. The runner materializes one selected retained root, works in it, and submits content-addressed changes; the owner validates acquisition, root and content and then atomically publishes the new root with the filtered journal result. That is §10.1's effect transaction with the mutation performed where the tools are and the publication performed where the authority is, so a runner crash exposes only a prior or a new complete transaction and the next acquisition resumes from the exact committed frontier. + +#### The transport between them is private to one release + +The runner client and the durable owner ship as one software-factory release identity, so the messages between them are not a compatibility boundary and are not a public contract. They are journaled by neither side, exported by neither, authored by nobody, and never expected to interoperate across independently versioned builds. Their decomposition is implementation detail. + +What replaces a wire contract is a release-identity check at admission. Connection admission validates an exact immutable client and server build or protocol fingerprint supplied by trusted deployment configuration, and a mismatch refuses closed — before request parsing, before acquisition, before any state access. There is no cross-version adaptation, no downgrade and no compatibility promise, because two builds that disagree about what was committed is the failure this check exists to prevent rather than to survive. + +Privacy of the transport is not privacy of the authority. These constraints are public and exact however the messages are decomposed: + +- one authenticated connection is one executor acquisition; +- every execution mutation validates that acquisition and the expected Workspace root inside the owner's transaction; +- the owner alone parses and adopts requests and alone opens and commits transactions; +- content-addressed data is validated before publication; +- delivery and inspection use separate authenticated paths that take no acquisition; +- credentials and raw transport payloads are never durable public records; +- a runner-to-owner release-identity mismatch refuses closed; and +- a completed replay may read its durable owner but attaches no execution or external-effect provider. + +#### Which side owns what + +| Concern | Owner | +| --- | --- | +| connection admission, including OIDC claim validation and acquisition registration | the durable owner | +| parsing every request | the durable owner; the runner parses only responses | +| opening, committing and rolling back transactions | the durable owner | +| content-addressed transfer | the runner produces content and names it; the owner validates and stores it | +| attaching Workspace, Agent, process, Git, Git-host, Issue and Project providers | the runner | +| cancellation | whichever side owns the scope being cancelled: the runner cancels its own document execution, and the owner cancels nothing on its behalf | +| closing the connection | either side; the owner invalidates the acquisition when it closes | +| stale-execution recovery | the durable owner, at the next acquisition | + +Delivery and inspection reach the owner without an acquisition, under §3.8, on authenticated paths of their own. + +#### What completed replay does and does not reach + +A completed run replays there as it does locally: it attaches no Workspace, Agent, process, Git, Git-host, Issue, Project or credential provider and performs no effect a second time. It does reach the run's durable owner, because that is where the retained result is; an ephemeral client holds nothing of its own to replay from. Reading retained completion from the owner that holds it is not attaching a provider, and the distinction is the whole point of the rule: what a completed replay must not do is contact an *external* service or repeat an effect, not refrain from reading its own history. + ## 14. Contract inventory | Contract | Status at this design revision | @@ -3174,5 +3819,15 @@ fetch operation requires its own language and durability contract. | generated-XMD mutation-proposal admission | built by #369 and #567: the standard Deno profile's write table is core's paired `File:write`, this package's lexical `Dir` and core's self-closing `File.Delete`, in that retained order and followed by any host extension; admitted mutations run as the ordinary components they are through the run's effect transactions, a generated deletion publishing the same `workspace_file` effect an authored one does; the evaluator adds no receipt or result entry, so a write-only fragment still binds `{ observations: [], output: "" }`; and approval is authored control flow before the element. Local Git, Git-host, issue, process, execution, credential and external-write effects are outside the class | | Deno-local DOFS persistence | POC proven by #349 / PR #350 | | scoped Deno Worker Shell | containment proven by #351 / PR #353 and transactions by #357 / PR #362; production integration unbuilt | +| `Git.Merge` ordered Workspace-local merge (§7.8) | specified by #710; implementation unbuilt | +| `Git.PublishTarget` compare-and-swap target publication (§7.9) | specified by #710; implementation unbuilt | +| `PullRequest.Comment`, `PullRequest.Ready`, `PullRequest.Close` (§7.10) | specified by #710; implementation unbuilt | +| `PullRequest.Merged` reconciled merged observation (§7.11) | specified by #710; implementation unbuilt | +| `Issue.Comment` and `Issue.Close` (§10.3) | specified by #710; implementation unbuilt | +| `Evidence.Run` trusted native evidence execution (§10.5) | specified by #710; implementation unbuilt | +| `Project.Status` and the Project-provider boundary (§10.6) | specified by #710; implementation unbuilt | +| factory protocol records consumed by these effects | specified by #710 and owned normatively by [the software factory](./github-actions-software-factory-spec.md) §11.2, which this specification links to rather than duplicating: `Git.Merge`'s publish ceiling reads the Stage 7 decision, `PullRequest.Merged`'s wait is one of those records, and `Project.Status` projects a stage through the configured stage-to-option table | +| remote lifecycle host, executor connection, versioned runner transport and remote topology (§3.8, §13.2) — the existing four-method `WorkflowHost` boundary, with a Cloudflare implementation beside the Deno one | specified by #710; implementation unbuilt | +| terminal-decision delivery on the delivery plane (§3.8) | specified by #710; implementation unbuilt | | Worker JavaScript | deferred | | bundled workerd local host | omitted; POC #347 / PR #348 retained as provider evidence | diff --git a/vitest.config.ts b/vitest.config.ts new file mode 100644 index 000000000..6517e2270 --- /dev/null +++ b/vitest.config.ts @@ -0,0 +1,30 @@ +import { defineConfig } from "vitest/config"; +import { cloudflareTest } from "@cloudflare/vitest-plugin"; + +/** + * The workerd suite. + * + * These tests run against a real Durable Object namespace, real SQLite storage + * and a real WebSocket, because acquisition lifetime, owner eviction and + * transaction atomicity are properties of that runtime rather than of any model + * of it. Nothing here is discoverable by `deno task test`: the corpus walks + * `*.test.ts`, and these are `*.vitest.ts`, so the Deno, Node and Bun shards + * never see a file importing `cloudflare:test`. + */ +export default defineConfig({ + test: { + projects: [ + { + test: { + name: "cloudflare", + include: ["packages/workflow/tests/cloudflare/**/*.vitest.ts"], + }, + plugins: [ + cloudflareTest({ + wrangler: { configPath: "./packages/workflow/tests/cloudflare/wrangler.jsonc" }, + }), + ], + }, + ], + }, +});