Quest: Resume workflows with their Workspace, repositories, and Agent state

[taras]

Story

As a workflow author, I want xmd workflow to retain one provider-backed
Workspace with the run's journal, repositories, and Agent state, so supervised
procedures resume after interruption without reconstructing hidden filesystem or
transcript state.

Settled contract

PR #358 records the normative architecture in architecture.md and
specs/workflow-workspace-spec.md:

• xmd run uses the caller's current environment and makes no restoration
  guarantee; xmd workflow owns one retained WorkflowRun and implicit root
  Workspace.
• One SQLite database holds logically separate filtered journal, versioned DOFS,
  Repository/Worktree metadata, and Agent-session stores.
• One expansion produces one effect and one Workspace transaction. Completed
  durable effects restore their results, ephemeral attachments rebuild the live
  Effection tree, and partial replay continues from the retained frontier.
• Missing or corrupt authoritative Workspace state is unrecoverable rather than
  silently replaced.
• Workflow Agents receive no Workspace materialization or additional
  directories. They request observations and propose mutations as constrained
  generated XMD executed by the host.
• Prompt, Git-host, Issue, and other external effects reconcile stable identity
  at their owning provider boundary; they do not claim cross-system atomicity.
• Public lifecycle operations remain host-neutral and preserve one executor
  authority.

This issue is the implementation umbrella. Child issues own executable
acceptance detail; this body owns the shared invariants and current closure map.

Current delivery state

The retained lifecycle, history, named Repository/Worktree composition, local
Git and Git-host effects, provider-neutral Issue and Fetch effects, ambient
authentication, authorized workflow component bundle, generated-XMD observation
and mutation admission, retained Agent sessions, durable answer delivery,
explicit ordinary-resume scheduling, adversarial composition, and end-to-end
certification are all implemented.

PR #181 contains the remaining integrated delivery. At exact head
e85479c33ce4c91d821e0eaf195150db374ae4a6 over base and merge base
6717e867c9467114e6b060620d5b18c87beb2c48, #299 recorded certification PASS.
All frozen commands passed, including both source and compiled entrypoints,
CF1-CF6, the 31-file owner matrix, Node and Bun WFH rows, typing,
publishability, and the frozen verification task.

The only remaining PR-local work is #292's truthful synchronization of the
living workflow and delivery records. #290's document proof, #301's complete
composition, and #300's explicit scheduling slice are implemented on PR #181;
those issues close with the PR rather than receiving another product pass.

Remaining order

1. Implement Make the adversarial implementation workflow accurately describe its behavior #292 against the exact certified PR head and obtain Planner PASS
   on one focused feedback commit.
2. Integrate that commit, refresh PR ✨ Compose and certify the supervised adversarial implementation workflow #181's title/body, and run the applicable
   delivery gate and required CI.
3. Mark the PR ready, obtain required review, and merge.
4. Close Test the adversarial planning workflow with shipped components #290, Deliver answers to suspended workflows for later resumption #300, Compose the supervised adversarial implementation workflow #301, Make the adversarial implementation workflow accurately describe its behavior #292, and this umbrella through the delivered merge.

No later hardening issue is inserted into this gate:

• Prevent xmd run filesystem operations from escaping the working directory #227 remains the host-filesystem provider race/portable-guarantee follow-up;
  retained Workflow Workspace operations already use their provider-owned
  logical namespace.
• An outer <PrintErrors> does not print a failure an <Output> region decided #327 remains the unrelated <PrintErrors>/<Output> correction; this
  workflow does not use that boundary.
• Run shell blocks in xmd workflows #363 remains transactional Worker Shell, deferred behind an upstream
  force-termination contract and excluded from PR ✨ Compose and certify the supervised adversarial implementation workflow #181.
• Require ACP adapters to enforce tool-free workflow Agent sessions #496 remains portable ACP adapter-level no-tool enforcement. Certification
  proves the initial empty-cwd, no-directory, no-MCP, empty-requested-tool,
  deny-all profile and refusal of native permission requests, not universal
  adapter-level tool absence.

Cross-cutting invariants

• WorkflowRun establishment and base pinning precede Workspace attachment.
• Shared production modules use contextual APIs and contain no runtime or
  provider detection.
• One authoritative host-owned DOFS connection serves a workflow database and
  serializes Workspace-local effects.
• Security filtering occurs before journal insertion; co-located Workspace
  content is not automatically journal or training data.
• Completed replay attaches no Workspace, Agent, process, Git-host, Issue, or
  Fetch provider.
• Status, list, and history are read-only and cannot advance a run.
• No required state exists only in a transcript, host path, provider handle,
  branch name, or Git sidecar ref.
• External effects reconcile stable identity and never claim atomicity across
  SQLite and a provider.
• Answer delivery records without executing. Manual and explicit trusted-host
  scheduling invoke the same ordinary resume path and executor lock. No watcher,
  delivery-to-resume wiring, unattended arbitration, or second executor ships.
• An Agent reaches the Workspace only as data: it names source and XMD admits
  and executes it. Generated Git, Git-host, Issue, process, credential, and
  other external effects remain outside the admitted classes.

Completion

This umbrella closes when PR #181 merges after #292 synchronization and the
normal delivery gate. Transactional Worker Shell, portable adapter-level
no-tool enforcement, the ordinary host-filesystem race, and the unrelated
error-printing correction remain independent follow-ups.

Intentionally excluded

• A public remote-host selector or deployed XMD service.
• Worker Shell or Worker JavaScript in the initial topology.
• Native subprocess execution, writable FUSE, bundled workerd, or Containers.
• Watchers, automatic answer-to-resume wiring, and unattended iteration
  arbitration.
• Human actor attestation.
• Automatic ingestion or arbitrary scanning of Workspace files.
• Recovery from deleted or corrupt authoritative Workspace state.
• Rewinding external systems during a history fork.

Authoritative evidence

• 📝 Specify retained workflow Workspaces #358: merged architecture and observable Workspace contract.
• Certify the adversarial implementation workflow end to end #299: closed end-to-end certification gate for PR ✨ Compose and certify the supervised adversarial implementation workflow #181.
• Make the adversarial implementation workflow accurately describe its behavior #292: frozen final synchronization plan and Implementor handoff.
• End goal: PR ✨ Compose and certify the supervised adversarial implementation workflow #181.

[taras]

Design-session record

This comment preserves the rationale and source audit behind the product contract in the issue body. The body remains the implementation contract; this is the reasoning that should not be rediscovered or silently reversed.

Why the original replay choices were replaced

The issue originally asked whether an effect recorded inside a replaced temporary environment should be re-executed, include the environment in its identity, or fork the journal.

That framing assumes current filesystem state can validate every historical event. It cannot. In this sequence:

<TempDir>
  <File path="abc.txt">content</File>
  <DeleteFile path="abc.txt" />
  <File path="xyz.txt">other content</File>
</TempDir>

an interruption after DeleteFile leaves abc.txt absent. That absence is the correct journal-frontier state. A replay guard on the earlier File cannot distinguish “the create never happened” from “a later recorded delete happened.” Current state can validate a frontier invariant, not every intermediate postcondition.

Executable.md is a repeatable workflow system, not a general agent harness that repairs an arbitrarily broken machine. Normal restoration should return to a state from which the recorded workflow can continue. Missing or corrupt provider state is an explicit stale/unrecoverable condition.

Replay and ephemeral work

The original mental model remains useful:

• replay rehydrates the Effection tree;
• durable effects are restored from the journal and are not repeated; and
• non-journaled ephemeral operations run again to reconstruct scope-owned runtime structure.

That matches @executablemd/durable-streams’ existing ephemeral(operation): it emits no journal entry, consumes no replay position, reruns on replay, and remains owned by the surrounding Effection scope.

For Workspace, the rerun operation establishes an attachment. It does not create a fresh empty filesystem. The durable provider state and ephemeral attachment have different lifetimes.

Existing provider precedent

The ACPX integration already separates a provider-neutral capability from a provider-specific adapter:

• core owns the Agent API, provider-factory seam, registry, and Markdown composition;
• @executablemd/acp exports createAcpxProvider() and owns provider handles;
• the host registers and selects a root provider;
• provider installation is lazy and scope-owned; and
• tests can install the provider directly with injected dependencies.

Workspace should reuse that shape where it fits. Its important difference is that the attachment is document-execution scoped while the logical Workspace state outlives that execution.

Workspace supplies implementations for the existing runtime contexts. It does not create a second filesystem or process surface.

Cloudflare Computer capability audit

Official sources inspected during the design session:

• package overview
• VFS layout
• filesystem schema
• runtime interface
• lifecycle
• Artifacts interface

The system already provides:

• a Workspace constructed inside a Durable Object from ctx.storage;
• a SQLite-backed VFS as the durable source of truth;
• a transient container filesystem synchronized with that VFS;
• filesystem-only operation or pluggable container, shell, and JavaScript execution backends;
• push -> execute -> pull synchronization for command backends that need it;
• retained execution records and reattachment for the container and JavaScript backends; and
• optional Git and Cloudflare Artifacts capabilities.

It does not currently provide a generic point-in-time Workspace snapshot/history API. VFS revisions and tombstones are synchronization machinery, not an audit history. Artifacts is a separate, opt-in Git repository service rather than automatic Workspace capture.

That does not block normal resumption: reattach to the current durable VFS for the same run. It does rule out rollback, arbitrary journal-prefix rewind, and branching from an intermediate filesystem state.

The package is preview-only and its documentation is partly forward-looking. Its lifecycle document also identifies reconnect/hibernation gaps: a dead capnweb session currently requires callers to reconstruct the Workspace, and some mismatch recovery remains future work. The adapter must expose only behavior the selected version actually ships.

Cross-system durability

The XMD journal and Cloudflare VFS do not share a transaction. A crash can occur after a Workspace mutation commits but before its XMD result is appended. The design therefore depends on repeatable mutation, stable provider execution identity, or explicit uncertain-completion failure—not historical filesystem replay.

Cloudflare’s runtime result can report a completed command with post-command synchronization still pending. The adapter cannot turn that into a successful durable XMD effect until the Workspace source of truth is synchronized.

Retention, training, and secrets

The run-scoped Workspace is retained because its resulting environment is useful evidence and may contribute to future training. Scope cleanup releases attachment resources; it does not delete durable state.

The journal was given default secret detection because durable run history is intended to be retained and reused. #218 does not invent another capture/export pipeline or security policy. #199 and #291 continue to own the persistence boundaries they create. If a future training exporter promotes raw Workspace contents into another store, that exporter becomes a new persistence boundary and must be reviewed then.

Relationship to PR #341

PR #341 is the correct foundation and remains safe to merge:

• it creates the stable run ID and deterministic expansion ID that Workspace provenance needs;
• it keeps both values independent of provider-specific state;
• it scopes live metadata to one document execution, matching the attachment lifetime;
• completed replay restores metadata without invoking providers; and
• it explicitly says its contextual values are not authority boundaries.

The last point is load-bearing. The runId + expansionId pair identifies a logical Workspace, but provider authorization must be rooted in a host-owned installation. A document-rebound context value must not grant access to another run’s provider resources.

Workflow-run establishment also precedes root import, while <Workspace> is reached during root expansion. Base pinning must therefore remain independent of Workspace acquisition. The intended order is: host pins the base and records the run, Workspace attaches using that run, then #293 reconciles the Worktree inside it.

One host-topology question remains

@cloudflare/computer is designed to construct a Workspace inside a Cloudflare Durable Object using Workers bindings. This repository currently has no Cloudflare Worker, Durable Object, or Wrangler host.

Before implementation is delegated, the host boundary must be selected: either #218 introduces a Cloudflare Workers/Durable Object host for Executable.md, or an existing XMD host reaches a separately deployed provider through an explicit transport. Importing @cloudflare/computer into the current local CLI is not by itself a viable provider topology.

This question does not affect whether #341 can merge, but it materially changes #218’s implementation scope and should be decided by Taras rather than by its implementor.