diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 6bd8e2c..8008d13 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -23,6 +23,8 @@ jobs: with: node-version: 22 cache: pnpm + - run: sudo apt-get update && sudo apt-get install -y bubblewrap + - run: sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0 - run: pnpm install --frozen-lockfile - run: pnpm typecheck - run: pnpm build @@ -68,6 +70,35 @@ jobs: apps/web/test-results/ retention-days: 7 + g1-local: + name: G1 real Git, runner and owner browser journey + runs-on: ubuntu-latest + timeout-minutes: 15 + steps: + - uses: actions/checkout@v5 + - uses: pnpm/action-setup@v6.1.0 + - uses: actions/setup-node@v5 + with: + node-version: 22 + cache: pnpm + - run: sudo apt-get update && sudo apt-get install -y bubblewrap + # Ubuntu's AppArmor default may deny unprivileged namespaces on ephemeral CI hosts. + - run: sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0 + - run: pnpm install --frozen-lockfile + - run: pnpm --filter @commonforge-org/web exec playwright install --with-deps chromium + - run: pnpm test:g1:local + - if: always() + uses: actions/upload-artifact@v4 + with: + name: g1-sanitized-results + path: | + .g1-artifacts/local-ci/report.json + .g1-artifacts/local-ci/recovery-report.json + apps/web/test-results/native/**/*.png + apps/web/test-results/native/**/source-acceptance.json + apps/web/test-results/native/**/recovery-acceptance.json + retention-days: 7 + # Secrets are not readable in a job-level `if`, so detect them in a tiny job first. # Fork PRs never receive secrets, so the live job is skipped there. live-gate: diff --git a/.gitignore b/.gitignore index 499d6ab..158177e 100644 --- a/.gitignore +++ b/.gitignore @@ -44,3 +44,11 @@ session.json # run outputs of the live harness and demo scripts (commit a run deliberately with `git add -f`) evidence/ + +# Task-local verification artifacts (may contain local coordinates) +.g1-artifacts/ + +# Reviewed, sanitized G1 evidence summaries are deliberately public; raw run evidence stays ignored. +!/docs/agent-native/evidence/ +!/docs/agent-native/evidence/*.json +!/docs/agent-native/evidence/*.diff diff --git a/README.md b/README.md index 528917e..161f9b3 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,7 @@

-

Your clients build. Their agents help. You stay in control.

+

Your project’s understanding survives the agent.

CI @@ -14,10 +14,20 @@ Runs on Cloudflare Workers and Artifacts

-CommonForge is a self-hosted workspace where an agency, its clients and their coding agents change -a client's website together. Clients ask for changes as tasks. Agents work on each task in an -isolated copy of the code. Every change is checked, previewed on a staging site and approved -before it reaches production, and every accepted version can be restored. +CommonForge is a public, self-hosted platform for people and coding agents to own software +and its project knowledge. The reference product now includes confirmed rules, behavioral scenarios, +pinned task context, independently verified changes, and durable correction history. The existing +agency workflow remains available while a separate private agency consumer is developed. + +The **bounded G1 workflow is verified**, including real local execution, a fresh-agent handoff, +and an isolated Cloudflare release/expiry/recovery rehearsal. Run `pnpm dev:real-local` for the real Git/Worker/Node +stack, or `pnpm test:g1:local` for its bounded API and browser rehearsal. Knowledge, Outcomes and +Review are available to owners/developers. New web projects default to this workflow; existing +legacy projects retain explicit enablement. +See [execution status](docs/agent-native/execution-status.md) for measured gates and +[the local workflow](docs/agent-native/local-workflow.md) for setup and limits. Production native +release and complete Git bundle restoration have additional requirements; existing agency +production behavior is documented below. For developers: it runs entirely on **Cloudflare Workers** and **Cloudflare Artifacts**. You deploy it into your own Cloudflare account with one script, and agents work through a small CLI, diff --git a/apps/cli/README.md b/apps/cli/README.md index 73b9892..e1e3203 100644 --- a/apps/cli/README.md +++ b/apps/cli/README.md @@ -41,6 +41,37 @@ The session is stored with mode 0600 in your config directory (`~/.config/common Linux, `~/Library/Application Support/commonforge` on macOS, `%APPDATA%\commonforge` on Windows; override with `CFORGE_CONFIG_DIR`). +## Project understanding handoff + +For native projects, use the owner-issued task code instead of a broad user login: + +```sh +cforge handoff --code '' --api-url https://forge.example.com +cforge checkout ./work +cforge context +# Edit and commit inside ./work, then: +cforge submit --workspace ./work +cforge propose +cforge results +``` + +Checkout carries protected `.commonforge/context.json` and `.md` with the exact rules, scenarios, +mandate and context digest. Submit refuses stale context. Propose may report pending integration; +repeat once the actual runner finishes. Results distinguish current applicable evidence from +superseded runs and failures; unavailable execution is not a successful behavioral check. + +When an owner authorizes renewal, exchange the replacement into a new empty checkout: + +```sh +cforge renew --code '' --out ./work-renewed +``` + +Renewal revokes prior task authority and pins new context. Preserve the old workspace as history; +it is not silently rewritten. See [renewal](../../docs/agent-native/renewal.md) and +[the reference workflow](../../docs/agent-native/local-workflow.md) for scope and limits. +These tools also have structured MCP equivalents. Task authority cannot confirm rules, approve, +accept, deploy, or access another task. + ## Find a task ```sh diff --git a/apps/cli/src/checkout.ts b/apps/cli/src/checkout.ts index 21d3b08..3280375 100644 --- a/apps/cli/src/checkout.ts +++ b/apps/cli/src/checkout.ts @@ -12,7 +12,7 @@ * No token is ever written to disk, to git config, to workspace.json or to the output. */ import { mkdir, readdir, realpath, rm, stat } from "node:fs/promises"; -import { resolve } from "node:path"; +import { join, resolve } from "node:path"; import { MaterializationPlanSchema, type MaterializationPlan } from "@commonforge-org/core"; import { materialize, validatePlan, verifyMaterialization, type AssetObjectRequest } from "@commonforge-org/runner-protocol"; import { createNodeIo } from "@commonforge-org/runner-protocol/node"; @@ -21,6 +21,7 @@ import { int, parse, positional, type CommandContext } from "./command.js"; import { CliError, scrub } from "./context.js"; import type { CheckoutRecord } from "./config.js"; import { gitEnv, helperConfigEntries, helperConfigEnv, installHelperConfig, lfsConfigEntries } from "./git.js"; +import { fetchNativeContext, writeNativeContext, type NativeContext } from "./native.js"; import { acquireLease, leaseTtl } from "./project.js"; const USAGE = "cforge checkout [--lease-ttl ] [--wait ] [--json]"; @@ -80,9 +81,10 @@ export async function checkout(ctx: CommandContext, args: readonly string[]): Pr let userApi: ApiClient | null = null; let leaseInfo: Record; let leaseToken: string; - const envLease = await leaseBearer(ctx.io, ctx.store, taskId, apiUrl).then((b) => (b?.source === "env" ? b : null)); + const nativeStored = await stat(join(ctx.store.dir, "native", `${taskId}.json`)).then(() => true, () => false); + const envLease = await leaseBearer(ctx.io, ctx.store, taskId, apiUrl).then(b => b?.source === "env" || nativeStored ? b : null); if (envLease) { - leaseInfo = { source: "env" }; + leaseInfo = { source: envLease.source }; leaseToken = envLease.token; } else { userApi = (await sessionClient(ctx.io, ctx.store, apiUrl)).api; @@ -115,7 +117,10 @@ export async function checkout(ctx: CommandContext, args: readonly string[]): Pr } if (plan.workspaceRoot !== root) throw new CliError("invalid_plan", `the plan is for ${plan.workspaceRoot}, not ${root}`); checkRemotes(plan); - await materializeAndVerify(ctx, plan, { apiUrl, taskId, root, leaseApi, leaseInfo, setRecord: (r) => void (recordPath = r) }); + const task = await leaseApi.get<{ projectProtocolVersion?: number }>(`/api/tasks/${encodeURIComponent(taskId)}`); + if (nativeStored && task.projectProtocolVersion !== 1 || task.projectProtocolVersion !== undefined && task.projectProtocolVersion !== 1) throw new CliError("unsupported_version", "unsupported project protocol"); + const nativeContext = task.projectProtocolVersion === 1 ? await fetchNativeContext(leaseApi, taskId) : null; + await materializeAndVerify(ctx, plan, { apiUrl, taskId, root, leaseApi, leaseInfo, setRecord: (r) => void (recordPath = r), ...(nativeContext ? { nativeContext } : {}) }); } catch (e) { await cleanup(); throw e; @@ -125,7 +130,7 @@ export async function checkout(ctx: CommandContext, args: readonly string[]): Pr async function materializeAndVerify( ctx: CommandContext, plan: MaterializationPlan, - o: { apiUrl: string; taskId: string; root: string; leaseApi: ApiClient; leaseInfo: Record; setRecord: (path: string) => void }, + o: { apiUrl: string; taskId: string; root: string; leaseApi: ApiClient; leaseInfo: Record; nativeContext?: NativeContext; setRecord: (path: string) => void }, ): Promise { const { apiUrl, taskId, root } = o; // The lease goes to an LFS URL only on the API's own origin (Tokens, folders phase 4). @@ -213,7 +218,12 @@ async function materializeAndVerify( ...(m.taskWorkspaceId ? { taskWorkspaceId: m.taskWorkspaceId } : {}), ...(m.source.kind === "git" ? { kind: "git", sha: m.source.sha, remoteUrl: m.source.remoteUrl } : { kind: "assets", manifestId: m.source.manifest.manifestId }), })); - ctx.out.result({ taskId, workspaceRoot: root, mounts, lease: o.leaseInfo, verified: true }, () => + if(o.nativeContext){ + const current=await fetchNativeContext(o.leaseApi,taskId); + if(current.attemptId!==o.nativeContext.attemptId||current.context.digest!==o.nativeContext.context.digest)throw new CliError('context_invalid','task authority changed during checkout'); + await writeNativeContext(root,current); + } + ctx.out.result({ ...(o.nativeContext ? { projectProtocolVersion: 1, attemptId: o.nativeContext.attemptId, contextDigest: o.nativeContext.context.digest } : {}), taskId, workspaceRoot: root, mounts, lease: o.leaseInfo, verified: true }, () => [ `Checked out ${taskId} into ${root} (verified).`, ...mounts.map((m) => ` ${m.mountName.padEnd(20)} ${m.writable ? "read-write" : "read-only "} ${"sha" in m ? m.sha : m.manifestId}`), diff --git a/apps/cli/src/main.ts b/apps/cli/src/main.ts index 62194bb..50f0778 100644 --- a/apps/cli/src/main.ts +++ b/apps/cli/src/main.ts @@ -2,6 +2,7 @@ import { createInterface as createLineReader } from "node:readline"; import { createInterface } from "node:readline/promises"; import { fileURLToPath } from "node:url"; +import { handoff, contextCommand, nativeCommand, propose, nativeResults, renew } from "./native.js"; import { checkout } from "./checkout.js"; import { checkpoint } from "./checkpoint.js"; import type { CommandContext } from "./command.js"; @@ -26,6 +27,12 @@ Usage: cforge [options] login --email | --code sign in (CLI code flow); the session is stored 0600 in the config dir logout | whoami + renew --code --out renew task authority into a fresh verified checkout + handoff --code exchange one-time task authority (no broad user session) + results inspect this attempt’s proposal, evidence and decision history + propose integrate submitted candidates, then freeze exact project review + context [--out ] inspect and export pinned project understanding + native --command propose knowledge or an exact project change project list | show | tasks task create --title [--request ] [--write ...] | lease | release checkout materialize a task workspace (verified); git uses the credential helper @@ -51,6 +58,12 @@ const COMMANDS: Record = { project, task, checkout, + handoff, + renew, + context: contextCommand, + native: nativeCommand, + propose, + results: nativeResults, status, checkpoint, submit, diff --git a/apps/cli/src/mcp.ts b/apps/cli/src/mcp.ts index 753c6d1..70765bd 100644 --- a/apps/cli/src/mcp.ts +++ b/apps/cli/src/mcp.ts @@ -18,7 +18,8 @@ export type InvokeCli = (argv: string[]) => Promise<{ code: number; stdout: stri export const MCP_PROTOCOL_VERSIONS = ["2025-11-25", "2025-06-18", "2025-03-26", "2024-11-05"] as const; const INSTRUCTIONS = `CommonForge tools for working on a website or app project an agency manages. -Start with sign_in if a tool answers login_required: the person gets a one-time code from the CommonForge app (Account page, or the task's Give to agent). +For agent-native projects start with exchange_task_handoff and inspect_project_context; preserve the confirmed correction and submit only within the mandate. Never use a broad user login to work around denied agent authority. +For legacy projects start with sign_in if a tool answers login_required: the person gets a one-time code from the CommonForge app (Account page, or the task's Give to agent). To ask for a change: list_projects, then create_task with a title and the request in plain words. The project's owner decides which folders a task can change. To do the work (needs local files and git): checkout_task into an empty folder, edit only inside it, then submit_task. save_progress keeps work safe without handing it in.`; @@ -78,6 +79,48 @@ function absolute(path: string | undefined, what: string): string { } export const TOOLS: ToolDef[] = [ + { + name: "renew_task_authority", title: "Renew into a fresh task checkout", + description: "Exchange the human-issued replacement renewal code and materialize a new empty workspace with fresh attempt/context. Prior mandate and token remain revoked. Reports success only after checkout and final authority validation; never extends an old pinned checkout.", + inputSchema: { type: "object", properties: { code: text("Human-issued one-use replacement task code."), folder: text("Absolute NEW empty workspace path."), api_url: text("CommonForge server origin.") }, required: ["code", "folder"], additionalProperties: false }, + annotations: { idempotentHint: false, openWorldHint: true }, + argv: a => ({ command: "renew", options: [opt("code", a.code ?? ""), opt("out", absolute(a.folder, "folder")), ...(a.api_url ? [opt("api-url", a.api_url)] : [])] }), + }, + { + name: "inspect_attempt_results", title: "Inspect own attempt results", + description: "Read this task-scoped attempt's exact proposals, evidence, human decisions and accepted manifests. No access to other task attempts.", + inputSchema: { type: "object", properties: { task_id: text("Assigned task id.") }, required: ["task_id"], additionalProperties: false }, + annotations: { readOnlyHint: true, openWorldHint: true }, + argv: a => ({ command: "results", args: [a.task_id ?? ""] }), + }, + { + name: "prepare_project_proposal", title: "Prepare exact project review", + description: "Integrate this task's submitted candidates. If queued, invoke again after completion to freeze the exact project envelope with current knowledge, source and runtime subjects. Does not approve or accept.", + inputSchema: { type: "object", properties: { task_id: text("Assigned task id."), inner_proposal_id: text("Optional exact integrated proposal id already returned by this task.") }, required: ["task_id"], additionalProperties: false }, + annotations: { idempotentHint: true, openWorldHint: true }, + argv: a => ({ command: "propose", options: a.inner_proposal_id ? [opt("inner-proposal", a.inner_proposal_id)] : [], args: [a.task_id ?? ""] }), + }, + { + name: "exchange_task_handoff", title: "Use task-scoped authority", + description: "Exchange a one-time project handoff code for only this task. Does not create a broad user session.", + inputSchema: { type: "object", properties: { code: text("The org.project.cfh_ task handoff code."), api_url: text("The CommonForge server origin.") }, required: ["code"], additionalProperties: false }, + annotations: { idempotentHint: false, openWorldHint: true }, + argv: a => ({ command: "handoff", options: [opt("code", a.code ?? ""), ...(a.api_url ? [opt("api-url", a.api_url)] : [])] }), + }, + { + name: "inspect_project_context", title: "Inspect pinned project understanding", + description: "Read exact confirmed rules, scenarios, assumptions and attempt authority; export protected JSON and Markdown context. Refuses stale knowledge.", + inputSchema: { type: "object", properties: { task_id: text("The assigned task id."), folder: text("Absolute workspace directory to receive .commonforge/context files.") }, required: ["task_id", "folder"], additionalProperties: false }, + annotations: { idempotentHint: true, openWorldHint: true }, + argv: a => ({ command: "context", options: [opt("out", absolute(a.folder, "folder"))], args: [a.task_id ?? ""] }), + }, + { + name: "propose_project_record", title: "Propose knowledge or an exact change", + description: "Submit a structured draftKnowledge, draftScenario, or prepareProposal command with expectedGeneration and idempotencyKey. Agent scope cannot confirm, approve, accept or deploy.", + inputSchema: { type: "object", properties: { task_id: text("The assigned task id."), command: text("Strict JSON project command including record, expectedGeneration and idempotencyKey.") }, required: ["task_id", "command"], additionalProperties: false }, + annotations: { idempotentHint: false, openWorldHint: true }, + argv: a => ({ command: "native", options: [opt("command", a.command ?? "")], args: [a.task_id ?? ""] }), + }, { name: "sign_in", title: "Sign in to CommonForge", @@ -264,7 +307,7 @@ export class McpServer { } catch (e) { return fail(e instanceof CliError ? e.code : "invalid_arguments", (e as Error).message); } - if (tool.name === "sign_in") { + if (tool.name === "sign_in" || tool.name === "exchange_task_handoff" || tool.name === "renew_task_authority") { const refused = await this.#refuseSwitch(cmd); if (refused) return fail("signed_in_elsewhere", refused); } diff --git a/apps/cli/src/native.ts b/apps/cli/src/native.ts new file mode 100644 index 0000000..1ab29c9 --- /dev/null +++ b/apps/cli/src/native.ts @@ -0,0 +1,138 @@ +/** Task-scoped G1 handoff and immutable pinned context; no broad login is created. */ +import { chmod, lstat, mkdir, readFile, readdir, realpath, rename, writeFile } from 'node:fs/promises'; +import { join, resolve } from 'node:path'; +import { z } from 'zod'; +import { canonicalJson, materializeProjectContext, type ProjectContextPacket } from '@commonforge-org/core'; +import { ApiClient, agentClient, resolveApiUrl } from './api.js'; +import { parse, positional, str, type CommandContext } from './command.js'; +import { CliError, rememberSecret } from './context.js'; + +const SafeTaskId = z.string().regex(/^[A-Za-z0-9][A-Za-z0-9._-]{0,200}$/); +const ContextWire = z.object({ packet: z.unknown(), digest: z.string().regex(/^[a-f0-9]{64}$/), json: z.string(), markdown: z.string() }).strict(); +const Reply = z.object({ protocolVersion: z.literal(1), attemptId: z.string(), mandateId: z.string(), context: ContextWire, knowledgeStale: z.boolean().optional(), generation: z.number().int().nonnegative().optional(), agentId: z.string().optional() }); +export type NativeContext = { attemptId: string; mandateId: string; context: ReturnType; knowledgeStale?: boolean; generation?: number; agentId?: string }; +export function validateNativeContext(raw: unknown, taskId: string): NativeContext { + if (!SafeTaskId.safeParse(taskId).success) throw new CliError('context_invalid', 'invalid task identifier'); + const r = Reply.safeParse(raw); + if (!r.success) throw new CliError('unsupported_version', 'server returned unsupported project context'); + const materialized = materializeProjectContext(r.data.context.packet as ProjectContextPacket); + if (materialized.packet.taskId !== taskId || materialized.packet.attemptId !== r.data.attemptId || materialized.packet.mandate.id !== r.data.mandateId || materialized.digest !== r.data.context.digest || materialized.json !== r.data.context.json || materialized.markdown !== r.data.context.markdown) throw new CliError('context_invalid', 'project context digest or subject does not match'); + return { attemptId: r.data.attemptId, mandateId: r.data.mandateId, context: materialized, ...(r.data.generation === undefined ? {} : { generation: r.data.generation }), ...(r.data.agentId === undefined ? {} : { agentId: r.data.agentId }), ...(r.data.knowledgeStale === undefined ? {} : { knowledgeStale: r.data.knowledgeStale }) }; +} +async function protectedDirectory(path: string): Promise { + await mkdir(path, { recursive: true, mode: 0o700 }); + if ((await lstat(path)).isSymbolicLink() || !(await lstat(path)).isDirectory()) throw new CliError('context_path_invalid', 'context destination must be a real directory'); + await chmod(path, 0o700); +} +async function safeWrite(dir: string, name: string, value: string): Promise { + if (!/^[A-Za-z0-9][A-Za-z0-9._-]{0,220}$/.test(name)) throw new CliError('context_path_invalid', 'invalid context filename'); + await protectedDirectory(dir); + const temp = join(dir, `.${name}.${crypto.randomUUID()}.tmp`); + await writeFile(temp, value, { mode: 0o600, flag: 'wx' }); + await rename(temp, join(dir, name)); +} +export async function writeNativeContext(root: string, data: NativeContext): Promise { + const dir = join(await realpath(root), '.commonforge'); + await safeWrite(dir, 'context.json', data.context.json + '\n'); + await safeWrite(dir, 'context.md', data.context.markdown); + await safeWrite(dir, 'attempt.json', canonicalJson({ protocolVersion: 1, taskId: data.context.packet.taskId, attemptId: data.attemptId, mandateId: data.mandateId, contextDigest: data.context.digest }) + '\n'); +} +export async function fetchNativeContext(api: ApiClient, taskId: string): Promise { + const data = validateNativeContext(await api.get(`/api/tasks/${encodeURIComponent(taskId)}/native/context`), taskId); + if (data.knowledgeStale) throw new CliError('knowledge_stale', 'project requirements changed; obtain a new task handoff and rerun verification'); + return data; +} +export async function checkPinnedNativeContext(root: string, data: NativeContext): Promise { + let pin: unknown; + try { pin = JSON.parse(await readFile(join(root, '.commonforge', 'attempt.json'), 'utf8')); } catch { throw new CliError('context_missing', 'native workspace has no pinned attempt; check out the task again'); } + const expected = { protocolVersion: 1, taskId: data.context.packet.taskId, attemptId: data.attemptId, mandateId: data.mandateId, contextDigest: data.context.digest }; + if (canonicalJson(pin) !== canonicalJson(expected)) throw new CliError('context_invalid', 'workspace attempt does not match current scoped authority'); + const raw = JSON.parse(await readFile(join(root, '.commonforge', 'context.json'), 'utf8')) as unknown; + if (materializeProjectContext(raw as ProjectContextPacket).digest !== data.context.digest) throw new CliError('context_invalid', 'workspace context was altered'); +} +async function exchangeNativeAuthority(ctx:CommandContext,code:string) { + rememberSecret(code); + const apiUrl=await resolveApiUrl(ctx.io,ctx.store,ctx.global.apiUrl); + const raw=await new ApiClient(ctx.io,apiUrl,null).post>('/api/auth/task/verify',{code}); + const lease=z.object({taskId:SafeTaskId,leaseToken:z.string().startsWith('cfl_'),leaseGeneration:z.number().int().nonnegative(),expiresAt:z.string().datetime({offset:true})}).parse(raw); + rememberSecret(lease.leaseToken); + const data=validateNativeContext(raw,lease.taskId); + if(Date.parse(lease.expiresAt)!==Date.parse(data.context.packet.mandate.expiresAt))throw new CliError('context_invalid','task lease expiry does not match pinned replacement authority'); + await ctx.store.saveConfig({apiUrl});await ctx.store.saveLease({apiUrl,...lease}); + await safeWrite(join(ctx.store.dir,'native'),`${lease.taskId}.json`,canonicalJson(data)); + return {lease,data}; +} +export async function handoff(ctx: CommandContext, args: readonly string[]): Promise { + const p=parse(args,{code:{type:'string'}},'cforge handoff --code '); + const code=str(p,'code');if(!code)throw new CliError('invalid_arguments','a task-scoped --code is required'); + const {lease,data}=await exchangeNativeAuthority(ctx,code); + ctx.out.result({taskId:lease.taskId,attemptId:data.attemptId,mandateId:data.mandateId,contextDigest:data.context.digest,expiresAt:lease.expiresAt},`Task-scoped handoff ready for ${lease.taskId}. Run cforge checkout, then read .commonforge/context.md.`); +} +/** A renewed code grants a NEW attempt; require a new materialized checkout before success. */ +export async function renew(ctx:CommandContext,args:readonly string[]):Promise { + const usage='cforge renew --code --out '; + const p=parse(args,{code:{type:'string'},out:{type:'string'}},usage); + const code=str(p,'code'),out=str(p,'out');if(!code||!out)throw new CliError('invalid_arguments',usage); + const root=resolve(ctx.io.cwd,out); + try { + const metadata=await lstat(root); + if(metadata.isSymbolicLink()||!metadata.isDirectory()||(await readdir(root)).length)throw new CliError('workspace_not_empty','renewal requires a new empty directory; preserve the previous checkout and its recorded context'); + } catch(error){if(error instanceof CliError||(error as {code?:string}).code!=='ENOENT')throw error;} + // Fail BEFORE consuming a single-use code if the current shell would override the new lease. + if(ctx.io.env.CFORGE_LEASE_TOKEN)throw new CliError('lease_override_active','unset CFORGE_LEASE_TOKEN before exchanging replacement authority; renewal stores its own task lease'); + const {lease}=await exchangeNativeAuthority(ctx,code); + try {const {checkout}=await import('./checkout.js');await checkout(ctx,[lease.taskId,root]);} + catch(error){throw new CliError(error instanceof CliError?error.code:'renewal_checkout_failed',`Replacement authority was exchanged but a verified fresh checkout did not complete. Retry cforge checkout ${lease.taskId} into a new empty directory; if authority expired or was revoked, ask for another replacement handoff. ${error instanceof Error?error.message:'Checkout failed.'}`);} +} +export async function contextCommand(ctx: CommandContext, args: readonly string[]): Promise { + const p = parse(args, { out: { type: 'string' } }, 'cforge context [--out ]'); + const taskId = positional(p, 0, 'taskId', 'cforge context '); + const apiUrl = await resolveApiUrl(ctx.io, ctx.store, ctx.global.apiUrl); + const api = await agentClient(ctx.io, ctx.store, taskId, apiUrl); + const data = await fetchNativeContext(api, taskId); + const root = resolve(ctx.io.cwd, str(p, 'out') ?? '.'); + await mkdir(root, { recursive: true }); + await writeNativeContext(root, data); + ctx.out.result({ taskId, attemptId: data.attemptId, mandateId: data.mandateId, contextDigest: data.context.digest, packet: data.context.packet, generation: data.generation, agentId: data.agentId, directory: join(root, '.commonforge') }, `Pinned project context written to ${join(root, '.commonforge')}.`); +} +/** Structured agent correction/proposal command. Server refuses authority beyond mandate. */ +export async function nativeCommand(ctx: CommandContext, args: readonly string[]): Promise { + const p = parse(args, { command: { type: 'string' } }, 'cforge native --command '); + const taskId = positional(p, 0, 'taskId', 'cforge native --command '); + let body: unknown; try { body = JSON.parse(str(p, 'command') ?? ''); } catch { throw new CliError('invalid_arguments', '--command must be structured JSON'); } + const apiUrl = await resolveApiUrl(ctx.io, ctx.store, ctx.global.apiUrl); + const api = await agentClient(ctx.io, ctx.store, taskId, apiUrl); + const data = await fetchNativeContext(api, taskId); + if (body && typeof body === "object" && !Array.isArray(body)) body = { expectedGeneration: data.generation, idempotencyKey: crypto.randomUUID(), ...body }; + const result = await api.post(`/api/tasks/${encodeURIComponent(taskId)}/native/commands`, body); + ctx.out.result(result, 'Project command recorded. Drafts require separate human confirmation.'); +} + + +export async function propose(ctx: CommandContext, args: readonly string[]): Promise { + const p = parse(args, { 'inner-proposal': { type: 'string' } }, 'cforge propose [--inner-proposal ]'); + const taskId = positional(p, 0, 'taskId', 'cforge propose '); + const apiUrl = await resolveApiUrl(ctx.io, ctx.store, ctx.global.apiUrl); + const api = await agentClient(ctx.io, ctx.store, taskId, apiUrl); + let context = await fetchNativeContext(api, taskId); + let innerId = str(p, 'inner-proposal'); + if (!innerId) { + const inner = await api.post<{ innerProposalId: string; jobId: string; state: string }>(`/api/tasks/${encodeURIComponent(taskId)}/native/submit`, {}); + if (inner.state !== 'integrated') { ctx.out.result(inner, `Artifact integration queued (${inner.jobId}). Run cforge propose ${taskId} again after integration completes.`); return; } + innerId = inner.innerProposalId; + context = await fetchNativeContext(api, taskId); + } + if (context.generation === undefined) throw new CliError('unsupported_version', 'server must provide project generation for scoped proposal preparation'); + const result = await api.post(`/api/tasks/${encodeURIComponent(taskId)}/native/proposals`, { innerProposalId: innerId, expectedGeneration: context.generation, idempotencyKey: crypto.randomUUID(), outcome: null }); + ctx.out.result(result, 'Exact project proposal prepared. A human must request verification and review the evidence before accepting.'); +} + +export async function nativeResults(ctx: CommandContext, args: readonly string[]): Promise { + const p = parse(args, {}, 'cforge results '); + const taskId = positional(p, 0, 'taskId', 'cforge results '); + const apiUrl = await resolveApiUrl(ctx.io, ctx.store, ctx.global.apiUrl); + const api = await agentClient(ctx.io, ctx.store, taskId, apiUrl); + const result = await api.get>(`/api/tasks/${encodeURIComponent(taskId)}/native/results`); + if (result.protocolVersion !== 1) throw new CliError('unsupported_version', 'unsupported project results protocol'); + ctx.out.result(result, () => JSON.stringify(result, null, 2)); +} diff --git a/apps/cli/src/submit.ts b/apps/cli/src/submit.ts index feaac1b..a23a13f 100644 --- a/apps/cli/src/submit.ts +++ b/apps/cli/src/submit.ts @@ -4,12 +4,14 @@ * (`POST /api/tasks/:id/workspaces/:ws/freeze {expectedSha}`), which revokes the write tokens and * records the candidate. Every push happens before any freeze, so a failed push freezes nothing. */ -import { resolve } from "node:path"; -import { normalizeApiUrl, taskClient } from "./api.js"; +import { join, resolve } from "node:path"; +import { stat } from "node:fs/promises"; +import { agentClient, normalizeApiUrl, taskClient } from "./api.js"; import { list, parse, str, type CommandContext } from "./command.js"; import { CliError } from "./context.js"; import { commitAll } from "./checkpoint.js"; import { gitOk, pushOneRef, type GitRunner } from "./git.js"; +import { checkPinnedNativeContext, fetchNativeContext } from "./native.js"; import { pickWritableMounts, resolveWorkspace } from "./workspace.js"; const USAGE = "cforge submit [--mount ]... [--commit ] [--tool ] [--workspace ] [--json]"; @@ -40,6 +42,12 @@ export async function submit(ctx: CommandContext, args: readonly string[]): Prom const mounts = pickWritableMounts(ws, explicit); const taskId = ws.record.taskId; + const nativeStored = await stat(join(ctx.store.dir, "native", `${taskId}.json`)).then(() => true, () => false) || await stat(join(ws.root, ".commonforge", "attempt.json")).then(() => true, () => false); + const api = nativeStored ? await agentClient(ctx.io, ctx.store, taskId, normalizeApiUrl(ws.record.apiUrl)) : await taskClient(ctx.io, ctx.store, taskId, normalizeApiUrl(ws.record.apiUrl), (l) => ctx.out.warn(l)); + const task = await api.get<{ projectProtocolVersion?: number }>(`/api/tasks/${encodeURIComponent(taskId)}`); + if (nativeStored && task.projectProtocolVersion !== 1 || task.projectProtocolVersion !== undefined && task.projectProtocolVersion !== 1) throw new CliError("unsupported_version", "unsupported project protocol"); + if (task.projectProtocolVersion === 1) await checkPinnedNativeContext(ws.root, await fetchNativeContext(api, taskId)); + const pushed: Array<{ mountName: string; taskWorkspaceId: string; sha: string; ref: string; committed: boolean }> = []; const skipped: string[] = []; for (const m of mounts) { @@ -60,7 +68,6 @@ export async function submit(ctx: CommandContext, args: readonly string[]): Prom } if (pushed.length === 0) throw new CliError("nothing_to_submit", `no writable mount has commits beyond its base (${skipped.join(", ") || "none"}); commit first or pass --mount`); - const api = await taskClient(ctx.io, ctx.store, taskId, normalizeApiUrl(ws.record.apiUrl), (l) => ctx.out.warn(l)); const candidates: Array = []; const summary = () => candidates.map((c) => ({ mountName: c.mountName, candidateId: c.candidateId, taskWorkspaceId: c.taskWorkspaceId, sha: c.sha, changedPaths: c.changedPaths, state: c.state })); for (const x of pushed) { diff --git a/apps/cli/test/mcp.test.ts b/apps/cli/test/mcp.test.ts index 397b219..3655121 100644 --- a/apps/cli/test/mcp.test.ts +++ b/apps/cli/test/mcp.test.ts @@ -55,7 +55,7 @@ describe("protocol", () => { expect(replies.get(1).result).toMatchObject({ protocolVersion: "2025-06-18", capabilities: { tools: {} }, serverInfo: { name: "commonforge", version: "0.1.0" } }); expect(replies.get(1).result.instructions).toContain("sign_in"); const tools = replies.get(2).result.tools; - expect(tools.map((t: { name: string }) => t.name)).toEqual(["sign_in", "who_am_i", "list_projects", "list_tasks", "create_task", "get_task", "checkout_task", "save_progress", "submit_task"]); + expect(tools.map((t: { name: string }) => t.name)).toEqual(["renew_task_authority", "inspect_attempt_results", "prepare_project_proposal", "exchange_task_handoff", "inspect_project_context", "propose_project_record", "sign_in", "who_am_i", "list_projects", "list_tasks", "create_task", "get_task", "checkout_task", "save_progress", "submit_task"]); for (const t of tools) { expect(t.inputSchema.type).toBe("object"); expect(t).not.toHaveProperty("argv"); diff --git a/apps/cli/test/native.test.ts b/apps/cli/test/native.test.ts new file mode 100644 index 0000000..9cd1d45 --- /dev/null +++ b/apps/cli/test/native.test.ts @@ -0,0 +1,109 @@ +import { mkdir, mkdtemp, readFile, rm, stat, symlink, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { describe, expect, it } from 'vitest'; +import { materializeProjectContext, type ProjectContextPacket } from '@commonforge-org/core'; +import { checkPinnedNativeContext, validateNativeContext, writeNativeContext } from '../src/native.js'; +import { harness } from './helpers.js'; +import { ConfigStore } from '../src/config.js'; +import { run } from '../src/main.js'; + +const T = '2026-10-17T12:00:00Z'; +const D = 'a'.repeat(64); +const packet: ProjectContextPacket = { domain: 'cf.project-context/1', schemaVersion: 1, organizationId: 'org', projectId: 'project', taskId: 'task-1', attemptId: 'attempt-1', mandate: { schemaVersion: 1, organizationId: 'org', projectId: 'project', id: 'mandate-1', author: 'owner', createdAt: T, revision: 1, taskId: 'task-1', agentId: 'agent', issuer: 'owner', operations: ['context:read'], resourceIds: [], issuedAt: T, expiresAt: '2026-10-17T13:00:00Z', revocationGeneration: 0, revokedAt: null, maxAttempts: 1, maxJobs: 5, timeoutSeconds: 60 }, knowledgeHead: D, knowledgeGeneration: 1, outcomes: [], rules: [], assumptions: [], scenarios: [], sourceHeads: [], omissions: [] }; +const wire = { protocolVersion: 1 as const, attemptId: 'attempt-1', mandateId: 'mandate-1', context: materializeProjectContext(packet) }; +describe('task scoped project context', () => { + it('refuses changed digest, task subject and unnegotiated protocol', () => { + expect(validateNativeContext(wire, 'task-1').attemptId).toBe('attempt-1'); + expect(() => validateNativeContext(wire, 'another-task')).toThrow('subject'); + expect(() => validateNativeContext(wire, '../outside')).toThrow('identifier'); + expect(() => validateNativeContext({ ...wire, protocolVersion: 2 }, 'task-1')).toThrow('unsupported'); + expect(() => validateNativeContext({ ...wire, context: { ...wire.context, digest: 'b'.repeat(64) } }, 'task-1')).toThrow('digest'); + }); + it('stores context without credentials in protected files and detects tampering', async () => { + const root = await mkdtemp(join(tmpdir(), 'cf-native-')); + try { + const data = validateNativeContext(wire, 'task-1'); + await writeNativeContext(root, data); + expect((await stat(join(root, '.commonforge'))).mode & 0o777).toBe(0o700); + expect((await stat(join(root, '.commonforge/context.json'))).mode & 0o777).toBe(0o600); + await expect(checkPinnedNativeContext(root, data)).resolves.toBeUndefined(); + expect(await readFile(join(root, '.commonforge/context.md'), 'utf8')).toContain('Project context'); + await expect(checkPinnedNativeContext(root, { ...data, attemptId: 'altered' })).rejects.toThrow('authority'); + } finally { await rm(root, { recursive: true, force: true }); } + }); + it('refuses symbolic link context destinations', async () => { + const root = await mkdtemp(join(tmpdir(), 'cf-native-')); + const outside = await mkdtemp(join(tmpdir(), 'cf-outside-')); + try { + await symlink(outside, join(root, '.commonforge')); + await expect(writeNativeContext(root, validateNativeContext(wire, 'task-1'))).rejects.toThrow('real directory'); + } finally { await rm(root, { recursive: true, force: true }); await rm(outside, { recursive: true, force: true }); } + }); + it.each([false, true])('refuses unsafe native submission before Git, including copied workspace without global marker (%s)', async copied => { + const root = await mkdtemp(join(tmpdir(), 'cf-native-submit-')); + const workspace = join(root, 'workspace'); const config = join(root, 'config'); + const apiUrl = 'http://localhost:8787'; let gitCalls = 0; + try { + await mkdir(join(workspace, '.commonforge'), { recursive: true }); + await writeFile(join(workspace, '.commonforge/workspace.json'), JSON.stringify({ snapshotId: 's', projectId: 'project', organizationId: 'org', mounts: [] })); + const store = new ConfigStore({}, config); + await store.saveCheckout({ apiUrl, taskId: 'task-1', workspaceRoot: workspace, mounts: [{ mountName: 'source', remoteUrl: 'file:///synthetic', writable: true, credential: 'none', taskWorkspaceId: 'ws', baseSha: 'base' }], createdAt: T }); + await store.saveLease({ apiUrl, taskId: 'task-1', leaseToken: 'cfl_onlytask123456', leaseGeneration: 1, expiresAt: '2050-01-01T00:00:00Z' }); + await store.saveSession({ apiUrl, sessionToken: 'cfs_broaduser123456', expiresAt: '2050-01-01T00:00:00Z', user: { userId: 'owner', email: 'synthetic@example.test' } }); + if (copied) await writeFile(join(workspace, '.commonforge/attempt.json'), '{}'); + else { await mkdir(join(config, 'native')); await writeFile(join(config, 'native/task-1.json'), '{}'); } + let output = ''; + const code = await run(['submit', '--commit', 'Must not commit', '--json'], { cwd: workspace, env: { CFORGE_CONFIG_DIR: config, CFORGE_API_URL: apiUrl }, stdout: x => { output += x; }, stderr: () => {}, fetch: async (url, options) => { + expect((options?.headers as Record).authorization).toBe('Bearer cfl_onlytask123456'); + if (String(url).endsWith('/native/context')) return Response.json({ ...wire, protocolVersion: 1, knowledgeStale: true }); + if (String(url).endsWith('/tasks/task-1')) return Response.json(copied ? {} : { projectProtocolVersion: 1 }); + throw new Error('unexpected mutation'); + } }, { git: async () => { gitCalls++; return { code: 0, stdout: '', stderr: '' }; } }); + expect(code).toBe(1); expect(output).toContain(copied ? 'unsupported_version' : 'knowledge_stale'); expect(gitCalls).toBe(0); + } finally { await rm(root, { recursive: true, force: true }); } + }); + it.each([false,true])('reports renewal success only after real Git checkout and final active context (%s)', async revoked => { + const h=harness(); + try { + h.world.addResource('source',{'index.txt':'actual local Git checkout'}); + const task=h.world.createTask('Renewed bounded task',['source']); + const token='cfl_renewedtask123456'; + task.generation=1;task.lease={token,generation:1,expiresAt:Date.now()+3600000,holder:'owner'};task.state='running'; + const pinned={...packet,organizationId:task.snapshot.organizationId,projectId:task.snapshot.projectId,taskId:task.taskId,attemptId:'renewed-attempt',mandate:{...packet.mandate,organizationId:task.snapshot.organizationId,projectId:task.snapshot.projectId,taskId:task.taskId,id:'replacement-mandate',expiresAt:new Date(task.lease.expiresAt).toISOString()}}; + const next={protocolVersion:1,attemptId:pinned.attemptId,mandateId:pinned.mandate.id,context:materializeProjectContext(pinned)}; + h.world.overrides.set('POST /api/auth/task/verify',()=>({status:200,body:{...next,taskId:task.taskId,leaseToken:token,leaseGeneration:1,expiresAt:new Date(task.lease!.expiresAt).toISOString()}})); + h.world.overrides.set(`GET /api/tasks/${task.taskId}`,()=>({status:200,body:{state:'running',projectProtocolVersion:1}})); + let reads=0; + h.world.overrides.set(`GET /api/tasks/${task.taskId}/native/context`,()=>{reads++;return revoked&&reads>1?{status:403,body:{error:{code:'mandate_revoked',message:'Replacement revoked during clone'}}}:{status:200,body:next};}); + const root=join(h.home,'renewed-workspace'); + const result=await h.cli(['renew','--code','org.project.cfh_renewal123456','--out',root,'--json']); + if(revoked){expect(result.code).toBe(1);expect(result.json().error.code).toBe('mandate_revoked');expect(result.stdout).not.toContain('\"verified\":true');await expect(stat(root)).rejects.toThrow();return;} + expect(result.code,result.stdout+result.stderr).toBe(0);expect(result.json().attemptId).toBe('renewed-attempt'); + expect(await readFile(join(root,'source/index.txt'),'utf8')).toBe('actual local Git checkout'); + expect(JSON.parse(await readFile(join(root,'.commonforge/attempt.json'),'utf8')).mandateId).toBe('replacement-mandate'); + expect(reads).toBe(2);await expect(stat(join(h.configDir,'session.json'))).rejects.toThrow(); + }finally{await h.world.close();} + }); + it('rejects renewal into an existing checkout before consuming its one-use code', async () => { + const root=await mkdtemp(join(tmpdir(),'cf-renewal-existing-'));let called=false;let out=''; + try{ + await writeFile(join(root,'preserved.txt'),'old work'); + const code=await run(['renew','--code','org.project.cfh_renewal123456','--out',root,'--json'],{cwd:root,env:{CFORGE_CONFIG_DIR:join(root,'config')},stdout:x=>{out+=x;},stderr:()=>{},fetch:async()=>{called=true;throw new Error('must not exchange');}}); + expect(code).toBe(1);expect(called).toBe(false);expect(out).toContain('workspace_not_empty');expect(await readFile(join(root,'preserved.txt'),'utf8')).toBe('old work'); + }finally{await rm(root,{recursive:true,force:true});} + }); + it('exchanges only task authority and never writes a broad user session or outputs the code/token', async () => { + const root = await mkdtemp(join(tmpdir(), 'cf-native-')); + const code = 'org.project.cfh_singleuse123456'; + const token = 'cfl_taskonly123456'; + let stdout = ''; let stderr = ''; + try { + const exit = await run(['handoff', '--code', code, '--json'], { cwd: root, env: { CFORGE_CONFIG_DIR: root, CFORGE_API_URL: 'http://localhost:8787' }, stdout: x => { stdout += x; }, stderr: x => { stderr += x; }, fetch: async (url, options) => { expect(String(url)).toContain('/api/auth/task/verify'); expect(options?.body).toBe(JSON.stringify({ code })); return Response.json({ ...wire, taskId: 'task-1', leaseToken: token, leaseGeneration: 1, expiresAt: '2026-10-17T13:00:00Z' }); } }); + expect(exit, stdout + stderr).toBe(0); + expect(stdout + stderr).not.toContain(code); expect(stdout + stderr).not.toContain(token); + await expect(stat(join(root, 'session.json'))).rejects.toThrow(); + expect((await stat(join(root, 'leases/task-1.json'))).mode & 0o777).toBe(0o600); + } finally { await rm(root, { recursive: true, force: true }); } + }); +}); diff --git a/apps/runner/src/g1-accepted-health.ts b/apps/runner/src/g1-accepted-health.ts new file mode 100644 index 0000000..4436fe6 --- /dev/null +++ b/apps/runner/src/g1-accepted-health.ts @@ -0,0 +1,26 @@ +/** Bind supplemental health observation to the immutable accepted integration revision. */ +import { createHash } from 'node:crypto'; +import { mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { parseRevisionId } from '@commonforge-org/core'; +import { inspectIsolatedHealth } from './g1-local-verifier.js'; +const resultPath = process.argv[2]!, result = JSON.parse(await readFile(resultPath,'utf8')); +const operator = JSON.parse(await readFile(process.argv[3]!,'utf8')); +const owner = JSON.parse(await readFile(process.argv[4]!,'utf8')); +const head = result.acceptedSourceHeads[0], revision = parseRevisionId(head.revisionId); +if (revision.kind !== 'art') throw new Error('expected real Artifacts-compatible Git source'); +const resourceResponse = await fetch(`${operator.workerUrl}/api/orgs/${result.input.organizationId}/resources/${head.resourceId}`, {headers:{authorization:`Bearer ${owner.ownerToken}`}}); +if(!resourceResponse.ok)throw new Error('accepted resource unavailable');const resource=await resourceResponse.json() as {coords:{namespace:string;repoName:string}}; +const response = await fetch(`${operator.sidecarUrl}/rpc`,{method:'POST',headers:{authorization:`Bearer ${operator.sidecarKey}`,'content-type':'application/json'},body:JSON.stringify({method:'readFile',args:[resource.coords.namespace,resource.coords.repoName,revision.sha,'fixtures/scheduling/worker.mjs']})}); +if(!response.ok)throw new Error('accepted integration source unavailable');const data=await response.json() as {result:{bytes:string}}; +const bytes=Buffer.from(data.result.bytes,'base64'),fileDigest=createHash('sha256').update(bytes).digest('hex'); +const root=await mkdtemp(join(tmpdir(),'g1-accepted-health-')); +try{ + await writeFile(join(root,'worker.mjs'),bytes,{mode:0o400}); + const health=await inspectIsolatedHealth(root) as {supportContact?:string;ruleVersion?:number}; + if(health.supportContact!=='support@example.test'||health.ruleVersion!==2||fileDigest!==result.implementationAfterDigest)throw new Error('accepted integration changed the observed fresh behavior'); + result.supplementalAcceptedIntegration={revisionId:head.revisionId,implementationPath:'fixtures/scheduling/worker.mjs',fileDigest,equalsCandidateFileDigest:true,observedHealth:health,execution:'bubblewrap-linux-http',network:'none',controlPlaneMounted:false}; + await writeFile(resultPath,JSON.stringify(result,null,2)); + process.stdout.write(`${JSON.stringify(result.supplementalAcceptedIntegration,null,2)}\n`); +}finally{await rm(root,{recursive:true,force:true});} diff --git a/apps/runner/src/g1-artifacts-sidecar.ts b/apps/runner/src/g1-artifacts-sidecar.ts new file mode 100644 index 0000000..e1974ed --- /dev/null +++ b/apps/runner/src/g1-artifacts-sidecar.ts @@ -0,0 +1,140 @@ +/** Local Artifacts transport compatibility over REAL LocalGitProvider bare repos and smart HTTP. */ +import { randomBytes, timingSafeEqual } from 'node:crypto'; +import { spawn } from 'node:child_process'; +import { createServer, type IncomingMessage, type ServerResponse } from 'node:http'; +import { copyFile, mkdir, readFile, rename, rm, writeFile } from 'node:fs/promises'; +import { basename, dirname, join, resolve } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { checkPath, hashCanonicalSync, IdSegmentSchema, localRevisionId, parseRevisionId, type ResourceRef } from '@commonforge-org/core'; +import { LocalGitProvider } from '@commonforge-org/provider-local-git'; +import { ArtifactsProviderError, type ArtifactsCommit, type ArtifactsCreatedRepo, type ArtifactsRepoInfo, type ArtifactsTokenInfo, type ArtifactsTokenMinted, type ArtifactsTransport, type ArtifactsTreeItem } from '@commonforge-org/provider-cloudflare'; +import { createGit, RUNNER_IDENTITY_ENV } from './git.js'; +interface Repo { ns: string; name: string; resource: ResourceRef; info: ArtifactsRepoInfo; tokens: Array } +export class RealLocalArtifacts implements ArtifactsTransport { + readonly kind = 'rest' as const; + readonly repos = new Map(); + readonly git = createGit({ allowFile: true }); + baseUrl = ''; + constructor(readonly provider: LocalGitProvider, readonly stateDir: string) {} + private repo(ns: string, name: string) { const r = this.repos.get(`${ns}/${name}`); if (!r) throw new ArtifactsProviderError('not_found', 'local repository not found'); return r; } + private remote(r: Repo) { if (r.resource.coords.kind !== 'local-git') throw new Error('local coords required'); return r.resource.coords.remoteUrl; } + private path(r: Repo) { return fileURLToPath(this.remote(r)); } + private pendingPersist: Promise = Promise.resolve(); + private async persist() { + const serialized = JSON.stringify([...this.repos]); + const write = this.pendingPersist.then(async () => { + const temporary = join(this.stateDir, `registry-${randomBytes(8).toString('hex')}.tmp`); + await writeFile(temporary, serialized, { mode: 0o600 }); + await rename(temporary, join(this.stateDir, 'registry.json')); + }); + this.pendingPersist = write.catch(() => undefined); return write; + } + async load() { + try { for (const [key, repo] of JSON.parse(await readFile(join(this.stateDir, 'registry.json'), 'utf8')) as Array<[string, Repo]>) this.repos.set(key, repo); } + catch (e) { if ((e as NodeJS.ErrnoException).code !== 'ENOENT') throw e; } + } + async createRepo(ns: string, name: string, opts: { description?: string; defaultBranch?: string } = {}): Promise { + IdSegmentSchema.parse(ns); IdSegmentSchema.parse(name); + if (this.repos.has(`${ns}/${name}`)) throw new ArtifactsProviderError('already_exists', 'local repository exists'); + const resource = await this.provider.createCanonicalResource({ organizationId: 'local-adapter', resourceId: `r${hashCanonicalSync({ ns, name }).slice(0, 40)}`, branch: opts.defaultBranch ?? 'main' }); + const info = { id: resource.resourceId, name, defaultBranch: opts.defaultBranch ?? 'main', remote: `${this.baseUrl}/git/${ns}/${name}.git`, source: null, readOnly: false }; + const repo: Repo = { ns, name, resource, info, tokens: [] }; this.repos.set(`${ns}/${name}`, repo); + const token = await this.createToken(ns, name, 'write', 3600); await this.persist(); + return { ...info, token: token.plaintext }; + } + async getRepo(ns: string, name: string) { return this.repos.get(`${ns}/${name}`)?.info ?? null; } + async deleteRepo(ns: string, name: string) { + const repo = this.repos.get(`${ns}/${name}`); if (!repo) return false; + // Task-owned state only. Provider registry retains historical identity; no reuse after delete. + await rm(this.path(repo), { recursive: true, force: true }); this.repos.delete(`${ns}/${name}`); await this.persist(); return true; + } + async createToken(ns: string, name: string, scope: 'read' | 'write', ttl: number): Promise { + if (!['read', 'write'].includes(scope) || !Number.isInteger(ttl) || ttl < 60 || ttl > 31536000) throw new Error('invalid token grant'); + const token = { id: `tok-${randomBytes(12).toString('hex')}`, plaintext: `art_v2_x_${randomBytes(32).toString('hex')}`, scope, expiresAt: new Date(Date.now() + ttl * 1000).toISOString(), state: 'active' as const, createdAt: new Date().toISOString() }; + this.repo(ns, name).tokens.push(token); await this.persist(); return token; + } + async listTokens(ns: string, name: string) { return this.repo(ns, name).tokens.map(({ plaintext: _secret, ...t }) => ({ ...t, state: t.state === 'active' && Date.parse(t.expiresAt) <= Date.now() ? 'expired' as const : t.state })); } + async revokeToken(ns: string, name: string, id: string) { const t = this.repo(ns, name).tokens.find(t => t.id === id); if (!t) return false; t.state = 'revoked'; await this.persist(); return true; } + async log(ns: string, name: string, opts: { ref: string; limit?: number }): Promise { + if (opts.ref.startsWith('refs/') || opts.ref.startsWith('-') || /[\x00-\x20]/.test(opts.ref)) return []; + const r = this.repo(ns, name), found = await this.git.run(['rev-parse', '--verify', '--end-of-options', `${opts.ref}^{commit}`], this.path(r)); + if (found.code !== 0) return []; + const shas = (await this.git.ok(['rev-list', '--first-parent', `--max-count=${Math.min(opts.limit ?? 20, 1000)}`, found.stdout.trim()], this.path(r))).trim().split('\n').filter(Boolean); + return (await Promise.all(shas.map(s => this.readCommit(ns, name, s)))).filter((c): c is ArtifactsCommit => c !== null); + } + async readCommit(ns: string, name: string, sha: string): Promise { + if (!/^[a-f0-9]{40,64}$/.test(sha)) return null; + const r = await this.git.run(['show', '-s', '--format=%H%x00%T%x00%P%x00%an%x00%ae%x00%cn%x00%ce%x00%at%x00%ct%x00%B', sha], this.path(this.repo(ns, name))); + if (r.code !== 0) return null; + const [hash, treeHash, parents, an, ae, cn, ce, at, ct, ...msg] = r.stdout.split('\0'); + return { hash: hash!, treeHash: treeHash!, parents: parents ? parents.split(' ') : [], author: { name: an!, email: ae! }, committer: { name: cn!, email: ce! }, authoredAt: Number(at), committedAt: Number(ct), message: msg.join('\0').trimEnd() }; + } + async readTree(ns: string, name: string, treeSha: string): Promise { + if (!/^[a-f0-9]{40,64}$/.test(treeSha)) return null; + const r = await this.git.run(['ls-tree', '-z', treeSha], this.path(this.repo(ns, name))); if (r.code !== 0) return null; + return r.stdout.split('\0').filter(Boolean).map(s => { const tab = s.indexOf('\t'), [mode, type, hash] = s.slice(0, tab).split(' '); return { name: s.slice(tab + 1), mode: mode === '040000' ? '40000' : mode!, hash: hash!, type: mode === '120000' ? 'symlink' : mode === '160000' ? 'gitlink' : mode === '100755' ? 'exec' : type as 'tree' | 'blob' }; }); + } + async readFile(ns: string, name: string, ref: string, path: string): Promise { + if (!checkPath(path).ok) return null; const [commit] = await this.log(ns, name, { ref, limit: 1 }); if (!commit) return null; + const repo = this.repo(ns, name); return this.provider.readFile(repo.resource, localRevisionId(repo.resource.resourceId, commit.hash), path); + } + async seedScheduling(ns: string, name: string) { + const repo = this.repo(ns, name), seed = join(this.stateDir, `seed-${randomBytes(8).toString('hex')}`); await mkdir(seed); + try { + for (const file of ['worker.mjs', 'build.mjs']) await copyFile(new URL(`../../../fixtures/scheduling/${file}`, import.meta.url), join(seed, file)); + await this.git.ok(['init', '-q', '-b', repo.info.defaultBranch], seed); await this.git.ok(['add', '.'], seed); + await this.git.ok(['commit', '-q', '-m', 'Synthetic scheduling fixture'], seed, { env: RUNNER_IDENTITY_ENV(new Date().toISOString()) }); + await this.git.ok(['push', '-q', this.remote(repo), `HEAD:refs/heads/${repo.info.defaultBranch}`], seed, { env: await this.provider.serviceGitEnv() }); + return { sha: (await this.git.ok(['rev-parse', 'HEAD'], seed)).trim() }; + } finally { await rm(seed, { recursive: true, force: true }); } + } + async candidateData(organizationId: string, resourceId: string, revisionId: string, namespacePrefix: string): Promise { + const revision = parseRevisionId(revisionId); + if (revision.kind !== 'art') throw new Error('expected Artifacts-compatible revision'); + const repo = [...this.repos.values()].find(r => r.info.id === revision.repoId && r.ns === `${namespacePrefix}-${organizationId}-canonical` && r.name === resourceId); + if (!repo) throw new Error('source scope mismatch'); + const revisionRef = localRevisionId(repo.resource.resourceId, revision.sha); + const data = await this.provider.readFile(repo.resource, revisionRef, 'worker.mjs') ?? await this.provider.readFile(repo.resource, revisionRef, 'fixtures/scheduling/worker.mjs'); + if (!data) throw new Error('missing candidate module'); return data; + } + async gitHttp(req: IncomingMessage, res: ServerResponse, url: URL) { + const match = /^\/git\/([A-Za-z0-9._-]+)\/([A-Za-z0-9._-]+)\.git\/(info\/refs|git-upload-pack|git-receive-pack)$/.exec(url.pathname); + if (!match) { res.writeHead(404); res.end(); return; } + const repo = this.repo(match[1]!, match[2]!); + const secret = req.headers.authorization?.replace(/^Bearer /, '') ?? ''; + const token = repo.tokens.find(t => t.plaintext.length === secret.length && timingSafeEqual(Buffer.from(t.plaintext), Buffer.from(secret))); + const write = match[3] === 'git-receive-pack' || url.searchParams.get('service') === 'git-receive-pack'; + if (!token || token.state !== 'active' || Date.parse(token.expiresAt) <= Date.now() || (write && token.scope !== 'write')) { res.writeHead(403); res.end('repository access denied'); return; } + const chunks: Buffer[] = []; let count = 0; + for await (const chunk of req) { count += (chunk as Buffer).length; if (count > 33554432) { res.writeHead(413); res.end(); return; } chunks.push(chunk as Buffer); } + const body = Buffer.concat(chunks); + const child = spawn('git', ['http-backend'], { env: { PATH: process.env.PATH ?? '/usr/bin:/bin', GIT_CONFIG_NOSYSTEM: '1', HOME: '/nonexistent', GIT_PROJECT_ROOT: dirname(this.path(repo)), PATH_INFO: `/${basename(this.path(repo))}/${match[3]}`, QUERY_STRING: url.searchParams.toString(), REQUEST_METHOD: req.method ?? 'GET', CONTENT_TYPE: req.headers['content-type'] ?? '', CONTENT_LENGTH: String(body.length), REMOTE_USER: 'commonforge-local', GIT_HTTP_EXPORT_ALL: '1', COMMONFORGE_TOKEN: await this.provider.serviceToken() }, stdio: ['pipe', 'pipe', 'pipe'] }); + const out: Buffer[] = []; let bytes = 0; const timer = setTimeout(() => child.kill('SIGKILL'), 30000); + child.stdout.on('data', (b: Buffer) => { bytes += b.length; if (bytes > 67108864) child.kill('SIGKILL'); else out.push(b); }); child.stderr.resume(); + child.once('error', () => { clearTimeout(timer); if (!res.headersSent) { res.writeHead(503); res.end(); } }); + child.once('close', () => { clearTimeout(timer); if (res.headersSent) return; const result = Buffer.concat(out), split = result.indexOf('\r\n\r\n'); if (split < 0) { res.writeHead(503); res.end(); return; } const lines = result.subarray(0, split).toString().split('\r\n'), headers: Record = {}; let status = 200; for (const line of lines) { const i = line.indexOf(':'); if (i < 0) continue; const key = line.slice(0, i).toLowerCase(), value = line.slice(i + 1).trim(); if (key === 'status') status = Number(value.slice(0, 3)); else headers[key] = value; } res.writeHead(status, headers); res.end(result.subarray(split + 4)); }); + child.stdin.on('error', () => undefined); child.stdin.end(body); + } +} +export async function startArtifactsSidecar(options: { stateDir: string; key: string; port?: number }) { + const stateDir = resolve(options.stateDir); await mkdir(stateDir, { recursive: true, mode: 0o700 }); + const provider = await LocalGitProvider.open({ root: join(stateDir, 'repos'), namespacePrefix: 'commonforge-sidecar' }); + const artifacts = new RealLocalArtifacts(provider, stateDir); await artifacts.load(); + const server = createServer((req, res) => { void (async () => { + const url = new URL(req.url ?? '/', 'http://localhost'); + if (req.headers.origin) { res.writeHead(403); res.end(); return; } + if (url.pathname.startsWith('/git/')) return artifacts.gitHttp(req, res, url); + if (req.headers.authorization !== `Bearer ${options.key}` || req.method !== 'POST' || url.pathname !== '/rpc') { res.writeHead(403); res.end(); return; } + const chunks: Buffer[] = []; let count = 0; for await (const chunk of req) { count += (chunk as Buffer).length; if (count > 1048576) throw new Error('RPC too large'); chunks.push(chunk as Buffer); } + const { method, args } = JSON.parse(Buffer.concat(chunks).toString('utf8')) as { method: string; args: unknown[] }; + const allowed = ['createRepo', 'getRepo', 'deleteRepo', 'createToken', 'listTokens', 'revokeToken', 'log', 'readCommit', 'readTree', 'readFile', 'seedScheduling']; + if (!allowed.includes(method) || !Array.isArray(args) || args.length > 4) throw new Error('unsupported RPC'); + const result = await (artifacts[method as keyof RealLocalArtifacts] as (...args: unknown[]) => Promise).apply(artifacts, args); + res.writeHead(200, { 'content-type': 'application/json' }); res.end(JSON.stringify({ result: result instanceof Uint8Array ? { bytes: Buffer.from(result).toString('base64') } : result })); + })().catch(e => { if (!res.headersSent) { res.writeHead(e instanceof ArtifactsProviderError ? 409 : 400, { 'content-type': 'application/json' }); res.end(JSON.stringify({ error: { code: e instanceof ArtifactsProviderError ? e.code : 'provider_unavailable', message: 'local source operation refused' } })); } }); }); + await new Promise((resolve, reject) => { server.once('error', reject); server.listen(options.port ?? 0, '127.0.0.1', resolve); }); + artifacts.baseUrl = `http://127.0.0.1:${(server.address() as { port: number }).port}`; + // Rebind persisted remotes to this startup's actual local address. + for (const repo of artifacts.repos.values()) repo.info.remote = `${artifacts.baseUrl}/git/${repo.ns}/${repo.name}.git`; + return { provider, artifacts, url: artifacts.baseUrl, close: () => new Promise((resolve, reject) => server.close(e => e ? reject(e) : resolve())) }; +} diff --git a/apps/runner/src/g1-bwrap-commands.ts b/apps/runner/src/g1-bwrap-commands.ts new file mode 100644 index 0000000..d534761 --- /dev/null +++ b/apps/runner/src/g1-bwrap-commands.ts @@ -0,0 +1,17 @@ +/** Narrow unprivileged Linux adapter: allow only Node entrypoints, all inside mount/PID/net namespaces. */ +import { chmod, mkdir, realpath, writeFile } from 'node:fs/promises'; +import { join } from 'node:path'; +import type { CommandSettings } from './context.js'; +export async function localNamespaceCommands(binDir: string): Promise { + await mkdir(binDir, { recursive: true, mode: 0o700 }); + const node = await realpath(process.execPath); + const shellQuote = (value: string) => `'${value.replace(/'/g, `'"'"'`)}'`; + const script = `#!/usr/bin/bash\nset -euo pipefail\nexec /usr/bin/bwrap --unshare-all --die-with-parent --new-session --cap-drop ALL --ro-bind /usr /usr --ro-bind /lib /lib --ro-bind /lib64 /lib64 --ro-bind ${shellQuote(node)} /node --proc /proc --dev /dev --tmpfs /tmp --bind "$PWD" /candidate --chdir /candidate /node "$@"\n`; + await writeFile(join(binDir, 'node'), script, { mode: 0o700 }); await chmod(join(binDir, 'node'), 0o700); + return { + // Existing runner uid guard is bypassed only because this pinned wrapper supplies the stronger + // namespace boundary. Candidate code sees /candidate and system runtime files, never the supervisor. + unsafeSameUid: true, path: binDir, allowlist: [['node', '{path}', '{args*}']], + network: 'inherit', install: { enabled: false }, maxOutputBytes: 65536, + }; +} diff --git a/apps/runner/src/g1-candidate-host.mjs b/apps/runner/src/g1-candidate-host.mjs new file mode 100644 index 0000000..8c43183 --- /dev/null +++ b/apps/runner/src/g1-candidate-host.mjs @@ -0,0 +1,16 @@ +// This file and candidate code execute together in a namespace with no control-plane mounts. +import { createServer } from 'node:http'; +process.stdout.write('SANDBOX_STARTED\n'); +const { default: worker } = await import('/candidate/worker.mjs'); +const server = createServer(async (req, res) => { + try { + const chunks = []; + for await (const chunk of req) chunks.push(chunk); + const request = new Request('http://fixture' + req.url, { method: req.method, ...(req.method === 'POST' ? { body: Buffer.concat(chunks), headers: { 'content-type': 'application/json' } } : {}) }); + const response = await worker.fetch(request); + res.writeHead(response.status, Object.fromEntries(response.headers)); + res.end(Buffer.from(await response.arrayBuffer())); + } catch { res.writeHead(500); res.end('candidate exception'); } +}); +// Separate socket handed to the trusted supervisor; no host network exists inside the namespace. +server.listen('/scratch/preview.sock', () => process.stdout.write('READY\n')); diff --git a/apps/runner/src/g1-cloud-rehearsal.ts b/apps/runner/src/g1-cloud-rehearsal.ts new file mode 100644 index 0000000..dee94e6 --- /dev/null +++ b/apps/runner/src/g1-cloud-rehearsal.ts @@ -0,0 +1,309 @@ +/** Dedicated G1 live CI rehearsal. Phased operator execution; importing/planning never contacts cloud. */ +import { randomUUID } from 'node:crypto'; +import { access, mkdir, mkdtemp, readFile, rename, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { dirname, join, resolve } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { z } from 'zod'; +import { configDigest } from '@commonforge-org/runner-protocol'; +import { canonicalJson, ProjectStateSchema, type BuildRecipe, type ProjectState } from '@commonforge-org/core'; +import { createCloudflareSchedulingAdapter } from './g1-cloudflare-source.js'; +import type { CloudflareReleaseEnvironment } from './g1-cloudflare-verifier.js'; +import { cases } from './g1-scheduling-harness.js'; +import { createGit, RUNNER_IDENTITY_ENV, type Git } from './git.js'; + +export const G1CloudRehearsalConfigSchema = z.object({ + schemaVersion: z.literal(1), runId: z.string().regex(/^[a-z0-9][a-z0-9-]{0,31}$/), + apiUrl: z.string().url(), accountId: z.string().regex(/^[a-f0-9]{32}$/), organizationId: z.string().regex(/^o-[a-f0-9]{12}$/), + ownerSessionFile: z.string().min(1), stateDirectory: z.string().min(1), + controlWorker: z.literal('commonforge-ci-worker'), targetPrefix: z.literal('cfci'), + mutationsApproved: z.boolean().default(false), deploymentApproved: z.boolean().default(false), + trustedSupervisorUrl: z.string().url().optional(), supervisorPort: z.number().int().min(1024).max(65535).default(8971), + pin: z.object({ binary: z.string().min(1), sha256: z.string().regex(/^[a-f0-9]{64}$/) }).strict(), schema: z.string().min(1), +}).strict().superRefine((c, context) => { + const url = new URL(c.apiUrl); + if (url.protocol !== 'https:' || !url.hostname.startsWith('commonforge-ci-worker.') || !url.hostname.endsWith('.workers.dev') || url.pathname !== '/' || url.search || url.hash || url.username || url.password) context.addIssue({ code: 'custom', message: 'API must be the exact HTTPS commonforge-ci-worker workers.dev origin' }); + if (c.trustedSupervisorUrl) { const supervisor = new URL(c.trustedSupervisorUrl); if (supervisor.protocol !== 'https:' || supervisor.username || supervisor.password || supervisor.search || supervisor.hash || supervisor.pathname !== '/') context.addIssue({ code: 'custom', message: 'supervisor must be an operator-pinned HTTPS origin' }); } +}); +export type G1CloudRehearsalConfig = z.infer; +export function cloudRehearsalContracts(raw: G1CloudRehearsalConfig) { + const config = G1CloudRehearsalConfigSchema.parse(raw); + const workerName = `cfci-${config.organizationId}-g1-${config.runId}`; + if (workerName.length > 63) throw new Error('CI release Worker name exceeds provider bound'); + const recipe: BuildRecipe = { recipeId: 'g1-cloud-scheduling', revision: 1, steps: [{ stepId: 'build', mountName: 'api', command: 'node build.mjs' }] }; + const targets = [{ targetId: 'api', kind: 'worker' as const, mountName: 'api', outputPath: 'dist', main: 'worker.mjs', workerName }]; + const targetEnvironment = { api: { vars: { APP_VERSION: 'g1-synthetic-scheduling' }, kvNamespaces: [] } }; + const healthCheck = { targetId: 'api', path: '/version' }; + const environment: CloudflareReleaseEnvironment = { targets, environment: targetEnvironment, healthCheck, accountId: config.accountId }; + const policy = { checks: [], contracts: [], rules: [], approvalRoles: ['owner'], publishRoles: ['owner'], release: { recipe, targets, environment: targetEnvironment, healthCheck, migrations: [] } }; + return { config, workerName, recipe, environment, policy }; +} +export function cloudRehearsalPlan(config: G1CloudRehearsalConfig) { + const c = cloudRehearsalContracts(config); + return { schemaVersion: 1, execution: 'planned-not-executed', controlWorker: c.config.controlWorker, apiUrl: c.config.apiUrl, + releaseWorker: c.workerName, scope: 'existing owner-approved isolated commonforge-ci infrastructure', + phases: ['setup: create fresh native project and dedicated source folder; configure vars-only recipe; emit supervisor registration', + 'run: confirm corrected rule and nine pinned scenarios; scoped real Git change; actual integration/verification/acceptance; explicit release build/publish/deploy and healthy observation', + 'expiry: read the actual first release after freshUntil; require unknown (no clock override)', + 'recover: accept and deploy a second source variant; recover exact first manifest under current knowledge; fresh nine-scenario verification; explicit third release and actual health', + 'inspect: read authoritative accepted/release/observation state and write sanitized results'], + persistentBindings: [], migrations: [], outboundFixtureEffects: [], + prerequisite: 'parent must authorize exact isolated CI target before enabling mutation/deployment flags; owner session and supervisor credentials remain private', + noProvisioning: true, noProductionTarget: true, noCloudClaimFromLocalTests: true }; +} +export function requireFreshCloudProjectName(projects: Array<{name: string}>, name: string): void { + if (projects.some(project => project.name === name)) throw new Error('matching project exists without this run checkpoint; refuse name-only adoption and require operator inspection'); +} +export interface CloudSourcePushCheckpoint { workspace?: string; intendedSha?: string; expectedRemoteSha?: string; sha?: string } +/** Persist immutable commit intent BEFORE a possibly ambiguous push. Retry observes the exact remote ref. */ +export async function pushResumableCloudSource(options: { + cycle: CloudSourcePushCheckpoint; remoteUrl: string; credentialSecret: string; files: Record; label: string; + checkpointDirectory: string; persist(): Promise; git?: Git; +}): Promise { + const { cycle, files } = options, git = options.git ?? createGit(); + const env = { GIT_CONFIG_COUNT: '1', GIT_CONFIG_KEY_0: `http.${options.remoteUrl}.extraHeader`, GIT_CONFIG_VALUE_0: `Authorization: Bearer ${options.credentialSecret}` }; + const validSha = (sha: string | undefined): sha is string => !!sha && /^(?:[a-f0-9]{40}|[a-f0-9]{64})$/.test(sha); + const remoteHead = async () => { + const refs = await git.ok(['ls-remote', options.remoteUrl, 'refs/heads/main'], options.checkpointDirectory, { env }); + const sha = refs.trim().split(/\s+/)[0]; if (!validSha(sha)) throw new Error('task source remote main is absent or malformed'); return sha; + }; + let remote = await remoteHead(); + // Legacy complete checkpoints are also rechecked rather than silently trusting a push label. + if (!cycle.intendedSha && cycle.sha) cycle.intendedSha = cycle.sha; + if (cycle.intendedSha && !validSha(cycle.intendedSha)) throw new Error('saved source intent is not an immutable full Git SHA'); + if (cycle.intendedSha && remote === cycle.intendedSha) { cycle.sha = cycle.intendedSha; await options.persist(); return cycle.sha; } + if (cycle.expectedRemoteSha && remote !== cycle.expectedRemoteSha) throw new Error('task remote moved from checkpointed base; refuse to overwrite another change'); + if (!cycle.workspace) { + const root = await mkdtemp(join(tmpdir(), 'commonforge-g1-cloud-source-')); cycle.workspace = join(root, 'candidate'); + cycle.expectedRemoteSha = remote; await options.persist(); + } + const exists = await access(join(cycle.workspace, '.git')).then(() => true, () => false); + if (!exists) { + if (cycle.intendedSha) throw new Error('intended source commit is not pushed and its checkpointed workspace is unavailable'); + // A partial failed clone is left as task-owned evidence; retry gets a new private workspace. + if (await access(cycle.workspace).then(() => true, () => false)) { + const root = await mkdtemp(join(tmpdir(), 'commonforge-g1-cloud-source-')); cycle.workspace = join(root, 'candidate'); await options.persist(); + } + await git.ok(['clone', '-q', options.remoteUrl, cycle.workspace], dirname(cycle.workspace), { env }); + } + if (!cycle.expectedRemoteSha) { cycle.expectedRemoteSha = remote; await options.persist(); } + const work = cycle.workspace; + if (!cycle.intendedSha) { + for (const [path, value] of Object.entries(files)) await writeFile(join(work, path), value); + await git.ok(['add', '--', ...Object.keys(files)], work); + const staged = await git.run(['diff', '--cached', '--quiet'], work); + const head = (await git.ok(['rev-parse', 'HEAD'], work)).trim(); + if (staged.code === 1) await git.ok(['commit', '-q', '-m', `G1 isolated live fixture ${options.label}`], work, { env: RUNNER_IDENTITY_ENV(new Date().toISOString()) }); + else if (staged.code !== 0 || head === cycle.expectedRemoteSha) throw new Error('no distinct intended source change; refuse an unrecorded no-op commit'); + cycle.intendedSha = (await git.ok(['rev-parse', 'HEAD'], work)).trim(); + if (!validSha(cycle.intendedSha)) throw new Error('prepared source commit is malformed'); + await options.persist(); + } + await git.ok(['cat-file', '-e', `${cycle.intendedSha}^{commit}`], work); + for (const [path, expected] of Object.entries(files)) if (await git.ok(['show', `${cycle.intendedSha}:${path}`], work) !== expected) throw new Error('checkpointed intended source differs from the exact rehearsal variant'); + remote = await remoteHead(); + if (remote !== cycle.intendedSha) { + if (remote !== cycle.expectedRemoteSha) throw new Error('task remote moved before push; refuse to overwrite another change'); + await git.ok(['push', '-q', `--force-with-lease=refs/heads/main:${cycle.expectedRemoteSha}`, options.remoteUrl, `${cycle.intendedSha}:refs/heads/main`], work, { env }); + } + if (await remoteHead() !== cycle.intendedSha) throw new Error('actual remote ref does not match checkpointed intended source'); + cycle.sha = cycle.intendedSha; await options.persist(); return cycle.sha; +} +interface Checkpoint { schemaVersion: 1; runId: string; configDigestInput: string; projectId?: string; resourceId?: string; setupComplete?: boolean; knowledgeComplete?: boolean; registration?: unknown; verifierDigest?: string; runtimeRecipeDigest?: string; cycles: Record; releases: Record; expiry?: unknown; recovery?: any; steps: Array<{ phase: string; operation: string; at: string; status: number }> } +const rememberedSecrets: string[] = []; +export class G1CloudRehearsal { + private livePoll?: (label: string, read: () => Promise, ready: (data: any) => boolean, timeout: number, interval: number) => Promise; + readonly contracts; checkpoint!: Checkpoint; private token = ''; private author = ''; phase = ''; + constructor(readonly config: G1CloudRehearsalConfig) { this.contracts = cloudRehearsalContracts(config); } + private get directory() { return resolve(this.config.stateDirectory); } + private get root() { if (!this.checkpoint.projectId) throw new Error('setup phase required'); return `/orgs/${this.config.organizationId}/projects/${this.checkpoint.projectId}`; } + private get native() { return `${this.root}/native`; } + async open() { + if (!this.config.mutationsApproved) throw new Error('operator has not enabled this reviewed isolated CI rehearsal'); + await mkdir(this.directory, { recursive: true, mode: 0o700 }); + const configDigestInput = canonicalJson({ runId: this.config.runId, apiUrl: this.config.apiUrl, organizationId: this.config.organizationId, accountId: this.config.accountId, workerName: this.contracts.workerName, recipe: this.contracts.recipe, environment: this.contracts.environment }); + this.checkpoint = await readFile(join(this.directory, 'checkpoint.json'), 'utf8').then(s => JSON.parse(s) as Checkpoint, () => ({ schemaVersion: 1, runId: this.config.runId, configDigestInput, cycles: {}, releases: {}, steps: [] })); + if (this.checkpoint.configDigestInput !== configDigestInput) throw new Error('checkpoint belongs to a different exact CI scope/runtime'); + process.env.CI_API_URL = this.config.apiUrl.replace(/\/$/, ''); + const helpers = await import(new URL('../../../scripts/ci-live/lib.ts', import.meta.url).href); + helpers.requireCiTarget('G1 isolated cloud rehearsal'); this.livePoll = helpers.poll; + const session = JSON.parse(await readFile(resolve(this.config.ownerSessionFile), 'utf8')); + this.token = session.ownerToken ?? session.sessionToken; + if (typeof this.token !== 'string' || !this.token.startsWith('cfs_')) throw new Error('private owner session is missing'); rememberedSecrets.push(this.token); helpers.secret(this.token); + const me = await this.api('GET', '/me'); this.author = me.user.userId; + const org = await this.api('GET', `/orgs/${this.config.organizationId}`); if (org.role !== 'owner') throw new Error('owner role required'); + } + private async save() { const temp = join(this.directory, `checkpoint-${randomUUID()}.tmp`); await writeFile(temp, JSON.stringify(this.checkpoint, null, 2), { mode: 0o600 }); await rename(temp, join(this.directory, 'checkpoint.json')); } + private async api(method: string, path: string, body?: unknown, token = this.token): Promise { + const response = await fetch(`${this.config.apiUrl.replace(/\/$/, '')}/api${path}`, { method, headers: { authorization: `Bearer ${token}`, ...(body === undefined ? {} : { 'content-type': 'application/json' }) }, ...(body === undefined ? {} : { body: JSON.stringify(body) }), redirect: 'manual', signal: AbortSignal.timeout(130000) }); + const data: any = await response.json().catch(() => null); + this.checkpoint.steps.push({ phase: this.phase, operation: `${method} ${path}`, at: new Date().toISOString(), status: response.status }); await this.save(); + if (!response.ok) throw new Error(`CI API ${method} ${path}: HTTP ${response.status} ${data?.error?.code ?? 'unclassified_error'}`); return data; + } + private async waitFor(read: () => Promise, predicate: (data: any) => boolean, label: string, milliseconds = 600000) { + if (!this.livePoll) throw new Error('existing CI polling helpers not initialized'); + return this.livePoll(label, read, data => { + if (predicate(data)) return true; + if (data?.state === 'failed' || data?.buildState === 'failed' || data?.state === 'recovery_required') throw new Error(`${label} failed: ${data?.error?.code ?? data.state}`); + return false; + }, milliseconds, 1500); + } + private state(): Promise { return this.api('GET', this.native).then(r => ProjectStateSchema.parse(r.state)); } + private base(id: string) { return { schemaVersion: 1, organizationId: this.config.organizationId, projectId: this.checkpoint.projectId!, id, author: this.author, createdAt: new Date().toISOString() }; } + private async command(command: Record, key: string) { const state = await this.state(); return this.api('POST', `${this.native}/commands`, { ...command, expectedGeneration: state.generation, idempotencyKey: key }); } + async setup() { + this.phase = 'setup'; + if (this.checkpoint.setupComplete) return this.checkpoint.registration; + const name = `G1 isolated cloud rehearsal ${this.config.runId}`; + if (!this.checkpoint.projectId) { + const projects = await this.api('GET', `/orgs/${this.config.organizationId}/projects`); + requireFreshCloudProjectName(projects.projects, name); + this.checkpoint.projectId = (await this.api('POST', `/orgs/${this.config.organizationId}/projects`, { name, mode: 'agent-native' })).projectId; await this.save(); + } + if (!this.checkpoint.resourceId) { + const folders = await this.api('GET', `${this.root}/folders`); + const existing = folders.folders.find((f: any) => f.mountName === 'api'); + this.checkpoint.resourceId = existing?.resourceId ?? (await this.api('POST', `${this.root}/folders`, { name: 'api', kind: 'code' })).resourceId; await this.save(); + } + const resource = await this.waitFor(() => this.api('GET', `/orgs/${this.config.organizationId}/resources/${this.checkpoint.resourceId}`), r => r.state === 'active' && !!r.head, 'real Sandbox folder initialization'); + await this.api('PUT', `${this.root}/policy`, this.contracts.policy); + const runtime = await createCloudflareSchedulingAdapter({ pin: this.config.pin, schema: this.config.schema, recipe: this.contracts.recipe, environment: this.contracts.environment }); + this.checkpoint.verifierDigest = runtime.verifierDigest; this.checkpoint.runtimeRecipeDigest = runtime.runtimeRecipeDigest; + const registration = { port: this.config.supervisorPort, accountId: this.config.accountId, pin: this.config.pin, schema: this.config.schema, recipe: this.contracts.recipe, environment: this.contracts.environment, + resources: [{ organizationId: this.config.organizationId, projectId: this.checkpoint.projectId, resourceId: this.checkpoint.resourceId, namespace: resource.coords.namespace, repoName: resource.coords.repoName, repoId: resource.coords.repoId, pathPrefix: '' }] }; + this.checkpoint.registration = registration; this.checkpoint.setupComplete = true; await this.save(); + await writeFile(join(this.directory, 'supervisor-registration.json'), JSON.stringify(registration, null, 2), { mode: 0o600 }); + return { projectId: this.checkpoint.projectId, resourceId: this.checkpoint.resourceId, releaseWorker: this.contracts.workerName, verifierDigest: runtime.verifierDigest, runtimeRecipeDigest: runtime.runtimeRecipeDigest, registrationFile: join(this.directory, 'supervisor-registration.json'), next: 'Parent starts trusted supervisor with this registration, configures exact HTTPS verifier origin, then authorizes run.' }; + } + private async verifySupervisor() { + if (!this.config.trustedSupervisorUrl) throw new Error('trusted HTTPS supervisor origin is not configured'); + const response = await fetch(`${this.config.trustedSupervisorUrl.replace(/\/$/, '')}/capabilities`, { redirect: 'manual', signal: AbortSignal.timeout(10000) }); + if (!response.ok) throw new Error('trusted supervisor capabilities unavailable'); const cap = await response.json() as any; + if (cap.verifierDigest !== this.checkpoint.verifierDigest || cap.runtimeRecipeDigest !== this.checkpoint.runtimeRecipeDigest || canonicalJson([...cap.scenarioIds].sort()) !== canonicalJson(cases.map(c => c[0]).sort())) throw new Error('supervisor runtime/verifier/scenario set differs from approved full release environment'); + } + private async knowledge() { + if (this.checkpoint.knowledgeComplete) return; + const ruleId = 'cloud-scheduling-rule'; let s = await this.state(); + if (!s.knowledge.some(k => k.id === ruleId)) await this.command({ type: 'draftKnowledge', record: { ...this.base(ruleId), revision: 1, predecessor: null, kind: 'rule', text: 'Customers may reschedule at least 24 hours before an appointment. Travel blocks everybody. Managers may override the cutoff only with a retained nonempty reason.', rationale: 'Synthetic corrected owner requirement for exact cloud release verification', sources: [{ kind: 'text', ref: 'G1 synthetic owner-confirmed scheduling requirement', digest: null }], confirmation: null, archived: false } }, 'live-rule-draft'); + s = await this.state(); if (!s.knowledge.some(k => k.id === ruleId && k.confirmation)) await this.command({ type: 'confirmKnowledge', id: ruleId, revision: 1, decisionId: 'confirm-cloud-rule', explanation: 'Confirm explicit boundary, authority, travel and reason requirements' }, 'live-rule-confirm'); + for (const [id] of cases) { + s = await this.state(); + if (!s.scenarios.some(q => q.id === id)) await this.command({ type: 'draftScenario', record: { ...this.base(id), revision: 1, predecessor: null, title: id, given: 'Synthetic UTC scheduling input', when: 'Rescheduling is requested', then: 'The independently pinned exact scenario expectation is observed', ruleRefs: [{ id: ruleId, revision: 2 }], required: true, method: 'automated', verifierDigest: this.checkpoint.verifierDigest, confirmation: null, archived: false } }, `live-scenario-draft-${id}`); + s = await this.state(); if (!s.scenarios.some(q => q.id === id && q.confirmation)) await this.command({ type: 'confirmScenario', id, revision: 1, decisionId: `confirm-${id}`, explanation: 'Confirm independently executable synthetic scheduling example' }, `live-scenario-confirm-${id}`); + } + this.checkpoint.knowledgeComplete = true; await this.save(); + } + async sourceCycle(label: 'initial' | 'later') { + if (this.checkpoint.cycles[label]?.acceptedManifest) return this.checkpoint.cycles[label]; + const cycle = this.checkpoint.cycles[label] ??= {}; await this.save(); + if (!cycle.taskId) { cycle.taskId = (await this.api('POST', `${this.root}/tasks`, { title: `G1 live ${label} source`, request: `Publish synthetic fixture variant ${label} while preserving all confirmed rules.`, writeMounts: ['api'] })).taskId; await this.save(); } + if (!cycle.leaseToken) { + await this.waitFor(() => this.api('GET', `/tasks/${cycle.taskId}`), t => t.state === 'ready', 'actual cloud source task'); + const handoff = await this.api('POST', `/tasks/${cycle.taskId}/native/handoff`, { agentId: `g1-cloud-rehearsal-${label}` }); + const agent = await this.api('POST', '/auth/task/verify', { code: handoff.code }); + cycle.leaseToken = agent.leaseToken; cycle.contextDigest = agent.context.digest; rememberedSecrets.push(agent.leaseToken); await this.save(); + } + if (!cycle.candidateId) { + const credential = await this.api('POST', `/tasks/${cycle.taskId}/credential`, { mountName: 'api', access: 'write' }, cycle.leaseToken); rememberedSecrets.push(credential.secret); + await pushResumableCloudSource({ cycle, remoteUrl: credential.remoteUrl, credentialSecret: credential.secret, + files: await cloudFixtureFiles(label), label, checkpointDirectory: this.directory, persist: () => this.save() }); + } + if (!cycle.candidateId) { + const task = await this.api('GET', `/tasks/${cycle.taskId}`), workspace = task.workspaces.find((w: any) => w.mountName === 'api'); + const candidate = await this.api('POST', `/tasks/${cycle.taskId}/workspaces/${workspace.taskWorkspaceId}/freeze`, { expectedSha: cycle.sha, tool: 'g1-authorized-cloud-rehearsal' }, cycle.leaseToken); cycle.candidateId = candidate.candidateId; await this.save(); + } + if (!cycle.innerProposalId) { cycle.innerProposalId = (await this.api('POST', `/orgs/${this.config.organizationId}/proposals`, { taskId: cycle.taskId, candidateIds: [cycle.candidateId] })).proposalId; await this.save(); } + const innerPath = `/orgs/${this.config.organizationId}/proposals/${cycle.innerProposalId}`; + const inner = await this.waitFor(() => this.api('GET', innerPath), p => p.state === 'prepared' || !!p.integration, 'cloud candidate preparation'); + if (!inner.integration) { await this.api('POST', `${innerPath}/integrate`, {}); await this.waitFor(() => this.api('GET', innerPath), p => !!p.integration, 'real Sandbox integration'); } + if (!cycle.proposalId) { const s = await this.state(); const prepared = await this.api('POST', `/tasks/${cycle.taskId}/native/proposals`, { innerProposalId: cycle.innerProposalId, expectedGeneration: s.generation, idempotencyKey: `live-outer-${label}` }, cycle.leaseToken); cycle.proposalId = prepared.proposal.id; await this.save(); } + const accepted = await this.verifyAccept(cycle.proposalId, `source-${label}`); Object.assign(cycle, accepted); await this.save(); return cycle; + } + private async verifyAccept(proposalId: string, label: string) { + let state = await this.state(); const proposal = state.proposals.find(p => p.id === proposalId); if (!proposal) throw new Error('prepared exact proposal missing'); + if (proposal.envelope.runtimeRecipeDigest !== this.checkpoint.runtimeRecipeDigest || proposal.envelope.requiredScenarios.length !== 9 || proposal.envelope.requiredScenarios.some(s => s.verifierDigest !== this.checkpoint.verifierDigest || s.method !== 'automated')) throw new Error('project review envelope differs from pinned nine-scenario cloud runtime'); + const already = state.manifests.find(m => m.envelopeDigest === proposal.digest); if (already) return { acceptedManifest: already.hash, sourceHeads: already.sourceHeads, envelopeDigest: proposal.digest, knowledgeHead: already.knowledgeHead }; + const checked = await this.api('POST', `${this.native}/proposals/${encodeURIComponent(proposalId)}/verify`, { idempotencyKey: `live-verify-${label}` }); + if (checked.evidence.length !== 9 || checked.evidence.some((e: any) => e.result !== 'pass' || e.provenance !== 'runner-verified' || e.envelopeDigest !== proposal.digest)) throw new Error('nine exact trusted remote-source scenario passes required'); + state = ProjectStateSchema.parse(checked.state); const decisionId = `live-decision-${label}`; + if (!state.decisions.some(d => d.id === decisionId)) await this.command({ type: 'recordDecision', record: { ...this.base(decisionId), envelopeDigest: proposal.digest, innerProposalHash: proposal.envelope.innerProposalHash, action: 'approve', explanation: 'Synthetic owner approves exact isolated CI source after real independently pinned workerd checks; deploy approval remains separate.', authorityRevision: state.authorities.find(a => a.principalId === this.author)!.revision } }, `record-${decisionId}`); + await this.command({ type: 'accept', proposalId, decisionId }, `accept-${decisionId}`); + state = await this.state(); const accepted = state.manifests.at(-1)!; + return { acceptedManifest: accepted.hash, sourceHeads: accepted.sourceHeads, envelopeDigest: proposal.digest, knowledgeHead: state.knowledgeHead, evidence: checked.evidence }; + } + async release(label: string, expectedManifest: string) { + if (!this.config.deploymentApproved) throw new Error(`deployment of exact isolated target ${this.contracts.workerName} is not enabled by operator`); + if (this.checkpoint.releases[label]?.healthState === 'healthy') return this.checkpoint.releases[label]; + const checkpoint = this.checkpoint.releases[label] ??= {}; + if (!checkpoint.releaseId) { const release = await this.api('POST', `${this.root}/releases`, {}); if (release.manifestHash !== expectedManifest) throw new Error('release prepared a different accepted manifest'); checkpoint.releaseId = release.releaseId; await this.save(); } + const path = `/orgs/${this.config.organizationId}/releases/${checkpoint.releaseId}`; + const built = await this.waitFor(() => this.api('GET', path), r => r.buildState === 'ready' || !!r.outputDigest, 'real cloud release build'); + if (built.envDigest !== configDigest(this.contracts.environment) || built.manifestHash !== expectedManifest || built.targets.length !== 1 || built.targets[0].workerName !== this.contracts.workerName || built.targets[0].config.kvNamespaces.length || built.targets[0].config.d1Databases?.length) throw new Error('release build escaped exact manifest/vars-only CI target'); + if (!built.publishApproved) await this.api('POST', `${path}/publish-approvals`, { outputDigest: built.outputDigest }); + let current = await this.api('GET', path); + if (!current.live && !['deploying', 'health_verification'].includes(current.state)) await this.api('POST', `${path}/deploy`, { idempotencyKey: `live-deploy-${this.config.runId}-${label}` }); + current = await this.waitFor(() => this.api('GET', path), r => r.live && r.healthState === 'healthy', 'actual deployed release observation'); + const deployment = current.deployments[0]; if (!deployment || deployment.workerName !== this.contracts.workerName) throw new Error('deployed target mismatch'); + const deploymentUrl = new URL(deployment.url); if (deploymentUrl.protocol !== 'https:' || !deploymentUrl.hostname.startsWith(`${this.contracts.workerName}.`) || !deploymentUrl.hostname.endsWith('.workers.dev')) throw new Error('release receipt URL escapes exact isolated CI Worker'); + const response = await fetch(`${deployment.url.replace(/\/$/, '')}/version`, { redirect: 'manual', signal: AbortSignal.timeout(15000) }); + const observed = await response.json() as { release?: string; variant?: string }; if (!response.ok || observed.release !== current.releaseId) throw new Error('live target does not identify the exact release'); + const deployedScenarios = []; + for (const [scenarioId, input, expected] of cases) { + const live = await fetch(`${deployment.url.replace(/\/$/, '')}/reschedule`, { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ now: '2026-10-17T12:00:00.000Z', ...input }), redirect: 'manual', signal: AbortSignal.timeout(15000) }); + const actual = await live.json(); if (!live.ok || canonicalJson(actual) !== canonicalJson(expected)) throw new Error(`actual deployed scheduling scenario failed: ${scenarioId}`); + deployedScenarios.push({ scenarioId, result: 'pass', execution: 'actual Cloudflare HTTPS Worker', releaseId: current.releaseId, outputDigest: current.outputDigest }); + } + const recorded = (await this.state()).observations.find(o => o.releaseId === current.releaseId && o.manifestHash === expectedManifest && o.buildDigest === current.outputDigest && o.result === 'healthy'); + if (!recorded) throw new Error('actual release observation is not durably bound to exact accepted manifest and output'); + Object.assign(checkpoint, current, { liveVersionObservation: observed, deployedScenarios, recordedObservationId: recorded.id }); await this.save(); return checkpoint; + } + async run() { this.phase = 'run'; if (!this.checkpoint.setupComplete) throw new Error('setup registration required'); await this.verifySupervisor(); await this.knowledge(); const source = await this.sourceCycle('initial'); return this.release('initial', source.acceptedManifest); } + async expiry() { + this.phase = 'expiry'; const initial = this.checkpoint.releases.initial; if (!initial?.freshUntil) throw new Error('initial observed release required'); + const current = await this.api('GET', `/orgs/${this.config.organizationId}/releases/${initial.releaseId}`); + if (Date.now() <= Date.parse(initial.freshUntil)) return { status: 'not_run', reason: 'actual freshness deadline not reached', waitUntil: initial.freshUntil }; + if (current.healthState !== 'unknown') throw new Error('expired release observation remains falsely healthy'); + this.checkpoint.expiry = { releaseId: current.releaseId, observedAt: new Date().toISOString(), freshUntil: initial.freshUntil, healthState: current.healthState, actualClock: true }; await this.save(); return this.checkpoint.expiry; + } + async recover() { + this.phase = 'recover'; if (!this.checkpoint.expiry) throw new Error('observe actual first-release expiry before recovery phase'); await this.verifySupervisor(); + const later = await this.sourceCycle('later'); await this.release('later', later.acceptedManifest); + const before = await this.state(); const initial = this.checkpoint.cycles.initial; + if (!this.checkpoint.recovery?.proposalId) { const prepared = await this.api('POST', `${this.native}/recoveries`, { manifestHash: initial.acceptedManifest, expectedGeneration: before.generation, idempotencyKey: 'live-historical-recovery' }); this.checkpoint.recovery = { proposalId: prepared.proposal.id, beforeKnowledgeHead: before.knowledgeHead, beforeKnowledge: before.knowledge }; await this.save(); } + const recovery = await this.verifyAccept(this.checkpoint.recovery.proposalId, 'historical-recovery'); const state = await this.state(); + if (state.knowledgeHead !== this.checkpoint.recovery.beforeKnowledgeHead || canonicalJson(state.knowledge) !== canonicalJson(this.checkpoint.recovery.beforeKnowledge) || canonicalJson(recovery.sourceHeads) !== canonicalJson(initial.sourceHeads)) throw new Error('historical source recovery changed current knowledge or exact artifacts'); + Object.assign(this.checkpoint.recovery, recovery); await this.save(); + const released = await this.release('recovery', recovery.acceptedManifest); if (released.liveVersionObservation.variant !== 'initial') throw new Error('recovery deployment did not run historical variant'); return this.inspect(); + } + async inspect() { + this.phase = 'inspect'; const state = await this.state(); + const releases = await this.api('GET', `${this.root}/releases`); + const report = { schemaVersion: 1, mode: 'actual-isolated-cloud-api', controlWorker: this.config.controlWorker, releaseWorker: this.contracts.workerName, + organizationId: this.config.organizationId, projectId: this.checkpoint.projectId, resourceId: this.checkpoint.resourceId, + cycles: Object.fromEntries(Object.entries(this.checkpoint.cycles).map(([label, c]) => [label, { acceptedManifest: c.acceptedManifest ?? null, sourceHeads: c.sourceHeads ?? null, envelopeDigest: c.envelopeDigest ?? null, knowledgeHead: c.knowledgeHead ?? null, contextDigest: c.contextDigest ?? null, scenarioCount: c.evidence?.length ?? 0 }])), + releases: Object.fromEntries(Object.entries(this.checkpoint.releases).map(([label, r]) => [label, { releaseId: r.releaseId, manifestHash: r.manifestHash, acceptedVersionId: r.acceptedVersionId, outputDigest: r.outputDigest, envDigest: r.envDigest, healthStateAtObservation: r.healthState, freshUntil: r.freshUntil, liveVersionObservation: r.liveVersionObservation, deployedScenarios: r.deployedScenarios, recordedObservationId: r.recordedObservationId }])), + expiry: this.checkpoint.expiry ?? { result: 'not_run' }, recovery: this.checkpoint.recovery ? { proposalId: this.checkpoint.recovery.proposalId, acceptedManifest: this.checkpoint.recovery.acceptedManifest ?? null, sourceHeads: this.checkpoint.recovery.sourceHeads ?? null, knowledgePreserved: state.knowledgeHead === this.checkpoint.recovery.beforeKnowledgeHead } : null, + currentManifest: state.manifests.at(-1)?.hash, currentKnowledgeHead: state.knowledgeHead, verifierDigest: this.checkpoint.verifierDigest, runtimeRecipeDigest: this.checkpoint.runtimeRecipeDigest, observations: state.observations, + liveReleaseId: releases.live?.releaseId ?? null, + gates: { actualInitialCloudRelease: this.checkpoint.releases.initial?.healthState === 'healthy' ? 'pass' : 'not_run', actualFreshnessExpiry: this.checkpoint.expiry ? 'pass' : 'not_run', actualRecoveryCloudRelease: this.checkpoint.releases.recovery?.healthState === 'healthy' ? 'pass' : 'not_run' }, + noProductionTarget: true, newPersistentInfrastructureProvisioned: false, noExternalEffectsReversed: true }; + await writeFile(join(this.directory, 'public-report.json'), JSON.stringify(report, null, 2)); return report; + } +} +export async function cloudFixtureFiles(variant: 'initial' | 'later') { + let worker = await readFile(new URL('../../../fixtures/scheduling/worker.mjs', import.meta.url), 'utf8'); + worker = `export const LIVE_VARIANT = '${variant}';\n` + worker.replace('async fetch(request)', 'async fetch(request, env)'); + worker = worker.replace("if (url.pathname === '/health')", "if (url.pathname === '/version') return Response.json({ release: env.COMMONFORGE_RELEASE_ID ?? null, version: env.APP_VERSION, variant: LIVE_VARIANT });\n if (url.pathname === '/health')"); + const build = "import{mkdir,readFile,writeFile}from'node:fs/promises';await mkdir('dist',{recursive:true});const source=await readFile('worker.mjs','utf8');await writeFile('dist/worker.mjs',source.replaceAll('export const ','const ').replaceAll('export function ','function '));\n"; + return { 'worker.mjs': worker, 'build.mjs': build, 'wrangler.json': JSON.stringify({ main: 'worker.mjs', compatibility_date: '2026-10-01', vars: {} }) }; +} +async function main() { + const phase = process.argv[2], path = process.argv[3]; + if (!phase || !path || !['plan', 'setup', 'run', 'expiry', 'recover', 'inspect'].includes(phase)) throw new Error('usage: g1-cloud-rehearsal.ts plan|setup|run|expiry|recover|inspect '); + const config = G1CloudRehearsalConfigSchema.parse(JSON.parse(await readFile(resolve(path), 'utf8'))); + if (phase === 'plan') { process.stdout.write(JSON.stringify(cloudRehearsalPlan(config), null, 2) + '\n'); return; } + const rehearsal = new G1CloudRehearsal(config); await rehearsal.open(); + const result = await (rehearsal[phase as 'setup' | 'run' | 'expiry' | 'recover' | 'inspect'])(); + process.stdout.write(JSON.stringify(result, null, 2) + '\n'); +} +if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) void main().catch(error => { let message = error instanceof Error ? error.message : String(error); for (const secret of rememberedSecrets) message = message.replaceAll(secret, ''); process.stderr.write(message + '\n'); process.exitCode = 1; }); diff --git a/apps/runner/src/g1-cloudflare-server.ts b/apps/runner/src/g1-cloudflare-server.ts new file mode 100644 index 0000000..664eef3 --- /dev/null +++ b/apps/runner/src/g1-cloudflare-server.ts @@ -0,0 +1,52 @@ +/** Operator-owned Artifacts -> isolated build/workerd supervisor. Serve loopback behind TLS. */ +import { readFile } from 'node:fs/promises'; +import { resolve } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { z } from 'zod'; +import { BuildRecipeSchema, parseRevisionId } from '@commonforge-org/core'; +import { RestArtifactsTransport, type ArtifactsTransport } from '@commonforge-org/provider-cloudflare'; +import type { ScenarioVerificationRequest } from '@commonforge-org/runner-protocol'; +import { createCloudflareSchedulingAdapter } from './g1-cloudflare-source.js'; +import type { CloudflareReleaseEnvironment } from './g1-cloudflare-verifier.js'; +import { startLocalScenarioBridge } from './g1-local-bridge.js'; + +export const CloudflareScenarioRegistrationSchema = z.object({ + organizationId:z.string().min(1),projectId:z.string().min(1),resourceId:z.string().min(1), + namespace:z.string().min(1),repoName:z.string().min(1),repoId:z.string().min(1), + pathPrefix:z.string().regex(/^(?:[A-Za-z0-9_-]+\/)*$/).default(''), +}).strict(); +export type CloudflareScenarioRegistration=z.infer; +/** Paths, namespaces and repositories come only from operator registrations. */ +export function registeredArtifactsSource(transport:ArtifactsTransport,resources:CloudflareScenarioRegistration[]) { + return async (request:ScenarioVerificationRequest,paths:string[]):Promise>=>{ + if(request.sourceHeads.length!==1)throw new Error('one registered source required'); + const head=request.sourceHeads[0]!,revision=parseRevisionId(head.revisionId); + const r=resources.find(r=>r.organizationId===request.organizationId&&r.projectId===request.projectId&&r.resourceId===head.resourceId); + if(!r||revision.kind!=='art'||revision.repoId!==r.repoId)throw new Error('registered source scope mismatch'); + const repo=await transport.getRepo(r.namespace,r.repoName); + if(!repo||repo.id!==r.repoId)throw new Error('registered repository identity changed'); + const files:Record={}; + for(const path of paths){ + if(!['worker.mjs','build.mjs','wrangler.json'].includes(path))throw new Error('unsupported verifier source path'); + const bytes=await transport.readFile(r.namespace,r.repoName,revision.sha,r.pathPrefix+path); + if(!bytes||bytes.length>1048576)throw new Error('registered source missing or oversized'); + files[path]=bytes; + } + return files; + }; +} +export async function main():Promise { + const configPath=process.argv[2];if(!configPath)throw new Error('usage: tsx src/g1-cloudflare-server.ts '); + const config=z.object({port:z.number().int().min(1024).max(65535),accountId:z.string().regex(/^[a-f0-9]{32}$/), + pin:z.object({binary:z.string().min(1),sha256:z.string().regex(/^[a-f0-9]{64}$/)}).strict(),schema:z.string().min(1), + recipe:BuildRecipeSchema,environment:z.record(z.string(),z.unknown()),resources:z.array(CloudflareScenarioRegistrationSchema).min(1).max(100), + }).strict().parse(JSON.parse(await readFile(configPath,'utf8'))); + const apiToken=process.env.CF_API_TOKEN,secret=process.env.COMMONFORGE_RUNNER_JOB_KEY; + if(!apiToken||!secret||secret.length<32)throw new Error('CF_API_TOKEN and COMMONFORGE_RUNNER_JOB_KEY required in supervisor environment'); + const runtime=await createCloudflareSchedulingAdapter({...config,environment:config.environment as unknown as CloudflareReleaseEnvironment}); + const transport=new RestArtifactsTransport({accountId:config.accountId,apiToken}); + const bridge=await startLocalScenarioBridge({port:config.port,resources:[],runtime,jobKey:{kid:'k1',secret},readCandidateFiles:registeredArtifactsSource(transport,config.resources)}); + process.stdout.write(JSON.stringify({url:bridge.url,runtimeRecipeDigest:runtime.runtimeRecipeDigest,verifierDigest:runtime.verifierDigest,cloudDeployment:false})+'\n'); + for(const signal of ['SIGINT','SIGTERM'] as const)process.once(signal,()=>{void bridge.close();}); +} +if(process.argv[1]&&resolve(process.argv[1])===fileURLToPath(import.meta.url))void main().catch(()=>{process.stderr.write('Cloudflare scenario supervisor refused startup; validate private operator configuration and toolchain.\n');process.exit(1);}); diff --git a/apps/runner/src/g1-cloudflare-source.ts b/apps/runner/src/g1-cloudflare-source.ts new file mode 100644 index 0000000..4e0e953 --- /dev/null +++ b/apps/runner/src/g1-cloudflare-source.ts @@ -0,0 +1,111 @@ +import { CandidateExecutionError, candidateErrorResults } from './g1-scenario-failure.js'; +/** Bounded actual source -> build -> workerd adapter for the scheduling reference target. + * Deliberately not a general build system. Source files are fetched by the supervisor at the + * signed exact Git revision; the candidate cannot substitute a test harness or build receipt. */ +import { constants } from 'node:fs'; +import { spawn } from 'node:child_process'; +import { chmod, mkdtemp, open, readFile, readdir, realpath, rm, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { canonicalJson, sha256HexSync, type BuildRecipe } from '@commonforge-org/core'; +import { releaseOutputDigest, releaseTargetDigest, ReleaseTargetConfigSchema, type ReleaseBuildReceipt } from '@commonforge-org/runner-protocol'; +import { cases } from './g1-scheduling-harness.js'; +import { cloudflareRuntimeRecipeDigest, cloudflareSchedulingVerifierDigest, verifyCloudflareBuiltScheduling, type CloudflareReleaseEnvironment, type CloudflareWorkerdPin } from './g1-cloudflare-verifier.js'; + +export const CLOUDFLARE_SCHEDULING_SOURCE_FILES = ['worker.mjs', 'build.mjs', 'wrangler.json']; +export interface CloudflareSchedulingAdapterOptions { + pin: CloudflareWorkerdPin; + schema: string; + recipe: BuildRecipe; + environment: CloudflareReleaseEnvironment; +} +function validateRecipe(options: CloudflareSchedulingAdapterOptions): void { + const { recipe, environment } = options; + const step = recipe.steps[0]; + const target = environment.targets[0]; + if (recipe.steps.length !== 1 || !step || step.command !== 'node build.mjs' || step.env && Object.keys(step.env).length || + environment.targets.length !== 1 || !target || target.kind !== 'worker' || target.mountName !== step.mountName || target.outputPath !== 'dist' || target.main !== 'worker.mjs') { + throw new Error('Cloudflare scheduling adapter requires one node build.mjs step and dist/worker.mjs target'); + } +} +async function buildIsolated(files: Record): Promise { + if (Object.keys(files).sort().join(',') !== [...CLOUDFLARE_SCHEDULING_SOURCE_FILES].sort().join(',') || Object.values(files).some(f => f.length > 1048576)) throw new Error('bounded exact source files required'); + const scratch = await mkdtemp(join(tmpdir(), 'commonforge-g1-cloud-build-')); + await chmod(scratch, 0o700); + let child: ReturnType | undefined; let sandboxStarted = false; + try { + for (const path of CLOUDFLARE_SCHEDULING_SOURCE_FILES) await writeFile(join(scratch, path), files[path]!, { mode: 0o400 }); + const node = await realpath(process.execPath); + child = spawn('bwrap', ['--unshare-all', '--die-with-parent', '--new-session', '--cap-drop', 'ALL', + '--ro-bind', '/usr', '/usr', '--ro-bind', '/lib', '/lib', '--ro-bind', '/lib64', '/lib64', + '--ro-bind', node, '/node', '--proc', '/proc', '--dev', '/dev', '--tmpfs', '/tmp', + '--bind', scratch, '/candidate', '--chdir', '/candidate', '/usr/bin/sh', '-c', 'printf \'SANDBOX_STARTED\\n\'; exec /node build.mjs'], + { env: { PATH: '/usr/bin:/bin', HOME: '/tmp', TZ: 'UTC' }, stdio: ['ignore','pipe','ignore'], detached: true }); + let startupOutput=''; + child.stdout?.on('data', chunk => { startupOutput=(startupOutput+String(chunk)).slice(-1024); if(startupOutput.includes('SANDBOX_STARTED\n'))sandboxStarted=true; }); + await new Promise((resolve, reject) => { + const timer = setTimeout(() => { if (child?.pid) try { process.kill(-child.pid, 'SIGKILL'); } catch {} reject(sandboxStarted ? new CandidateExecutionError() : new Error('build isolation unavailable')); }, 10000); + child!.once('error', e => { clearTimeout(timer); reject(e); }); + child!.once('close', code => { clearTimeout(timer); code === 0 ? resolve() : reject(sandboxStarted ? new CandidateExecutionError() : new Error('build isolation unavailable')); }); + }); + // bwrap's PID namespace dies with its init; no candidate process survives to race this read. + const dist = join(scratch, 'dist'); + if ((await readdir(dist)).sort().join(',') !== 'worker.mjs') throw new Error('only one bundled Worker output is supported'); + if (await realpath(dist) !== dist) throw new Error('build output directory may not be a symlink'); + const handle = await open(join(dist, 'worker.mjs'), constants.O_RDONLY | constants.O_NOFOLLOW); + try { + const st = await handle.stat(); + if (!st.isFile() || st.nlink !== 1 || st.size > 1048576) throw new Error('build output must be a bounded plain file'); + const bytes = await handle.readFile(); + if (bytes.length > 1048576) throw new Error('build output exceeds bound'); + return bytes; + } finally { await handle.close(); } + } finally { + if (child?.pid) { try { process.kill(-child.pid, 'SIGKILL'); } catch {} + if (child.exitCode === null && child.signalCode === null) await new Promise(resolve => child!.once('exit', () => resolve())); } + await rm(scratch, { recursive: true, force: true }); + } +} + +/** Full environment is the same server-owned snapshot releases.ts hashes, not just target vars. + * Source-only cfg fields match releaseBuild's read-before-build rules; unsupported resources fail. + * The adapter actually builds candidate code in a namespace, then executes those output bytes in + * workerd. Production releaseBuild will rebuild; this does not assert identical future build bytes. */ +export async function createCloudflareSchedulingAdapter(options: CloudflareSchedulingAdapterOptions) { + // Snapshot operator configuration so later accidental mutations cannot change a pinned adapter. + options = JSON.parse(JSON.stringify(options)) as CloudflareSchedulingAdapterOptions; + validateRecipe(options); + const sourceSupervisorDigest = sha256HexSync(await readFile(new URL(import.meta.url))); + const verifierDigest = sha256HexSync(canonicalJson({ domain: 'cf.cloudflare-source-verifier/1', + workerdVerifierDigest: await cloudflareSchedulingVerifierDigest(options.pin, options.schema), + sourceSupervisorDigest, nodeDigest: sha256HexSync(await readFile(process.execPath)), nodeVersion: process.versions.node, architecture: process.arch })); + return { + runtimeRecipeDigest: cloudflareRuntimeRecipeDigest(options.recipe, options.environment), verifierDigest, actor: 'local-workerd-runner', + sourceFiles: [...CLOUDFLARE_SCHEDULING_SOURCE_FILES], scenarioIds: cases.map(c => c[0]), execution: 'bubblewrap-linux-http' as const, + async verify(input: Record) { + if (Object.keys(input).sort().join(',') !== [...CLOUDFLARE_SCHEDULING_SOURCE_FILES].sort().join(',') || Object.values(input).some(bytes => bytes.length > 1048576)) throw new Error('bounded exact source files required'); + const files = Object.fromEntries(Object.entries(input).map(([path, bytes]) => [path, Uint8Array.from(bytes)])); + const src: Record = JSON.parse(new TextDecoder('utf8', { fatal: true }).decode(files['wrangler.json']!)); + if (!src || typeof src !== 'object' || Array.isArray(src)) throw new Error('source config must be JSON object'); + const target = options.environment.targets[0]!; + const env = options.environment.environment[target.targetId] ?? { vars: {}, kvNamespaces: [] }; + const vars: Record = {}; + if (src.vars && typeof src.vars === 'object' && !Array.isArray(src.vars)) for (const [key, value] of Object.entries(src.vars)) if (['string','number','boolean'].includes(typeof value)) vars[key] = String(value); + const added = env.compatibilityFlags ?? []; + const flags = [...(Array.isArray(src.compatibility_flags) ? src.compatibility_flags : []).filter(f => !added.some(a => f === `no_${a}`)), ...added]; + const config = ReleaseTargetConfigSchema.parse({ compatibilityDate: src.compatibility_date, compatibilityFlags: [...new Set(flags)], + vars: { ...vars, ...env.vars, COMMONFORGE_RELEASE_ID: 'scenario-verification' }, kvNamespaces: env.kvNamespaces, + ...(env.d1Databases?.length ? { d1Databases: env.d1Databases } : {}) }); + let bytes: Uint8Array; + try { bytes = await buildIsolated(files); } catch (error) { + if (!(error instanceof CandidateExecutionError)) throw error; + return { verifierDigest, runtimeRecipeDigest: cloudflareRuntimeRecipeDigest(options.recipe, options.environment), execution: 'bubblewrap-linux-http' as const, network: 'none' as const, controlPlaneMounted: false as const, results: candidateErrorResults(cases.map(c => c[0])) }; + } + const outputFiles = [{ path: 'worker.mjs', sha256: sha256HexSync(bytes), size: bytes.length }]; + const targets = [{ targetId: target.targetId, files: outputFiles, config, digest: releaseTargetDigest(outputFiles, config) }]; + const built: ReleaseBuildReceipt = { jobId: 'scenario-build', releaseId: 'scenario-verification', outputDigest: releaseOutputDigest(targets), targets, builtAt: new Date().toISOString() }; + const verified = await verifyCloudflareBuiltScheduling({ ...options, built, readOutput: async path => path === 'worker.mjs' ? bytes : null }); + return { ...verified, verifierDigest }; + }, + }; +} diff --git a/apps/runner/src/g1-cloudflare-verifier.ts b/apps/runner/src/g1-cloudflare-verifier.ts new file mode 100644 index 0000000..c071f3a --- /dev/null +++ b/apps/runner/src/g1-cloudflare-verifier.ts @@ -0,0 +1,163 @@ +import { CandidateExecutionError, candidateErrorResults } from './g1-scenario-failure.js'; +/** Trusted scheduling harness against immutable Cloudflare release outputs in real workerd. + * No cloud deploy, binding emulation, candidate tests, Node fallback or credentials. */ +import { spawn } from 'node:child_process'; +import { chmod, lstat, mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'; +import { request } from 'node:http'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { BuildRecipeSchema, canonicalJson, hashCanonicalSync, sha256HexSync, type BuildRecipe } from '@commonforge-org/core'; +import { releaseOutputDigest, releaseTargetDigest, ReleaseTargetConfigSchema, type ReleaseBuildReceipt, type ReleaseTarget } from '@commonforge-org/runner-protocol'; +import { cases } from './g1-scheduling-harness.js'; +import type { LocalScenarioResult } from './g1-local-verifier.js'; + +/** Supervisor configuration, never candidate-provided. Pin the actual installed executable bytes. */ +export interface CloudflareWorkerdPin { binary: string; sha256: string } +export interface CloudflareReleaseEnvironment { + targets: ReleaseTarget[]; + environment: Record; kvNamespaces: unknown[]; d1Databases?: unknown[]; data?: string[]; compatibilityFlags?: string[] }>; + healthCheck: unknown; + accountId: string; + migrations?: unknown[]; +} +export interface CloudflareBuiltVerification { + verifierDigest: string; + runtimeRecipeDigest: string; + outputDigest: string; + targetDigest: string; + execution: 'bubblewrap-linux-http'; + engine: 'workerd'; + engineDigest: string; + network: 'none'; + controlPlaneMounted: false; + results: LocalScenarioResult[]; +} + +export function cloudflareRuntimeRecipeDigest(recipe: BuildRecipe, environment: CloudflareReleaseEnvironment): string { + BuildRecipeSchema.parse(recipe); + return hashCanonicalSync({ domain: 'cf.project-runtime/1', recipe, environment }); +} +export async function cloudflareSchedulingVerifierDigest(pin: CloudflareWorkerdPin, schema: string): Promise { + return hashCanonicalSync({ domain: 'cf.cloudflare-scheduling-verifier/2', cases, binaryDigest: pin.sha256, + supervisorDigest: sha256HexSync(await readFile(new URL(import.meta.url))), failureSupervisorDigest: sha256HexSync(await readFile(new URL(`./g1-scenario-failure.${import.meta.url.endsWith('.ts')?'ts':'js'}`,import.meta.url))), schemaDigest: sha256HexSync(await readFile(schema)) }); +} + +function capnpText(value: string): string { + // JSON string escaping is not identical to Cap'n Proto (notably \uXXXX); refuse unsupported controls. + if (/[\x00-\x1f\x7f]/.test(value)) throw new Error('unsupported control character in runtime configuration'); + return JSON.stringify(value); +} +function configuration(module: Uint8Array, config: ReturnType): string { + const text = new TextDecoder('utf-8', { fatal: true }).decode(module); + if (text.includes('\0')) throw new Error('unsupported null byte in Worker module'); + // Explicit empty internet allowlist. Even a workerd vulnerability remains + // confined by the surrounding mount/PID/network namespace and clean environment. + return `using Workerd = import "workerd.capnp";\nconst config :Workerd.Config = ( + services = [(name = "internet", network = (allow = [])), (name = "candidate", worker = ( + modules = [(name = "worker.mjs", esModule = embed "worker.mjs")], + compatibilityDate = ${capnpText(config.compatibilityDate)}, + compatibilityFlags = [${config.compatibilityFlags.map(capnpText).join(',')}], + bindings = [${Object.entries(config.vars).map(([name, value]) => `(name = ${capnpText(name)}, text = ${capnpText(value)})`).join(',')}] + ))], + sockets = [(name = "http", address = "unix:/scratch/preview.sock", http = (), service = "candidate")] + );\n`; +} +async function observe(socketPath: string, body: unknown): Promise { + return new Promise((resolve, reject) => { + const req = request({ socketPath, path: '/reschedule', method: 'POST', timeout: 1500, + headers: { host: 'fixture', 'content-type': 'application/json' } }, res => { + const chunks: Buffer[] = []; let size = 0; + res.on('data', (chunk: Buffer) => { size += chunk.length; if (size > 65536) res.destroy(new Error('response exceeds bound')); else chunks.push(chunk); }); + res.on('error', reject); + res.on('end', () => { try { if (res.statusCode !== 200) throw new Error('non-success response'); resolve(JSON.parse(Buffer.concat(chunks).toString())); } catch (e) { reject(e); } }); + }); + req.on('timeout', () => req.destroy(new Error('request timed out'))); req.on('error', reject); req.end(JSON.stringify(body)); + }); +} +async function waitForSocket(path: string, exited: () => boolean): Promise { + const deadline = Date.now() + 5000; + while (Date.now() < deadline) { + if (exited()) throw new Error('isolated workerd exited before startup'); + if ((await lstat(path).catch(() => null))?.isSocket()) return; + await new Promise(resolve => setTimeout(resolve, 20)); + } + throw new Error('isolated workerd startup timed out'); +} + +/** Receipt and bytes must come from the supervisor's real release-build job, not an API body. + * Narrow supported production contract: one already bundled ES Worker, vars only, no persistent + * bindings or migrations, no additional modules. Refuse instead of emulating unavailable services. + * Hashes verify built bytes; they do not themselves authenticate who produced the receipt. */ +export async function verifyCloudflareBuiltScheduling(options: { + pin: CloudflareWorkerdPin; + /** The schema shipped alongside the pinned workerd installation, supervisor-owned. */ + schema: string; + recipe: BuildRecipe; + environment: CloudflareReleaseEnvironment; + built: ReleaseBuildReceipt; + readOutput(path: string): Promise; +}): Promise { + const { pin, recipe, environment, built } = options; + if (!/^[a-f0-9]{64}$/.test(pin.sha256)) throw new Error('workerd binary requires an exact digest'); + if (environment.targets.length !== 1 || built.targets.length !== 1 || (environment.migrations?.length ?? 0)) throw new Error('unsupported Cloudflare target set or migrations'); + const target = environment.targets[0]!; + const receipt = built.targets[0]!; + if (target.kind !== 'worker' || !target.main || receipt.targetId !== target.targetId || receipt.files.length !== 1 || receipt.files[0]!.path !== target.main || !/\.(?:mjs|js)$/.test(target.main) || (receipt.dataFiles?.length ?? 0)) throw new Error('only one bundled Worker output is supported'); + const config = ReleaseTargetConfigSchema.parse(receipt.config); + const env = environment.environment[target.targetId] ?? { vars: {}, kvNamespaces: [] }; + if (Object.keys(environment.environment).some(k => k !== target.targetId) || env.kvNamespaces.length || env.d1Databases?.length || env.data?.length || config.kvNamespaces.length || config.d1Databases?.length || config.notFoundHandling) throw new Error('persistent bindings and assets are unsupported'); + if (Object.entries(env.vars).some(([key, value]) => config.vars[key] !== value) || (env.compatibilityFlags ?? []).some(flag => !config.compatibilityFlags.includes(flag))) throw new Error('release environment/config mismatch'); + if (config.vars.COMMONFORGE_RELEASE_ID !== built.releaseId) throw new Error('release identity mismatch'); + if (releaseTargetDigest(receipt.files, config, receipt.dataFiles ?? []) !== receipt.digest || releaseOutputDigest(built.targets) !== built.outputDigest) throw new Error('release output digest mismatch'); + const file = receipt.files[0]!; + if (file.size > 1048576) throw new Error('Worker output exceeds bound'); + const output = await options.readOutput(file.path); + if (!output || output.length !== file.size || sha256HexSync(output) !== file.sha256) throw new Error('release output bytes mismatch'); + const executable = await readFile(pin.binary); + if (sha256HexSync(executable) !== pin.sha256) throw new Error('installed workerd differs from supervisor pin'); + const schemaBytes = await readFile(options.schema); + const verifierDigest = await cloudflareSchedulingVerifierDigest(pin, options.schema); + const scratch = await mkdtemp(join(tmpdir(), 'commonforge-g1-workerd-')); + await chmod(scratch, 0o700); + let child: ReturnType | undefined; + try { + // Snapshot all executable/configuration inputs before launch; no path race with candidate files. + await mkdir(join(scratch, 'socket'), { mode: 0o700 }); + await writeFile(join(scratch, 'workerd'), executable, { mode: 0o500 }); + await writeFile(join(scratch, 'workerd.capnp'), schemaBytes, { mode: 0o400 }); + await writeFile(join(scratch, 'worker.mjs'), output, { mode: 0o400 }); + await writeFile(join(scratch, 'config.capnp'), configuration(output, config), { mode: 0o400 }); + child = spawn('bwrap', ['--unshare-all', '--die-with-parent', '--new-session', '--cap-drop', 'ALL', + '--ro-bind', '/usr', '/usr', '--ro-bind', '/lib', '/lib', '--ro-bind', '/lib64', '/lib64', + '--proc', '/proc', '--dev', '/dev', '--tmpfs', '/tmp', + '--ro-bind', join(scratch, 'workerd'), '/workerd', '--ro-bind', join(scratch, 'workerd.capnp'), '/candidate/workerd.capnp', + '--ro-bind', join(scratch, 'worker.mjs'), '/candidate/worker.mjs', '--ro-bind', join(scratch, 'config.capnp'), '/candidate/config.capnp', + '--bind', join(scratch, 'socket'), '/scratch', '--chdir', '/candidate', '/usr/bin/sh', '-c', 'printf \'SANDBOX_STARTED\\n\'; exec /workerd serve /candidate/config.capnp'], + { env: { PATH: '/usr/bin:/bin', HOME: '/tmp', TZ: 'UTC' }, stdio: ['ignore', 'pipe', 'pipe'], detached: true }); + let failure: Error | undefined; let sandboxStarted = false; + let startupOutput=''; + child.stdout?.on('data', chunk => { startupOutput=(startupOutput+String(chunk)).slice(-1024); if(startupOutput.includes('SANDBOX_STARTED\n'))sandboxStarted=true; }); + child.once('error', e => { failure = e; }); + // Drain bounded diagnostics without exposing candidate-controlled logs or secrets in results. + child.stderr?.on('data', () => {}); + try { await waitForSocket(join(scratch, 'socket', 'preview.sock'), () => !!failure || child!.exitCode !== null || child!.signalCode !== null); } catch (error) { if (sandboxStarted) throw new CandidateExecutionError(); throw error; } + const results: LocalScenarioResult[] = []; + for (const [scenarioId, input, expected] of cases) { + try { + const actual = await observe(join(scratch, 'socket', 'preview.sock'), { now: '2026-10-17T12:00:00.000Z', ...input }); + const pass = canonicalJson(actual) === canonicalJson(expected); + results.push({ scenarioId, result: pass ? 'pass' : 'fail', detail: pass ? 'expected workerd HTTP behavior observed' : 'workerd response differs from trusted expectation' }); + } catch { results.push({ scenarioId, result: 'error', detail: 'isolated workerd HTTP request failed' }); } + } + return { verifierDigest, runtimeRecipeDigest: cloudflareRuntimeRecipeDigest(recipe, environment), outputDigest: built.outputDigest, + targetDigest: receipt.digest, execution: 'bubblewrap-linux-http', engine: 'workerd', engineDigest: pin.sha256, + network: 'none', controlPlaneMounted: false, results }; + } catch (error) { + if (!(error instanceof CandidateExecutionError)) throw error; + return { verifierDigest, runtimeRecipeDigest: cloudflareRuntimeRecipeDigest(recipe, environment), outputDigest: built.outputDigest, targetDigest: receipt.digest, execution: 'bubblewrap-linux-http', engine: 'workerd', engineDigest: pin.sha256, network: 'none', controlPlaneMounted: false, results: candidateErrorResults(cases.map(c => c[0])) }; + } finally { + if (child?.pid) { try { process.kill(-child.pid, 'SIGKILL'); } catch { /* already exited */ } + if (child.exitCode === null && child.signalCode === null) await new Promise(resolve => child!.once('exit', () => resolve())); } + await rm(scratch, { recursive: true, force: true }); + } +} diff --git a/apps/runner/src/g1-dev-worker.ts b/apps/runner/src/g1-dev-worker.ts new file mode 100644 index 0000000..015da4c --- /dev/null +++ b/apps/runner/src/g1-dev-worker.ts @@ -0,0 +1,91 @@ +/** Start a disposable real-local Worker + Node source sidecar + pull runner. */ +import { spawn } from 'node:child_process'; +import { randomBytes } from 'node:crypto'; +import { mkdtemp, readFile, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join, resolve } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { startArtifactsSidecar } from './g1-artifacts-sidecar.js'; +import { startLocalScenarioBridge } from './g1-local-bridge.js'; +import { startLocalPreview } from './g1-local-preview.js'; +import { localNamespaceCommands } from './g1-bwrap-commands.js'; +import { brokerCredentials } from './credentials.js'; +import { createRunner } from './runner.js'; +import { createWorkerClient } from './worker-client.js'; +import { workerBlobStore } from './blob-store.js'; + +async function command(args: string[], cwd: string) { + await new Promise((resolve, reject) => { const child = spawn('pnpm', args, { cwd, stdio: ['ignore', process.stderr, process.stderr] }); child.once('error', reject); child.once('exit', code => code === 0 ? resolve() : reject(new Error(`command exited ${code}`))); }); +} +async function main() { + const repo = fileURLToPath(new URL('../../../', import.meta.url)), workerDir = join(repo, 'apps/worker'); + const root = process.env.G1_STATE_DIR ? resolve(process.env.G1_STATE_DIR) : await mkdtemp(join(tmpdir(), 'commonforge-g1-real-worker-')); + const previous = await readFile(join(root, 'worker.json'), 'utf8').then(s => JSON.parse(s) as { vars: Record }, () => null); + const priorOperator = previous ? JSON.parse(await readFile(join(root, 'operator.json'), 'utf8')) as { sidecarUrl: string; sidecarKey: string; bridgeUrl: string; jobKey: {kid:string;secret:string} } : null; + const port = Number(process.env.PORT ?? 8899), previewPort = Number(process.env.PREVIEW_PORT ?? 8900); + if (!Number.isInteger(port) || port <= 1024 || port > 65535 || !Number.isInteger(previewPort) || previewPort <= 1024 || previewPort > 65535 || port === previewPort) throw new Error('PORT/PREVIEW_PORT must be distinct unprivileged ports'); + const workerUrl = `http://127.0.0.1:${port}`, previewUrl = `http://127.0.0.1:${previewPort}`; + for (const url of [workerUrl, previewUrl]) { + try { const r = await fetch(`${url}/health`, { signal: AbortSignal.timeout(500) }); await r.body?.cancel(); throw new Error(`port already occupied: ${url}`); } + catch (e) { if (e instanceof Error && e.message.startsWith('port already')) throw e; } + } + const random = () => randomBytes(32).toString('hex'), key = priorOperator?.sidecarKey ?? random(), runnerBearer = previous?.vars.COMMONFORGE_RUNNER_TOKEN ?? random(); + const jobKey = priorOperator?.jobKey ?? { kid: 'k1', secret: random() }, namespacePrefix = 'commonforge-real'; + const sidecar = await startArtifactsSidecar({ stateDir: join(root, 'source'), key, port: priorOperator ? Number(new URL(priorOperator.sidecarUrl).port) : undefined }); + const bridge = await startLocalScenarioBridge({ provider: sidecar.provider, jobKey, resources: [], port: priorOperator ? Number(new URL(priorOperator.bridgeUrl).port) : undefined, readCandidate: async request => { + const head = request.sourceHeads[0]!; return sidecar.artifacts.candidateData(request.organizationId, head.resourceId, head.revisionId, namespacePrefix); + } }); + const previewKey = previous?.vars.COMMONFORGE_LOCAL_PREVIEW_KEY ?? random(); + const localPreview = await startLocalPreview({ key: previewKey, port: previous?.vars.COMMONFORGE_LOCAL_PREVIEW_URL ? Number(new URL(previous.vars.COMMONFORGE_LOCAL_PREVIEW_URL).port) : undefined }); + const vars = { + COMMONFORGE_NAMESPACE_PREFIX: namespacePrefix, COMMONFORGE_ASSETS_BUCKET: 'commonforge-assets-real', + COMMONFORGE_APP_URL: workerUrl, COMMONFORGE_PUBLIC_URL: workerUrl, COMMONFORGE_EMAIL_FROM: 'login@example.test', COMMONFORGE_EMAIL_MODE: 'log', + COMMONFORGE_ADMIN_EMAILS: process.env.ADMIN_EMAIL ?? 'owner@agency.e2e.test', COMMONFORGE_DEPLOY_WORKER_PREFIX: 'cfapp', + COMMONFORGE_SESSION_SECRET: previous?.vars.COMMONFORGE_SESSION_SECRET ?? random(), COMMONFORGE_RUNNER_JOB_KEY: jobKey.secret, COMMONFORGE_RUNNER_TOKEN: runnerBearer, + COMMONFORGE_LOCAL_ARTIFACTS_URL: sidecar.url, COMMONFORGE_LOCAL_ARTIFACTS_KEY: key, COMMONFORGE_LOCAL_RUNNER_URL: bridge.url, COMMONFORGE_LOCAL_PREVIEW_URL: localPreview.url, COMMONFORGE_LOCAL_PREVIEW_KEY: previewKey, + COMMONFORGE_PREVIEW_URL: previewUrl, COMMONFORGE_PREVIEW_KEY: previous?.vars.COMMONFORGE_PREVIEW_KEY ?? random(), CF_ACCOUNT_ID: '00000000000000000000000000000000', CF_API_TOKEN: 'local-only-no-cloud-token', + }; + const bindings = { compatibility_date: '2026-10-01', compatibility_flags: ['nodejs_compat'], workers_dev: false, + r2_buckets: [{ binding: 'COMMONFORGE_ASSETS', bucket_name: 'commonforge-assets-real' }], vars }; + const config = { ...bindings, name: 'commonforge-real-local', main: join(workerDir, 'e2e/real-stack.ts'), + assets: { directory: join(repo, 'apps/web/dist'), not_found_handling: 'single-page-application', run_worker_first: ['/api/*', '/health'] }, + durable_objects: { bindings: [{ name: 'ORG_LEDGER', class_name: 'OrgLedger' }, { name: 'OPS_GATE', class_name: 'OpsGate' }] }, + migrations: [{ tag: 'v1', new_sqlite_classes: ['OrgLedger'] }, { tag: 'v2', new_sqlite_classes: ['OpsGate'] }], + d1_databases: [{ binding: 'DB', database_name: 'commonforge-real', database_id: '00000000-0000-4000-8000-0000000000e3', migrations_dir: join(workerDir, 'migrations') }], + queues: { producers: [{ binding: 'OUTBOX', queue: 'commonforge-real-outbox' }, { binding: 'STEPS', queue: 'commonforge-real-steps' }, { binding: 'RUNNER_JOBS', queue: 'commonforge-real-runner-jobs' }], consumers: [ { queue: 'commonforge-real-outbox', max_batch_size: 4, max_batch_timeout: 1 }, { queue: 'commonforge-real-steps', max_batch_size: 4, max_batch_timeout: 1 }, { queue: 'commonforge-real-runner-jobs', max_batch_size: 1, max_batch_timeout: 1 }] }, + }; + const configPath = join(root, 'worker.json'), previewConfig = join(root, 'preview.json'), persist = join(root, 'miniflare'); + await writeFile(configPath, JSON.stringify(config, null, 2), { mode: 0o600 }); + await writeFile(previewConfig, JSON.stringify({ ...bindings, vars: { COMMONFORGE_PREVIEW_KEY: vars.COMMONFORGE_PREVIEW_KEY, COMMONFORGE_LOCAL_PREVIEW_URL: localPreview.url, COMMONFORGE_LOCAL_PREVIEW_KEY: previewKey }, name: 'commonforge-real-preview', main: join(workerDir, 'e2e/real-preview.ts') }, null, 2), { mode: 0o600 }); + await writeFile(join(root, 'operator.json'), JSON.stringify({ sidecarUrl: sidecar.url, sidecarKey: key, bridgeUrl: bridge.url, jobKey, workerUrl, namespacePrefix }), { mode: 0o600 }); + const baseArgs = ['--filter', '@commonforge-org/worker', 'exec', 'wrangler']; + await command([...baseArgs, 'd1', 'migrations', 'apply', 'commonforge-real', '--local', '-c', configPath, '--persist-to', persist, '--env-file', join(workerDir, 'e2e/no-secrets.env')], repo); + // Both processes need their own ephemeral inspector port as well as distinct HTTP ports. + const start = (path: string, listen: number) => spawn('pnpm', [...baseArgs, 'dev', '-c', path, '--persist-to', persist, '--env-file', join(workerDir, 'e2e/no-secrets.env'), '--ip', '127.0.0.1', '--port', String(listen), '--inspector-port', '0', '--show-interactive-dev-session=false'], { cwd: repo, stdio: ['ignore', 'pipe', 'pipe'] }); + const child = start(configPath, port), preview = start(previewConfig, previewPort); + for (const c of [child, preview]) { c.stdout.pipe(process.stderr); c.stderr.pipe(process.stderr); } + let stopped = false; + const stop = async () => { if (stopped) return; stopped = true; child.kill('SIGTERM'); preview.kill('SIGTERM'); await bridge.close(); await localPreview.close(); await sidecar.close(); }; + process.once('SIGINT', () => { void stop(); }); process.once('SIGTERM', () => { void stop(); }); + for (const [name, process] of [['API', child], ['preview', preview]] as const) { + process.once('exit', (code, signal) => { + if (!stopped) console.error(`G1 ${name} Worker exited (code=${code}, signal=${signal}); stopping the local stack`); + void stop(); + }); + } + const headers = { authorization: `Bearer ${runnerBearer}` }, client = createWorkerClient({ workerUrl, headers, attempts: 2 }); + const runner = createRunner({ workDir: join(root, 'runner'), journalDir: join(root, 'journal'), trustedWorkerOrigin: workerUrl, + credentials: brokerCredentials({ headers }), jobKeyring: [jobKey], git: { allowHttp: true }, + commands: await localNamespaceCommands(join(root, 'sandbox-bin')), evidenceKey: job => client.evidenceKey(job), currentHeads: job => client.heads(job), + blobStore: workerBlobStore({ workerUrl, headers }), previewBaseUrl: previewUrl, runnerId: 'real-local-pull-runner', log: line => process.stderr.write(`${line}\n`) }); + const report = { mode: 'real-local-worker', pid: process.pid, root, workerUrl, previewUrl, sourceUrl: sidecar.url, bridgeUrl: bridge.url, operatorFile: join(root, 'operator.json'), + real: ['Worker API', 'SQLite Durable Objects', 'D1', 'local R2', 'Git smart HTTP', 'Node pull runner', 'isolated scheduling scenarios', 'static and isolated API previews'], + limitations: ['Cloud release is unavailable.', 'Only Node build/check entrypoints are supported; no dependency installs.', 'API previews support standalone Node-compatible modules and fresh synthetic state per request; Workers-only runtime APIs require cloud verification.'] }; + await writeFile(join(root, 'report.json'), JSON.stringify(report, null, 2)); process.stdout.write(`${JSON.stringify(report)}\n`); + while (!stopped) { + try { const delivery = await client.nextJob('real-local-pull-runner'); if (delivery) { const result = await runner.execute(delivery); const job = runner.lastVerified.get(result.jobId); if (job) await client.postResult(job, result); } } + catch (e) { if (!stopped) process.stderr.write(`[real-local runner] ${e instanceof Error ? e.message : String(e)}\n`); } + if (!stopped) await new Promise(resolve => setTimeout(resolve, 250)); + } +} +if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) void main().catch(e => { process.stderr.write(`${e instanceof Error ? e.stack : String(e)}\n`); process.exit(1); }); diff --git a/apps/runner/src/g1-fresh-session.ts b/apps/runner/src/g1-fresh-session.ts new file mode 100644 index 0000000..130af05 --- /dev/null +++ b/apps/runner/src/g1-fresh-session.ts @@ -0,0 +1,35 @@ +/** Prepare scoped persisted inputs for a real, separately invoked agent. Does NOT simulate an agent. */ +import { createHash } from 'node:crypto'; +import { mkdir, mkdtemp, readFile, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { createGit } from './git.js'; +const session = JSON.parse(await readFile(process.argv[2]!, 'utf8')) as { ownerToken: string; organizationId: string; projectId: string }; +const operator = JSON.parse(await readFile(process.argv[3]!, 'utf8')) as { workerUrl: string }; +async function api(method: string, path: string, body?: unknown, token = session.ownerToken): Promise { + const res = await fetch(`${operator.workerUrl}/api${path}`, { method, headers: { authorization: `Bearer ${token}`, ...(body === undefined ? {} : { 'content-type': 'application/json' }) }, ...(body === undefined ? {} : { body: JSON.stringify(body) }) }); + const data = await res.json(); if (!res.ok) throw new Error(`${method} ${path}: ${res.status} ${JSON.stringify(data)}`); return data; +} +const taskText = 'Add supportContact metadata to health response with value support@example.test; keep existing project behavior'; +const root = await mkdtemp(join(tmpdir(), 'g1-fresh-agent-')); +const task = await api('POST', `/orgs/${session.organizationId}/projects/${session.projectId}/tasks`, { title: 'Add support contact to fixture health', request: taskText, writeMounts: ['app'] }); +let ready: any; const end = Date.now() + 60000; +while (Date.now() < end) { ready = await api('GET', `/tasks/${task.taskId}`); if (ready.state === 'ready') break; await new Promise(r => setTimeout(r, 250)); } +if (ready.state !== 'ready') throw new Error('task source seeding timed out'); +const handoff = await api('POST', `/tasks/${task.taskId}/native/handoff`, { agentId: 'actual-fresh-session-codex' }); +const agent = await api('POST', '/auth/task/verify', { code: handoff.code }); +const context = await api('GET', `/tasks/${task.taskId}/native/context`, undefined, agent.leaseToken); +const credential = await api('POST', `/tasks/${task.taskId}/credential`, { mountName: 'app', access: 'write' }, agent.leaseToken); +const work = join(root, 'workspace'), git = createGit({ allowHttp: true }); +const env = { GIT_CONFIG_COUNT: '1', GIT_CONFIG_KEY_0: `http.${credential.remoteUrl}.extraHeader`, GIT_CONFIG_VALUE_0: `Authorization: Bearer ${credential.secret}` }; +await git.ok(['clone', '-q', credential.remoteUrl, work], root, { env }); +await mkdir(join(work, '.commonforge'), { recursive: true }); +const inputs = { 'task.md': taskText + '\n', '.commonforge/context.json': context.context.json, '.commonforge/context.md': context.context.markdown }; +for (const [path, text] of Object.entries(inputs)) await writeFile(join(work, path), text); +const inputDigests = Object.fromEntries(Object.entries(inputs).map(([path, text]) => [path, createHash('sha256').update(text).digest('hex')])); +const sourceFile = 'fixtures/scheduling/worker.mjs'; +const sourceDigest = createHash('sha256').update(await readFile(join(work, sourceFile))).digest('hex'); +const report = { root, workspace: work, taskId: task.taskId, organizationId: session.organizationId, projectId: session.projectId, allowedInputFiles: Object.keys(inputs), allowedImplementationPath: sourceFile, inputDigests, implementationBeforeDigest: sourceDigest, contextDigest: context.context.digest, agentIdentity: null, freshAgentExecuted: false }; +await writeFile(join(root, 'agent-control.json'), JSON.stringify({ ...report, credential, agent, ownerSessionFile: process.argv[2], workerUrl: operator.workerUrl, workspaceId: ready.workspaces.find((w: any) => w.mountName === 'app').taskWorkspaceId }), { mode: 0o600 }); +await writeFile(join(root, 'report.json'), JSON.stringify(report, null, 2)); +process.stdout.write(`${JSON.stringify(report, null, 2)}\n`); diff --git a/apps/runner/src/g1-fresh-submit.ts b/apps/runner/src/g1-fresh-submit.ts new file mode 100644 index 0000000..f166a2e --- /dev/null +++ b/apps/runner/src/g1-fresh-submit.ts @@ -0,0 +1,43 @@ +/** Supervisor submission AFTER an actual external fresh agent edited its allowed source file. */ +import { createHash } from 'node:crypto'; +import { readFile, writeFile } from 'node:fs/promises'; +import { dirname, join } from 'node:path'; +import { createGit, RUNNER_IDENTITY_ENV } from './git.js'; +import { inspectIsolatedHealth } from './g1-local-verifier.js'; +const controlFile = process.argv[2]!, control = JSON.parse(await readFile(controlFile,'utf8')); +const owner = JSON.parse(await readFile(control.ownerSessionFile,'utf8')); +const agentIdentity = process.argv[3]; +if (!agentIdentity || !/^\/[A-Za-z0-9_/-]+$/.test(agentIdentity)) throw new Error('record the actual independently invoked collaboration agent identity'); +async function api(method:string,path:string,body?:unknown,token=owner.ownerToken):Promise{ + const response=await fetch(`${control.workerUrl}/api${path}`,{method,headers:{authorization:`Bearer ${token}`,...(body===undefined?{}:{'content-type':'application/json'})},...(body===undefined?{}:{body:JSON.stringify(body)})}); + const data=await response.json();if(!response.ok)throw new Error(`${method} ${path}: ${response.status} ${JSON.stringify(data)}`);return data; +} +async function waitFor(path:string,ready:(x:any)=>boolean){const end=Date.now()+60000;while(Date.now()setTimeout(r,250));}throw new Error('real integration timed out');} +const git=createGit({allowHttp:true}),source=join(control.workspace,control.allowedImplementationPath); +for(const [path,digest]of Object.entries(control.inputDigests))if(createHash('sha256').update(await readFile(join(control.workspace,path))).digest('hex')!==digest)throw new Error('fresh agent altered pinned task/context inputs'); +const afterDigest=createHash('sha256').update(await readFile(source)).digest('hex');if(afterDigest===control.implementationBeforeDigest)throw new Error('fresh source was not changed'); +const changed=(await git.ok(['diff','--name-only'],control.workspace)).trim().split('\n').filter(Boolean);if(JSON.stringify(changed)!==JSON.stringify([control.allowedImplementationPath]))throw new Error('fresh agent changed source outside permitted file'); +const diff=await git.ok(['diff','--',control.allowedImplementationPath],control.workspace); +const health=await inspectIsolatedHealth(dirname(source)) as {supportContact?:string;ruleVersion?:number}; +if(health.supportContact!=='support@example.test'||health.ruleVersion!==2)throw new Error('fresh implementation health behavior did not match fixed task'); +await git.ok(['add','--',control.allowedImplementationPath],control.workspace);await git.ok(['commit','-q','-m','Fresh scoped agent adds support contact while preserving confirmed rules'],control.workspace,{env:RUNNER_IDENTITY_ENV(new Date().toISOString())}); +const sha=(await git.ok(['rev-parse','HEAD'],control.workspace)).trim(),cred=control.credential; +await git.ok(['push','-q','origin','HEAD:refs/heads/main'],control.workspace,{env:{GIT_CONFIG_COUNT:'1',GIT_CONFIG_KEY_0:`http.${cred.remoteUrl}.extraHeader`,GIT_CONFIG_VALUE_0:`Authorization: Bearer ${cred.secret}`}}); +const candidate=await api('POST',`/tasks/${control.taskId}/workspaces/${control.workspaceId}/freeze`,{expectedSha:sha,tool:'actual-fresh-codex-session'},control.agent.leaseToken); +const inner=await api('POST',`/orgs/${control.organizationId}/proposals`,{taskId:control.taskId,candidateIds:[candidate.candidateId]}); +const innerPath=`/orgs/${control.organizationId}/proposals/${inner.proposalId}`;await waitFor(innerPath,p=>p.state==='prepared');await api('POST',`${innerPath}/integrate`,{});const integrated=await waitFor(innerPath,p=>!!p.integration); +const native=`/orgs/${control.organizationId}/projects/${control.projectId}/native`; +let state=(await api('GET',native)).state; +const outer=await api('POST',`/tasks/${control.taskId}/native/proposals`,{innerProposalId:inner.proposalId,expectedGeneration:state.generation,idempotencyKey:`fresh-outer-${control.taskId}`},control.agent.leaseToken); +const verification=await api('POST',`${native}/proposals/${encodeURIComponent(outer.proposal.id)}/verify`,{idempotencyKey:`fresh-verify-${control.taskId}`});state=verification.state; +if(verification.evidence.length!==9||!verification.evidence.every((e:any)=>e.result==='pass'&&e.provenance==='runner-verified'))throw new Error('fresh source did not preserve every trusted scenario'); +const me=await api('GET','/me'),decisionId=`fresh-decision-${Date.now()}`; +const decision={schemaVersion:1,organizationId:control.organizationId,projectId:control.projectId,id:decisionId,author:me.user.userId,createdAt:new Date().toISOString(),envelopeDigest:outer.proposal.digest,innerProposalHash:outer.proposal.envelope.innerProposalHash,action:'approve',explanation:'Synthetic owner API decision after inspecting exact fresh source diff, independently executed scheduling evidence and isolated supportContact response.',authorityRevision:state.authorities.find((a:any)=>a.principalId===me.user.userId).revision}; +state=(await api('POST',`${native}/commands`,{type:'recordDecision',record:decision,expectedGeneration:state.generation,idempotencyKey:decisionId})).state; +state=(await api('POST',`${native}/commands`,{type:'accept',proposalId:outer.proposal.id,decisionId,expectedGeneration:state.generation,idempotencyKey:`accept-${decisionId}`})).state; +const report={mode:'actual-fresh-agent-real-local-api',agent:{canonicalIdentity:agentIdentity,facility:'Codex collaboration.spawn_agent',fork_turns:'none',model:'inherited default; independent model identifier not exposed by collaboration API',crossVendorClaim:false,originalConversationProvided:false,humanCorrectionRestated:false}, + input:JSON.parse(await readFile(join(control.root,'report.json'),'utf8')),implementationAfterDigest:afterDigest,sourceRevision:sha,proposalId:outer.proposal.id,envelopeDigest:outer.proposal.digest,acceptedManifest:state.manifests.at(-1).hash,acceptedSourceHeads:state.sourceHeads, + supplementalIsolatedHealth:health,scenarioEvidence:verification.evidence,artifactIntegration:integrated.integration,decision:{id:decisionId,actor:'synthetic owner role through API; not an observed human usability session'}, + gates:{freshAgentExecution:'pass',correctedRulePreserved:'pass',realGit:'pass',signedRunnerIntegration:'pass',independentScenarios:'pass',supplementalHealthBehavior:'pass',atomicAcceptance:'pass',cloudRelease:'not_run'}}; +await writeFile(join(control.root,'implementation.diff'),diff);await writeFile(join(control.root,'result.json'),JSON.stringify(report,null,2)); +process.stdout.write(`${JSON.stringify({reportFile:join(control.root,'result.json'),diffFile:join(control.root,'implementation.diff'),sourceRevision:sha,contextDigest:control.contextDigest,acceptedManifest:report.acceptedManifest,gates:report.gates},null,2)}\n`); diff --git a/apps/runner/src/g1-local-bridge.ts b/apps/runner/src/g1-local-bridge.ts new file mode 100644 index 0000000..0cc03d3 --- /dev/null +++ b/apps/runner/src/g1-local-bridge.ts @@ -0,0 +1,91 @@ +/** Local-only, signed G1 verifier bridge over real LocalGitProvider revisions. */ +import { createServer, type Server } from 'node:http'; +import { mkdtemp, rm, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { hashCanonicalSync, type RevisionId, type ScenarioEvidence } from '@commonforge-org/core'; +import { LocalGitProvider } from '@commonforge-org/provider-local-git'; +import { signScenarioResult, verifyScenarioRequest, type JobSigningKey } from '@commonforge-org/runner-protocol'; +import { schedulingVerifierDigest, verifySchedulingCandidate, type LocalScenarioResult } from './g1-local-verifier.js'; +import { cases } from './g1-scheduling-harness.js'; + +export const SCHEDULING_RUNTIME_RECIPE_DIGEST = hashCanonicalSync({ domain: 'cf.local-scheduling-runtime/1', runtime: 'node-http', nodeVersion: process.versions.node, architecture: process.arch, isolation: 'bubblewrap-linux', clock: '2026-10-17T12:00:00.000Z', timezone: 'UTC' }); +export interface ScenarioRuntimeAdapter { + actor?: string; runtimeRecipeDigest: string; verifierDigest: string; scenarioIds: string[]; sourceFiles: string[]; + verify(files: Record): Promise<{verifierDigest?:string;runtimeRecipeDigest?:string;results:LocalScenarioResult[];execution:'bubblewrap-linux-http';network:'none';controlPlaneMounted:false}>; +} +export interface LocalBridgeOptions { + provider?: LocalGitProvider; jobKey: JobSigningKey; port?: number; + /** Operator config, not caller-selected local paths. */ + readCandidate?: (request: import('@commonforge-org/runner-protocol').ScenarioVerificationRequest) => Promise; + /** Pinned operator runtime, never populated from an incoming request. */ + runtime?: ScenarioRuntimeAdapter; + readCandidateFiles?: (request: import('@commonforge-org/runner-protocol').ScenarioVerificationRequest, paths: string[]) => Promise>; + resources: Array<{ organizationId: string; projectId: string; resourceId: string; workerPath?: string }>; +} +export async function startLocalScenarioBridge(options: LocalBridgeOptions): Promise<{ server: Server; url: string; close(): Promise }> { + const verifierDigest = options.runtime?.verifierDigest ?? await schedulingVerifierDigest(); + const runtimeRecipeDigest=options.runtime?.runtimeRecipeDigest??SCHEDULING_RUNTIME_RECIPE_DIGEST; + const scenarioIds=options.runtime?.scenarioIds??cases.map(c=>c[0]); + const runs = new Map }>(); + const server = createServer((req, res) => { + const send = (status: number, data: unknown) => { res.writeHead(status, { 'content-type': 'application/json', 'cache-control': 'no-store' }); res.end(JSON.stringify(data)); }; + void (async () => { + if (req.headers.origin) return send(403, { error: 'browser requests refused' }); + if (req.method === 'GET' && (req.url === '/health' || req.url === '/capabilities')) return send(200, { + mode: 'real-local', protocolVersion: 1, verifierDigest, runtimeRecipeDigest, + scenarioIds, isolation: 'bubblewrap-linux-http', cloudRelease: false, + }); + if (req.method !== 'POST' || req.url !== '/verify') return send(404, { error: 'not_found' }); + const chunks: Buffer[] = []; let bytes = 0; + for await (const chunk of req) { bytes += (chunk as Buffer).length; if (bytes > 262144) return send(413, { error: 'body_too_large' }); chunks.push(chunk as Buffer); } + const request = await verifyScenarioRequest(JSON.parse(Buffer.concat(chunks).toString('utf8')), options.jobKey); + if (request.runtimeRecipeDigest !== runtimeRecipeDigest || request.sourceHeads.length !== 1 || request.requiredScenarios.some(s => s.method !== 'automated' || s.verifierDigest !== verifierDigest || !scenarioIds.includes(s.id))) return send(422, { error: 'unsupported_pinned_verifier' }); + const head = request.sourceHeads[0]!; + const registration = options.resources.find(r => r.organizationId === request.organizationId && r.projectId === request.projectId && r.resourceId === head.resourceId); + if ((!registration || !options.provider) && !options.readCandidate && !options.readCandidateFiles) return send(403, { error: 'resource_scope_denied' }); + const digest = hashCanonicalSync(request); + const previous = runs.get(request.runId); + if (previous && previous.digest !== digest) return send(409, { error: 'run_id_conflict' }); + if (!previous && runs.size >= 1000) return send(503, { error: 'run_limit_reached_restart_or_rotate_bridge' }); + const execute = async () => { + const paths=options.runtime?.sourceFiles??[registration?.workerPath??'worker.mjs']; + const files=options.readCandidateFiles ? await options.readCandidateFiles(request,paths) : options.readCandidate ? (()=>{if(options.runtime)throw new Error('runtime requires complete exact source files');return null;})() : null; + const sourceFiles=files??await (async()=>{ + if(options.readCandidate)return {[paths[0]!]:await options.readCandidate(request)}; + const resource=await options.provider!.openCanonicalResource(registration!.organizationId,registration!.resourceId); + if(!resource)throw new Error('registered resource missing'); + return Object.fromEntries(await Promise.all(paths.map(async path=>[path,await options.provider!.readFile(resource,head.revisionId as RevisionId,path)]))); + })(); + if(Object.keys(sourceFiles).length!==paths.length || paths.some(path=>!sourceFiles[path]||sourceFiles[path]!.length>1048576))throw new Error('candidate source absent or oversized'); + const candidate = await mkdtemp(join(tmpdir(), 'commonforge-g1-bridge-')); + const startedAt = new Date().toISOString(); + try { + if(!options.runtime)await writeFile(join(candidate, 'worker.mjs'), sourceFiles[paths[0]!]!, { mode: 0o400 }); + const verified = options.runtime ? await options.runtime.verify(sourceFiles as Record) : await verifySchedulingCandidate(candidate); + if(verified.verifierDigest!==undefined&&verified.verifierDigest!==verifierDigest || 'runtimeRecipeDigest' in verified&&verified.runtimeRecipeDigest!==undefined&&verified.runtimeRecipeDigest!==runtimeRecipeDigest)throw new Error('runtime adapter subject changed'); + const actor=options.runtime?.actor??'local-node-runner'; + const completedAt = new Date().toISOString(); + if (Date.parse(completedAt) > Date.parse(request.expiresAt)) throw new Error('verification expired'); + const evidence: ScenarioEvidence[] = request.requiredScenarios.map(s => { + const result = verified.results.find(r => r.scenarioId === s.id)!; + return { schemaVersion: 1, organizationId: request.organizationId, projectId: request.projectId, + id: `ev-${request.runId}-${s.id}`, author: actor, createdAt: completedAt, + envelopeDigest: request.envelopeDigest, buildDigest: request.buildDigest, runtimeRecipeDigest: request.runtimeRecipeDigest, + scenarioId: s.id, scenarioRevision: s.revision, verifierDigest: s.verifierDigest, method: 'automated', result: result.result, + runId: request.runId, actor, startedAt, completedAt, provenance: 'runner-verified', supersedes: null, + attachments: [], summary: result.detail, + }; + }); + return signScenarioResult({ protocolVersion: 1, requestDigest: digest, runId: request.runId, evidence, completedAt, execution: verified.execution, network: verified.network, controlPlaneMounted: false }, options.jobKey); + } finally { await rm(candidate, { recursive: true, force: true }); } + }; + const running = previous?.result ?? execute(); + if (!previous) runs.set(request.runId, { digest, result: running }); + return send(200, await running); + })().catch(() => send(400, { error: 'local_verification_refused', detail: 'request signature, source, runtime, or isolation check failed' })); + }); + server.requestTimeout = 15000; + await new Promise((resolve, reject) => { server.once('error', reject); server.listen(options.port ?? 0, '127.0.0.1', resolve); }); + return { server, url: `http://127.0.0.1:${(server.address() as { port: number }).port}`, close: () => new Promise((resolve, reject) => server.close(e => e ? reject(e) : resolve())) }; +} diff --git a/apps/runner/src/g1-local-preview.ts b/apps/runner/src/g1-local-preview.ts new file mode 100644 index 0000000..139108f --- /dev/null +++ b/apps/runner/src/g1-local-preview.ts @@ -0,0 +1,22 @@ +/** Narrow standalone-module preview backend. No source-provider/control-plane authority. */ +import { createServer } from 'node:http'; +import { mkdtemp, rm, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { requestIsolatedCandidate } from './g1-local-verifier.js'; +export async function startLocalPreview(options: { key: string; port?: number }) { + const server = createServer((req, res) => { void (async () => { + if (req.method !== 'POST' || req.url !== '/serve' || req.headers.origin || req.headers.authorization !== `Bearer ${options.key}`) { res.writeHead(403); res.end(); return; } + let size = 0; const chunks: Buffer[] = []; for await (const chunk of req) { size += (chunk as Buffer).length; if (size > 1600000) throw new Error('preview too large'); chunks.push(chunk as Buffer); } + const input = JSON.parse(Buffer.concat(chunks).toString('utf8')) as { module: string; path: string; method: string; body?: string }; + if (typeof input.module !== 'string' || input.module.length > 1400000 || typeof input.path !== 'string' || typeof input.method !== 'string' || (input.body !== undefined && (typeof input.body !== 'string' || input.body.length > 65536))) throw new Error('invalid preview request'); + const root = await mkdtemp(join(tmpdir(), 'g1-preview-api-')); + try { + await writeFile(join(root, 'worker.mjs'), Buffer.from(input.module, 'base64'), { mode: 0o400 }); + const response = await requestIsolatedCandidate(root, input.path, input.method, input.body); + res.writeHead(200, { 'content-type': 'application/json' }); res.end(JSON.stringify(response)); + } finally { await rm(root, { recursive: true, force: true }); } + })().catch(() => { if (!res.headersSent) { res.writeHead(503); res.end('isolated local preview unavailable'); } }); }); + await new Promise((resolve, reject) => { server.once('error', reject); server.listen(options.port ?? 0, '127.0.0.1', resolve); }); + return { url: `http://127.0.0.1:${(server.address() as { port:number }).port}`, close: () => new Promise((resolve, reject) => server.close(e => e ? reject(e) : resolve())) }; +} diff --git a/apps/runner/src/g1-local-verifier.ts b/apps/runner/src/g1-local-verifier.ts new file mode 100644 index 0000000..5883a57 --- /dev/null +++ b/apps/runner/src/g1-local-verifier.ts @@ -0,0 +1,125 @@ +import { CandidateExecutionError, candidateErrorResults } from './g1-scenario-failure.js'; +/** Real Linux verifier. The supervisor owns scenarios; only candidate code enters the sandbox. */ +import { createHash } from 'node:crypto'; +import { constants } from 'node:fs'; +import { spawn } from 'node:child_process'; +import { chmod, mkdir, mkdtemp, open, readFile, realpath, rm, writeFile } from 'node:fs/promises'; +import { request } from 'node:http'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { cases } from './g1-scheduling-harness.js'; + +export interface LocalScenarioResult { scenarioId: string; result: 'pass' | 'fail' | 'error'; detail: string } +export interface LocalVerification { + verifierDigest: string; + sourceDigest: string; + execution: 'bubblewrap-linux-http'; + network: 'none'; + controlPlaneMounted: false; + results: LocalScenarioResult[]; +} + +const hostPath = fileURLToPath(new URL('../src/g1-candidate-host.mjs', import.meta.url)); +export async function schedulingVerifierDigest(): Promise { + // Bind the supervisor's comparison, isolation and timeout logic too, not just the examples. + // Source-mode and compiled supervisors intentionally have different verifier definitions. + return createHash('sha256').update('cf.scheduling-verifier/3\0').update(JSON.stringify(cases)) + .update(await readFile(hostPath)).update(await readFile(new URL(import.meta.url))) + .update(await readFile(new URL(`./g1-scenario-failure.${import.meta.url.endsWith('.ts')?'ts':'js'}`,import.meta.url))).digest('hex'); +} + +interface SocketResponse { status: number; headers: Record; body: string } +function socketResponse(socketPath: string, path: string, method: string, body?: string): Promise { + return new Promise((resolve, reject) => { + const req = request({ socketPath, path, method, headers: { 'content-type': 'application/json' }, timeout: 2000 }, (res) => { + const chunks: Buffer[] = []; let bytes = 0; + res.on('data', (chunk: Buffer) => { bytes += chunk.length; if (bytes > 64 * 1024) res.destroy(new Error('response too large')); else chunks.push(chunk); }); + res.on('error', reject); + res.on('end', () => resolve({ status: res.statusCode ?? 500, headers: Object.fromEntries(Object.entries(res.headers).filter((e): e is [string,string] => typeof e[1] === 'string')), body: Buffer.concat(chunks).toString('base64') })); + }); + req.on('timeout', () => req.destroy(new Error('candidate timeout'))); req.on('error', reject); + req.end(method === 'GET' ? undefined : body); + }); +} +async function socketRequest(socketPath: string, body: unknown, path = '/reschedule', method = 'POST'): Promise { + const response = await socketResponse(socketPath, path, method, JSON.stringify(body)); + if (response.status !== 200) throw new Error(`HTTP ${response.status}`); + return JSON.parse(Buffer.from(response.body,'base64').toString('utf8')); +} + +/** No unsafe fallback. Missing namespaces/bwrap is an unavailable environmental check. */ +async function withCandidateSandbox(candidateDir: string, observe: (socketPath: string, sourceDigest: string) => Promise): Promise { + const dir = await realpath(candidateDir); + const worker = join(dir, 'worker.mjs'); + const handle = await open(worker, constants.O_RDONLY | constants.O_NOFOLLOW); + let bytes: Buffer; + try { + const stat = await handle.stat(); + if (!stat.isFile() || stat.nlink !== 1 || stat.size > 1024 * 1024) throw new Error('candidate worker must be a bounded plain file'); + bytes = await handle.readFile(); + if (bytes.length > 1024 * 1024) throw new Error('candidate grew beyond size bound'); + } finally { await handle.close(); } + const sourceDigest = createHash('sha256').update(bytes).digest('hex'); + const scratch = await mkdtemp(join(tmpdir(), 'commonforge-g1-sandbox-')); + await chmod(scratch, 0o700); + const snapshot = join(scratch, 'worker.mjs'); + const socketDir = join(scratch, 'socket'); + await writeFile(snapshot, bytes, { mode: 0o400 }); + await mkdir(socketDir, { mode: 0o700 }); + const node = await realpath(process.execPath); + const child = spawn('bwrap', [ + '--unshare-all', '--die-with-parent', '--new-session', '--cap-drop', 'ALL', + '--ro-bind', '/usr', '/usr', '--ro-bind', '/lib', '/lib', '--ro-bind', '/lib64', '/lib64', + '--ro-bind', node, '/node', '--proc', '/proc', '--dev', '/dev', '--tmpfs', '/tmp', + '--ro-bind', snapshot, '/candidate/worker.mjs', '--ro-bind', hostPath, '/host.mjs', + '--bind', socketDir, '/scratch', '--chdir', '/candidate', '/node', '/host.mjs', + ], { env: { PATH: '/usr/bin:/bin', HOME: '/tmp', TZ: 'UTC' }, stdio: ['ignore', 'pipe', 'pipe'], detached: true }); + let sandboxStarted = false; + child.stderr.on('data', () => { /* Drain diagnostics without retaining candidate-controlled secrets. */ }); + try { + await new Promise((resolve, reject) => { + const timer = setTimeout(() => reject(sandboxStarted ? new CandidateExecutionError() : new Error('candidate isolation unavailable')), 5000); + child.once('error', (e) => { clearTimeout(timer); reject(e); }); + child.once('close', (code) => { clearTimeout(timer); reject(sandboxStarted ? new CandidateExecutionError() : new Error('candidate isolation unavailable')); }); + let output = ''; + child.stdout.on('data', (chunk: Buffer) => { output = (output + chunk.toString()).slice(-1024); if (output.includes('SANDBOX_STARTED\n')) sandboxStarted = true; if (output.includes('READY\n')) { clearTimeout(timer); resolve(); } }); + }); + return await observe(join(socketDir, 'preview.sock'), sourceDigest); + } catch (error) { + if (error instanceof CandidateExecutionError) throw new CandidateExecutionError(sourceDigest); + throw error; + } finally { + try { process.kill(-child.pid!, 'SIGKILL'); } catch { /* already exited */ } + if (child.pid && child.exitCode === null && child.signalCode === null) await new Promise((resolve) => child.once('exit', () => resolve())); + await rm(scratch, { recursive: true, force: true }); + } +} + +export async function verifySchedulingCandidate(candidateDir: string): Promise { + try { return await withCandidateSandbox(candidateDir, async (socketPath, sourceDigest) => { + const results: LocalScenarioResult[] = []; + for (const [scenarioId, input, expected] of cases) { + try { + const actual = await socketRequest(socketPath, { now: '2026-10-17T12:00:00.000Z', ...input }); + const same = JSON.stringify(actual) === JSON.stringify(expected); + results.push({ scenarioId, result: same ? 'pass' : 'fail', detail: same ? 'expected HTTP behavior observed' : 'HTTP response differs from trusted expectation' }); + } catch { results.push({ scenarioId, result: 'error', detail: 'candidate HTTP request failed' }); } + } + return { verifierDigest: await schedulingVerifierDigest(), sourceDigest, execution: 'bubblewrap-linux-http', network: 'none', controlPlaneMounted: false, results }; + }); } catch (error) { + if (!(error instanceof CandidateExecutionError)) throw error; + return { verifierDigest: await schedulingVerifierDigest(), sourceDigest: error.sourceDigest!, execution: 'bubblewrap-linux-http', network: 'none', controlPlaneMounted: false, results: candidateErrorResults(cases.map(c => c[0])) }; + } +} + +/** Supplemental real HTTP observation; does not claim to replace signed required scenarios. */ +export async function inspectIsolatedHealth(candidateDir: string): Promise { + return withCandidateSandbox(candidateDir, socket => socketRequest(socket, undefined, '/health', 'GET')); +} + +/** One bounded real preview HTTP request in a fresh isolated candidate process. */ +export async function requestIsolatedCandidate(candidateDir: string, path: string, method: string, body?: string): Promise { + if (!['GET','POST'].includes(method) || !path.startsWith('/') || /[\r\n\0]/.test(path) || path.length > 2048) throw new Error('unsupported preview request'); + return withCandidateSandbox(candidateDir, socket => socketResponse(socket, path, method, body)); +} diff --git a/apps/runner/src/g1-pending-review.ts b/apps/runner/src/g1-pending-review.ts new file mode 100644 index 0000000..4342f05 --- /dev/null +++ b/apps/runner/src/g1-pending-review.ts @@ -0,0 +1,37 @@ +/** Prepare an actual independently verified source proposal for owner browser review. */ +import { mkdtemp, readFile, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { createGit, RUNNER_IDENTITY_ENV } from './git.js'; +const session = JSON.parse(await readFile(process.argv[2]!, 'utf8')) as { ownerToken: string; organizationId: string; projectId: string }; +const operator = JSON.parse(await readFile(process.argv[3]!, 'utf8')) as { workerUrl: string }; +async function api(method: string, path: string, body?: unknown, token = session.ownerToken): Promise { + const res = await fetch(`${operator.workerUrl}/api${path}`, { method, headers: { authorization: `Bearer ${token}`, ...(body === undefined ? {} : { 'content-type': 'application/json' }) }, ...(body === undefined ? {} : { body: JSON.stringify(body) }) }); + const data = await res.json(); if (!res.ok) throw new Error(`${method} ${path}: ${res.status} ${JSON.stringify(data)}`); return data; +} +async function waitFor(path: string, ready: (data: any) => boolean) { const end = Date.now() + 60000; while (Date.now() < end) { const data = await api('GET', path); if (ready(data)) return data; await new Promise(r => setTimeout(r, 250)); } throw new Error('timed out'); } +const native = `/orgs/${session.organizationId}/projects/${session.projectId}/native`; +const task = await api('POST', `/orgs/${session.organizationId}/projects/${session.projectId}/tasks`, { title: 'Review health metadata follow-up', request: 'Expose review readiness in synthetic fixture health while preserving all corrected scheduling rules.', writeMounts: ['app'] }); +const ready = await waitFor(`/tasks/${task.taskId}`, data => data.state === 'ready'); +const handoff = await api('POST', `/tasks/${task.taskId}/native/handoff`, { agentId: 'browser-review-preparation-client' }); +const agent = await api('POST', '/auth/task/verify', { code: handoff.code }); +const credential = await api('POST', `/tasks/${task.taskId}/credential`, { mountName: 'app', access: 'write' }, agent.leaseToken); +const root = await mkdtemp(join(tmpdir(), 'g1-pending-browser-review-')), work = join(root, 'candidate'), git = createGit({ allowHttp: true }); +const env = { GIT_CONFIG_COUNT: '1', GIT_CONFIG_KEY_0: `http.${credential.remoteUrl}.extraHeader`, GIT_CONFIG_VALUE_0: `Authorization: Bearer ${credential.secret}` }; +await git.ok(['clone', '-q', credential.remoteUrl, work], root, { env }); +const nested = join(work, 'fixtures/scheduling/worker.mjs'); +const file = await readFile(nested).then(() => nested, () => join(work, 'worker.mjs')); +await writeFile(file, (await readFile(file, 'utf8')).replace('ruleVersion: 2', 'ruleVersion: 2, reviewReady: true')); +await git.ok(['add', '.'], work); await git.ok(['commit', '-q', '-m', 'Expose review readiness in fixture health'], work, { env: RUNNER_IDENTITY_ENV(new Date().toISOString()) }); +const sha = (await git.ok(['rev-parse', 'HEAD'], work)).trim(); await git.ok(['push', '-q', 'origin', 'HEAD:refs/heads/main'], work, { env }); +const ws = ready.workspaces.find((w: any) => w.mountName === 'app'); +const candidate = await api('POST', `/tasks/${task.taskId}/workspaces/${ws.taskWorkspaceId}/freeze`, { expectedSha: sha, tool: 'real-browser-review-preparation' }, agent.leaseToken); +const inner = await api('POST', `/orgs/${session.organizationId}/proposals`, { taskId: task.taskId, candidateIds: [candidate.candidateId] }); +const innerPath = `/orgs/${session.organizationId}/proposals/${inner.proposalId}`; +await waitFor(innerPath, data => data.state === 'prepared'); await api('POST', `${innerPath}/integrate`, {}); await waitFor(innerPath, data => !!data.integration); +const state = (await api('GET', native)).state; +const outer = await api('POST', `/tasks/${task.taskId}/native/proposals`, { innerProposalId: inner.proposalId, expectedGeneration: state.generation, idempotencyKey: `browser-outer-${task.taskId}` }, agent.leaseToken); +const verification = await api('POST', `${native}/proposals/${encodeURIComponent(outer.proposal.id)}/verify`, { idempotencyKey: `browser-verify-${task.taskId}` }); +if (!verification.evidence.every((e: any) => e.result === 'pass')) throw new Error('behavioral verification failed'); +const report = { root, organizationId: session.organizationId, projectId: session.projectId, taskId: task.taskId, innerProposalId: inner.proposalId, proposalId: outer.proposal.id, envelopeDigest: outer.proposal.digest, sourceRevision: sha, scenarioCount: verification.evidence.length, accepted: false, status: 'awaiting-owner-review' }; +await writeFile(join(root, 'report.json'), JSON.stringify(report, null, 2)); process.stdout.write(`${JSON.stringify(report, null, 2)}\n`); diff --git a/apps/runner/src/g1-real-local.ts b/apps/runner/src/g1-real-local.ts new file mode 100644 index 0000000..ebe4b46 --- /dev/null +++ b/apps/runner/src/g1-real-local.ts @@ -0,0 +1,93 @@ +/** Reproducible real Git/Node-runner/HTTP-verifier spike; does not simulate cloud release. */ +import { randomBytes } from 'node:crypto'; +import { createServer } from 'node:http'; +import { copyFile, mkdir, mkdtemp, readFile, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join, resolve } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { LocalGitProvider } from '@commonforge-org/provider-local-git'; +import { localRevisionId } from '@commonforge-org/core'; +import { scenarioBuildDigest, signScenarioRequest, verifyScenarioResult, type JobSigningKey, type ScenarioVerificationRequest } from '@commonforge-org/runner-protocol'; +import { createGit, RUNNER_IDENTITY_ENV } from './git.js'; +import { createRunner } from './runner.js'; +import { staticCredentials } from './credentials.js'; +import { fsBlobStore } from './blob-store.js'; +import { createRunnerSubmit } from './submit.js'; +import { schedulingVerifierDigest } from './g1-local-verifier.js'; +import { SCHEDULING_RUNTIME_RECIPE_DIGEST, startLocalScenarioBridge } from './g1-local-bridge.js'; +import { localNamespaceCommands } from './g1-bwrap-commands.js'; +import { cases } from './g1-scheduling-harness.js'; + +export async function runRealLocal(options: { port?: number; bridgePort?: number; jobKey?: JobSigningKey; organizationId?: string; projectId?: string } = {}) { + const root = await mkdtemp(join(tmpdir(), 'commonforge-g1-real-local-')); + const seed = join(root, 'seed'); await mkdir(seed); + for (const file of ['worker.mjs', 'build.mjs']) await copyFile(new URL(`../../../fixtures/scheduling/${file}`, import.meta.url), join(seed, file)); + const git = createGit({ allowFile: true }); + await git.ok(['init', '-q', '-b', 'main'], seed); + await git.ok(['add', '.'], seed); + await git.ok(['commit', '-q', '-m', 'Synthetic scheduling reference'], seed, { env: RUNNER_IDENTITY_ENV(new Date().toISOString()) }); + const provider = await LocalGitProvider.open({ root: join(root, 'repos'), namespacePrefix: 'commonforge-g1' }); + const organizationId = options.organizationId ?? 'g1-local-org', projectId = options.projectId ?? 'g1-scheduling'; + const resource = await provider.createCanonicalResource({ organizationId, resourceId: 'scheduling', source: seed }); + if (resource.coords.kind !== 'local-git') throw new Error('local coords expected'); + const revision = await provider.resolveRevision(resource, { kind: 'ref', ref: 'refs/heads/main' }); + const jobKey = options.jobKey ?? { kid: 'k1', secret: randomBytes(32).toString('hex') }; + await writeFile(join(root, 'job-key.json'), JSON.stringify(jobKey), { mode: 0o600 }); + const serviceEnv = await provider.serviceGitEnv(); + // Only the fixed audited fixture build executes here. Hostile code uses the namespace verifier. + const runner = createRunner({ workDir: join(root, 'runner'), credentials: staticCredentials(() => serviceEnv), + jobKeyring: [jobKey], git: { allowFile: true }, commands: await localNamespaceCommands(join(root, 'sandbox-bin')), + blobStore: fsBlobStore(join(root, 'blobs')), previewBaseUrl: 'http://127.0.0.1', runnerId: 'g1-real-local-node', log: () => undefined }); + const submit = createRunnerSubmit({ runner, jobKey }); + const buildResult = await submit({ kind: 'preview-build', previewId: 'scheduling', + integrationSha: { scheduling: revision.contentId }, sources: [{ resourceId: 'scheduling', mountName: 'app', canonical: resource.coords }], + recipe: { recipeId: 'audited-scheduling-build', revision: 1, steps: [{ stepId: 'build', mountName: 'app', command: 'node build.mjs' }], output: { mountName: 'app', path: 'dist' } }, + bundles: [{ name: 'worker', mountName: 'app', path: 'dist' }], + }); + if (!buildResult.ok) throw new Error(`real runner preview build failed: ${JSON.stringify(buildResult)}`); + const preview = createServer((_req, res) => { void readFile(join(root, 'blobs/previews/scheduling/site/index.html')).then(data => { res.writeHead(200, { 'content-type': 'text/html; charset=utf-8' }); res.end(data); }, () => { res.writeHead(503); res.end('preview not available'); }); }); + await new Promise((resolve, reject) => { preview.once('error', reject); preview.listen(options.port ?? 0, '127.0.0.1', resolve); }); + const previewUrl = `http://127.0.0.1:${(preview.address() as { port: number }).port}/`; + let bridge: Awaited>; + try { bridge = await startLocalScenarioBridge({ provider, jobKey, port: options.bridgePort, resources: [{ organizationId, projectId, resourceId: 'scheduling' }] }); } + catch (e) { preview.close(); throw e; } + try { + const now = new Date(); + const sourceHeads = [{ resourceId: 'scheduling', revisionId: localRevisionId(resource.coords.repoId, revision.contentId) }]; + const request: ScenarioVerificationRequest = { protocolVersion: 1, organizationId, projectId, + runId: `local-${Date.now()}`, envelopeDigest: 'a'.repeat(64), buildDigest: scenarioBuildDigest(sourceHeads), + runtimeRecipeDigest: SCHEDULING_RUNTIME_RECIPE_DIGEST, sourceHeads, + requiredScenarios: cases.map(c => ({ id: c[0], revision: 1, verifierDigest: '', method: 'automated' })), + issuedAt: now.toISOString(), expiresAt: new Date(now.getTime() + 60000).toISOString() }; + const verifierDigest = await schedulingVerifierDigest(); + request.requiredScenarios.forEach(s => { s.verifierDigest = verifierDigest; }); + const result = await fetch(`${bridge.url}/verify`, { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify(await signScenarioRequest(request, jobKey)) }); + if (!result.ok) throw new Error(`bridge verification HTTP ${result.status}: ${await result.text()}`); + const signedResult = await result.json(); + const verified = await verifyScenarioResult(signedResult, jobKey, request); + const previewResponse = await fetch(previewUrl); + if (!previewResponse.ok || !(await previewResponse.text()).includes('Synthetic scheduling')) throw new Error('real HTTP preview failed'); + const report = { protocolVersion: 1, mode: 'real-local', root, organizationId, projectId, sourceHeads, + previewUrl, bridgeUrl: bridge.url, verifierDigest, runtimeRecipeDigest: SCHEDULING_RUNTIME_RECIPE_DIGEST, + gates: { realGit: 'pass', signedNodeRunner: 'pass', localBlobStore: 'pass', httpPreview: 'pass', trustedIsolatedScenarios: verified.evidence.every(e => e.result === 'pass') ? 'pass' : 'fail', workerWorkflow: 'not_run', cloudRelease: 'not_run', freshAgent: 'not_run' }, + runnerBuild: buildResult, verification: verified, limitations: ['Only Node build/check entrypoints are supported by the local namespace wrapper; package installs and other command families fail closed.', 'Standalone provider/runner rehearsal does not itself prove the Worker acceptance workflow.', 'Local preview is not a cloud release.'], + }; + await writeFile(join(root, 'report.json'), JSON.stringify(report, null, 2)); + await writeFile(join(root, 'signed-verification.json'), JSON.stringify(signedResult, null, 2)); + return { report, provider, resource, close: async () => { await bridge.close(); await new Promise((resolve, reject) => preview.close(e => e ? reject(e) : resolve())); } }; + } catch (e) { await bridge.close(); preview.close(); throw e; } +} + +async function main() { + const keyFile = process.env.COMMONFORGE_LOCAL_JOB_KEY_FILE; + const jobKey = keyFile ? JSON.parse(await readFile(resolve(keyFile), 'utf8')) as JobSigningKey : undefined; + const running = await runRealLocal({ port: Number(process.env.PREVIEW_PORT ?? 0), bridgePort: Number(process.env.BRIDGE_PORT ?? 0), jobKey, + organizationId: process.env.G1_ORGANIZATION_ID, projectId: process.env.G1_PROJECT_ID }); + process.stdout.write(`${JSON.stringify(running.report, null, 2)}\n`); + if (!process.argv.includes('--serve')) await running.close(); + else { + const stop = async () => { await running.close(); process.exit(0); }; + process.once('SIGINT', () => { void stop(); }); process.once('SIGTERM', () => { void stop(); }); + } +} +if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) void main().catch(e => { process.stderr.write(`${e instanceof Error ? e.message : String(e)}\n`); process.exitCode = 1; }); diff --git a/apps/runner/src/g1-recovery-rehearsal.ts b/apps/runner/src/g1-recovery-rehearsal.ts new file mode 100644 index 0000000..8c46990 --- /dev/null +++ b/apps/runner/src/g1-recovery-rehearsal.ts @@ -0,0 +1,77 @@ +/** Post-browser real recovery proof: authoritative state, Git smart HTTP and isolated restored behavior. */ +import { createHash } from 'node:crypto'; +import { mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { canonicalJson, parseRevisionId, ProjectStateSchema } from '@commonforge-org/core'; +import { createGit } from './git.js'; +import { inspectIsolatedHealth, verifySchedulingCandidate } from './g1-local-verifier.js'; + +interface OwnerSession { ownerToken: string; organizationId: string; projectId: string } +interface Operator { workerUrl: string; sidecarUrl: string; sidecarKey: string } +interface BrowserEvidence { organizationId: string; projectId: string; historicalManifest: string; acceptedManifest: string; sourceHeads: Array<{ resourceId: string; revisionId: string }>; knowledgeHead: string; scenarioCount: number } +export async function inspectRecoveredSource(ownerFile: string, operatorFile: string, browserFile: string) { + const owner = JSON.parse(await readFile(ownerFile, 'utf8')) as OwnerSession; + const operator = JSON.parse(await readFile(operatorFile, 'utf8')) as Operator; + const browser = JSON.parse(await readFile(browserFile, 'utf8')) as BrowserEvidence; + if (owner.organizationId !== browser.organizationId || owner.projectId !== browser.projectId) throw new Error('owner/recovery project scope mismatch'); + const api = async (path: string) => { + const response = await fetch(`${operator.workerUrl}/api${path}`, { headers: { authorization: `Bearer ${owner.ownerToken}` } }); + if (!response.ok) throw new Error(`recovery read HTTP ${response.status}`); return response.json(); + }; + const rpc = async (method: string, args: unknown[]): Promise => { + const response = await fetch(`${operator.sidecarUrl}/rpc`, { method: 'POST', headers: { authorization: `Bearer ${operator.sidecarKey}`, 'content-type': 'application/json' }, body: JSON.stringify({ method, args }) }); + if (!response.ok) throw new Error(`real source operation ${method} HTTP ${response.status}`); return (await response.json() as { result: T }).result; + }; + const initial = await api(`/orgs/${owner.organizationId}/projects/${owner.projectId}/native`) as { state: unknown }; + const state = ProjectStateSchema.parse(initial.state), accepted = state.manifests.at(-1); + const target = state.manifests.find(m => m.hash === browser.historicalManifest); + const proposal = state.proposals.find(p => p.digest === accepted?.envelopeDigest); + if (!accepted || !target || !proposal || proposal.envelope.recovery?.manifestHash !== target.hash || accepted.hash !== browser.acceptedManifest || accepted.behavior !== 'verified') throw new Error('exact recovery acceptance binding absent'); + if (canonicalJson(state.sourceHeads) !== canonicalJson(target.sourceHeads) || canonicalJson(state.sourceHeads) !== canonicalJson(browser.sourceHeads) || state.knowledgeHead !== browser.knowledgeHead || accepted.knowledgeHead !== state.knowledgeHead) throw new Error('recovery changed current knowledge or source mapping'); + const evidence = accepted.evidenceIds.map(id => state.evidence.find(e => e.id === id)); + if (evidence.length !== browser.scenarioCount || evidence.length !== 9 || evidence.some(e => !e || e.result !== 'pass' || e.provenance !== 'runner-verified' || e.envelopeDigest !== proposal.digest || e.buildDigest !== proposal.envelope.buildDigest || e.runtimeRecipeDigest !== proposal.envelope.runtimeRecipeDigest)) throw new Error('fresh required verified evidence unavailable'); + const root = await mkdtemp(join(tmpdir(), 'g1-recovery-readonly-')); + const sourceChecks: Array> = []; + try { + for (const head of state.sourceHeads) { + const parsed = parseRevisionId(head.revisionId); if (parsed.kind !== 'art') throw new Error('real Git Artifacts-compatible revision required'); + let resource: any; const end = Date.now() + 90000; + do { + resource = await api(`/orgs/${owner.organizationId}/resources/${head.resourceId}`); + if (resource.head?.revisionId === head.revisionId && resource.head.reconciliation === 'synced') break; + await new Promise(resolve => setTimeout(resolve, 250)); + } while (Date.now() < end); + if (resource.head?.revisionId !== head.revisionId || resource.head.reconciliation !== 'synced') throw new Error('restored Git ref reconciliation did not become synced'); + const token = await rpc<{ id: string; plaintext: string }>('createToken', [resource.coords.namespace, resource.coords.repoName, 'read', 300]); + let observedSha: string; + try { + const git = createGit({ allowHttp: true }); + const refs = await git.ok(['ls-remote', resource.coords.remoteUrl, `refs/heads/${resource.branch ?? 'main'}`], root, { env: { GIT_CONFIG_COUNT: '1', GIT_CONFIG_KEY_0: `http.${resource.coords.remoteUrl}.extraHeader`, GIT_CONFIG_VALUE_0: `Authorization: Bearer ${token.plaintext}` } }); + observedSha = refs.trim().split(/\s+/)[0]!; + if (observedSha !== parsed.sha) throw new Error('actual Git smart HTTP canonical ref differs from restored accepted source'); + } finally { await rpc('revokeToken', [resource.coords.namespace, resource.coords.repoName, token.id]); } + let path = 'worker.mjs'; + let data = await rpc<{ bytes: string } | null>('readFile', [resource.coords.namespace, resource.coords.repoName, parsed.sha, path]); + if (!data) { path = 'fixtures/scheduling/worker.mjs'; data = await rpc<{ bytes: string } | null>('readFile', [resource.coords.namespace, resource.coords.repoName, parsed.sha, path]); } + if (!data) throw new Error('restored source module absent'); + const bytes = Buffer.from(data.bytes, 'base64'); await writeFile(join(root, 'worker.mjs'), bytes, { mode: 0o400 }); + const repeated = await verifySchedulingCandidate(root); + if (repeated.results.length !== 9 || repeated.results.some(r => r.result !== 'pass')) throw new Error('exact restored canonical source fails independent namespace scenarios'); + const health = await inspectIsolatedHealth(root) as { ruleVersion?: number; reviewReady?: boolean }; + if (health.ruleVersion !== 2 || health.reviewReady !== undefined) throw new Error('recovery did not restore corrected historical fixture health exactly'); + sourceChecks.push({ resourceId: head.resourceId, revisionId: head.revisionId, canonicalGitSha: observedSha, actualGitSmartHttp: 'pass', reconciliation: resource.head.reconciliation, + implementationPath: path, moduleDigest: createHash('sha256').update(bytes).digest('hex'), isolatedHealth: health, repeatedScenarioResults: repeated.results, execution: repeated.execution, network: repeated.network, controlPlaneMounted: repeated.controlPlaneMounted }); + } + const final = ProjectStateSchema.parse((await api(`/orgs/${owner.organizationId}/projects/${owner.projectId}/native`) as { state: unknown }).state); + if (final.knowledgeHead !== state.knowledgeHead || canonicalJson(final.knowledge) !== canonicalJson(state.knowledge) || canonicalJson(final.sourceHeads) !== canonicalJson(state.sourceHeads)) throw new Error('project authority changed while observing recovery'); + return { schemaVersion: 1, execution: 'real-browser-acceptance-followed-by-real-Git-and-isolated-HTTP', organizationId: owner.organizationId, projectId: owner.projectId, + historicalManifest: target.hash, recoveryAcceptedManifest: accepted.hash, recoveryEnvelopeDigest: proposal.digest, acceptedArtifactVersion: sourceChecks.length ? (await api(`/orgs/${owner.organizationId}/resources/${state.sourceHeads[0]!.resourceId}`) as any).head.acceptedVersionId : null, + knowledgeHead: state.knowledgeHead, knowledgeGeneration: state.knowledgeGeneration, knowledgePreserved: true, exactHistoricalSourceRestored: true, + evidence: evidence.map(e => ({ id: e!.id, runId: e!.runId, scenarioId: e!.scenarioId, scenarioRevision: e!.scenarioRevision, verifierDigest: e!.verifierDigest, result: e!.result, provenance: e!.provenance, envelopeDigest: e!.envelopeDigest, buildDigest: e!.buildDigest })), + sourceChecks, gates: { ownerBrowserRecovery: 'pass', freshSignedScenarios: 'pass', exactAcceptedHistoricalRefs: 'pass', realGitReconciliation: 'pass', independentRestoredRuntime: 'pass', currentKnowledgePreserved: 'pass', cloudDeployment: 'not_run' }, releaseClaimed: false, externalEffectsReversed: false }; + } finally { await rm(root, { recursive: true, force: true }); } +} +if (process.argv[1]?.endsWith('g1-recovery-rehearsal.ts') || process.argv[1]?.endsWith('g1-recovery-rehearsal.js')) { + void inspectRecoveredSource(process.argv[2]!, process.argv[3]!, process.argv[4]!).then(report => process.stdout.write(`${JSON.stringify(report, null, 2)}\n`)).catch(error => { process.stderr.write(`${error instanceof Error ? error.stack : String(error)}\n`); process.exitCode = 1; }); +} diff --git a/apps/runner/src/g1-scenario-failure.ts b/apps/runner/src/g1-scenario-failure.ts new file mode 100644 index 0000000..3cebeec --- /dev/null +++ b/apps/runner/src/g1-scenario-failure.ts @@ -0,0 +1,8 @@ +/** Only supervisors that observed namespace startup may classify a candidate execution error. + * Messages are fixed/redacted: candidate diagnostics, file paths and stacks never enter evidence. */ +export class CandidateExecutionError extends Error { + constructor(readonly sourceDigest?: string) { super('isolated candidate execution failed or timed out'); this.name = 'CandidateExecutionError'; } +} +export const candidateErrorResults = (ids: readonly string[]) => ids.map(scenarioId => ({ + scenarioId, result: 'error' as const, detail: 'Isolated candidate execution failed or timed out; no passing behavior was established.', +})); diff --git a/apps/runner/src/g1-scheduling-harness.ts b/apps/runner/src/g1-scheduling-harness.ts new file mode 100644 index 0000000..124bf95 --- /dev/null +++ b/apps/runner/src/g1-scheduling-harness.ts @@ -0,0 +1,14 @@ +// Trusted supervisor fixture, mounted read-only OUTSIDE the candidate tree. +// Candidate code runs only in the sibling process; it cannot change these expectations. +const cases: Array<[string, Record, Record]> = [ + ['exact-cutoff', { appointment: '2026-10-18T12:00:00.000Z', actor: 'customer' }, { allowed: true, override: null }], + ['inside-cutoff', { appointment: '2026-10-18T11:59:59.999Z', actor: 'customer' }, { allowed: false, code: 'cutoff' }], + ['outside-cutoff', { appointment: '2026-10-18T12:00:00.001Z', actor: 'customer' }, { allowed: true, override: null }], + ['travel-block', { appointment: '2026-10-19T12:00:00.000Z', actor: 'customer', traveling: true }, { allowed: false, code: 'technician_traveling' }], + ['unauthorized-override', { appointment: '2026-10-17T13:00:00.000Z', actor: 'customer', reason: 'Please' }, { allowed: false, code: 'cutoff' }], + ['manager-reason-required', { appointment: '2026-10-17T13:00:00.000Z', actor: 'manager', reason: ' ' }, { allowed: false, code: 'override_reason_required' }], + ['manager-reason-retained', { appointment: '2026-10-17T13:00:00.000Z', actor: 'manager', reason: ' Customer emergency ' }, { allowed: true, override: { actor: 'manager', reason: 'Customer emergency', recordedAt: '2026-10-17T12:00:00.000Z' } }], + ['manager-no-travel-bypass', { appointment: '2026-10-17T13:00:00.000Z', actor: 'manager', reason: 'Emergency', traveling: true }, { allowed: false, code: 'technician_traveling' }], + ['explicit-offset', { appointment: '2026-10-18T07:00:00.000-05:00', actor: 'customer' }, { allowed: true, override: null }], +]; +export { cases }; diff --git a/apps/runner/src/g1-worker-rehearsal.ts b/apps/runner/src/g1-worker-rehearsal.ts new file mode 100644 index 0000000..f6ed7d3 --- /dev/null +++ b/apps/runner/src/g1-worker-rehearsal.ts @@ -0,0 +1,141 @@ +/** Concrete public API + real Git rehearsal against scripts/dev-real-worker.sh. No fixture API. */ +import { copyFile, lstat, mkdir, mkdtemp, readFile, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { dirname, join, resolve } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { createGit, RUNNER_IDENTITY_ENV } from './git.js'; +import { inspectIsolatedHealth } from './g1-local-verifier.js'; +import { cases } from './g1-scheduling-harness.js'; +interface Operator { sidecarUrl: string; sidecarKey: string; bridgeUrl: string; workerUrl: string; namespacePrefix: string } +export async function rehearseWorker(operatorFile: string, logFile: string, options: { selfDevelopment?: boolean; ownerSessionFile?: string; preview?: boolean } = {}) { + const operator = JSON.parse(await readFile(operatorFile, 'utf8')) as Operator; + const root = await mkdtemp(join(tmpdir(), 'commonforge-g1-worker-rehearsal-')); + process.stderr.write(`real Worker rehearsal artifacts: ${root}\n`); + const events: Array<{ operation: string; status: number }> = []; + let ownerToken = ''; + async function api(method: string, path: string, body?: unknown, token = ownerToken): Promise { + const res = await fetch(`${operator.workerUrl}/api${path}`, { method, headers: { ...(token ? { authorization: `Bearer ${token}` } : {}), ...(body !== undefined ? { 'content-type': 'application/json' } : {}) }, ...(body !== undefined ? { body: JSON.stringify(body) } : {}) }); + events.push({ operation: `${method} ${path}`, status: res.status }); + const result = await res.json(); if (!res.ok) throw new Error(`${method} ${path}: ${res.status} ${JSON.stringify(result)}`); return result; + } + async function waitFor(operation: () => Promise, accepted: (v: T) => boolean, label: string, timeout = 60000): Promise { + const end = Date.now() + timeout; let last: T | undefined; + while (Date.now() < end) { last = await operation(); if (accepted(last)) return last; await new Promise(r => setTimeout(r, 250)); } + throw new Error(`timeout waiting for ${label}: ${JSON.stringify(last)}`); + } + let login: any; + if (options.ownerSessionFile) { + ownerToken = JSON.parse(await readFile(options.ownerSessionFile, 'utf8')).ownerToken; + login = await api('GET', '/me'); + } else { + await api('POST', '/auth/magic-link', { email: 'owner@agency.e2e.test', client: 'cli' }, ''); + const code = await waitFor(async () => (await readFile(logFile, 'utf8')).match(/magic link for owner@agency\.e2e\.test: .*?[?&](?:code|token)=(cfm_[A-Za-z0-9_-]+)/g)?.at(-1)?.match(/(cfm_[A-Za-z0-9_-]+)/)?.[1], v => !!v, 'local magic link'); + login = await api('POST', '/auth/cli/verify', { code }, ''); ownerToken = login.sessionToken; + } + const org = await api('POST', '/orgs', { name: 'G1 real-source rehearsal' }), orgId = org.orgId; + const project = await api('POST', `/orgs/${orgId}/projects`, { name: options.selfDevelopment ? 'CommonForge own development' : 'Synthetic scheduling' }), projectId = project.projectId; + const resource = await api('POST', `/orgs/${orgId}/resources`, { name: 'Scheduling fixture', provider: 'artifacts' }); + let sourceSnapshot: { commit: string; fileCount: number; exclusions: string[] } | null = null; + if (options.selfDevelopment) { + const publicRepo = fileURLToPath(new URL('../../../', import.meta.url)); + const seed = join(root, 'public-source-snapshot'); await mkdir(seed); + const git = createGit({ allowHttp: true }); + const paths = (await git.ok(['ls-files', '-z', '--cached', '--others', '--exclude-standard'], publicRepo)).split('\0').filter(Boolean); + let fileCount = 0; const exclusions: string[] = []; + for (const path of paths) { + if (/(?:^|\/)(?:\.env(?:$|\.)|\.dev\.vars|credentials|node_modules|\.wrangler|\.g1-artifacts)(?:\/|$)/.test(path) || path.startsWith('.git/')) { exclusions.push(path); continue; } + const source = join(publicRepo, path), stat = await lstat(source).catch(() => null); + if (!stat?.isFile()) { exclusions.push(path); continue; } + await mkdir(dirname(join(seed, path)), { recursive: true }); await copyFile(source, join(seed, path)); fileCount++; + } + await git.ok(['init', '-q', '-b', 'main'], seed); await git.ok(['add', '.'], seed); + await git.ok(['commit', '-q', '-m', 'Isolated public CommonForge source snapshot for G1 self-development'], seed, { env: RUNNER_IDENTITY_ENV(new Date().toISOString()) }); + const tokenResponse = await fetch(`${operator.sidecarUrl}/rpc`, { method: 'POST', headers: { authorization: `Bearer ${operator.sidecarKey}`, 'content-type': 'application/json' }, body: JSON.stringify({ method: 'createToken', args: [resource.coords.namespace, resource.coords.repoName, 'write', 300] }) }); + if (!tokenResponse.ok) throw new Error('real source bootstrap credential refused'); + const token = (await tokenResponse.json() as { result: { plaintext: string; id: string } }).result; + await git.ok(['push', '-q', resource.coords.remoteUrl, 'HEAD:refs/heads/main'], seed, { env: { GIT_CONFIG_COUNT: '1', GIT_CONFIG_KEY_0: `http.${resource.coords.remoteUrl}.extraHeader`, GIT_CONFIG_VALUE_0: `Authorization: Bearer ${token.plaintext}` } }); + await fetch(`${operator.sidecarUrl}/rpc`, { method: 'POST', headers: { authorization: `Bearer ${operator.sidecarKey}`, 'content-type': 'application/json' }, body: JSON.stringify({ method: 'revokeToken', args: [resource.coords.namespace, resource.coords.repoName, token.id] }) }); + sourceSnapshot = { commit: (await git.ok(['rev-parse', 'HEAD'], seed)).trim(), fileCount, exclusions }; + } else { + const rpc = await fetch(`${operator.sidecarUrl}/rpc`, { method: 'POST', headers: { authorization: `Bearer ${operator.sidecarKey}`, 'content-type': 'application/json' }, body: JSON.stringify({ method: 'seedScheduling', args: [resource.coords.namespace, resource.coords.repoName] }) }); + if (!rpc.ok) throw new Error('real Git fixture bootstrap failed'); + } + await api('POST', `/orgs/${orgId}/resources/${resource.resourceId}/adopt-head`, {}); + await api('POST', `/orgs/${orgId}/projects/${projectId}/mounts`, { mountName: 'app', resourceId: resource.resourceId, operations: ['read', 'write'] }); + if (options.preview) await api('PUT', `/orgs/${orgId}/projects/${projectId}/policy`, { preview: { recipe: { recipeId: 'scheduling-preview', revision: 1, steps: [{ stepId: 'build', mountName: 'app', command: 'node build.mjs' }], output: { mountName: 'app', path: 'dist' } }, bundles: [{ name: 'api', mountName: 'app', path: 'dist' }], hostVars: {}, ttlSeconds: 3600 } }); + const native = `/orgs/${orgId}/projects/${projectId}/native`; + let state = (await api('POST', `${native}/enable`, { idempotencyKey: 'enable' })).state; + await writeFile(join(root, 'owner-session.json'), JSON.stringify({ ownerToken, organizationId: orgId, projectId, resourceId: resource.resourceId }), { mode: 0o600 }); + const author = login.user.userId; + const base = (id: string) => ({ schemaVersion: 1, organizationId: orgId, projectId, id, author, createdAt: new Date().toISOString() }); + async function command(command: Record): Promise { const result = await api('POST', `${native}/commands`, { ...command, expectedGeneration: state.generation, idempotencyKey: `cmd-${state.generation}` }); state = result.state; return result; } + await command({ type: 'draftKnowledge', record: { ...base('reschedule-rule'), revision: 1, predecessor: null, kind: 'rule', text: 'Customers may reschedule at least 24 hours before an appointment, before travel begins.', rationale: 'Owner requested deterministic scheduling rules', sources: [{ kind: 'text', ref: 'Synthetic owner request', digest: null }], confirmation: null, archived: false } }); + await command({ type: 'confirmKnowledge', id: 'reschedule-rule', revision: 1, decisionId: 'confirm-initial-rule', explanation: 'Confirmed explicit cutoff and travel boundary' }); + const capability = await (await fetch(`${operator.bridgeUrl}/capabilities`)).json() as { verifierDigest: string }; + for (const [id] of cases.slice(0, 4)) { + await command({ type: 'draftScenario', record: { ...base(id), revision: 1, predecessor: null, title: id, given: 'Synthetic UTC clock and appointment', when: 'A customer requests rescheduling', then: 'Enforce the cutoff and travel rule', ruleRefs: [{ id: 'reschedule-rule', revision: 2 }], required: true, method: 'automated', verifierDigest: capability.verifierDigest, confirmation: null, archived: false } }); + await command({ type: 'confirmScenario', id, revision: 1, decisionId: `confirm-${id}`, explanation: 'Confirmed synthetic behavioral example' }); + } + await command({ type: 'upsertOutcome', record: { ...base('health-metadata-outcome'), revision: 1, description: 'Expose the scheduling rule version in fixture health while preserving the owner correction.', owner: author, successCriteria: ['Health reports the implementation rule version', 'All confirmed scheduling scenarios still pass'], state: 'open', taskIds: [] } }); + async function cycle(label: string, edit: (source: string) => string) { + const task = await api('POST', `/orgs/${orgId}/projects/${projectId}/tasks`, { title: label, request: label, writeMounts: ['app'] }); + const ready = await waitFor(() => api('GET', `/tasks/${task.taskId}`), t => t.state === 'ready', 'real task seeding'); + const handoff = await api('POST', `/tasks/${task.taskId}/native/handoff`, { agentId: `rehearsal-${label}` }); + const agent = await api('POST', '/auth/task/verify', { code: handoff.code }, ''); + const context = await api('GET', `/tasks/${task.taskId}/native/context`, undefined, agent.leaseToken); + const credential = await api('POST', `/tasks/${task.taskId}/credential`, { mountName: 'app', access: 'write' }, agent.leaseToken); + const work = join(root, `work-${label}`), git = createGit({ allowHttp: true }); + const env = { GIT_CONFIG_COUNT: '1', GIT_CONFIG_KEY_0: `http.${credential.remoteUrl}.extraHeader`, GIT_CONFIG_VALUE_0: `Authorization: Bearer ${credential.secret}` }; + await git.ok(['clone', '-q', credential.remoteUrl, work], root, { env }); + const workerPath = join(work, options.selfDevelopment ? 'fixtures/scheduling/worker.mjs' : 'worker.mjs'); + await writeFile(workerPath, edit(await readFile(workerPath, 'utf8'))); + await git.ok(['add', '.'], work); await git.ok(['commit', '-q', '-m', label], work, { env: RUNNER_IDENTITY_ENV(new Date().toISOString()) }); + const sha = (await git.ok(['rev-parse', 'HEAD'], work)).trim(); await git.ok(['push', '-q', 'origin', 'HEAD:refs/heads/main'], work, { env }); + const ws = ready.workspaces.find((w: any) => w.mountName === 'app'); + const candidate = await api('POST', `/tasks/${task.taskId}/workspaces/${ws.taskWorkspaceId}/freeze`, { expectedSha: sha, tool: 'g1-real-api-rehearsal' }, agent.leaseToken); + const inner = await api('POST', `/orgs/${orgId}/proposals`, { taskId: task.taskId, candidateIds: [candidate.candidateId] }); + await waitFor(() => api('GET', `/orgs/${orgId}/proposals/${inner.proposalId}`), p => p.state === 'prepared', 'candidate preparation'); + await api('POST', `/orgs/${orgId}/proposals/${inner.proposalId}/integrate`, {}); + await waitFor(() => api('GET', `/orgs/${orgId}/proposals/${inner.proposalId}`), p => !!p.integration, 'real Node integration'); + state = (await api('GET', native)).state; + const outer = await api('POST', `/tasks/${task.taskId}/native/proposals`, { innerProposalId: inner.proposalId, expectedGeneration: state.generation, idempotencyKey: `outer-${label}` }, agent.leaseToken); + state = outer.state; + const verified = await api('POST', `${native}/proposals/${encodeURIComponent(outer.proposal.id)}/verify`, { idempotencyKey: `verify-${label}` }); state = verified.state; + if (!verified.evidence.every((e: any) => e.result === 'pass')) throw new Error('trusted scenarios did not pass'); + const health = await inspectIsolatedHealth(dirname(workerPath)) as { ruleVersion?: number }; + if (health.ruleVersion !== (label === 'initial' ? 1 : 2)) throw new Error('independent health metadata observation failed'); + const proposal = outer.proposal; + await command({ type: 'recordDecision', record: { ...base(`approve-${label}`), envelopeDigest: proposal.digest, innerProposalHash: proposal.envelope.innerProposalHash, action: 'approve', explanation: 'Reviewed exact real runner behavior', authorityRevision: state.authorities.find((a: any) => a.principalId === author).revision } }); + await command({ type: 'accept', proposalId: proposal.id, decisionId: `approve-${label}` }); + return { taskId: task.taskId, innerProposalId: inner.proposalId, proposalId: proposal.id, contextDigest: agent.context.digest, contextRuleRevisions: context.context.packet.rules.map((r: any) => ({ id: r.id, revision: r.revision })), sourceRevision: sha, supplementalObservedHealth: health, scenarioCount: verified.evidence.length, acceptedManifest: state.manifests.at(-1).hash }; + } + const initial = await cycle('initial', source => source.replace('fixture: true, clock: CLOCK', 'fixture: true, ruleVersion: 1, clock: CLOCK')); + await command({ type: 'draftKnowledge', record: { ...base('reschedule-rule'), revision: 3, predecessor: 2, kind: 'rule', text: 'Managers may override the 24-hour cutoff only with a retained nonempty reason. Travel still blocks everybody.', rationale: 'Owner correction persists beyond the original attempt', sources: [{ kind: 'text', ref: 'Synthetic owner correction', digest: null }], confirmation: null, archived: false } }); + await command({ type: 'confirmKnowledge', id: 'reschedule-rule', revision: 3, decisionId: 'confirm-manager-correction', explanation: 'Require manager exception and retain reason' }); + for (const [id] of cases.slice(4)) { + await command({ type: 'draftScenario', record: { ...base(id), revision: 1, predecessor: null, title: id, given: 'Synthetic manager/customer and appointment', when: 'Rescheduling is requested', then: 'Preserve authorization, reason, and travel restriction', ruleRefs: [{ id: 'reschedule-rule', revision: 4 }], required: true, method: 'automated', verifierDigest: capability.verifierDigest, confirmation: null, archived: false } }); + await command({ type: 'confirmScenario', id, revision: 1, decisionId: `confirm-${id}`, explanation: 'Confirmed corrected behavioral example' }); + } + const corrected = await cycle('corrected', source => source.replace('ruleVersion: 1', 'ruleVersion: 2')); + let previewEvidence: unknown = null; + if (options.preview) { + const queued = await api('POST', `/orgs/${orgId}/proposals/${corrected.innerProposalId}/previews`, {}); + const ready = await waitFor(() => api('GET', `/orgs/${orgId}/previews/${queued.previewId}`), p => p.state === 'ready', 'real control-plane preview'); + const staticResponse = await fetch(ready.url); + if (!staticResponse.ok || !(await staticResponse.text()).includes('Synthetic scheduling')) throw new Error('real R2 static preview failed'); + const healthResponse = await fetch(`${ready.url}api/health`); + const health = await healthResponse.json() as { ruleVersion?: number }; + if (!healthResponse.ok || health.ruleVersion !== 2) throw new Error('real isolated API preview health failed'); + const managerResponse = await fetch(`${ready.url}api/reschedule`, { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ actor: 'manager', appointment: '2026-10-17T13:00:00.000Z', reason: 'Preview override' }) }); + const manager = await managerResponse.json() as { allowed?: boolean; override?: {reason?:string} }; + if (!managerResponse.ok || !manager.allowed || manager.override?.reason !== 'Preview override') throw new Error('real isolated API preview manager rule failed'); + previewEvidence = { previewId: queued.previewId, sourceBinding: ready.integrationSha ?? null, separateOrigin: new URL(ready.url).origin !== new URL(operator.workerUrl).origin, staticHttpStatus: staticResponse.status, apiHealth: health, managerResponse: manager, execution: 'real Worker preview authorization + local R2 + isolated Node HTTP module', privateLinkExcluded: true }; + } + const report = { mode: 'real-local-worker-api', root, ownDevelopmentProject: !!options.selfDevelopment, sourceSnapshot, previewEvidence, workerUrl: operator.workerUrl, organizationId: orgId, projectId, resourceId: resource.resourceId, initial, corrected, + gates: { realSourceApiWorkflow: 'pass', independentlyVerifiedBehavior: 'pass', persistedCorrectionInLaterContext: 'pass', atomicAcceptance: 'pass', freshAgent: 'not_run', cloudRelease: 'not_run', browserOwnerJourney: 'not_run' }, events }; + await writeFile(join(root, 'report.json'), JSON.stringify(report, null, 2)); + // Local-only bootstrap for browser review; never exported as project context or committed. + await writeFile(join(root, 'owner-session.json'), JSON.stringify({ ownerToken, organizationId: orgId, projectId }), { mode: 0o600 }); + return report; +} +if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) void rehearseWorker(process.argv[2]!, process.argv[3]!, { selfDevelopment: process.argv.includes('--self-development'), ownerSessionFile: process.env.G1_OWNER_SESSION_FILE, preview: process.argv.includes('--preview') }).then(r => process.stdout.write(`${JSON.stringify(r, null, 2)}\n`)).catch(e => { process.stderr.write(`${e instanceof Error ? e.stack : String(e)}\n`); process.exitCode = 1; }); diff --git a/apps/runner/test/g1-artifacts-sidecar.test.ts b/apps/runner/test/g1-artifacts-sidecar.test.ts new file mode 100644 index 0000000..5e780f9 --- /dev/null +++ b/apps/runner/test/g1-artifacts-sidecar.test.ts @@ -0,0 +1,26 @@ +import { mkdtemp, mkdir, rm, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { expect, it } from 'vitest'; +import { createGit, RUNNER_IDENTITY_ENV } from '../src/git.js'; +import { startArtifactsSidecar } from '../src/g1-artifacts-sidecar.js'; +it('serves real Git smart HTTP with repository-scoped expiring/revocable credentials', async () => { + const root = await mkdtemp(join(tmpdir(), 'g1-real-sidecar-')); + const sidecar = await startArtifactsSidecar({ stateDir: root, key: 'local-rpc-key' }); + try { + const repo = await sidecar.artifacts.createRepo('test-canonical', 'site'); + const seeded = await sidecar.artifacts.seedScheduling('test-canonical', 'site'); + const work = join(root, 'checkout'), git = createGit({ allowHttp: true }); + const env = { GIT_CONFIG_COUNT: '1', GIT_CONFIG_KEY_0: `http.${repo.remote}.extraHeader`, GIT_CONFIG_VALUE_0: `Authorization: Bearer ${repo.token}` }; + await git.ok(['clone', '-q', repo.remote, work], root, { env }); + expect((await sidecar.artifacts.log('test-canonical', 'site', { ref: 'main' }))[0]?.hash).toBe(seeded.sha); + await writeFile(join(work, 'extra.txt'), 'real source update'); + await git.ok(['add', '.'], work); await git.ok(['commit', '-q', '-m', 'Real edit'], work, { env: RUNNER_IDENTITY_ENV(new Date().toISOString()) }); + await git.ok(['push', '-q', 'origin', 'HEAD:refs/heads/main'], work, { env }); + expect(new TextDecoder().decode(await sidecar.artifacts.readFile('test-canonical', 'site', 'main', 'extra.txt') ?? undefined)).toBe('real source update'); + const tokens = await sidecar.artifacts.listTokens('test-canonical', 'site'); + await sidecar.artifacts.revokeToken('test-canonical', 'site', tokens[0]!.id); + expect((await git.run(['ls-remote', repo.remote], root, { env })).code).not.toBe(0); + expect((await fetch(`${sidecar.url}/rpc`, { method: 'POST', body: '{}' })).status).toBe(403); + } finally { await sidecar.close(); await rm(root, { recursive: true, force: true }); } +}); diff --git a/apps/runner/test/g1-bwrap-commands.test.ts b/apps/runner/test/g1-bwrap-commands.test.ts new file mode 100644 index 0000000..8f43f2f --- /dev/null +++ b/apps/runner/test/g1-bwrap-commands.test.ts @@ -0,0 +1,18 @@ +import { mkdtemp, mkdir, rm, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { expect, it } from 'vitest'; +import { localNamespaceCommands } from '../src/g1-bwrap-commands.js'; +import { runIsolated } from '../src/commands.js'; +it('real runner Node entrypoints cannot read controller state or reach host networking', async () => { + const root = await mkdtemp(join(tmpdir(), 'g1-command-boundary-')); + try { + const candidate = join(root, 'candidate'); await mkdir(candidate); + const secret = join(root, 'controller-key'); await writeFile(secret, 'controller must remain isolated'); + const commands = await localNamespaceCommands(join(root, 'bin')); + await writeFile(join(candidate, 'test.mjs'), `import {existsSync,writeFileSync} from 'node:fs'; import {networkInterfaces} from 'node:os'; if (existsSync(${JSON.stringify(secret)}) || Object.values(networkInterfaces()).flat().some(x=>x&&!x.internal)) process.exit(1); writeFileSync('passed.txt','isolated');`); + const result = await runIsolated({ argv: [join(root, 'bin/node'), 'test.mjs'], cwd: candidate, env: { PATH: commands.path!, HOME: '/tmp' }, timeoutMs: 5000 }); + expect(result.exitCode, result.stderr).toBe(0); + expect(commands.allowlist).toEqual([['node', '{path}', '{args*}']]); + } finally { await rm(root, { recursive: true, force: true }); } +}); diff --git a/apps/runner/test/g1-cloud-rehearsal.test.ts b/apps/runner/test/g1-cloud-rehearsal.test.ts new file mode 100644 index 0000000..489cfaa --- /dev/null +++ b/apps/runner/test/g1-cloud-rehearsal.test.ts @@ -0,0 +1,84 @@ +import { createRequire } from 'node:module'; +import { mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'; +import { dirname, join } from 'node:path'; +import { tmpdir } from 'node:os'; +import { pathToFileURL } from 'node:url'; +import { createGit, RUNNER_IDENTITY_ENV, type Git } from '../src/git.js'; +import { describe, expect, it } from 'vitest'; +import { canonicalJson, sha256HexSync } from '@commonforge-org/core'; +import { G1CloudRehearsalConfigSchema, cloudFixtureFiles, cloudRehearsalContracts, cloudRehearsalPlan, pushResumableCloudSource, requireFreshCloudProjectName, type CloudSourcePushCheckpoint, type G1CloudRehearsalConfig } from '../src/g1-cloud-rehearsal.js'; +import { createCloudflareSchedulingAdapter } from '../src/g1-cloudflare-source.js'; +const require = createRequire(new URL('../../worker/package.json', import.meta.url)); +const toolchain = dirname(require.resolve('workerd/package.json')); +const pin = { binary: join(toolchain,'bin/workerd'), sha256: sha256HexSync(await readFile(join(toolchain,'bin/workerd'))) }; +const config: G1CloudRehearsalConfig = { schemaVersion:1,runId:'an21-test',apiUrl:'https://commonforge-ci-worker.example.workers.dev',accountId:'0'.repeat(32),organizationId:'o-0123456789ab',ownerSessionFile:'/not-read-by-plan',stateDirectory:'/not-written-by-plan',controlWorker:'commonforge-ci-worker',targetPrefix:'cfci',mutationsApproved:false,deploymentApproved:false,supervisorPort:8971,pin,schema:join(toolchain,'workerd.capnp') }; +describe('dedicated G1 cloud rehearsal preparation',()=>{ + it('plans no network or provisioning and constrains the existing CI target',()=>{ + const plan=cloudRehearsalPlan(config);expect(plan.execution).toBe('planned-not-executed');expect(plan.releaseWorker).toBe('cfci-o-0123456789ab-g1-an21-test');expect(plan.noProvisioning).toBe(true); + for(const apiUrl of ['https://commonforge-worker.example.workers.dev','http://commonforge-ci-worker.example.workers.dev','https://commonforge-ci-worker.example.workers.dev/api','https://evil.example'])expect(G1CloudRehearsalConfigSchema.safeParse({...config,apiUrl}).success).toBe(false); + expect(G1CloudRehearsalConfigSchema.safeParse({...config,targetPrefix:'cfapp'}).success).toBe(false); + }); + it('pins the exact complete release environment used by the verifier',()=>{ + const c=cloudRehearsalContracts(config);expect(c.environment).toEqual({targets:c.policy.release.targets,environment:c.policy.release.environment,healthCheck:c.policy.release.healthCheck,accountId:config.accountId}); + expect(c.policy.release.migrations).toEqual([]);expect(c.policy.release.targets).toHaveLength(1);expect(c.environment.environment.api).toEqual({vars:{APP_VERSION:'g1-synthetic-scheduling'},kvNamespaces:[]}); + }); + // Two real builds and isolated workerd executions exceed the default 5 s on CI hosts. + it('builds both actual deployable source variants and executes all nine scenarios in pinned isolated workerd',async()=>{ + const c=cloudRehearsalContracts(config),adapter=await createCloudflareSchedulingAdapter({pin,schema:config.schema,recipe:c.recipe,environment:c.environment}); + const revisions=[]; + for(const variant of ['initial','later'] as const){ + const files=await cloudFixtureFiles(variant);expect(files['build.mjs']).not.toContain('index.html');expect(files['worker.mjs']).toContain(`LIVE_VARIANT = '${variant}'`);expect(files['worker.mjs']).toContain('env.COMMONFORGE_RELEASE_ID'); + const result=await adapter.verify(Object.fromEntries(Object.entries(files).map(([path,text])=>[path,new TextEncoder().encode(text)]))); + expect(result.results).toHaveLength(9);expect(result.results.every(r=>r.result==='pass')).toBe(true);expect(result.network).toBe('none');expect(result.controlPlaneMounted).toBe(false);revisions.push(canonicalJson(files)); + } + expect(revisions[0]).not.toBe(revisions[1]); + },30_000); +}); + +async function localSource() { + const root=await mkdtemp(join(tmpdir(),'g1-live-resume-')),seed=join(root,'seed'),remote=join(root,'remote.git'),git=createGit({allowFile:true}); + await mkdir(seed);await git.ok(['init','-q','-b','main'],seed);await writeFile(join(seed,'seed.txt'),'base');await git.ok(['add','.'],seed);await git.ok(['commit','-q','-m','base'],seed,{env:RUNNER_IDENTITY_ENV(new Date().toISOString())}); + await git.ok(['clone','-q','--bare',seed,remote],root); + return {root,git,remoteUrl:pathToFileURL(remote).href}; +} +describe('live source push continuation with real Git',()=>{ + it('fails closed on a name match without an owned checkpoint',()=>{ + expect(()=>requireFreshCloudProjectName([{name:'existing'}],'existing')).toThrow('name-only adoption'); + expect(()=>requireFreshCloudProjectName([{name:'different'}],'new')).not.toThrow(); + }); + it('resumes a push that landed before the client received success, even if the local workspace is gone',async()=>{ + const local=await localSource();let saved:CloudSourcePushCheckpoint={};let commits=0,pushes=0; + const git:Git={...local.git,async ok(args,cwd,options){if(args[0]==='commit')commits++;if(args[0]==='push'){pushes++;const output=await local.git.ok(args,cwd,options);throw new Error('injected disconnect after actual landed push');}return local.git.ok(args,cwd,options);}}; + try{ + const cycle:CloudSourcePushCheckpoint={},files={'worker.mjs':'export default {};\n','build.mjs':'// trusted builder\n','wrangler.json':'{}'}; + const persist=async()=>{saved=JSON.parse(JSON.stringify(cycle));}; + await expect(pushResumableCloudSource({cycle,files,label:'initial',checkpointDirectory:local.root,remoteUrl:local.remoteUrl,credentialSecret:'synthetic-file-transport',git,persist})).rejects.toThrow('disconnect'); + expect(saved.intendedSha).toMatch(/^[a-f0-9]{40}$/);expect(saved.sha).toBeUndefined();expect(saved.workspace).toBeTruthy();expect(saved.expectedRemoteSha).not.toBe(saved.intendedSha); + const observed=(await local.git.ok(['ls-remote',local.remoteUrl,'refs/heads/main'],local.root)).trim().split(/\s+/)[0];expect(observed).toBe(saved.intendedSha); + await rm(saved.workspace!,{recursive:true,force:true}); + const resumed={...saved};const result=await pushResumableCloudSource({cycle:resumed,files,label:'initial',checkpointDirectory:local.root,remoteUrl:local.remoteUrl,credentialSecret:'synthetic-file-transport',git,persist:async()=>{saved=JSON.parse(JSON.stringify(resumed));}}); + expect(result).toBe(observed);expect(saved.sha).toBe(observed);expect(commits).toBe(1);expect(pushes).toBe(1); + }finally{await rm(local.root,{recursive:true,force:true});} + }); + it('continues after the local commit landed before its intent checkpoint was saved',async()=>{ + const local=await localSource();let saved:CloudSourcePushCheckpoint={},commits=0;const cycle:CloudSourcePushCheckpoint={}; + const git:Git={...local.git,async ok(args,cwd,options){if(args[0]==='commit')commits++;return local.git.ok(args,cwd,options);}}; + const files={'worker.mjs':'export default {};\n'}; + try{ + await expect(pushResumableCloudSource({cycle,files,label:'initial',checkpointDirectory:local.root,remoteUrl:local.remoteUrl,credentialSecret:'synthetic-file-transport',git,persist:async()=>{if(cycle.intendedSha)throw new Error('injected crash before intent save');saved=JSON.parse(JSON.stringify(cycle));}})).rejects.toThrow('before intent save'); + expect(saved.intendedSha).toBeUndefined();expect(saved.workspace).toBeTruthy(); + const resumed={...saved};const sha=await pushResumableCloudSource({cycle:resumed,files,label:'initial',checkpointDirectory:local.root,remoteUrl:local.remoteUrl,credentialSecret:'synthetic-file-transport',git,persist:async()=>{saved=JSON.parse(JSON.stringify(resumed));}}); + expect(commits).toBe(1);expect(saved.sha).toBe(sha);expect((await local.git.ok(['ls-remote',local.remoteUrl,'refs/heads/main'],local.root)).split(/\s+/)[0]).toBe(sha); + }finally{await rm(local.root,{recursive:true,force:true});} + }); + it('retains intended commit before an unlanded push and refuses a later foreign remote change',async()=>{ + const local=await localSource();let saved:CloudSourcePushCheckpoint={};const cycle:CloudSourcePushCheckpoint={}; + const git:Git={...local.git,async ok(args,cwd,options){if(args[0]==='push')throw new Error('injected failure before push');return local.git.ok(args,cwd,options);}}; + const files={'worker.mjs':'export default {};\n'}; + try{ + await expect(pushResumableCloudSource({cycle,files,label:'initial',checkpointDirectory:local.root,remoteUrl:local.remoteUrl,credentialSecret:'synthetic-file-transport',git,persist:async()=>{saved=JSON.parse(JSON.stringify(cycle));}})).rejects.toThrow('before push'); + const foreign=join(local.root,'foreign');await local.git.ok(['clone','-q',local.remoteUrl,foreign],local.root);await writeFile(join(foreign,'foreign.txt'),'separate actor');await local.git.ok(['add','.'],foreign);await local.git.ok(['commit','-q','-m','foreign'],foreign,{env:RUNNER_IDENTITY_ENV(new Date().toISOString())});await local.git.ok(['push','-q','origin','HEAD:refs/heads/main'],foreign); + await expect(pushResumableCloudSource({cycle:{...saved},files,label:'initial',checkpointDirectory:local.root,remoteUrl:local.remoteUrl,credentialSecret:'synthetic-file-transport',git:local.git,persist:async()=>{}})).rejects.toThrow('refuse to overwrite'); + }finally{await rm(local.root,{recursive:true,force:true});} + }); +}); diff --git a/apps/runner/test/g1-cloudflare-bridge.test.ts b/apps/runner/test/g1-cloudflare-bridge.test.ts new file mode 100644 index 0000000..1c2568f --- /dev/null +++ b/apps/runner/test/g1-cloudflare-bridge.test.ts @@ -0,0 +1,51 @@ +/** Actual Git snapshot, HTTP signed request, isolated build/workerd, signed result. + * This is a local production-runtime bridge check, not a deployed Cloudflare release. */ +import { createRequire } from 'node:module'; +import { readFile } from 'node:fs/promises'; +import { dirname, join } from 'node:path'; +import { afterAll, beforeAll, describe, expect, it } from 'vitest'; +import { hashCanonicalSync, sha256HexSync } from '@commonforge-org/core'; +import { scenarioBuildDigest, signScenarioRequest, verifyScenarioResult, type ScenarioVerificationRequest } from '@commonforge-org/runner-protocol'; +import { KEY, World } from './helpers.js'; +import { startLocalScenarioBridge } from '../src/g1-local-bridge.js'; +import { createCloudflareSchedulingAdapter } from '../src/g1-cloudflare-source.js'; +const require = createRequire(new URL('../../worker/package.json', import.meta.url)); +const toolchain = dirname(require.resolve('workerd/package.json')); +const recipe = { recipeId: 'scheduling', revision: 1, steps: [{ stepId: 'bundle', mountName: 'api', command: 'node build.mjs' }] }; +const environment = { targets: [{ targetId: 'api', kind: 'worker' as const, mountName: 'api', outputPath: 'dist', main: 'worker.mjs', workerName: 'cfapp-g1-scheduling' }], environment: {}, accountId: '0'.repeat(32), healthCheck: { targetId: 'api', path: '/version' } }; +let world: World; +let bridge: Awaited>; +let request: ScenarioVerificationRequest; +beforeAll(async () => { + world = await World.create(); + const canonical = await world.canonical('res-api', { + 'worker.mjs': await readFile(new URL('../../../fixtures/scheduling/worker.mjs', import.meta.url), 'utf8'), + 'build.mjs': "import{mkdirSync,readFileSync,writeFileSync}from'node:fs';mkdirSync('dist');writeFileSync('dist/worker.mjs',readFileSync('worker.mjs','utf8').replaceAll('export const ','const '));", + 'wrangler.json': '{"compatibility_date":"2026-08-22"}', + }); + const runtime = await createCloudflareSchedulingAdapter({ pin: { binary: join(toolchain, 'bin/workerd'), sha256: sha256HexSync(await readFile(join(toolchain, 'bin/workerd'))) }, schema: join(toolchain, 'workerd.capnp'), recipe, environment }); + bridge = await startLocalScenarioBridge({ provider: world.provider, jobKey: KEY, runtime, resources: [{ organizationId: 'org1', projectId: 'project1', resourceId: 'res-api' }] }); + const now = new Date().toISOString(); + const sourceHeads = [{ resourceId: 'res-api', revisionId: canonical.revision }]; + request = { protocolVersion: 1, organizationId: 'org1', projectId: 'project1', runId: 'workerd-bridge-run', envelopeDigest: 'a'.repeat(64), sourceHeads, buildDigest: scenarioBuildDigest(sourceHeads), runtimeRecipeDigest: runtime.runtimeRecipeDigest, + requiredScenarios: runtime.scenarioIds.map(id => ({ id, revision: 1, verifierDigest: runtime.verifierDigest, method: 'automated' })), issuedAt: now, expiresAt: new Date(Date.parse(now) + 120000).toISOString() }; +}); +afterAll(async () => { await bridge?.close(); await world?.close(); }); +const submit = async (r: ScenarioVerificationRequest) => fetch(`${bridge.url}/verify`, { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify(await signScenarioRequest(r, KEY)) }); +describe('signed actual Git/workerd source workflow', () => { + it('binds all real results to exact revision, trusted verifier and release-compatible runtime', async () => { + const response = await submit(request); + expect(response.status).toBe(200); + const signed = await response.json(); + const verified = await verifyScenarioResult(signed, KEY, request); + expect(verified.evidence).toHaveLength(9); + expect(verified.evidence.every(e => e.result === 'pass' && e.provenance === 'runner-verified')).toBe(true); + expect(request.runtimeRecipeDigest).toBe(hashCanonicalSync({ domain: 'cf.project-runtime/1', recipe, environment })); + const retry = await submit(request); + expect(await retry.json()).toEqual(signed); + }); + it('refuses caller-selected runtime and cross-project source even with valid signing key', async () => { + expect((await submit({ ...request, runId: 'wrong-runtime', runtimeRecipeDigest: '0'.repeat(64) })).status).toBe(422); + expect((await submit({ ...request, runId: 'wrong-project', projectId: 'another-project' })).status).toBe(403); + }); +}); diff --git a/apps/runner/test/g1-cloudflare-failures.test.ts b/apps/runner/test/g1-cloudflare-failures.test.ts new file mode 100644 index 0000000..7777c9d --- /dev/null +++ b/apps/runner/test/g1-cloudflare-failures.test.ts @@ -0,0 +1,17 @@ +/** Actual isolated build/workerd failures; no cloud deployment or simulated success. */ +import { createRequire } from 'node:module'; +import { readFile } from 'node:fs/promises'; +import { dirname, join } from 'node:path'; +import { expect, it } from 'vitest'; +import { sha256HexSync } from '@commonforge-org/core'; +import { createCloudflareSchedulingAdapter } from '../src/g1-cloudflare-source.js'; +const require=createRequire(new URL('../../worker/package.json',import.meta.url)); +const toolchain=dirname(require.resolve('workerd/package.json')); +it('records isolated build and workerd startup errors without retaining candidate diagnostics',async()=>{ + const adapter=await createCloudflareSchedulingAdapter({pin:{binary:join(toolchain,'bin/workerd'),sha256:sha256HexSync(await readFile(join(toolchain,'bin/workerd')))},schema:join(toolchain,'workerd.capnp'),recipe:{recipeId:'test',revision:1,steps:[{stepId:'build',mountName:'api',command:'node build.mjs'}]},environment:{targets:[{targetId:'api',kind:'worker',mountName:'api',outputPath:'dist',main:'worker.mjs',workerName:'cfapp-g1-test'}],environment:{},accountId:'0'.repeat(32),healthCheck:{targetId:'api',path:'/version'}}}); + const encode=(s:string)=>new TextEncoder().encode(s); + const files=(build:string)=>({'worker.mjs':encode('export default { broken syntax'),'build.mjs':encode(build),'wrangler.json':encode('{"compatibility_date":"2026-08-22"}')}); + for(const build of ["throw Error('candidate-private-diagnostic');", "import{mkdirSync,copyFileSync}from'node:fs';mkdirSync('dist');copyFileSync('worker.mjs','dist/worker.mjs');"]){ + const result=await adapter.verify(files(build));expect(result.results).toHaveLength(9);expect(result.results.every(r=>r.result==='error')).toBe(true);expect(JSON.stringify(result)).not.toContain('candidate-private-diagnostic'); + } +}); diff --git a/apps/runner/test/g1-cloudflare-server.test.ts b/apps/runner/test/g1-cloudflare-server.test.ts new file mode 100644 index 0000000..56cf33b --- /dev/null +++ b/apps/runner/test/g1-cloudflare-server.test.ts @@ -0,0 +1,21 @@ +/** Source authority tests use a transport double; actual Git/workerd execution is in bridge tests. */ +import { describe, expect, it, vi } from 'vitest'; +import type { ArtifactsTransport } from '@commonforge-org/provider-cloudflare'; +import type { ScenarioVerificationRequest } from '@commonforge-org/runner-protocol'; +import { registeredArtifactsSource } from '../src/g1-cloudflare-server.js'; + +describe('operator-registered Cloudflare source',()=>{ + it('reads only the pinned repository, immutable revision and operator prefix',async()=>{ + const readFile=vi.fn().mockResolvedValue(new Uint8Array([1,2,3])); + const transport={getRepo:vi.fn().mockResolvedValue({id:'repo1'}),readFile} as unknown as ArtifactsTransport; + const read=registeredArtifactsSource(transport,[{organizationId:'org1',projectId:'p1',resourceId:'res1',namespace:'operator-ns',repoName:'operator-repo',repoId:'repo1',pathPrefix:'fixture/'}]); + const request={organizationId:'org1',projectId:'p1',sourceHeads:[{resourceId:'res1',revisionId:`art:repo1:${'a'.repeat(40)}`}]} as ScenarioVerificationRequest; + expect(await read(request,['worker.mjs'])).toEqual({'worker.mjs':new Uint8Array([1,2,3])}); + expect(readFile).toHaveBeenCalledWith('operator-ns','operator-repo','a'.repeat(40),'fixture/worker.mjs'); + readFile.mockClear(); + await expect(read({...request,projectId:'other'},['worker.mjs'])).rejects.toThrow(/scope/); + await expect(read({...request,sourceHeads:[{resourceId:'res1',revisionId:`art:other:${'a'.repeat(40)}`}]},['worker.mjs'])).rejects.toThrow(/scope/); + await expect(read(request,['../secret'])).rejects.toThrow(/path/); + expect(readFile).not.toHaveBeenCalled(); + }); +}); diff --git a/apps/runner/test/g1-cloudflare-source.test.ts b/apps/runner/test/g1-cloudflare-source.test.ts new file mode 100644 index 0000000..2f088ed --- /dev/null +++ b/apps/runner/test/g1-cloudflare-source.test.ts @@ -0,0 +1,43 @@ +import { createRequire } from 'node:module'; +import { mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { dirname, join } from 'node:path'; +import { afterAll, describe, expect, it } from 'vitest'; +import { sha256HexSync } from '@commonforge-org/core'; +import { createCloudflareSchedulingAdapter } from '../src/g1-cloudflare-source.js'; +const require = createRequire(new URL('../../worker/package.json', import.meta.url)); +const toolchain = dirname(require.resolve('workerd/package.json')); +const pin = { binary: join(toolchain, 'bin/workerd'), sha256: sha256HexSync(await readFile(join(toolchain, 'bin/workerd'))) }; +const recipe = { recipeId: 'scheduling', revision: 1, steps: [{ stepId: 'bundle', mountName: 'api', command: 'node build.mjs' }] }; +const environment = { targets: [{ targetId: 'api', kind: 'worker' as const, mountName: 'api', outputPath: 'dist', main: 'worker.mjs', workerName: 'cfapp-g1-scheduling' }], + environment: {}, accountId: '0'.repeat(32), healthCheck: { targetId: 'api', path: '/version' } }; +const adapter = await createCloudflareSchedulingAdapter({ pin, schema: join(toolchain, 'workerd.capnp'), recipe, environment }); +const encode = (s: string) => new TextEncoder().encode(s); +const module = await readFile(new URL('../../../fixtures/scheduling/worker.mjs', import.meta.url), 'utf8'); +const build = "import{mkdirSync,readFileSync,writeFileSync}from'node:fs';mkdirSync('dist');writeFileSync('dist/worker.mjs',readFileSync('worker.mjs','utf8').replaceAll('export const ','const '));"; +const files = (worker = module, builder = build) => ({ 'worker.mjs': encode(worker), 'build.mjs': encode(builder), 'wrangler.json': encode('{"compatibility_date":"2026-08-22"}') }); +const cleanup: string[] = []; +afterAll(async () => { for (const path of cleanup) await rm(path, { recursive: true, force: true }); }); +describe('trusted source-to-Cloudflare adapter', () => { + it('really executes candidate build then workerd checks; candidate tests cannot weaken it', async () => { + const good = await adapter.verify(files()); + expect(good.results.every(r => r.result === 'pass')).toBe(true); + expect(good.verifierDigest).toBe(adapter.verifierDigest); + const bad = await adapter.verify(files('export default {fetch(){return Response.json({allowed:true,override:null})}}')); + expect(bad.results.some(r => r.result === 'fail')).toBe(true); + }); + it('isolates build and runtime from controlling paths and network', async () => { + const root = await mkdtemp(join(tmpdir(), 'commonforge-control-key-')); cleanup.push(root); + const secret = join(root, 'key'); await writeFile(secret, 'control-supervisor-secret'); + const guardedBuild = `import{existsSync}from'node:fs';import{networkInterfaces}from'node:os';if(existsSync(${JSON.stringify(secret)})||Object.values(networkInterfaces()).flat().some(x=>x&&!x.internal))process.exit(5);${build}`; + // Runtime global fetch is explicitly deny-all as well as having no host network namespace. + const guardedModule = module.replace('export default', 'const candidate =') + '\nexport default {async fetch(request,env){try{await fetch("http://127.0.0.1:1/control");return Response.json({allowed:false,code:"network_leak"})}catch{}if(Object.values(env).includes("control-supervisor-secret"))throw Error("leak");return candidate.fetch(request,env)}};'; + const result = await adapter.verify(files(guardedModule, guardedBuild)); + expect(result.results.every(r => r.result === 'pass')).toBe(true); + }); + it('refuses symlink outputs, extra deployment modules and unsupported recipes', async () => { + await expect(adapter.verify(files(module, "import{mkdirSync,symlinkSync}from'node:fs';mkdirSync('dist');symlinkSync('../worker.mjs','dist/worker.mjs');"))).rejects.toThrow(); + await expect(adapter.verify(files(module, build + "writeFileSync('dist/extra.mjs','export default {}');"))).rejects.toThrow('one bundled'); + await expect(createCloudflareSchedulingAdapter({ pin, schema: join(toolchain, 'workerd.capnp'), recipe: { ...recipe, steps: [{ ...recipe.steps[0]!, command: 'curl example.test' }] }, environment })).rejects.toThrow('one node'); + }); +}); diff --git a/apps/runner/test/g1-cloudflare-verifier.test.ts b/apps/runner/test/g1-cloudflare-verifier.test.ts new file mode 100644 index 0000000..8b308da --- /dev/null +++ b/apps/runner/test/g1-cloudflare-verifier.test.ts @@ -0,0 +1,58 @@ +import { createRequire } from 'node:module'; +import { readFile } from 'node:fs/promises'; +import { dirname, join } from 'node:path'; +import { afterAll, beforeAll, describe, expect, it } from 'vitest'; +import { hashCanonicalSync, sha256HexSync } from '@commonforge-org/core'; +import type { ReleaseBuildReceipt } from '@commonforge-org/runner-protocol'; +import { World } from './helpers.js'; +import { cloudflareRuntimeRecipeDigest, verifyCloudflareBuiltScheduling, type CloudflareReleaseEnvironment } from '../src/g1-cloudflare-verifier.js'; + +// Resolve the actual pinned Worker toolchain; no downloads and no optional fake fallback. +const require = createRequire(new URL('../../worker/package.json', import.meta.url)); +const toolchain = dirname(require.resolve('workerd/package.json')); +const pin = { binary: join(toolchain, 'bin/workerd'), sha256: sha256HexSync(await readFile(join(toolchain, 'bin/workerd'))) }; +const schema = join(toolchain, 'workerd.capnp'); +let world: World; +let built: ReleaseBuildReceipt; +const recipe = { recipeId: 'scheduling-cloudflare', revision: 1, steps: [{ stepId: 'bundle', mountName: 'api', command: 'node build.mjs' }] }; +const target = { targetId: 'api', kind: 'worker' as const, mountName: 'api', outputPath: 'dist', main: 'worker.mjs', workerName: 'cfapp-g1-scheduling' }; +const environment: CloudflareReleaseEnvironment = { targets: [target], environment: {}, accountId: '0123456789abcdef0123456789abcdef', healthCheck: { targetId: 'api', path: '/version' } }; +const prefix = 'org/org1/releases/rel-cloudflare-test/'; +const readOutput = (path: string) => readFile(world.blobPath(`${prefix}api/${path}`)); +const verify = (extra: Partial[0]> = {}) => verifyCloudflareBuiltScheduling({ pin, schema, recipe, environment, built, readOutput, ...extra }); + +beforeAll(async () => { + world = await World.create(); + const fixture = await readFile(new URL('../../../fixtures/scheduling/worker.mjs', import.meta.url), 'utf8'); + const canonical = await world.canonical('res-api', { + 'worker.mjs': fixture, + 'wrangler.json': JSON.stringify({ compatibility_date: '2026-08-22' }), + 'build.mjs': "import{mkdirSync,readFileSync,writeFileSync}from'node:fs';mkdirSync('dist');writeFileSync('dist/worker.mjs',readFileSync('worker.mjs','utf8').replaceAll('export const ','const '));", + }); + const result = await world.runner().execute(await world.delivery({ kind: 'release-build', releaseId: 'rel-cloudflare-test', acceptedVersionId: 'av-g1', + sources: [{ resourceId: 'res-api', mountName: 'api', canonical: canonical.coords, sha: canonical.sha }], recipe, targets: [target], environment: {}, outputPrefix: prefix })); + if (!result.ok) throw new Error(`actual release-build failed: ${JSON.stringify(result)}`); + built = result.receipt as ReleaseBuildReceipt; +}); +afterAll(() => world?.close()); + +describe('Cloudflare scheduling runtime (actual release-build + isolated workerd)', () => { + it('executes all nine supervisor cases against exact immutable release output bytes', async () => { + const result = await verify(); + expect(result.results).toHaveLength(9); + expect(result.results.every(r => r.result === 'pass')).toBe(true); + expect(result).toMatchObject({ engine: 'workerd', network: 'none', controlPlaneMounted: false, outputDigest: built.outputDigest }); + expect(result.runtimeRecipeDigest).toBe(hashCanonicalSync({ domain: 'cf.project-runtime/1', recipe, environment })); + expect(result.verifierDigest).toMatch(/^[a-f0-9]{64}$/); + }); + it('rejects corrupted build bytes, changed output receipts and an unpinned executable', async () => { + await expect(verify({ readOutput: async () => new TextEncoder().encode('export default {}') })).rejects.toThrow('bytes mismatch'); + await expect(verify({ built: { ...built, outputDigest: '0'.repeat(64) } })).rejects.toThrow('digest mismatch'); + await expect(verify({ pin: { ...pin, sha256: '0'.repeat(64) } })).rejects.toThrow('supervisor pin'); + }); + it('refuses binding/migration emulation and binds environment changes to a different runtime', async () => { + await expect(verify({ environment: { ...environment, migrations: [{}] } })).rejects.toThrow('migrations'); + await expect(verify({ environment: { ...environment, environment: { api: { vars: {}, kvNamespaces: [{}] } } } })).rejects.toThrow('bindings'); + expect(cloudflareRuntimeRecipeDigest(recipe, { ...environment, accountId: 'f'.repeat(32) })).not.toBe(cloudflareRuntimeRecipeDigest(recipe, environment)); + }); +}); diff --git a/apps/runner/test/g1-isolation-unavailable.test.ts b/apps/runner/test/g1-isolation-unavailable.test.ts new file mode 100644 index 0000000..e0119d2 --- /dev/null +++ b/apps/runner/test/g1-isolation-unavailable.test.ts @@ -0,0 +1,22 @@ +/** Explicit spawn failure double: verifies refusal classification, not real namespace behavior. */ +import { describe, expect, it, vi } from 'vitest'; +import { mkdtemp, rm, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +vi.mock('node:child_process', async () => { + const { EventEmitter } = await import('node:events'); + const { PassThrough } = await import('node:stream'); + return { spawn: () => { + const child = Object.assign(new EventEmitter(), { stdout: new PassThrough(), stderr: new PassThrough(), exitCode:null, signalCode:null, pid:undefined }); + queueMicrotask(()=>child.emit('error',new Error('isolation unavailable'))); + return child; + } }; +}); +import { verifySchedulingCandidate } from '../src/g1-local-verifier.js'; +describe('unavailable isolation refusal',()=>{ + it('throws instead of fabricating signed scenario execution and cleans up without an exit event',async()=>{ + const dir=await mkdtemp(join(tmpdir(),'cf-no-isolation-')); + try {await writeFile(join(dir,'worker.mjs'),'export default {};');await expect(verifySchedulingCandidate(dir)).rejects.toThrow('isolation unavailable');} + finally {await rm(dir,{recursive:true,force:true});} + }); +}); diff --git a/apps/runner/test/g1-local-preview.test.ts b/apps/runner/test/g1-local-preview.test.ts new file mode 100644 index 0000000..0edceb2 --- /dev/null +++ b/apps/runner/test/g1-local-preview.test.ts @@ -0,0 +1,15 @@ +import { readFile } from 'node:fs/promises'; +import { expect, it } from 'vitest'; +import { startLocalPreview } from '../src/g1-local-preview.js'; +it('serves a real API module in namespaces with no candidate-controlled supervisor or network', async () => { + const preview = await startLocalPreview({ key: 'local-preview-auth-key' }); + try { + const module = (await readFile(new URL('../../../fixtures/scheduling/worker.mjs', import.meta.url))).toString('base64'); + const response = await fetch(`${preview.url}/serve`, { method: 'POST', headers: { authorization: 'Bearer local-preview-auth-key', 'content-type': 'application/json' }, body: JSON.stringify({ module, path: '/reschedule', method: 'POST', body: JSON.stringify({ actor: 'manager', appointment: '2026-10-17T13:00:00.000Z', reason: 'Preview emergency' }) }) }); + expect(response.status).toBe(200); + const result = await response.json() as { status: number; body: string }; + expect(result.status).toBe(200); + expect(JSON.parse(Buffer.from(result.body,'base64').toString('utf8'))).toMatchObject({ allowed: true, override: { actor: 'manager', reason: 'Preview emergency' } }); + expect((await fetch(`${preview.url}/serve`, { method: 'POST', body: '{}' })).status).toBe(403); + } finally { await preview.close(); } +}); diff --git a/apps/runner/test/g1-local-verifier.test.ts b/apps/runner/test/g1-local-verifier.test.ts new file mode 100644 index 0000000..948efd0 --- /dev/null +++ b/apps/runner/test/g1-local-verifier.test.ts @@ -0,0 +1,31 @@ +import { mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { afterAll, describe, expect, it } from 'vitest'; +import { verifySchedulingCandidate } from '../src/g1-local-verifier.js'; +const dirs: string[] = []; +afterAll(async () => { for (const d of dirs) await rm(d, { recursive: true, force: true }); }); +async function candidate(source: string) { const d = await mkdtemp(join(tmpdir(), 'commonforge-g1-test-')); dirs.push(d); await writeFile(join(d, 'worker.mjs'), source); return d; } +const fixture = new URL('../../../fixtures/scheduling/worker.mjs', import.meta.url); + +describe('trusted real scheduling verifier', () => { + it('executes HTTP behavior across the exact cutoff, travel and retained manager reason', async () => { + const r = await verifySchedulingCandidate(await candidate(await readFile(fixture, 'utf8'))); + expect(r.results).toHaveLength(9); + expect(r.results.every((r) => r.result === 'pass')).toBe(true); + expect(r).toMatchObject({ network: 'none', controlPlaneMounted: false }); + }); + it('candidate-owned tests cannot weaken the independently pinned harness', async () => { + const d = await candidate('export default { fetch() { return Response.json({allowed:true,override:null}); } };'); + await writeFile(join(d, 'test.mjs'), 'process.exit(0)'); + const r = await verifySchedulingCandidate(d); + expect(r.results.some((r) => r.result === 'fail')).toBe(true); + }); + it('candidate cannot read a controlling-instance secret or reach its network', async () => { + const secretDir = await mkdtemp(join(tmpdir(), 'commonforge-control-')); dirs.push(secretDir); + const secret = join(secretDir, 'supervisor-key'); await writeFile(secret, 'must-not-be-readable'); + const source = `import {existsSync} from 'node:fs';\nimport {networkInterfaces} from 'node:os';\nconst isolated = !existsSync(${JSON.stringify(secret)}) && !Object.values(networkInterfaces()).flat().some(x => x && !x.internal);\n${await readFile(fixture, 'utf8')}`.replace('return Response.json(reschedule(await request.json()))', "return Response.json(isolated ? reschedule(await request.json()) : {allowed:false,code:'isolation_broken'})"); + const r = await verifySchedulingCandidate(await candidate(source)); + expect(r.results.every((r) => r.result === 'pass')).toBe(true); + }); +}); diff --git a/apps/runner/test/g1-real-local.test.ts b/apps/runner/test/g1-real-local.test.ts new file mode 100644 index 0000000..16d13f6 --- /dev/null +++ b/apps/runner/test/g1-real-local.test.ts @@ -0,0 +1,15 @@ +import { describe, expect, it } from 'vitest'; +import { runRealLocal } from '../src/g1-real-local.js'; +describe('G1 real local source/runner/preview rehearsal', () => { + it('uses real local Git, a signed Node runner, disk blobs, HTTP preview, and isolated trusted scenarios', async () => { + const local = await runRealLocal(); + try { + expect(local.report.gates).toMatchObject({ realGit: 'pass', signedNodeRunner: 'pass', localBlobStore: 'pass', httpPreview: 'pass', trustedIsolatedScenarios: 'pass', workerWorkflow: 'not_run', cloudRelease: 'not_run' }); + const capabilities = await (await fetch(`${local.report.bridgeUrl}/capabilities`)).json() as { mode: string }; + expect(capabilities.mode).toBe('real-local'); + expect((await fetch(`${local.report.bridgeUrl}/verify`, { method: 'POST', body: '{}' })).status).toBe(400); + expect((await fetch(`${local.report.bridgeUrl}/capabilities`, { headers: { origin: 'https://hostile.example' } })).status).toBe(403); + expect(local.report.verification.evidence.every(e => e.provenance === 'runner-verified')).toBe(true); + } finally { await local.close(); } + }); +}); diff --git a/apps/runner/test/g1-scenario-failures.test.ts b/apps/runner/test/g1-scenario-failures.test.ts new file mode 100644 index 0000000..50b4dc8 --- /dev/null +++ b/apps/runner/test/g1-scenario-failures.test.ts @@ -0,0 +1,46 @@ +/** Actual namespace startup failure and retry; no mocked isolation success. */ +import { mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { describe, expect, it } from 'vitest'; +import { sha256HexSync } from '@commonforge-org/core'; +import { verifySchedulingCandidate } from '../src/g1-local-verifier.js'; +import { startLocalScenarioBridge, SCHEDULING_RUNTIME_RECIPE_DIGEST } from '../src/g1-local-bridge.js'; +import { signScenarioRequest, verifyScenarioResult, scenarioBuildDigest } from '@commonforge-org/runner-protocol'; +import { schedulingVerifierDigest } from '../src/g1-local-verifier.js'; + +describe('durable scenario execution failures', () => { + it('records syntax/startup errors after namespace startup and signs complete errors before a passing retry', async () => { + const bad = Buffer.from('export default { broken syntax'); + let bytes = bad; + const dir = await mkdtemp(join(tmpdir(),'cf-failed-candidate-')); + await writeFile(join(dir,'worker.mjs'),bad); + const key = { kid:'failure-test', secret:'f'.repeat(64) }; + const verifierDigest=await schedulingVerifierDigest(); + const bridge=await startLocalScenarioBridge({jobKey:key,readCandidate:async()=>bytes,resources:[]}); + try { + const result=await verifySchedulingCandidate(dir); + expect(result.sourceDigest).toBe(sha256HexSync(bad)); + expect(result.results).toHaveLength(9);expect(result.results.every(r=>r.result==='error')).toBe(true); + const sourceHeads=[{resourceId:'source',revisionId:'art:repo:'+ 'a'.repeat(40)}]; + const issue=(runId:string)=>({protocolVersion:1 as const,organizationId:'org',projectId:'project',runId,envelopeDigest:'a'.repeat(64),buildDigest:scenarioBuildDigest(sourceHeads),runtimeRecipeDigest:SCHEDULING_RUNTIME_RECIPE_DIGEST,sourceHeads,requiredScenarios:[{id:'cutoff-before',revision:1,verifierDigest,method:'automated' as const}],issuedAt:new Date().toISOString(),expiresAt:new Date(Date.now()+60000).toISOString()}); + const request=issue('failed-run'); + // Select an actual trusted scenario identity advertised by this bridge. + const caps=await (await fetch(bridge.url+'/capabilities')).json() as {scenarioIds:string[]};request.requiredScenarios[0]!.id=caps.scenarioIds[0]!; + const dispatch=async(q:typeof request)=>fetch(bridge.url+'/verify',{method:'POST',body:JSON.stringify(await signScenarioRequest(q,key))}); + const first=await dispatch(request);expect(first.status).toBe(200); + const failed=await verifyScenarioResult(await first.json(),key,request);expect(failed.evidence[0]!.result).toBe('error'); + expect(JSON.stringify(failed)).not.toContain('broken syntax'); + bytes=Buffer.from(await readFile(new URL('../../../fixtures/scheduling/worker.mjs',import.meta.url))); + const repeated=await dispatch(request);expect((await verifyScenarioResult(await repeated.json(),key,request)).evidence[0]!.result).toBe('error'); + const retry={...issue('fresh-run'),requiredScenarios:request.requiredScenarios}; + const passed=await dispatch(retry);expect((await verifyScenarioResult(await passed.json(),key,retry)).evidence[0]!.result).toBe('pass'); + } finally {await bridge.close();await rm(dir,{recursive:true,force:true});} + }); + it('records a candidate startup timeout as errors only after isolated supervisor startup',async()=>{ + const dir=await mkdtemp(join(tmpdir(),'cf-startup-timeout-')); + try {await writeFile(join(dir,'worker.mjs'),'while(true){};export default {};');const result=await verifySchedulingCandidate(dir);expect(result.results.every(r=>r.result==='error')).toBe(true);expect(result.results[0]!.detail).toContain('timed out');} + finally{await rm(dir,{recursive:true,force:true});} + },10000); + +}); diff --git a/apps/web/e2e/fixture.spec.ts b/apps/web/e2e/fixture.spec.ts index c8c92f5..25060c3 100644 --- a/apps/web/e2e/fixture.spec.ts +++ b/apps/web/e2e/fixture.spec.ts @@ -489,6 +489,7 @@ test.describe("fixture: eight-minute path", () => { await page.getByRole("button", { name: "New project" }).click(); await expect(page.getByLabel("Project name")).toHaveValue(""); await page.getByLabel("Project name").fill("Intranet"); + await page.getByLabel("Project workflow").selectOption("legacy"); await page.getByRole("button", { name: "Create project" }).click(); await expect(page.getByRole("region", { name: "Folders" })).toBeVisible(); await expect(page.getByText("This project has no folders yet.")).toBeVisible(); @@ -538,6 +539,7 @@ test.describe("fixture: eight-minute path", () => { await expect(page.getByRole("note")).toContainText("Fixture mode"); await page.getByRole("button", { name: "New project" }).click(); await page.getByLabel("Project name").fill("Intranet"); + await page.getByLabel("Project workflow").selectOption("legacy"); await page.getByRole("button", { name: "Create project" }).click(); const add = page.getByRole("form", { name: "New folder" }); await add.getByLabel("New folder").fill("media"); @@ -615,6 +617,7 @@ test.describe("fixture: eight-minute path", () => { await expect(page.getByRole("note")).toContainText("Fixture mode"); await page.getByRole("button", { name: "New project" }).click(); await page.getByLabel("Project name").fill("Intranet"); + await page.getByLabel("Project workflow").selectOption("legacy"); await page.getByRole("button", { name: "Create project" }).click(); const add = page.getByRole("form", { name: "New folder" }); await add.getByLabel("New folder").fill("media"); @@ -1069,6 +1072,7 @@ test.describe("fixture: getting started in a brand-new organization", () => { // 1. the checklist's New project opens the form await org.getByRole("button", { name: "New project" }).click(); await page.getByLabel("Project name").fill("Bakery site"); + await page.getByLabel("Project workflow").selectOption("legacy"); await page.getByRole("button", { name: "Create project" }).click(); await expect(page.getByRole("region", { name: "Folders" })).toBeVisible(); await expect(page.getByText("This project has no folders yet.")).toBeVisible(); diff --git a/apps/web/e2e/local-stack.spec.ts b/apps/web/e2e/local-stack.spec.ts index 44936da..d6a6463 100644 --- a/apps/web/e2e/local-stack.spec.ts +++ b/apps/web/e2e/local-stack.spec.ts @@ -700,6 +700,7 @@ test("Getting started on a fresh organization: Git import, release example, firs await expect(gs).toContainText("1 of 6 done"); await gs.getByRole("button", { name: "New project" }).click(); await page.getByLabel("Project name").fill("Bakery site"); + await page.getByLabel("Project workflow").selectOption("legacy"); await page.getByRole("button", { name: "Create project" }).click(); await expect(page.getByText("This project has no folders yet.")).toBeVisible(); const orgId = /\/o\/([^/]+)\/p\//.exec(page.url())![1]!; @@ -739,9 +740,9 @@ test("Getting started on a fresh organization: Git import, release example, firs await panel.getByRole("button", { name: "Publish this version" }).click(); await panel.getByRole("button", { name: "Approve publishing this release" }).click({ timeout: 30_000 }); await panel.getByRole("button", { name: "Publish to production" }).click(); - await expect(panel.locator(".badge", { hasText: /^Live$/ })).toBeVisible({ timeout: 45_000 }); - // the version list follows without a reload - await expect(page.locator("li.version-live").locator(".badge", { hasText: /^Live$/ })).toBeVisible({ timeout: 20_000 }); + // First publication intentionally removes the publish panel once it is live + // (Versions.tsx publishTarget). Wait for the durable timeline state, not that transient panel. + await expect(page.locator("li.version-live").locator(".badge", { hasText: /^Live$/ })).toBeVisible({ timeout: 45_000 }); // 4. staging: the Staging example; production is live, so staging starts at once (no waiting) await overview(); diff --git a/apps/web/e2e/native-outcomes.spec.ts b/apps/web/e2e/native-outcomes.spec.ts new file mode 100644 index 0000000..10fa090 --- /dev/null +++ b/apps/web/e2e/native-outcomes.spec.ts @@ -0,0 +1,51 @@ +/** Actual browser + Worker API; creates task-owned synthetic records with no mocked responses. */ +import { readFileSync } from 'node:fs'; +import { expect, test } from '@playwright/test'; + +test('owner revises outcome intent, sees concurrent-edit refusal, and retains exact history after reload',async({page},info)=>{ + const sessionPath=process.env.COMMONFORGE_NATIVE_SESSION_FILE; + if(!sessionPath)throw Error('Outcome owner journey requires the real-local task-owned owner session.'); + const session=JSON.parse(readFileSync(sessionPath,'utf8')) as {ownerToken:string}; + await page.setExtraHTTPHeaders({authorization:`Bearer ${session.ownerToken}`}); + await page.goto('/'); + const coordinates=await page.evaluate(async()=>{ + async function post(path:string,body:unknown){const response=await fetch('/api'+path,{method:'POST',headers:{'content-type':'application/json'},body:JSON.stringify(body)});if(!response.ok)throw Error('Synthetic outcome project creation failed');return response.json();} + const org=await post('/orgs',{name:`Outcome revision rehearsal ${Date.now()}`}); + const project=await post(`/orgs/${org.orgId}/projects`,{name:'Outcome revision history',mode:'agent-native'}); + return {orgId:org.orgId,projectId:project.projectId}; + }); + await page.goto(`/o/${coordinates.orgId}/p/${coordinates.projectId}/outcomes`); + await page.getByLabel('Desired improvement',{exact:true}).fill('Preserve the confirmed manager exception'); + await page.getByLabel('Success criteria (one per line)',{exact:true}).fill('A fresh agent retains the reason requirement'); + await page.getByRole('button',{name:'Create outcome',exact:true}).click(); + const article=page.locator('article.update-card').filter({has:page.getByRole('heading',{name:'Preserve the confirmed manager exception',exact:true})}); + await expect(article).toContainText('Status: open · revision 1'); + await article.getByRole('button',{name:'Revise outcome',exact:true}).click(); + await expect(page.getByLabel('Desired improvement',{exact:true})).toBeFocused(); + await page.getByLabel('Desired improvement',{exact:true}).fill('Unsaved text must be discarded only on cancel'); + await page.getByRole('button',{name:'Cancel outcome edit',exact:true}).click(); + await expect(page.getByLabel('Desired improvement',{exact:true})).toHaveValue(''); + await expect(article).toContainText('Status: open · revision 1'); + await article.getByRole('button',{name:'Revise outcome',exact:true}).click(); + await page.getByLabel('Desired improvement',{exact:true}).fill('Retain the manager exception and its audit reason'); + // A separate status action advances the same authoritative record while the form is open. + await article.getByRole('button',{name:'Mark achieved',exact:true}).click(); + await expect(page.getByRole('alert')).toContainText('This outcome changed while you were editing'); + await expect(page.getByRole('button',{name:'Save outcome revision',exact:true})).toBeDisabled(); + await expect(page.getByLabel('Desired improvement',{exact:true})).toHaveValue('Retain the manager exception and its audit reason'); + await page.getByRole('button',{name:'Cancel outcome edit',exact:true}).click(); + await article.getByRole('button',{name:'Revise outcome',exact:true}).click(); + await page.getByLabel('Desired improvement',{exact:true}).fill('Retain the manager exception and its audit reason'); + await page.getByLabel('Success criteria (one per line)',{exact:true}).fill('A fresh agent retains the reason requirement\nUnauthorized overrides stay denied'); + await page.getByLabel('Outcome status',{exact:true}).selectOption('deferred'); + await page.getByRole('button',{name:'Save outcome revision',exact:true}).click(); + await expect(page.getByRole('heading',{name:'Retain the manager exception and its audit reason',exact:true})).toBeVisible(); + await expect(page.getByLabel('Desired improvement',{exact:true})).toHaveValue(''); + await page.reload(); + await expect(page.locator('article.update-card')).toContainText('Status: deferred · revision 3'); + await page.locator('article.update-card summary').filter({hasText:'Outcome revision history'}).click(); + for(const text of ['Revision 1 · open','Revision 2 · achieved','Revision 3 · deferred','A fresh agent retains the reason requirement','Unauthorized overrides stay denied'])await expect(page.getByText(text,{exact:true}).first()).toBeVisible(); + await expect(page.locator('article.update-card details')).toContainText('Preserve the confirmed manager exception'); + await expect(page.locator('article.update-card details')).toContainText('recorded by'); + await page.screenshot({path:info.outputPath('outcome-revision-history.png'),fullPage:true}); +}); diff --git a/apps/web/e2e/native.spec.ts b/apps/web/e2e/native.spec.ts new file mode 100644 index 0000000..12ba6c9 --- /dev/null +++ b/apps/web/e2e/native.spec.ts @@ -0,0 +1,196 @@ +/** Actual browser + real Worker API. No intercepted requests or fixture provider results. */ +import { readFileSync, writeFileSync } from 'node:fs'; +import { expect, test, type Page } from '@playwright/test'; + +const OWNER = process.env.COMMONFORGE_NATIVE_OWNER ?? 'owner@agency.e2e.test'; +const LOG = process.env.COMMONFORGE_NATIVE_LOG ?? '/tmp/g1-real-worker.log'; +async function signedIn(page: Page) { + if (process.env.COMMONFORGE_NATIVE_SESSION_FILE) { + const session = JSON.parse(readFileSync(process.env.COMMONFORGE_NATIVE_SESSION_FILE, 'utf8')) as { ownerToken: string }; + await page.setExtraHTTPHeaders({ authorization: `Bearer ${session.ownerToken}` }); + await page.goto('/'); + await expect(page.getByRole('button', { name: 'Sign out' })).toBeVisible(); + return; + } + const start = readFileSync(LOG, 'utf8').length; + await page.goto('/signin'); + await page.getByLabel('Work email').fill(OWNER); + await page.getByRole('button', { name: 'Email me a sign-in link' }).click(); + await expect(page.getByRole('status')).toContainText('Check your email'); + let link: string | undefined; + await expect.poll(() => { + const lines = readFileSync(LOG, 'utf8').slice(start).split('\n'); + link = lines.filter(line => line.includes(`magic link for ${OWNER}: `)).at(-1)?.match(/https?:\/\/[^\s]+/)?.[0]; + return !!link; + }).toBe(true); + // Secret link is intentionally neither logged nor persisted in a test artifact. + await page.goto(link!); + await page.getByRole('button', { name: 'Sign in', exact: true }).click(); + await expect(page.getByRole('button', { name: 'Sign out' })).toBeVisible(); +} +async function api(page: Page, path: string, body: unknown) { + return page.evaluate(async ({ path, body }) => { + const r = await fetch('/api' + path, { method: 'POST', credentials: 'same-origin', headers: { 'content-type': 'application/json' }, body: JSON.stringify(body) }); + const data = await r.json(); + if (!r.ok) throw new Error(`API ${r.status}: ${data.error?.message ?? 'operation failed'}`); + return data; + }, { path, body }); +} + +test('owner preserves a correction and sees honest outcomes and review gaps using the real API', async ({ page }, info) => { + await signedIn(page); + const org = await api(page, '/orgs', { name: `Native browser rehearsal ${Date.now()}` }); + const project = await api(page, `/orgs/${org.orgId}/projects`, { name: 'Durable understanding browser rehearsal' }); + const root = `/o/${org.orgId}/p/${project.projectId}`; + await page.goto(root + '/knowledge'); + await page.getByRole('button', { name: 'Enable project understanding' }).click(); + await expect(page.getByRole('heading', { name: 'Knowledge', exact: true })).toBeVisible(); + const initial = 'Customers may reschedule until 24 hours before their appointment.'; + const correction = 'Managers may override the 24-hour cutoff only with a retained reason.'; + await page.getByLabel('Requirement or understanding').fill(initial); + await page.getByRole('textbox', { name: 'Rationale', exact: true }).fill('Predictable dispatch with recorded owner intent.'); + await page.getByLabel('Source or provenance').fill('Synthetic owner requirement in the real API browser rehearsal.'); + await page.getByRole('button', { name: 'Save draft', exact: true }).click(); + await expect(page.getByRole('button', { name: 'Confirm rule' })).toBeVisible(); + await page.getByRole('button', { name: 'Confirm rule' }).click(); + await expect(page.getByText('rule · revision 2 · Confirmed', { exact: true })).toBeVisible(); + await page.getByRole('button', { name: 'Correct this record' }).click(); + await page.getByLabel('Requirement or understanding').fill(correction); + await page.getByRole('textbox', { name: 'Rationale', exact: true }).fill('Manager exception must survive subsequent agent sessions.'); + await page.getByRole('button', { name: 'Save draft', exact: true }).click(); + await page.getByRole('button', { name: 'Confirm rule' }).click(); + await expect(page.getByText('rule · revision 4 · Confirmed', { exact: true })).toBeVisible(); + await page.reload(); + await expect(page.getByText(correction, { exact: true }).first()).toBeVisible(); + await page.getByLabel('Title', { exact: true }).fill('Manager override retains its reason'); + await page.getByLabel('Given', { exact: true }).fill('An appointment is less than 24 hours away and a manager is authorized'); + await page.getByLabel('When', { exact: true }).fill('The manager overrides the cutoff with a reason'); + await page.getByLabel('Then', { exact: true }).fill('The request succeeds and retains the reason'); + await page.getByRole('button', { name: 'Save scenario draft' }).click(); + await page.getByRole('button', { name: 'Confirm scenario' }).click(); + await expect(page.getByRole('heading', { name: 'Manager override retains its reason · revision 2 · Confirmed' })).toBeVisible(); + await page.screenshot({ path: info.outputPath('knowledge-confirmed-correction.png'), fullPage: true }); + await page.getByRole('button',{name:'Archive scenario',exact:true}).click(); + await expect(page.getByRole('heading',{name:'Manager override retains its reason · revision 3 · Archived'})).toBeVisible(); + await page.reload(); + await expect(page.getByRole('heading',{name:'Manager override retains its reason · revision 3 · Archived'})).toBeVisible(); + await expect(page.getByRole('button',{name:'Archive scenario',exact:true})).toHaveCount(0); + await page.screenshot({path:info.outputPath('scenario-archived.png'),fullPage:true}); + await page.goto(root + '/outcomes'); + await page.getByLabel('Desired improvement').fill('A fresh agent preserves the manager exception'); + await page.getByLabel('Success criteria (one per line)').fill('The manager reason remains mandatory\nUnauthorized overrides remain denied'); + await page.getByRole('button', { name: 'Create outcome' }).click(); + await expect(page.getByRole('heading', { name: 'A fresh agent preserves the manager exception' })).toBeVisible(); + await page.screenshot({ path: info.outputPath('outcome-open.png'), fullPage: true }); + await page.goto(root + '/review'); + await expect(page.getByText('No proposals submitted.', { exact: false })).toBeVisible(); + await expect(page.getByText('Health unknown.', { exact: false })).toBeVisible(); + await expect(page.getByText('Requirements recorded; implementation verification outstanding')).toHaveCount(4); + await page.screenshot({ path: info.outputPath('review-outstanding-unknown.png'), fullPage: true }); + await info.attach('real-api-journey', { contentType: 'application/json', body: JSON.stringify({ execution: 'real-browser-real-worker-api', orgId: org.orgId, projectId: project.projectId, correction, screenshots: ['knowledge-confirmed-correction.png', 'outcome-open.png', 'review-outstanding-unknown.png'], limitations: 'No source proposal or release is claimed by this UI correction test.' }) }); +}); + +test('owner reviews real runner evidence and accepts the exact source proposal', async ({ page }, info) => { + test.skip(!process.env.COMMONFORGE_NATIVE_REVIEW_REPORT, 'Requires an actual unaccepted verified source proposal prepared through the real-local workflow.'); + const report = JSON.parse(readFileSync(process.env.COMMONFORGE_NATIVE_REVIEW_REPORT!, 'utf8')) as { organizationId: string; projectId: string; proposalId: string; envelopeDigest: string; scenarioCount: number }; + await signedIn(page); + await page.goto(`/o/${report.organizationId}/p/${report.projectId}/review`); + const proposal = page.locator(`article[data-proposal-id="${report.proposalId}"]`); + await expect(proposal).toBeVisible(); + await expect(proposal.getByText('Verified runner execution', { exact: false })).toHaveCount(report.scenarioCount); + await expect(proposal.getByText('pass (current result) · Verified runner execution', { exact: false })).toHaveCount(report.scenarioCount); + if (process.env.COMMONFORGE_NATIVE_REQUIRE_PREVIEW === '1') { + await proposal.getByRole('button', { name: 'Create exact-source preview' }).click(); + const link = proposal.getByRole('link', { name: 'Open reviewed preview' }).first(); + await expect(link).toBeVisible({ timeout: 60000 }); + const opened = page.waitForEvent('popup'); + await link.click(); + const preview = await opened; + await expect(preview.getByRole('heading', { name: 'Synthetic scheduling' })).toBeVisible(); + await preview.screenshot({ path: info.outputPath('real-reviewed-preview.png') }); + // Preview URLs contain short-lived capability tokens and are never included in reports. + const health = await preview.request.get(preview.url() + 'api/health'); + expect(health.ok()).toBe(true); expect((await health.json()).ruleVersion).toBe(2); + const manager = await preview.request.post(preview.url() + 'api/reschedule', {data:{actor:'manager',appointment:'2026-10-17T13:00:00.000Z',reason:'Owner preview observation'}}); + expect(manager.ok()).toBe(true); expect((await manager.json()).override.reason).toBe('Owner preview observation'); + await preview.close(); + } + const alreadyAccepted = await proposal.getByText('Accepted', { exact: true }).isVisible(); + if (!alreadyAccepted) { + await expect(proposal.getByRole('button', { name: 'Approve exact change' })).toBeDisabled(); + await proposal.getByLabel('Review explanation').fill('Owner reviewed the exact self-development change and real independently pinned runner evidence; the manager exception retains its reason.'); + await page.screenshot({ path: info.outputPath('source-review-real-evidence.png'), fullPage: true }); + await proposal.getByRole('button', { name: 'Approve exact change' }).click(); + await expect(proposal.getByRole('button', { name: 'Accept project version' })).toBeVisible(); + await proposal.getByRole('button', { name: 'Accept project version' }).click(); + } + await expect(proposal.getByText('Accepted', { exact: true })).toBeVisible(); + await expect(page.getByText('Accepted with applicable behavioral evidence', { exact: true }).first()).toBeVisible(); + await page.screenshot({ path: info.outputPath('source-accepted-real-evidence.png'), fullPage: true }); + writeFileSync(info.outputPath('source-acceptance.json'), JSON.stringify({ execution:'real-browser-real-worker-real-git-real-runner', organizationId:report.organizationId, projectId:report.projectId, proposalId:report.proposalId, envelopeDigest:report.envelopeDigest, scenarioCount:report.scenarioCount, acceptedInBrowser:!alreadyAccepted, inspectedExistingAcceptance:alreadyAccepted,releaseClaimed:false },null,2)); + await info.attach('real-source-acceptance', { contentType: 'application/json', body: JSON.stringify({ execution: 'real-browser-real-worker-real-git-real-runner', ...report, acceptedInBrowser: !alreadyAccepted, inspectedExistingAcceptance: alreadyAccepted, releaseClaimed: false }) }); +}); + +test('new project onboarding defaults to native knowledge and initializes a dedicated source folder', async ({ page }, info) => { + await signedIn(page); + const organization = await api(page, '/orgs', { name:'Native onboarding reference' }); + await page.goto(`/o/${organization.orgId}`); + await page.getByRole('button', {name:'New project',exact:true}).click(); + await expect(page.getByLabel('Project workflow')).toHaveValue('agent-native'); + await page.getByLabel('Project name').fill('Public reference'); + await page.getByRole('button', {name:'Create project',exact:true}).click(); + await expect(page.getByRole('region', {name:'Folders'})).toBeVisible(); + await expect(page.getByRole('form', {name:'Attach an existing folder'})).toHaveCount(0); + const projectId = /\/p\/([^/]+)/.exec(new URL(page.url()).pathname)![1]!; + const folder = page.getByRole('form', {name:'New folder'}); + await folder.getByLabel('New folder', {exact:true}).fill('source'); + await folder.getByRole('button', {name:'Add folder',exact:true}).click(); + await expect(page.locator('.folder-row', {hasText:'source'})).toContainText('Ready', {timeout:60_000}); + await expect(page.getByRole('button', {name:'Detach source',exact:true})).toHaveCount(0); + await page.getByRole('navigation', {name:'Project sections'}).getByRole('link', {name:'Knowledge',exact:true}).click(); + await expect(page.getByRole('heading', {name:'Knowledge',exact:true,level:2})).toBeVisible(); + await expect(page.getByRole('button', {name:'Enable project understanding'})).toHaveCount(0); + const state = await page.evaluate(async ({org,pid}) => { + const response=await fetch(`/api/orgs/${org}/projects/${pid}/native`,{credentials:'same-origin'}); + if(!response.ok) throw new Error('native onboarding state unavailable'); + return (await response.json()).state; + },{org:organization.orgId,pid:projectId}); + expect(state.sourceHeads).toHaveLength(1); + expect(state.manifests.at(-1).sourceHeads).toEqual(state.sourceHeads); + expect(state.manifests.at(-1).behavior).toBe('historical-unverified'); + await page.screenshot({path:info.outputPath('native-onboarding.png'),fullPage:true}); +}); + +test('owner verifies and accepts historical source recovery under current requirements', async ({page},info)=>{ + test.skip(!process.env.COMMONFORGE_NATIVE_REVIEW_REPORT,'Requires the actual corrected source workflow and accepted later browser proposal.'); + const report=JSON.parse(readFileSync(process.env.COMMONFORGE_NATIVE_REVIEW_REPORT!,'utf8')) as {organizationId:string;projectId:string;scenarioCount:number}; + await signedIn(page); + await page.goto(`/o/${report.organizationId}/p/${report.projectId}/review`); + const before=await page.evaluate(async ({org,pid})=>(await (await fetch(`/api/orgs/${org}/projects/${pid}/native`,{credentials:'same-origin'})).json()).state,{org:report.organizationId,pid:report.projectId}); + const target=[...before.manifests].reverse().find((m:any)=>m.behavior==='verified'&&m.knowledgeHead===before.knowledgeHead&&m.sourceHeads.length===before.sourceHeads.length&&m.sourceHeads.some((head:any)=>before.sourceHeads.some((current:any)=>current.resourceId===head.resourceId&¤t.revisionId!==head.revisionId))); + expect(target,'rehearsal must include a prior corrected implementation and later accepted source').toBeTruthy(); + await page.getByRole('button',{name:`Review recovery to project version ${target.sequence}`,exact:true}).click(); + const recovery=page.locator('article[data-proposal-id]').filter({has:page.getByRole('heading',{name:'Historical source recovery',exact:true})}).first(); + await expect(recovery).toBeVisible(); + await expect(recovery.getByText('Restores exact historical source under current requirements',{exact:false})).toBeVisible(); + await recovery.getByRole('button',{name:'Run required automated scenarios',exact:true}).click(); + await expect(recovery.getByText('pass (current result) · Verified runner execution',{exact:false})).toHaveCount(report.scenarioCount,{timeout:120000}); + await recovery.getByLabel('Review explanation').fill('Reviewed exact historical source with fresh execution under the current corrected rule. This does not restore knowledge, deploy, or reverse external effects.'); + await recovery.getByRole('button',{name:'Approve exact change',exact:true}).click(); + await expect(recovery.getByRole('button',{name:'Accept project version',exact:true})).toBeVisible(); + await recovery.getByRole('button',{name:'Accept project version',exact:true}).click(); + await expect(recovery.getByText('Accepted',{exact:true})).toBeVisible(); + const after=await page.evaluate(async ({org,pid})=>(await (await fetch(`/api/orgs/${org}/projects/${pid}/native`,{credentials:'same-origin'})).json()).state,{org:report.organizationId,pid:report.projectId}); + expect(after.sourceHeads).toEqual(target.sourceHeads); + expect(after.knowledgeHead).toBe(before.knowledgeHead); + expect(after.knowledge).toEqual(before.knowledge); + expect(after.manifests.at(-1).behavior).toBe('verified'); + // Acceptance schedules durable ref reconciliation; keep the real stack alive until it finishes. + for(const head of target.sourceHeads)await expect.poll(()=>page.evaluate(async ({org,head})=>{ + const response=await fetch(`/api/orgs/${org}/resources/${head.resourceId}`,{credentials:'same-origin'}); + if(!response.ok)return false; + const resource=await response.json();return resource.head?.revisionId===head.revisionId&&resource.head?.reconciliation==='synced'; + },{org:report.organizationId,head}),{timeout:90000}).toBe(true); + await page.screenshot({path:info.outputPath('recovery-accepted.png'),fullPage:true}); + writeFileSync(info.outputPath('recovery-acceptance.json'),JSON.stringify({execution:'real-browser-real-worker-real-git-real-runner',organizationId:report.organizationId,projectId:report.projectId,historicalManifest:target.hash,acceptedManifest:after.manifests.at(-1).hash,sourceHeads:after.sourceHeads,knowledgeHead:after.knowledgeHead,scenarioCount:report.scenarioCount,releaseClaimed:false,externalEffectsReversed:false},null,2)); +}); diff --git a/apps/web/playwright.native.config.ts b/apps/web/playwright.native.config.ts new file mode 100644 index 0000000..8a60444 --- /dev/null +++ b/apps/web/playwright.native.config.ts @@ -0,0 +1,9 @@ +import { defineConfig, devices } from '@playwright/test'; + +/** External REAL Worker started by scripts/dev-real-worker.sh; no fake provider routes. */ +export default defineConfig({ + testDir: './e2e', testMatch: ['native.spec.ts','native-outcomes.spec.ts'], workers: 1, fullyParallel: false, + reporter: 'list', outputDir: 'test-results/native', timeout: 120000, + expect: { timeout: 20000 }, + use: { baseURL: process.env.COMMONFORGE_NATIVE_URL ?? 'http://127.0.0.1:8899', ...devices['Desktop Chrome'], trace: 'off' }, +}); diff --git a/apps/web/src/api/client.ts b/apps/web/src/api/client.ts index b66cdec..f3af940 100644 --- a/apps/web/src/api/client.ts +++ b/apps/web/src/api/client.ts @@ -146,7 +146,7 @@ export interface CommonForgeApi { inviteMember(org: string, body: { email: string; role: Role }): Promise; getProjectAccess(org: string, pid: string): Promise; putProjectAccess(org: string, pid: string, body: { userId: string; level: AccessLevel; folders?: Record }): Promise; - createProject(org: string, name: string): Promise<{ projectId: string; name: string }>; + createProject(org: string, name: string, mode?: "legacy" | "agent-native"): Promise<{ projectId: string; name: string }>; listResources(org: string): Promise; listFolders(org: string, pid: string): Promise; /** A new folder (a git folder; there is no kind to choose): `201` before its first version exists, listed as `settingUp` until then. */ @@ -465,8 +465,8 @@ export class HttpApi implements CommonForgeApi { putProjectAccess(org: string, pid: string, body: { userId: string; level: AccessLevel; folders?: Record }) { return this.json("PUT", `/orgs/${enc(org)}/projects/${enc(pid)}/access`, body); } - createProject(org: string, name: string) { - return this.json<{ projectId: string; name: string }>("POST", `/orgs/${enc(org)}/projects`, { name }); + createProject(org: string, name: string, mode?: "legacy" | "agent-native") { + return this.json<{ projectId: string; name: string }>("POST", `/orgs/${enc(org)}/projects`, { name, ...(mode ? { mode } : {}) }); } async listResources(org: string) { return (await this.json<{ resources: Resource[] }>("GET", `/orgs/${enc(org)}/resources`)).resources; diff --git a/apps/web/src/api/fake.ts b/apps/web/src/api/fake.ts index c154b80..ba9af7d 100644 --- a/apps/web/src/api/fake.ts +++ b/apps/web/src/api/fake.ts @@ -604,7 +604,7 @@ export class FakeApi implements CommonForgeApi { /** the empty project's folders (added by intake only) */ private readonly handoverMounts: Mount[] = []; /** projects created at runtime (the two fixture projects are fixed) */ - private readonly newProjects = new Map(); + private readonly newProjects = new Map(); /** * the policy of every project but the Customer Portal (whose policy is `policy`): a new project * starts with none of the demo's checks, release or staging @@ -1167,6 +1167,7 @@ export class FakeApi implements CommonForgeApi { this.project(org, pid); return { projectId: pid, + mode: this.newProjects.get(pid)?.mode ?? "legacy", name: this.projectName(pid), membershipRevision: 3, policyRevision: pid === FIXTURE_PROJECT ? this.policyRevision : this.otherPolicy(pid).revision, @@ -1235,12 +1236,12 @@ export class FakeApi implements CommonForgeApi { return { policyRevision: this.policyRevision, policy: structuredClone(this.policy) }; } - async createProject(org: string, name: string) { + async createProject(org: string, name: string, mode?: "legacy" | "agent-native") { this.log("createProject", org, name); this.org(org); this.needDev(); const projectId = `p-${(0xc0de0000 + this.newProjects.size + 10).toString(16)}`; - this.newProjects.set(projectId, { org, name: name.trim(), createdAt: new Date(this.now()).toISOString(), mounts: [] }); + this.newProjects.set(projectId, { org, name: name.trim(), createdAt: new Date(this.now()).toISOString(), mounts: [], ...(mode ? {mode} : {}) }); // the worker gives a developer creator read + write on the new project if (this.role === "developer") this.grants.set(`u-developer|project|${projectId}`, ["read", "write"]); return { projectId, name: name.trim() }; diff --git a/apps/web/src/api/native.ts b/apps/web/src/api/native.ts new file mode 100644 index 0000000..003a95e --- /dev/null +++ b/apps/web/src/api/native.ts @@ -0,0 +1,45 @@ +import { ProjectStateSchema, type ProjectState } from '@commonforge-org/core'; +import { ApiError } from './client.js'; + +async function request(org: string, pid: string, suffix = '', body?: unknown): Promise> { + const res = await fetch(`/api/orgs/${encodeURIComponent(org)}/projects/${encodeURIComponent(pid)}/native${suffix}`, { + method: body === undefined ? 'GET' : 'POST', credentials: 'same-origin', + headers: { accept: 'application/json', ...(body === undefined ? {} : { 'content-type': 'application/json' }) }, + ...(body === undefined ? {} : { body: JSON.stringify(body) }), + }); + const data = await res.json() as Record; + if (!res.ok) { + const error = data.error && typeof data.error === 'object' ? data.error as Record : data; + throw new ApiError(res.status, String(error.code ?? 'native_request_failed'), String(error.message ?? 'The operation could not be completed.')); + } + return data; +} +export async function loadNativeProject(org: string, pid: string): Promise { + const data = await request(org, pid); + if (data.protocolVersion !== 1) throw new ApiError(409, 'unsupported_version', 'This server does not support project context version 1.'); + return data.state === null ? null : ProjectStateSchema.parse(data.state); +} +export async function enableNativeProject(org: string, pid: string): Promise { + await request(org, pid, '/enable', { idempotencyKey: crypto.randomUUID() }); +} +export async function nativeCommand(org: string, pid: string, generation: number, body: Record): Promise { + if(body.type==='prepareRecovery'){ + await request(org,pid,'/recoveries',{manifestHash:body.manifestHash,expectedGeneration:generation,idempotencyKey:crypto.randomUUID()}); + return; + } + if (body.type === 'requestVerification') { + await request(org, pid, `/proposals/${encodeURIComponent(String(body.proposalId))}/verify`, { idempotencyKey: crypto.randomUUID() }); + return; + } + await request(org, pid, '/commands', { ...body, expectedGeneration: generation, idempotencyKey: crypto.randomUUID() }); +} + +export interface NativeVerificationJob { + runId:string; envelopeDigest:string; issuedAt:string; completedAt:string|null; + status:'pending'|'expired'|'completed'|'error'|'stale'; execution:string; + failure:{code:string;summary:string}|null; evidenceIds:string[]; +} +export async function loadNativeVerifications(org:string,pid:string,proposalId:string):Promise { + const data=await request(org,pid,`/proposals/${encodeURIComponent(proposalId)}/verifications?limit=100`); + return data.items as NativeVerificationJob[]; +} diff --git a/apps/web/src/api/types.ts b/apps/web/src/api/types.ts index b1523ac..bce6d92 100644 --- a/apps/web/src/api/types.ts +++ b/apps/web/src/api/types.ts @@ -104,6 +104,8 @@ export interface Mount { operations: Operation[]; /** the viewer may upload and change files here (owners; developers with write); absent from older servers: false */ canWrite?: boolean; + /** New dedicated source setup may be retried even when direct legacy edits are unavailable. */ + canRetrySetup?: boolean; /** * the viewer may start a task that writes here (the New task form; Worker `writeMountIssue`): * owners and developers with write on a folder opened for writing, clients in a staging-mode @@ -113,6 +115,7 @@ export interface Mount { } export interface Project { + mode?: "legacy" | "agent-native"; projectId: string; name: string; membershipRevision: number; diff --git a/apps/web/src/lib/router.ts b/apps/web/src/lib/router.ts index 56a0172..5260a02 100644 --- a/apps/web/src/lib/router.ts +++ b/apps/web/src/lib/router.ts @@ -2,7 +2,7 @@ import { useSyncExternalStore } from "react"; /** `tasks`: the Tasks board, shown only for staging-mode projects (staging phase 3). */ -export type ProjectTab = "overview" | "files" | "changes" | "versions" | "activity" | "tasks"; +export type ProjectTab = "overview" | "files" | "changes" | "versions" | "activity" | "tasks" | "outcomes" | "knowledge" | "review"; export type Route = | { name: "signin" } @@ -24,6 +24,9 @@ export type Route = | { name: "project"; org: string; pid: string; tab: "changes" } | { name: "project"; org: string; pid: string; tab: "overview" } | { name: "project"; org: string; pid: string; tab: "activity" } + | { name: "project"; org: string; pid: string; tab: "outcomes" } + | { name: "project"; org: string; pid: string; tab: "knowledge" } + | { name: "project"; org: string; pid: string; tab: "review" } /** the Tasks board (staging-mode projects): `/o/:org/p/:pid/tasks` */ | { name: "project"; org: string; pid: string; tab: "tasks" } | { name: "project"; org: string; pid: string; tab: "versions"; mount?: string; /** an accepted version id: open its restore sheet */ version?: string } @@ -63,6 +66,7 @@ export function parseRoute(pathname: string, search: string): Route { const intakeId = q.get("intake") ?? undefined; return { name: "intake", org, pid, ...(intakeId ? { intakeId } : {}) }; } + if ((tab === "outcomes" || tab === "knowledge" || tab === "review") && parts.length === 5) return { name: "project", org, pid, tab }; if (tab === "changes" && parts.length === 5) return { name: "project", org, pid, tab: "changes" }; if (tab === "overview" && parts.length === 5) return { name: "project", org, pid, tab: "overview" }; if (tab === "activity" && parts.length === 5) return { name: "project", org, pid, tab: "activity" }; diff --git a/apps/web/src/screens/AgentHandoff.tsx b/apps/web/src/screens/AgentHandoff.tsx index ef59956..1da4af1 100644 --- a/apps/web/src/screens/AgentHandoff.tsx +++ b/apps/web/src/screens/AgentHandoff.tsx @@ -29,6 +29,7 @@ import { loginCommand, mayRunAgent, submitCommand, + shellWord, taskFolders, type HandoffTask, } from "../lib/handoff.js"; @@ -63,7 +64,7 @@ export function useHandoffContext(role: Role, staging: boolean): HandoffContext } export function handoffTask(task: Pick & Partial>): HandoffTask { - return { taskId: task.taskId, title: task.title, request: task.request ?? "", ...taskFolders(task), ...(task.lease ? { lease: task.lease } : {}) }; + return { ...((task as { projectProtocolVersion?: number }).projectProtocolVersion ? { projectProtocolVersion: (task as { projectProtocolVersion?: number }).projectProtocolVersion } : {}), taskId: task.taskId, title: task.title, request: task.request ?? "", ...taskFolders(task), ...(task.lease ? { lease: task.lease } : {}) }; } /** A command (or several lines) with its copy button. */ @@ -152,6 +153,9 @@ function CopyPrompt({ task, ctx }: { task: HandoffTask; ctx: HandoffContext }) { } export function AgentHandoff({ task, ctx, ownTask = true, onDone, intro }: { task: HandoffTask; ctx: HandoffContext; ownTask?: boolean; onDone: () => void; intro?: string }) { + const projectProtocol = (task as HandoffTask & { projectProtocolVersion?: number }).projectProtocolVersion; + if (projectProtocol !== undefined && projectProtocol !== 1) return

This project requires an unsupported agent protocol. Update CommonForge before assigning an agent.

; + if (projectProtocol === 1) return ; const dir = checkoutDir(task.title); const canRun = mayRunAgent(ctx.role, { staging: ctx.staging, ownTask }); const client = !isDeveloperRole(ctx.role); @@ -282,3 +286,31 @@ export function TaskHandoffLoader({ taskId, ctx, load, ownTask = true, onDone }: if (!task) return ; return ; } + + +/** A replacement always starts in a unique empty checkout; existing context stays historical. */ +export function nativeAgentPrompt(task:HandoffTask,ctx:HandoffContext,code:string,nonce:string=crypto.randomUUID()):string { + const safe=(value:string)=>{const word=shellWord(value);if(!word)throw new Error('The handoff contains an unsupported command value.');return word;}; + const initialDir=checkoutDir(task.title),replacement=!!task.lease; + const dir=replacement?`${initialDir}-renewed-${safe(nonce)}`:initialDir; + const commands=replacement?`cforge renew --api-url ${safe(ctx.apiUrl)} --code ${safe(code)} --out ${safe(dir)}`:`cforge handoff --api-url ${safe(ctx.apiUrl)} --code ${safe(code)}\ncforge checkout ${safe(task.taskId)} ${safe(dir)}\ncforge context ${safe(task.taskId)} --out ${safe(dir)}`; + return `Work on CommonForge task ${task.taskId}: ${task.title}\n${task.request}\n\nUse only the task-scoped handoff below. Build cforge from the public CommonForge source if unavailable.\n${CLI_SOURCE_INSTALL}\n\n${commands}\n\nRead ${dir}/.commonforge/context.md and context.json before editing. Preserve confirmed rules and their corrections. Do not edit .commonforge files. Implement only the assigned task, then run applicable checks and cforge submit from the checkout. Propose corrections through structured native commands; human confirmation, acceptance and release are separate authority. No provider credentials or broad user login are required.${replacement?' Previous mandate stays revoked. Preserve the old checkout; this creates a fresh attempt and pinned context.':''}`; +} + +/** Native projects exchange one-use task authority; never create a broad CLI session. */ +function NativeAgentHandoff({ task, ctx, onDone }: { task: HandoffTask; ctx: HandoffContext; onDone: () => void }) { + const [agentId, setAgentId] = useState(''); + const [busy, setBusy] = useState(false); + const [error, setError] = useState(null); + const [copied, setCopied] = useState(false); + const copy = () => { + setBusy(true); setError(null); + const prompt = fetch(`/api/tasks/${encodeURIComponent(task.taskId)}/native/handoff`, { method: 'POST', credentials: 'same-origin', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ agentId }) }).then(async res => { + const body = await res.json() as { code?: string; error?: { code: string; message: string } }; + if (!res.ok || !body.code) throw new ApiError(res.status, body.error?.code ?? 'handoff_failed', body.error?.message ?? 'Task handoff could not be issued.'); + return nativeAgentPrompt(task,ctx,body.code); + }); + void copyLater(prompt).then(() => setCopied(true), e => setError(e)).finally(() => setBusy(false)); + }; + return

Task-scoped agent handoff

{task.title}

The agent receives this task's source access and pinned project understanding. It cannot confirm rules, accept changes or deploy.

{copied ? 'Copied. This handoff code works once within 15 minutes. Task authority expires after one hour unless renewed.' : 'The copied prompt includes a one-use task code. It does not sign the agent in as you.'}

{error ?

{errorMessage(error, true)}

: null}
; +} diff --git a/apps/web/src/screens/FoldersTab.tsx b/apps/web/src/screens/FoldersTab.tsx index 1212c84..9802b5c 100644 --- a/apps/web/src/screens/FoldersTab.tsx +++ b/apps/web/src/screens/FoldersTab.tsx @@ -72,6 +72,7 @@ export interface FoldersViewProps { org: string; pid: string; viewerRole: Role; + native?: boolean; folders: ProjectFolder[]; /** org folders not in this project (owners) */ attachable: Array>; @@ -108,14 +109,14 @@ export function FoldersView(p: FoldersViewProps) { icon="files" title="This project has no folders yet." action={ - p.viewerRole === "owner" ? ( + p.viewerRole === "owner" && !p.native ? ( Import from Git ) : undefined } > - {p.viewerRole === "owner" ? "Import the site's code from a public Git repository (it keeps its history), or add an empty folder below." : "Add an empty folder below; the project owner imports existing code."} + {p.native ? "Add a new source folder below. Its first version becomes part of this project’s accepted state." : p.viewerRole === "owner" ? "Import the site's code from a public Git repository (it keeps its history), or add an empty folder below." : "Add an empty folder below; the project owner imports existing code."} ) : (
    @@ -130,7 +131,7 @@ export function FoldersView(p: FoldersViewProps) { {!f.writable && !owner ?
    Read-only in this project
    : null}
    - {owner ? ( + {owner && !p.native ? ( setName(e.target.value)} /> + @@ -72,6 +80,7 @@ function NewProjectForm({ busy, error, onCreate, onCancel, startOpen = false }: onClick={() => { // a later New project starts empty, without the last attempt's error setName(""); + setMode("agent-native"); setOpen(false); onCancel?.(); }} @@ -137,15 +146,15 @@ export function OrgHome({ org, role, onSignOut }: { org: string; role: Role; onS const membership = memberships.find((m) => m.orgId === org) ?? { orgId: org, name: org, role }; const label = orgLabels(memberships).get(org) ?? membership.name; const load = useLoad(() => api.listProjects(org), org); - const act = useAction(async (name: string) => { - const p = await api.createProject(org, name); + const act = useAction(async (name: string, mode: "legacy" | "agent-native") => { + const p = await api.createProject(org, name, mode); navigate(settingsHref(org, p.projectId, "folders")); }); let content; if (load.error) content = ; else if (!load.data) content = ; - else content = void act.run(n), onCancel: act.clearError } } : {})} />; + else content = void act.run(n, mode), onCancel: act.clearError } } : {})} />; return ( {content} diff --git a/apps/web/src/screens/NativeReview.tsx b/apps/web/src/screens/NativeReview.tsx new file mode 100644 index 0000000..fe9baf7 --- /dev/null +++ b/apps/web/src/screens/NativeReview.tsx @@ -0,0 +1,67 @@ +import { loadNativeVerifications, type NativeVerificationJob } from '../api/native.js'; +import { useState } from 'react'; +import { currentConfirmed, observationHealth, isRevisionId, revisionCommitSha, type ProjectManifest, type ProjectProposal, type ProjectState } from '@commonforge-org/core'; +import { useAction, useApp, useLoad } from '../lib/hooks.js'; +import { projectHref } from '../lib/router.js'; +import { ErrorNote, Link } from '../ui.js'; +import type { PreviewView } from '../api/types.js'; +import { NativeProject } from './ProjectKnowledge.js'; + +export function NativeReview() { + return {(state, command, busy) => }; +} +export function Review({ state, command, busy }: { state: ProjectState; command: (body: Record) => Promise; busy: boolean }) { + return <>

    Behavioral change review

    Review the exact requirements, implementation, and observed behavior. Acceptance records a project version; release is a separate operation.

    {state.proposals.length === 0 ?

    No proposals submitted. An assigned agent submits a change using its pinned project context.

    : [...state.proposals].reverse().map(p => )}

    Accepted project history

    Recovery prepares a new review of historical source under current confirmed requirements. It does not restore old knowledge, deploy, or reverse database or external effects.

    {[...state.manifests].reverse().map(m =>

    Project version {m.sequence}

    {m.behavior === 'outstanding' ? 'Requirements recorded; implementation verification outstanding' : m.behavior === 'verified' ? 'Accepted with applicable behavioral evidence' : 'Historical source; behavior unverified'}

    {m.createdAt}

    {canPrepareRecovery(state,m)?:null}
    Exact accepted subject

    Manifest: {m.hash}

    Knowledge: {m.knowledgeHead}

    {m.sourceHeads.map(h =>

    {h.resourceId}: {h.revisionId}

    )}
    )}

    Release observations

    {state.observations.length === 0 ?

    Health unknown. No bounded release observation has been recorded.

    : state.observations.map(o =>

    {o.environment}: {observationHealth(o, new Date().toISOString())}

    {o.summary}

    Release {o.releaseId} · observed {o.observedAt} · fresh until {o.freshUntil}

    Manifest {o.manifestHash}

    )}
    ; +} +function Proposal({ proposal: p, state, command, busy }: { proposal: ProjectProposal; state: ProjectState; command: (body: Record) => Promise; busy: boolean }) { + const { me, api } = useApp(); + const previews = useLoad(() => p.envelope.innerProposalId ? api.listPreviews(state.organizationId, p.envelope.innerProposalId) : Promise.resolve([]), `${p.id}/previews`, 15000); + const verificationJobs = useLoad(() => loadNativeVerifications(state.organizationId,state.projectId,p.id), `${p.id}/verification-jobs/${state.generation}`,15000); + const requestPreview = useAction(async () => { if (p.envelope.innerProposalId) { await api.requestPreview(state.organizationId, p.envelope.innerProposalId); previews.reload(); } }); + const [explanation, setExplanation] = useState(''); + const e = p.envelope; + const accepted = state.manifests.some(m => m.envelopeDigest === p.digest); + const stale = e.expectedKnowledgeHead !== state.knowledgeHead || e.policyRevision!==state.policyRevision || !accepted&&e.expectedParent!==(state.manifests.at(-1)?.hash??null); + const decision = [...state.decisions].reverse().find(d => d.envelopeDigest === p.digest && d.action === 'approve'); + const evidence = state.evidence.filter(ev => ev.envelopeDigest === p.digest); + const purpose = e.outcome ? state.outcomes.find(o => o.id === e.outcome!.id && o.revision === e.outcome!.revision)?.description : null; + return

    {purpose ?? (e.recovery ? 'Historical source recovery' : e.innerProposalId ? 'Source change' : 'Knowledge change')}

    {new Date(p.createdAt).toISOString().slice(0, 16).replace('T', ' ')} UTC

    {accepted ? 'Accepted' : stale ? 'Requirements changed: refresh and verify this change again.' : 'Awaiting review'}

    Submitted by {p.author} · {p.createdAt}

    {e.outcome ?

    Outcome: {state.outcomes.find(o => o.id === e.outcome!.id && o.revision === e.outcome!.revision)?.description ?? 'Referenced outcome is unavailable'}

    : null}

    {e.recovery ? 'Restores exact historical source under current requirements' : e.innerProposalId ? 'Includes source changes' : 'Knowledge-only proposal'} · {e.requiredScenarios.length} required scenarios

    Working preview

    {e.innerProposalId ? : null}{!e.innerProposalId ?

    {e.recovery?'Recovery uses exact historical source. Inspect and verify the required behavior before accepting; deployment is separate.':'Knowledge-only change; no new preview.'}

    : previews.data?.length ? previews.data.map(preview =>

    {preview.state === 'ready' && preview.proposalHash === e.innerProposalHash && previewMatchesProjectSource(preview, e.sourceHeads) && preview.url && /^https?:\/\//.test(preview.url) ? Open reviewed preview : `Preview ${preview.state}${preview.proposalHash !== e.innerProposalHash || !previewMatchesProjectSource(preview, e.sourceHeads) ? ' (source subject does not match this review)' : ''}`}{preview.expiresAt ? ` · expires ${preview.expiresAt}` : ''}

    ) :

    No ready preview is available for this change.

    }
    {accepted ? 'Inspect accepted behavior evidence' : 'Requirements and observations'}{!accepted && e.requiredScenarios.some(s => s.method === 'automated') ? : null}{e.requiredScenarios.length === 0 ?

    No behavioral scenarios required by the current policy.

    : e.requiredScenarios.map(s => { + const definition = state.scenarios.find(r => r.id === s.id && r.revision === s.revision); + const results = evidence.filter(ev => ev.scenarioId === s.id && ev.scenarioRevision === s.revision); + return
    {definition?.title ?? s.id} · revision {s.revision} · {s.method}{definition ? <>

    Given {definition.given}; when {definition.when}; then {definition.then}

    {definition.ruleRefs.map(ref =>

    Rule: {state.knowledge.find(k => k.id === ref.id && k.revision === ref.revision)?.text ?? 'Referenced rule unavailable'}

    )} : null}{results.length === 0 ?

    Not run: evidence missing

    : results.map(ev =>

    {ev.result}{results.some(n=>n.supersedes===ev.id) ? ' (superseded; retained as history)' : ' (current result)'} · {ev.provenance === 'runner-verified' ? 'Verified runner execution' : ev.provenance === 'human-attested' ? 'Human observation' : 'Imported history, untrusted locally'} · {ev.actor} · {ev.completedAt}
    {ev.summary}{stale ? ' This check ran before the requirements changed.' : ''}

    )}{s.method === 'manual' && !accepted && !stale ? : null}
    ; + })}
    Exact review subject and provenance

    Proposal {p.id}

    Envelope {p.digest}

    Build {e.buildDigest}

    Runtime recipe {e.runtimeRecipeDigest}

    Knowledge {e.expectedKnowledgeHead}

    Inner proposal {e.innerProposalHash ?? 'none'}

    {e.recovery?

    Historical manifest {e.recovery.manifestHash}

    :null}{e.effects.length ?
      {e.effects.map(effect =>
    • {effect}
    • )}
    :

    No external effects declared.

    }
    {!accepted ? <>