Skip to content
Draft
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ Start the background process, continue launching the other lanes, then drain the

The runner and its preflight have no implicit timeout. Do not invent a duration from role, mode, or a convenient round number; real implementation lanes can run for 90 minutes or much longer. Pass `--timeout` only when the user, an external service deadline, or a measured task contract supplies a real bound. That value starts at wrapper entry, before module loading and argument parsing, and remains one absolute deadline across setup, preflight, model execution, and output capture. It is never a fresh allowance per child, and long waits are armed in runtime-safe chunks without shortening the supplied deadline. Otherwise supervise liveness through the retained background task/session handle and cancel manually only on evidence that the run is dead. Cancel through that retained handle so the runner receives SIGINT or SIGTERM, sends it to an active child when one remains, stops waiting on inherited output pipes, removes the empty output reservation, and writes a `cancelled` receipt. Preserve that receipt; a retry is a new attempt with new unique output and receipt paths. Unchanged running state is not a dropout, and Claude's ten-minute foreground ceiling is never a reason to terminate a healthy lane.

Read-only mode maps to Claude plan mode with project-only settings and an explicit tool list, Codex's read-only sandbox, and Grok plan mode plus its `read-only` sandbox and read-oriented tool list. Grok's built-in read-only profile deliberately keeps its own state and system temporary directories writable, so point a read-only Grok lane at the actual checkout rather than a worktree under `/tmp`, `/var/tmp`, or the host's temporary directory. `isolated-write` maps to Claude `acceptEdits` with project-only settings, Codex `workspace-write`, and Grok `acceptEdits` plus its `workspace` sandbox and write-capable tool list. Give every writer only a dedicated worktree or output directory. Never route a writer into the primary checkout.
Read-only mode maps to Claude plan mode with project-only settings and an explicit tool list, Codex's read-only sandbox, and Grok plan mode plus its `read-only` sandbox and read-oriented tool list. Grok's built-in read-only profile deliberately keeps its own state and system temporary directories writable, so point a read-only Grok lane at the actual checkout rather than a worktree under `/tmp`, `/var/tmp`, or the host's temporary directory. `isolated-write` maps to Claude `acceptEdits` with project-only settings, Codex `workspace-write`, and Grok `acceptEdits` plus its `workspace` sandbox, write-capable tool list, and `--allow Bash` and `--allow Edit` rules. Headless Grok cancels any tool call that would need an approval prompt and ends the turn, and its `acceptEdits` mode does not pre-approve Grok's file tools. `--allow Bash` pre-approves shell commands, and `--allow Edit` pre-approves both file tools, `search_replace` and `write`. Grok still prompts for a shell command that writes output into a file, such as `> out.txt` or `| tee out.txt`, and for a command it cannot split into simple segments, such as `$(...)`, `${VAR:-default}`, or a subshell. Headless mode cancels those commands, so tell a Grok writer to create and edit files with its file tools and to run build and test steps as plain commands. Deny rules, hooks, and Grok's built-in sensitive-path checks still apply, and a command they gate is still cancelled. The `workspace` sandbox confines writes to the worktree, `~/.grok`, and temporary directories, but it reads the whole host and, on macOS, never blocks child-process network. The runner therefore gives a Grok writer only an allowlisted environment (paths, user, shell, locale, terminal, proxy and CA settings, and `GROK_*` and `XAI_*`), so parent tokens and agent sockets never reach its shell. Treat a Grok writer as able to read host files and reach the network, and route it only prompts and repositories you trust. Give every writer only a dedicated worktree or output directory. Never route a writer into the primary checkout.

Every concurrent external lane needs distinct prompt, output, and receipt paths. The launcher reserves output and receipt paths exclusively and refuses to overwrite them.

Expand All @@ -95,6 +95,6 @@ Success requires all of these:

The receipt also carries elapsed time, token usage when the CLI exposes it, and cost when available. Keep it with the arena or review artifacts so parent-harness comparisons are evidence-based.

Any missing CLI, failed login, unavailable model, explicit timeout, cancellation, catchable post-reservation launcher failure, non-zero child exit, malformed result, or model mismatch is a receipt-bearing dropout. Record it and apply the calling skill's existing dropout policy. A `cancelled` receipt proves that the runner received the signal; its `signal` field is non-null only when the runner sent that signal to a still-active direct CLI child, and remains null when cancellation only stopped a post-exit pipe drain. The provider CLI owns any processes it starts beneath that direct child; the receipt does not claim a process-tree kill. Do not delete or overwrite the receipt. Never substitute the parent model, retry another provider, or reinterpret an external descriptor as a native model slug.
Any missing CLI, failed login, unavailable model, explicit timeout, cancellation, catchable post-reservation launcher failure, non-zero child exit, malformed result, provider-reported failure, or model mismatch is a receipt-bearing dropout. A Grok turn that ends because a tool call hit an approval prompt records `permission-cancelled`, names the tool, and carries the cancelled tool result and terminal event as evidence. Any other Grok error result records `child-failed` with its terminal event. Record it and apply the calling skill's existing dropout policy. A `cancelled` receipt proves that the runner received the signal; its `signal` field is non-null only when the runner sent that signal to a still-active direct CLI child, and remains null when cancellation only stopped a post-exit pipe drain. The provider CLI owns any processes it starts beneath that direct child; the receipt does not claim a process-tree kill. Do not delete or overwrite the receipt. Never substitute the parent model, retry another provider, or reinterpret an external descriptor as a native model slug.

Start native and external lanes in the same fan-out phase, then wait for all of them before judging. A judge must not read candidate paths while their owners are still writing.
Original file line number Diff line number Diff line change
Expand Up @@ -125,13 +125,16 @@ describe("invocationCommand", () => {
expect.arrayContaining([
"--permission-mode",
"acceptEdits",
"--allow",
"Bash",
"--sandbox",
"workspace",
"--tools",
"read_file,grep,list_dir,run_terminal_cmd,search_replace",
"read_file,grep,list_dir,run_terminal_cmd,search_replace,write",
])
);
expect(grok.args).not.toContain("--always-approve");
expect(grok.args).not.toContain("bypassPermissions");

const claude = invocationCommand(
options({ provider: "claude", model: "fable", mode: "isolated-write" })
Expand All @@ -146,6 +149,20 @@ describe("invocationCommand", () => {
);
});

it("pre-approves Grok shell commands and file edits only for writers", () => {
const reader = invocationCommand(
options({ provider: "grok", model: "grok-4.6", mode: "read-only" })
);
expect(reader.args).not.toContain("--allow");
const writer = invocationCommand(
options({ provider: "grok", model: "grok-4.6", mode: "isolated-write" })
);
const rules = writer.args.flatMap((arg, index) =>
arg === "--allow" ? [writer.args[index + 1]] : []
);
expect(rules).toEqual(["Bash", "Edit"]);
});

it("covers low, medium, and high for every external provider", () => {
const cases = [
{
Expand Down
7 changes: 6 additions & 1 deletion plugins/pstack/skills/poteto-mode/scripts/runner/commands.ts
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,11 @@ function grokSandbox(mode: AccessMode): string {

function grokTools(mode: AccessMode): string {
const readonly = ["read_file", "grep", "list_dir", "run_terminal_cmd"];
return [...readonly, ...(mode === "isolated-write" ? ["search_replace"] : [])].join(",");
return [...readonly, ...(mode === "isolated-write" ? ["search_replace", "write"] : [])].join(",");
}

function grokPermissionRules(mode: AccessMode): readonly string[] {
return mode === "isolated-write" ? ["--allow", "Bash", "--allow", "Edit"] : [];
}

function permissionMode(mode: AccessMode): string {
Expand Down Expand Up @@ -130,6 +134,7 @@ export function invocationCommand(options: RunnerOptions): CommandSpec {
options.effort,
"--permission-mode",
permissionMode(options.mode),
...grokPermissionRules(options.mode),
"--sandbox",
grokSandbox(options.mode),
"--tools",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -133,4 +133,26 @@ describe("parseProviderOutput", () => {
)
).toThrow("final agent message");
});

it("reports a Grok error result as a provider failure, not malformed output", () => {
const stdout = JSON.stringify({
type: "result",
subtype: "error_during_execution",
is_error: true,
stop_reason: "end_turn",
session_id: "grok-session",
modelUsage: { "grok-4.6-build": {} },
});
let thrown: unknown;
try {
parseProviderOutput("grok", stdout, "", "grok-4.6");
} catch (error) {
thrown = error;
}
expect(thrown).toMatchObject({
status: "child-failed",
evidence: stdout,
metadata: { reportedModel: "grok-4.6-build", sessionId: "grok-session" },
});
});
});
64 changes: 55 additions & 9 deletions plugins/pstack/skills/poteto-mode/scripts/runner/parse-output.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,19 @@ import {

type JsonObject = Record<string, unknown>;

export class ProviderReportedFailure extends Error {
constructor(
message: string,
readonly status: "permission-cancelled" | "child-failed",
readonly evidence: string,
readonly metadata: Omit<ParsedOutput, "text">
) {
super(message);
}
}

const GROK_PERMISSION_CANCELLED = /User cancelled the execution for tool `([^`]+)`/;

function object(value: unknown): JsonObject | null {
return value !== null && typeof value === "object" && !Array.isArray(value)
? (value as JsonObject)
Expand Down Expand Up @@ -84,8 +97,21 @@ function parseClaude(stdout: string, requestedModel: string): ParsedOutput {
};
}

function cancelledGrokToolResult(event: JsonObject): string | null {
const content = object(event.message)?.content;
if (!Array.isArray(content)) return null;
const block = content.find((item) => {
const toolResult = object(item);
return toolResult?.type === "tool_result"
&& GROK_PERMISSION_CANCELLED.test(JSON.stringify(toolResult.content));
});
return block === undefined ? null : JSON.stringify(block);
}

function parseGrok(stdout: string, requestedModel: string): ParsedOutput {
let result: JsonObject | null = null;
let resultLine = "";
let cancelledToolResult: string | null = null;
for (const line of stdout.split("\n")) {
if (line.trim().length === 0) continue;
let raw: unknown;
Expand All @@ -95,23 +121,43 @@ function parseGrok(stdout: string, requestedModel: string): ParsedOutput {
throw new Error("grok emitted a non-JSON event");
}
const event = object(raw);
if (event?.type === "result") result = event;
if (event?.type === "result") {
result = event;
resultLine = line;
}
if (event?.type === "user") {
cancelledToolResult = cancelledGrokToolResult(event) ?? cancelledToolResult;
}
}

if (result === null) throw new Error("grok result did not contain a terminal event");
if (result.is_error === true || result.subtype !== "success") {
throw new Error("grok reported an error result");
}
const text = nullableString(result.result);
if (text === null) throw new Error("grok result did not contain final text");

return {
text,
const metadata = {
reportedModel: modelFromUsage(result.modelUsage, "grok", requestedModel),
sessionId: nullableString(result.session_id),
usage: normalizedUsage(result.usage),
costUsd: finiteNumber(result.total_cost_usd) ?? null,
};
if (result.is_error === true || result.subtype !== "success") {
if (result.stop_reason === "cancelled" && cancelledToolResult !== null) {
const tool = GROK_PERMISSION_CANCELLED.exec(cancelledToolResult)?.[1];
throw new ProviderReportedFailure(
`grok cancelled ${tool} at a permission prompt it cannot show headless`,
"permission-cancelled",
`${cancelledToolResult}\n${resultLine}`,
metadata
);
}
throw new ProviderReportedFailure(
"grok reported an error result",
"child-failed",
resultLine,
metadata
);
}
const text = nullableString(result.result);
if (text === null) throw new Error("grok result did not contain final text");

return { text, ...metadata };
}

function parseCodex(stdout: string): ParsedOutput {
Expand Down
63 changes: 60 additions & 3 deletions plugins/pstack/skills/poteto-mode/scripts/runner/run.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,10 @@ if (name === "claude") {
console.log(JSON.stringify({type:"thread.started",thread_id:"o1"}));
console.log(JSON.stringify({type:"item.completed",item:{type:"agent_message",text:"CODEX_OK"}}));
console.log(JSON.stringify({type:"turn.completed",usage:{input_tokens:20,cached_input_tokens:5,output_tokens:3,reasoning_output_tokens:1}}));
} else if (process.env.FAKE_GROK_PERMISSION_CANCELLED === "1") {
console.log(JSON.stringify({type:"system",subtype:"init",session_id:"g2",tools:["run_terminal_command"],slash_commands:Array(800).fill("skill")}));
console.log(JSON.stringify({type:"user",message:{role:"user",content:[{type:"tool_result",tool_use_id:"t0",content:"x".repeat(5000),is_error:false},{type:"tool_result",tool_use_id:"t1",content:"[{\\"type\\":\\"content\\",\\"content\\":{\\"type\\":\\"text\\",\\"text\\":\\"User cancelled the execution for tool \`run_terminal_command\`\\"}}]",is_error:true}]},session_id:"g2"}));
console.log(JSON.stringify({type:"result",subtype:"error_during_execution",is_error:true,stop_reason:"cancelled",errors:["cancelled"],session_id:"g2",usage:{input_tokens:30,output_tokens:4},total_cost_usd:0.02,modelUsage:{[model + "-build"]:{}}}));
} else {
console.log(JSON.stringify({type:"assistant",message:{content:[{type:"text",text:"progress"}]}}));
console.log(JSON.stringify({type:"result",subtype:"success",is_error:false,result:"GROK_OK",session_id:"g1",usage:{input_tokens:30,output_tokens:4,total_tokens:34},total_cost_usd:0.02,modelUsage:{[model + "-build"]:{}}}));
Expand Down Expand Up @@ -241,6 +245,7 @@ beforeEach(() => {
delete process.env.FAKE_GROK_TRANSIENT_UNAUTH_PATH;
delete process.env.FAKE_GROK_PREFLIGHT_LOG_PATH;
delete process.env.FAKE_GROK_MISSING_MODEL;
delete process.env.FAKE_GROK_PERMISSION_CANCELLED;
delete process.env.FAKE_DESCENDANT_HOLDS_PIPES_MS;
delete process.env.FAKE_DESCENDANT_PID_PATH;
delete process.env.FAKE_SELF_SIGNAL;
Expand All @@ -265,6 +270,7 @@ afterEach(() => {
delete process.env.FAKE_GROK_TRANSIENT_UNAUTH_PATH;
delete process.env.FAKE_GROK_PREFLIGHT_LOG_PATH;
delete process.env.FAKE_GROK_MISSING_MODEL;
delete process.env.FAKE_GROK_PERMISSION_CANCELLED;
delete process.env.FAKE_DESCENDANT_HOLDS_PIPES_MS;
delete process.env.FAKE_DESCENDANT_PID_PATH;
delete process.env.FAKE_SELF_SIGNAL;
Expand Down Expand Up @@ -294,6 +300,29 @@ describe("runLane", () => {
});
}

it("classifies a Grok permission cancellation with the cancellation as evidence", async () => {
process.env.FAKE_GROK_PERMISSION_CANCELLED = "1";
const input = options("grok", "grok-permission-cancelled");
const result = await runLane(input);
expect(result.exitCode).toBe(77);
expect(existsSync(input.outputPath)).toBe(false);
const recorded = receipt(input.receiptPath);
expect(recorded).toMatchObject({
status: "permission-cancelled",
reportedModel: "grok-4.6-build",
modelVerified: true,
sessionId: "g2",
usage: { inputTokens: 30, outputTokens: 4 },
error: {
message: "grok cancelled run_terminal_command at a permission prompt it cannot show headless",
},
});
expect(recorded.error?.evidence).toContain("User cancelled the execution for tool");
expect(recorded.error?.evidence).toContain('"stop_reason":"cancelled"');
expect(recorded.error?.evidence).not.toContain('"subtype":"init"');
expect(recorded.error?.evidence).not.toContain('"tool_use_id":"t0"');
});

it("records Codex's exact argv without fabricating a reported model", async () => {
const input = options("codex");
const result = await runLane(input);
Expand Down Expand Up @@ -916,21 +945,49 @@ describe("childEnvironment", () => {
CLAUDE_CODE_CHILD_SESSION: "1",
KEEP_ME: "yes",
};
expect(childEnvironment("claude", source)).toEqual({
expect(childEnvironment("claude", "isolated-write", source)).toEqual({
PATH: "/bin",
CLAUDECODE: "1",
CLAUDE_CODE_CHILD_SESSION: "1",
KEEP_ME: "yes",
});
expect(childEnvironment("codex", source)).toEqual({
expect(childEnvironment("codex", "isolated-write", source)).toEqual({
PATH: "/bin",
CODEX_THREAD_ID: "codex",
CODEX_CI: "1",
KEEP_ME: "yes",
});
expect(childEnvironment("grok", source)).toEqual({
expect(childEnvironment("grok", "read-only", source)).toEqual({
PATH: "/bin",
KEEP_ME: "yes",
});
});

it("passes a Grok writer, whose shell is pre-approved, only an allowlisted environment", () => {
const source = {
PATH: "/bin",
HOME: "/home/dev",
TMPDIR: "/tmp/dev",
LANG: "en_US.UTF-8",
LC_ALL: "en_US.UTF-8",
HTTPS_PROXY: "http://proxy:8080",
GROK_HOME: "/home/dev/.grok",
XAI_API_KEY: "xai-key",
GH_TOKEN: "gh-secret",
AWS_SECRET_ACCESS_KEY: "aws-secret",
SSH_AUTH_SOCK: "/tmp/agent.sock",
CLAUDECODE: "1",
KEEP_ME: "yes",
};
expect(childEnvironment("grok", "isolated-write", source)).toEqual({
PATH: "/bin",
HOME: "/home/dev",
TMPDIR: "/tmp/dev",
LANG: "en_US.UTF-8",
LC_ALL: "en_US.UTF-8",
HTTPS_PROXY: "http://proxy:8080",
GROK_HOME: "/home/dev/.grok",
XAI_API_KEY: "xai-key",
});
});
});
Loading