Retire the CLI, reduce the tool menu to eight entries, and fix four provider bugs - #83
Merged
Conversation
Everything the CLI built because it needed an agent front-end -- sessions and resume, skills, the TUI, print and RPC modes, model selection -- pi supplies, so the extension replaces it rather than reimplementing it. The cua act executor path and the --print -o jsonl telemetry go with it. Removes a third of the rename surface before the rename touches it: 45 of 122 files that mention cua, and 8,946 lines. ptywright stays: it is independently useful and published, but it now has no in-repo consumer, which the architecture doc records.
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.
Autofix Details
Bugbot Autofix prepared fixes for both issues found in the latest run.
- ✅ Fixed: CI dropped workspace typecheck
- Added a
npx tsc -bstep to thepi-extension-unitCI job so the full TypeScript project reference graph is typechecked in CI again.
- Added a
- ✅ Fixed: Stale CLI lockfile entry
- Removed the stale
packages/cli@onkernel/cua-cliextraneous block frompackage-lock.jsonto complete the workspace deletion cleanup.
- Removed the stale
Or push these changes by commenting:
@cursor push 465e38220c
Preview (465e38220c)
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -59,6 +59,7 @@ jobs:
- run: npm ci
- run: npm run build --workspace @onkernel/cua-ai
- run: npm run build --workspace @onkernel/cua-agent
+ - run: npx tsc -b
- name: Pi extension unit tests
run: npm test --workspace @onkernel/cua-pi-extension
@@ -59,6 +59,7 @@ jobs:
- run: npm ci
- run: npm run build --workspace @onkernel/cua-ai
- run: npm run build --workspace @onkernel/cua-agent
+ - run: npx tsc -b
- name: Pi extension unit tests
run: npm test --workspace @onkernel/cua-pi-extension
diff --git a/package-lock.json b/package-lock.json
--- a/package-lock.json
+++ b/package-lock.json
@@ -6309,31 +6309,6 @@
"node": ">=22.19.0"
}
},
- "packages/cli": {
- "name": "@onkernel/cua-cli",
- "version": "0.9.0",
- "extraneous": true,
- "license": "MIT",
- "dependencies": {
- "@earendil-works/pi-agent-core": "0.83.0",
- "@earendil-works/pi-coding-agent": "0.83.0",
- "@earendil-works/pi-tui": "0.83.0",
- "@onkernel/cua-agent": "0.10.0",
- "@onkernel/cua-ai": "0.10.0",
- "@onkernel/sdk": "0.49.0"
- },
- "bin": {
- "cua": "dist/cli.js"
- },
- "devDependencies": {
- "@onkernel/ptywright": "0.1.0",
- "tsdown": "^0.22.2",
- "vitest": "^3.2.4"
- },
- "engines": {
- "node": ">=22.19.0"
- }
- },
"packages/pi-extension": {
"name": "@onkernel/cua-pi-extension",
"version": "0.10.0",
@@ -6309,31 +6309,6 @@
"node": ">=22.19.0"
}
},
- "packages/cli": {
- "name": "@onkernel/cua-cli",
- "version": "0.9.0",
- "extraneous": true,
- "license": "MIT",
- "dependencies": {
- "@earendil-works/pi-agent-core": "0.83.0",
- "@earendil-works/pi-coding-agent": "0.83.0",
- "@earendil-works/pi-tui": "0.83.0",
- "@onkernel/cua-agent": "0.10.0",
- "@onkernel/cua-ai": "0.10.0",
- "@onkernel/sdk": "0.49.0"
- },
- "bin": {
- "cua": "dist/cli.js"
- },
- "devDependencies": {
- "@onkernel/ptywright": "0.1.0",
- "tsdown": "^0.22.2",
- "vitest": "^3.2.4"
- },
- "engines": {
- "node": ">=22.19.0"
- }
- },
"packages/pi-extension": {
"name": "@onkernel/cua-pi-extension",
"version": "0.10.0",You can send follow-ups to the cloud agent here.
The deleted cli-unit job was the only one running tsc -b over the whole project graph, and also the only one building and testing ptywright -- so deleting it dropped both. A typecheck-and-ptywright job restores them, Zig setup included. Regenerating the lockfile drops the extraneous packages/cli entry and 2,400 lines of CLI-only dependencies. That exposed a second problem: pi was reaching node_modules only through npm's automatic peer install once the CLI stopped depending on it, while the extension's end-to-end test spawns the pi binary. The extension now declares the pi packages it tests against, and drops the pi-tui peer it never imports.
The menu is now one entry per capability: browser and computer (each including its batch form), browser-act, playwright, and the four provider-native surfaces. mixed, the standalone batch selectors, and the 37 individual tool names are gone -- they were packaging, not capability. Three fixes behind it, each verified against a live API rather than inferred: Anthropic rejects its native browser and computer tools in one request (400: viewport frame vs display frame). The compiler now refuses that pair instead of letting it reach the wire, and the menu reports it as a conflict rather than as unavailability. Google's schema quirk was not a model limit but a serialization gap: the Gemini API rejects the JSON Schema keywords const and additionalProperties outright rather than ignoring them. A payload transform narrows both for Google, so Gemini now takes every browser tool including browser_act, and the quirk is deleted. A deactivated selection now warns on stderr outside TUI mode. Previously a scripted run lost its tools silently, created no browser, and let the model answer from memory with exit 0. /cua-tools compiles each entry on its own. It had been passing the current selection to the tool menu, whose verdicts are relative to that selection, so one failing selection marked everything unavailable.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.
Autofix Details
Bugbot Autofix prepared fixes for both issues found in the latest run.
- ✅ Fixed: Gemini transform misses flat tools
- Extended the Gemini schema payload transform to also narrow flat Google Interactions function-tool
parameters, with coverage added by a regression test.
- Extended the Gemini schema payload transform to also narrow flat Google Interactions function-tool
- ✅ Fixed: Resume breaks on old selectors
- Added legacy persisted-selector migration and safe restore handling so session_start no longer throws on retired selector names and instead restores with warning output.
Or push these changes by commenting:
@cursor push a50e486d12
Preview (a50e486d12)
diff --git a/packages/ai/src/tool-catalog.ts b/packages/ai/src/tool-catalog.ts
--- a/packages/ai/src/tool-catalog.ts
+++ b/packages/ai/src/tool-catalog.ts
@@ -570,21 +570,21 @@
* `enum`, and `additionalProperties: false` only tightens validation the model
* never performs. Rewriting them is what lets Google take the same declarations
* every other provider gets, verified against the live API.
+ *
+ * Google's transports carry function schemas in two shapes:
+ * `tools[].functionDeclarations[].parameters` (Generative AI) and
+ * `tools[].parameters` (Interactions). This transform narrows both.
*/
function createGeminiSchemaTransform(): CuaPayloadTransform {
return {
identity: "provider.google.function-declaration-schema",
- writes: ["tools.functionDeclarations"],
+ writes: ["tools.functionDeclarations", "tools.parameters", "tools.function.parameters"],
phase: "tool-declarations",
apply(payload) {
if (!isRecord(payload) || !Array.isArray(payload.tools)) return payload;
return {
...payload,
- tools: payload.tools.map((tool) =>
- isRecord(tool) && Array.isArray(tool.functionDeclarations)
- ? { ...tool, functionDeclarations: tool.functionDeclarations.map(narrowToGeminiSchema) }
- : tool,
- ),
+ tools: payload.tools.map((tool) => narrowGeminiToolDeclaration(tool)),
};
},
};
@@ -592,6 +592,20 @@
const GEMINI_UNSUPPORTED_SCHEMA_KEYWORDS = new Set(["additionalProperties", "$schema", "$defs", "definitions"]);
+function narrowGeminiToolDeclaration(tool: unknown): unknown {
+ if (!isRecord(tool)) return tool;
+ if (Array.isArray(tool.functionDeclarations)) {
+ return { ...tool, functionDeclarations: tool.functionDeclarations.map(narrowToGeminiSchema) };
+ }
+ if ("parameters" in tool) {
+ return { ...tool, parameters: narrowToGeminiSchema(tool.parameters) };
+ }
+ if (isRecord(tool.function) && "parameters" in tool.function) {
+ return { ...tool, function: { ...tool.function, parameters: narrowToGeminiSchema(tool.function.parameters) } };
+ }
+ return tool;
+}
+
function narrowToGeminiSchema(node: unknown): unknown {
if (Array.isArray(node)) return node.map(narrowToGeminiSchema);
if (!isRecord(node)) return node;
diff --git a/packages/ai/test/tool-catalog.test.ts b/packages/ai/test/tool-catalog.test.ts
--- a/packages/ai/test/tool-catalog.test.ts
+++ b/packages/ai/test/tool-catalog.test.ts
@@ -409,6 +409,29 @@
expect(sent).toContain('"enum":["text"]');
});
+ it("also rewrites flat function tools on the Interactions transport", async () => {
+ const [googleNative] = cua.providers.google.toolsets.browser();
+ const catalog = compileCuaToolCatalog({
+ model: getCuaModel("google:gemini-3.6-flash"),
+ requestedTools: [googleNative!, cua.tools.browser.waitFor()],
+ });
+ expect(catalog.model.api).toBe(GOOGLE_CUA_INTERACTIONS_API);
+ const waitFor = catalog.toolDeclarations.find((tool) => tool.name === "browser_wait_for");
+ if (!waitFor) throw new Error("expected browser_wait_for declaration");
+ const raw = {
+ tools: [
+ { type: "function", name: googleNative!.name, parameters: { type: "object" } },
+ { type: "function", name: waitFor.name, description: waitFor.description, parameters: waitFor.parameters },
+ ],
+ };
+ const sent = JSON.stringify(await catalog.payload.apply(raw, catalog.model));
+ expect(JSON.stringify(raw)).toContain('"const"');
+ expect(sent).toContain('"type":"computer_use"');
+ expect(sent).toContain('"name":"browser_wait_for"');
+ expect(sent).not.toContain('"const"');
+ expect(sent).not.toContain('"additionalProperties"');
+ });
+
it("leaves other providers' declarations untouched", async () => {
const catalog = compileCuaToolCatalog({
model: getCuaModel("openai:gpt-5.6-sol"),
diff --git a/packages/pi-extension/src/index.ts b/packages/pi-extension/src/index.ts
--- a/packages/pi-extension/src/index.ts
+++ b/packages/pi-extension/src/index.ts
@@ -14,6 +14,7 @@
allSelectableSpecs,
compileSpecs,
expandSelection,
+ parsePersistedSelection,
parseSelection,
selectorAvailability,
type CuaSelection,
@@ -218,7 +219,11 @@
selection = flags.selection;
browserOptions = flags.browserOptions;
const saved = restoreConfig(ctx.sessionManager.getBranch());
- if (saved) selection = parseSelection(saved.selectors.join(","), saved.coordinates);
+ if (saved) {
+ const restored = parsePersistedSelection(saved.selectors, saved.coordinates);
+ selection = restored.selection;
+ if (restored.warning) process.stderr.write(`cua: ${restored.warning}\n`);
+ }
configureDeclarations();
installTools();
initialized = false;
diff --git a/packages/pi-extension/src/selection.ts b/packages/pi-extension/src/selection.ts
--- a/packages/pi-extension/src/selection.ts
+++ b/packages/pi-extension/src/selection.ts
@@ -48,6 +48,15 @@
export const CUA_SELECTORS: readonly string[] = Object.freeze(Object.keys(MENU));
+const LEGACY_SELECTOR_ALIASES: Readonly<Record<string, readonly string[]>> = Object.freeze({
+ mixed: ["browser", "computer"],
+ "browser-batch": ["browser"],
+ "browser_batch": ["browser"],
+ "computer-batch": ["computer"],
+ "computer_batch": ["computer"],
+});
+const LEGACY_TOOL_SELECTOR_MAP: Readonly<Record<string, string>> = legacyToolSelectorMap();
+
export function parseSelection(value: string | undefined, coordinates: string | undefined): CuaSelection {
const coordinateMode = coordinates ?? "pixels";
if (coordinateMode !== "pixels" && coordinateMode !== "normalized-1000") {
@@ -65,6 +74,20 @@
return Object.freeze({ selectors: Object.freeze(selectors), coordinates: coordinateMode });
}
+export function parsePersistedSelection(
+ selectors: readonly string[],
+ coordinates: string | undefined,
+): { selection: CuaSelection; warning?: string } {
+ const migrated = migrateLegacySelectors(selectors);
+ const selection = parseSelection(migrated.selectors.join(","), coordinates);
+ if (!migrated.legacy.length && !migrated.dropped.length) return { selection };
+ const notes = [
+ ...(migrated.legacy.length ? [`mapped legacy selectors: ${migrated.legacy.join(", ")}`] : []),
+ ...(migrated.dropped.length ? [`ignored unknown selectors: ${migrated.dropped.join(", ")}`] : []),
+ ];
+ return { selection, warning: `restored saved CUA selection (${notes.join("; ")})` };
+}
+
/**
* Every tool any menu entry can contribute, for the up-front registration pi
* requires before a tool can be activated. Keyed by model-facing name, so the
@@ -163,3 +186,51 @@
function message(error: unknown): string {
return error instanceof Error ? error.message : String(error);
}
+
+function legacyToolSelectorMap(): Readonly<Record<string, string>> {
+ const coordinates = cua.coordinates.pixels();
+ const mapping = new Map<string, string>();
+ const ambiguous = new Set<string>();
+ for (const [selector, resolver] of Object.entries(MENU)) {
+ for (const spec of resolver(coordinates)) {
+ const existing = mapping.get(spec.name);
+ if (existing && existing !== selector) {
+ ambiguous.add(spec.name);
+ mapping.delete(spec.name);
+ continue;
+ }
+ if (!ambiguous.has(spec.name)) mapping.set(spec.name, selector);
+ }
+ }
+ return Object.freeze(Object.fromEntries(mapping));
+}
+
+function migrateLegacySelectors(selectors: readonly string[]): {
+ selectors: string[];
+ legacy: string[];
+ dropped: string[];
+} {
+ const next: string[] = [];
+ const seen = new Set<string>();
+ const legacy: string[] = [];
+ const dropped: string[] = [];
+ const append = (selector: string): void => {
+ if (seen.has(selector)) return;
+ seen.add(selector);
+ next.push(selector);
+ };
+ for (const selector of selectors) {
+ if (CUA_SELECTORS.includes(selector)) {
+ append(selector);
+ continue;
+ }
+ const mapped = LEGACY_SELECTOR_ALIASES[selector] ?? (LEGACY_TOOL_SELECTOR_MAP[selector] ? [LEGACY_TOOL_SELECTOR_MAP[selector]] : undefined);
+ if (mapped) {
+ legacy.push(selector);
+ for (const mappedSelector of mapped) append(mappedSelector);
+ continue;
+ }
+ dropped.push(selector);
+ }
+ return { selectors: next, legacy, dropped };
+}
diff --git a/packages/pi-extension/test/extension.test.ts b/packages/pi-extension/test/extension.test.ts
--- a/packages/pi-extension/test/extension.test.ts
+++ b/packages/pi-extension/test/extension.test.ts
@@ -246,6 +246,47 @@
expect(legacy.active).not.toContain("computer_click");
});
+ it("restores legacy saved selectors without crashing session_start", async () => {
+ const written: string[] = [];
+ const write = vi.spyOn(process.stderr, "write").mockImplementation(((chunk: string) => {
+ written.push(String(chunk));
+ return true;
+ }) as never);
+ try {
+ const pi = makePi({
+ "cua-tools": "playwright",
+ "cua-coordinates": "pixels",
+ "cua-browser-timeout": "300",
+ "cua-profile-save-changes": false,
+ });
+ const restoredCtx = {
+ ...ctx,
+ sessionManager: {
+ getBranch: () => [
+ {
+ type: "custom",
+ customType: "cua-pi-config-v1",
+ data: {
+ version: 1,
+ origin: "command",
+ selectors: ["mixed", "browser_snapshot", "not-a-selector"],
+ coordinates: "pixels",
+ },
+ },
+ ],
+ },
+ } as unknown as ExtensionContext;
+ extension(pi.api);
+ await getHandler(pi, "session_start")({}, restoredCtx);
+ expect(pi.active).toContain("browser_snapshot");
+ expect(pi.active).toContain("computer_click");
+ expect(pi.active).not.toContain("playwright_execute");
+ expect(written.join("")).toMatch(/restored saved CUA selection/);
+ } finally {
+ write.mockRestore();
+ }
+ });
+
it("removes stale incompatible CUA schemas from the provider payload", async () => {
// A provider-native surface is the incompatibility that survives the model
// allowlist's removal: an unknown provider now compiles fine, but Anthropic's
diff --git a/packages/pi-extension/test/selection.test.ts b/packages/pi-extension/test/selection.test.ts
--- a/packages/pi-extension/test/selection.test.ts
+++ b/packages/pi-extension/test/selection.test.ts
@@ -1,6 +1,13 @@
import { getCuaModel } from "@onkernel/cua-ai";
import { describe, expect, it } from "vitest";
-import { compileSpecs, CUA_SELECTORS, expandSelection, parseSelection, selectorAvailability } from "../src/selection";
+import {
+ compileSpecs,
+ CUA_SELECTORS,
+ expandSelection,
+ parsePersistedSelection,
+ parseSelection,
+ selectorAvailability,
+} from "../src/selection";
describe("CUA pi selectors", () => {
it("has stable exact browser and computer entry membership, batch included", () => {
@@ -65,6 +72,16 @@
"playwright_execute",
]);
});
+
+ it("migrates legacy persisted selectors and warns when unknown names are dropped", () => {
+ const restored = parsePersistedSelection(
+ ["mixed", "browser_snapshot", "playwright_execute", "not-a-selector"],
+ "pixels",
+ );
+ expect(restored.selection.selectors).toEqual(["browser", "computer", "playwright"]);
+ expect(restored.warning).toMatch(/mapped legacy selectors/);
+ expect(restored.warning).toMatch(/ignored unknown selectors/);
+ });
it("compiles Anthropic native computer use only for supported Anthropic models", () => {
const specs = expandSelection(parseSelection("anthropic-computer", "pixels"));
const catalog = compileSpecs(getCuaModel("anthropic:claude-fable-5"), specs);You can send follow-ups to the cloud agent here.
Found by running OpenAI's native computer path end to end for the first time. A computer result with no screenshot put its failure text in an error key inside computer_call_output.output, which the Responses API refuses outright: 400 Unknown parameter: 'input[4].output.error' So a single failed action poisoned the rest of the conversation. The output now always carries a valid computer_screenshot and the failure text follows as a user message, so the model still learns what happened. A 1x1 placeholder is not enough: the Responses API rejects it even though the vision endpoint accepts one, hence 64x64.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
Autofix Details
Bugbot Autofix prepared a fix for the issue found in the latest run.
- ✅ Fixed: Broken error-key regression assertion
- Replaced the escaped-string check with a key-pattern assertion (
/"error"\s*:/) so the test now fails if a forbiddenerrorfield is serialized.
- Replaced the escaped-string check with a key-pattern assertion (
Or push these changes by commenting:
@cursor push 0afcbac8cb
Preview (0afcbac8cb)
diff --git a/packages/ai/test/openai-native-provider.test.ts b/packages/ai/test/openai-native-provider.test.ts
--- a/packages/ai/test/openai-native-provider.test.ts
+++ b/packages/ai/test/openai-native-provider.test.ts
@@ -191,7 +191,7 @@
// Responses API answer 400 `Unknown parameter: 'input[N].output.error'`, which
// poisoned every later request in the conversation.
const sent = await sendWithResult([{ type: "text", text: "click failed" }], true);
- expect(sent).not.toContain('\\"error\\"');
+ expect(sent).not.toMatch(/"error"\s*:/);
expect(sent).toContain("computer_screenshot");
expect(sent).toContain("image_url");
// The text still reaches the model, as a message rather than an invalid field.You can send follow-ups to the cloud agent here.
An unresolvable ref throws before any input is sent, but the step's expect was still awaited afterwards, so a model that invented a ref burned a full global timeout per attempt -- 5 calls and ~5 minutes where the primitives took 2 calls and 15 seconds. Scoped to actions that provably never dispatched. Two existing tests caught a broader first attempt: when delivery is merely uncertain, a lost acknowledgement may mean the input landed, and the expectation is how that gets discovered, so those still wait.
…uous assertion
The Gemini schema narrowing only walked tools[].functionDeclarations, but
the Interactions transport serializes function tools flat as
{ type: "function", parameters }. Selecting a native surface alongside a
function tool derives that transport, so const and additionalProperties
still reached the API as a 400 -- after the menu said the selection was
fine. It now narrows both shapes.
A session persisted before the menu shrank can name a selector that no
longer exists, and restore ran it straight through parseSelection, so the
session threw instead of starting. Retired selectors are now dropped with
a stderr note.
The new computer_call_output guard searched for a backslash-escaped
"error", which JSON.stringify never emits, so reintroducing the invalid
field would have passed. That assertion was worthless; it now fails
against the old shape.
rgarcia
changed the title
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
Bugbot Autofix prepared a fix for the issue found in the latest run.
- ✅ Fixed: Empty restore ignores saved selection
- Restore now always reparses the persisted command selection (even when filtering leaves zero selectors), so resumed sessions keep an intentional empty or fully-retired selection instead of falling back to CLI flags.
Or push these changes by commenting:
@cursor push 366ecaca2e
Preview (366ecaca2e)
diff --git a/packages/pi-extension/src/index.ts b/packages/pi-extension/src/index.ts
--- a/packages/pi-extension/src/index.ts
+++ b/packages/pi-extension/src/index.ts
@@ -229,7 +229,7 @@
if (dropped.length) {
process.stderr.write(`cua: ignoring retired tool selector(s) from this session: ${dropped.join(", ")}\n`);
}
- if (known.length) selection = parseSelection(known.join(","), saved.coordinates);
+ selection = parseSelection(known.join(","), saved.coordinates);
}
configureDeclarations();
installTools();
diff --git a/packages/pi-extension/test/extension.test.ts b/packages/pi-extension/test/extension.test.ts
--- a/packages/pi-extension/test/extension.test.ts
+++ b/packages/pi-extension/test/extension.test.ts
@@ -333,6 +333,32 @@
}
});
+ it("restores an intentionally empty command selection on resume", async () => {
+ const pi = makePi({
+ "cua-tools": "playwright",
+ "cua-coordinates": "pixels",
+ "cua-browser-timeout": "300",
+ "cua-profile-save-changes": false,
+ });
+ extension(pi.api);
+ const resumedCtx = {
+ ...ctx,
+ sessionManager: {
+ getBranch: () => [
+ {
+ type: "custom",
+ customType: "cua-pi-config-v1",
+ data: { version: 1, origin: "command", selectors: [], coordinates: "pixels" },
+ },
+ ],
+ },
+ } as unknown as ExtensionContext;
+
+ await getHandler(pi, "session_start")({}, resumedCtx);
+
+ expect(pi.active).toEqual(["bash"]);
+ });
+
it("warns on stderr when a selection deactivates outside TUI mode", async () => {
const written: string[] = [];
const write = vi.spyOn(process.stderr, "write").mockImplementation(((chunk: string) => {You can send follow-ups to the cloud agent here.
Reviewed by Cursor Bugbot for commit e451e5f. Configure here.
A persisted selection comes from /cua-tools, which deliberately overrides the flags. Falling back to the flags when every persisted selector had been retired re-enabled tools that session had explicitly replaced, and made the stderr note misleading: it said retired selectors were ignored while the whole selection had been.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Retires the CLI, replaces its selection surface with an eight-entry menu, and fixes four provider bugs that only live testing could find. Net −10,300 lines.
This started as "delete the CLI" and grew, because the menu is only honest if the compiler underneath it is. Every provider claim below was verified against a live API, not inferred from a compile.
1. The CLI is gone
Everything the CLI built because it needed an agent front-end — sessions and resume, skills, the TUI, print and RPC modes, model selection — pi supplies, so #82's extension replaces it rather than reimplementing it.
cua actand the--print -o jsonltelemetry go with it.It was also 45 of the 122 files that mention
cuaand 8,946 lines, so deleting it first means the rename in 5c sweeps a tree a third smaller instead of renaming code on its way to the bin.@onkernel/ptywrightstays — independently useful and published — but it now has no in-repo consumer. Worth deciding whether a PTY testing library belongs in this repo once the repo is "browser tools for your agent."2. The menu is eight capability entries
browser,computer(each including its batch form),browser-act,playwright, and the four provider-native surfaces.mixed, the standalone batch selectors, and all 37 individual tool names are gone — packaging, not capability.selection.tswent from 246 lines to 145.3. Four bugs, all found by running it
Anthropic's two native surfaces cannot coexist. The live API answers 400:
browser_20260701 cannot be declared alongside a computer_* tool ... the browser tool's viewport coordinate frame is not compatible with the computer tool's display coordinate frame. The compiler let the pair through to the wire; it now refuses locally, and the menu reports it as a conflict rather than as unavailability.Google's schema quirk was never a model limitation. The Gemini API rejects the JSON Schema keywords
constandadditionalPropertiesoutright rather than ignoring them. Both have exact equivalents, so a payload transform rewrites them — and Gemini then accepts every function tool includingbrowser_actandbrowser_wait_for, which the quirk had marked unavailable. Verified end-to-end: a Gemini agent with--cua-tools browserread example.com's link text asLearn more, the page's real text, where the same run previously returned the staleMore information...from training data with no browser involved.OpenAI's native computer path was unusable past the first failed action. A
computer_call_outputwhose result carried no image put the failure text in anerrorkey, which the Responses API refuses:400 Unknown parameter: 'input[4].output.error'. One failed action poisoned every later request. The output now always carries a validcomputer_screenshotwith the text following as a user message. (A 1×1 placeholder is not enough — the Responses API rejects it even though the vision endpoint accepts one.)browser_actburned a full deadline on actions that never dispatched. An unresolvable ref throws immediately, but the step'sexpectwas still awaited, so a model that invented a ref spent 5 calls and ~5 minutes where the primitives took 2 calls and 15 seconds. Scoped carefully: two existing tests caught a broader first attempt, because uncertain delivery may mean the input landed and the expectation is how that gets discovered.4. Two QA findings about honesty
A deactivated selection was silent outside the TUI. The reason reached only the status line, so a scripted run lost its tools, created no browser, and let the model answer from memory with exit 0. It now warns on stderr, once per distinct reason. Exit code stays 0 deliberately — the agent may have other tools.
/cua-toolslied when it mattered most. It passed the current selection into a menu whose verdicts are relative to that selection, so one failing selection marked every entry unavailable — including entries that then activated fine. Each entry now compiles standalone, with pairwise conflicts reported separately.Verified live
openai-computer175.29.112.234anthropic-computer91.124.0.159anthropic-browser149.57.247.38google-browser185.133.242.58browserLearn moreoff the live pageThis VM's egress is
23.19.247.81, so each drove a real cloud browser.playwright,browser-act, andcomputerwere exercised on xAI in an earlier hands-on pass. Every browser created was deleted; final count zero.typecheck 0, cua-ai 99, cua-agent 283, cua-pi-extension 30. CI gainstypecheck-and-ptywright, which restores the whole-projecttsc -band ptywright's build and tests — the deletedcli-unitjob was the only place those ran.Follow-ups, not here
cacheRead/cacheWrite/cacheWrite1hper assistant message and shipscomputeCacheWaste/detectCacheMissinternally, so the deleted JSONL schema has a better replacement already in the host. Reading it is a small addition, not a rewrite.--cua-tools→--browser-toolsand the five browser flags collapsing into one--browser-optionsJSON.Note
Medium Risk
Dropping the CLI release workflow and CI smoke tests removes the main automated guard for the published
cuabinary; remaining risk is mostly doc/CI drift unless library and pi-extension tests cover the same surface.Overview
Retires the
cuaCLI from shipping and documentation. The rootREADMEis rewritten around@onkernel/cua-agent’sattach()/compile()flow and installing@onkernel/cua-pi-extensionin pi, dropping quickstart, CLI reference, named sessions, transcripts, skills, and subcommands.CI and releases follow the same cut.
.github/workflows/release-cua-cli.ymlis deleted;ci.ymldrops CLI build, pack, and bin smoke tests and renames the remaining job totypecheck-and-ptywright(whole-workspacetsc -bplus ptywright only). Agent skills (release,update-docs) anddocs/npm-releases.mdnow cover@onkernel/cua-ai,@onkernel/cua-agent, and manual-first publish for@onkernel/cua-pi-extension.Architecture docs swap the composition root.
docs/architecture.mdremoves the CLI composition and TUI selector sections and documents pi-extension composition instead;docs/cua-cli-harness-migration.mdis removed.@onkernel/ptywrightis noted as having no in-repo consumer after the CLI removal.Reviewed by Cursor Bugbot for commit ee413ab. Bugbot is set up for automated code reviews on this repo. Configure here.