feat(compact): expose upload_file

[alectimison-maker]

Summary

• expose upload_file to Compact Act in both Chrome and Firefox
• give Compact a narrow, platform-accurate upload schema without advertising unavailable download tools
• add direct-upload, exact-selector, and guarded lazy-widget guidance with cross-browser boundary tests

Closes #413.

Design notes

Compact gains only upload_file; Ask and the rest of the Compact tool boundary stay unchanged. Chrome accepts a current user attachmentId or an absolute filePath explicitly supplied by the user. Firefox accepts a current attachmentId or falls back to WebBrain's existing picker. Neither Compact schema exposes downloadId or points the model toward download_files/list_downloads.

The Compact prompts direct the model to use an existing file input first, avoid generic selectors when multiple inputs exist, allow one guarded initializer click for a lazy widget, and verify the page attachment before submitting.

Testing

• node --check src/chrome/src/agent/tools.js
• node --check src/firefox/src/agent/tools.js
• node --check test/run.js
• node test/run.js — 1450 passed; the sole unrelated failure is the existing package.json 26.0.10 vs newest CHANGELOG 26.0.0 mismatch
• npm run test:security — 60/60 checks passed

[vercel]

@alectimison-maker is attempting to deploy a commit to the esokullu's projects Team on Vercel.

A member of the Team first needs to authorize it.

[Copilot]

Pull request overview

Exposes a platform-specific upload_file workflow to Compact Act mode across Chrome and Firefox.

Changes:

• Adds narrow Compact upload schemas and prompt guidance.
• Preserves Ask mode and excludes download workflows.
• Adds cross-browser tool-boundary and guidance tests.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File	Description
src/chrome/src/agent/tools.js	Adds Chrome Compact upload support.
src/firefox/src/agent/tools.js	Adds Firefox Compact upload and picker support.
test/run.js	Tests Compact upload schemas, prompts, and boundaries.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[esokullu]

PR 2678 — feat(compact): expose upload_file

What it does: adds upload_file to COMPACT_TOOL_NAMES in both browsers, and rewrites the tool schema for the compact tier only (via compactUploadFileTool) so the model sees a narrower parameter set — Chrome: selector/attachmentId/filePath; Firefox: selector/attachmentId. downloadId is hidden in both. Compact prompts get a matching upload_file line, and the attachment-notice test flips to expect upload guidance in compact.

The tier-scoped schema override is a nice pattern — the notice builder (agent.js:20313) already derives canUseUploadTool from the live tool catalog, so the [UNTRUSTED USER ATTACHMENTS] handles start appearing in compact automatically with no extra wiring.

Blocking: compact has no way out of the ambiguous-selector gate

upload_file sets a per-tab recovery latch on an ambiguous selector, and the only thing that clears it is a successful get_interactive_elements returning a file-input record:

• src/chrome/src/agent/agent.js:17031 sets it; agent.js:16964 hard-fails every later upload_file with recoveryRequired: 'get_interactive_elements'
• agent.js:1684 _clearUploadSelectorRecoveryAfterInspection — clears only on name === 'get_interactive_elements'
• Firefox is identical: agent.js:13805 / 14055

get_interactive_elements is not in COMPACT_TOOL_NAMES (tools.js:1697-1705). So in compact, one ambiguous selector permanently bricks uploads for that tab until a page replacement, and the error text instructs the model to call a tool it doesn't have — the exact "point at an unavailable tool" problem the PR's schema work is trying to avoid. Also note agent.js:17028 (input-not-found) names get_interactive_elements and get_accessibility_tree; the latter is in compact, so that one is only half-wrong.

Pick one:
- add get_accessibility_tree (or read_page) as a clearing inspection when the tier lacks get_interactive_elements, and make the error text tier-aware; or

• skip the latch entirely when the compact catalog is active and just return the ambiguity error without persisting it.

Either way it needs a test — the new test only asserts schema/prompt shape, never exercises the handler.

Security: drop filePath from the Chrome compact schema

Compact has no downloads, no list_downloads, no read_downloaded_file — so there is no legitimate in-run source for a local path other than the user literally typing one. Meanwhile compact drops the full-tier hardening that exists precisely to stop invented paths (tools.js:1562, the /Users/Shared/… failure note). filePath on Chrome is a CDP-backed arbitrary local-file read that lands in a page input on an untrusted site; compact Act is also the tier most exposed to page-content injection. The schema string "explicitly supplied by the user" is guidance, not enforcement.

Recommendation: make compact attachmentId-only on Chrome, matching Firefox. That is a one-line schema change and the compact prompt line loses |filePath. If you keep it, add an explicit "never construct or guess a path; only a path the user typed in this conversation" clause.

Related, weaker point: hiding downloadId from the compact schema is prompt-level only — _executeTool still honors it if the model emits it. Harmless today (no downloadIds exist in compact), but it's a soft boundary, not a real one, and the test name (must hide downloadId) reads stronger than what's enforced.

Smaller notes

• assert.match(prompt, /upload_file[\s\S]{0,180}do not click the page upload control/i) and friends encode character distances between prompt phrases. Any reword of that bullet shifts the offsets and these fail for no real reason. Assert on the extracted upload_file line instead.
• assert.doesNotMatch(prompt, /download_files|list_downloads|downloadId/i) scans the whole compact prompt, so it now blocks any future unrelated mention. Probably intended, worth being deliberate about.
• compactUploadFileTool replaces parameters wholesale rather than deleting keys — fine, but it means a future param added to the base upload_file silently won't reach compact. A delete-based transform would fail more loudly.

[esokullu]

Pushed three commits to this branch addressing the ambiguity dead-end and tightening the compact schema. Copilot flagged the same recovery problem on both tools.js files, so this covers that too.

The dead-end. An ambiguous selector latches upload_file via _uploadSelectorRecoveryRequired, and the only thing that clears it is a get_interactive_elements response carrying a verified file-input selector (agent.js:1684). That tool wasn't in the compact catalog, so one ambiguous selector left compact unable to upload for the rest of the page's life, with the handler repeatedly asking for a tool the model had never been given.

I checked whether the latch could clear on something compact already had. It can't: read_page's forms output carries type/name/id/placeholder/value but no CSS selector (content.js:186), and get_accessibility_tree returns ref_id lines. get_interactive_elements is the only tool that produces a verified unique selector, so clearing on anything else would just let the model resume guessing.

So it's in COMPACT_TOOL_NAMES now, in both browsers, with a comment tying its presence to upload_file. Both compact prompts gained a - get_interactive_elements bullet — the compact prompt is a closed "use ONLY these" list, so without it the model wouldn't know it had the tool. I also restored the recovery sentence the PR had stripped from the compact description; it was right to strip when the tool was missing, and it's accurate now.

filePath dropped from the Chrome compact schema. Compact has no download tools, so the only file it can legitimately reach is the one the user attached to this run. A path could only come from the model inventing one — and compact drops the full-tier guidance that exists to stop exactly that (tools.js:1562, the /Users/Shared/… note). On Chrome filePath is a CDP-backed read of any local file into an untrusted page's input, and compact Act is the tier most exposed to page-content injection. Compact is now selector + attachmentId in both browsers; full tier is unchanged.

While there I switched the transform from rebuilding parameters to deleting a named COMPACT_UPLOAD_HIDDEN_PARAMS list off the base schema, so a future upload_file parameter still reaches compact instead of vanishing silently.

Tests. The character-distance regexes (/upload_file[\s\S]{0,180}…/) are gone — those break when neighbouring bullets are reworded, not when behaviour changes. Assertions now run against the prompt's own - upload_file( line. Added an exact-keys check on the compact schema, and a test for the invariant rather than the instance: every tier exposing upload_file also exposes get_interactive_elements, iterating compact/mid/full across both browsers, so it catches the regression class.

Copilot also asked for coverage of the ambiguous-selector retry itself. The existing handler test proved the latch clears but stopped there, so I extended it through the step the recovery exists for: after the inspection supplies a unique selector, the corrected upload dispatches and leaves the latch clear.

Full suite 1451 passed, security corpus 60/60. The one failure is the pre-existing package.json 26.0.10 vs CHANGELOG 26.0.0 mismatch, unrelated to this PR — worth noting it only surfaces in npm test, which PR CI doesn't run, so it'll bite at release time rather than here.

One judgment call worth a second opinion: adding get_interactive_elements grows the compact catalog by one tool that returns every interactive element, which isn't cheap for the tier whose whole point is a small context. The alternative was dropping the latch in compact and accepting selector-guessing. I went with the catalog change because the guess-prevention seemed worth more than the tokens, but it's your call.