Repository navigation
feat: expose 45 MCP tools with image, select, dialog and task controls - #52
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
|
||
| async function remoteElement(tabId: number, ref: string, frameId: number, fileInput = false): Promise<string> { | ||
| if (!ref) throw new Error('Missing element ref'); | ||
| const expression = `(() => {const el=(${targetElement.toString()})(${JSON.stringify(ref)}); ${fileInput ? `if(el.matches('input[type="file"]')) return el; if(el.tagName==='LABEL' && el.control?.type==='file') return el.control; const child=el.querySelector('input[type="file"]') ?? el.parentElement?.querySelector('input[type="file"]'); if(!child) throw new Error('No file input at ref'); return child;` : `el.scrollIntoView({block:'center',inline:'center',behavior:'instant'}); if(!el.checkVisibility({checkOpacity:true,checkVisibilityCSS:true})) throw new Error('Element not visible after scroll'); return el;`} })()`; |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 35fb4c22a3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const frame = /^f(\d+)/.exec(refPrefix); | ||
| node.frameId = frame ? Number(frame[1]) : 0; |
There was a problem hiding this comment.
When a caller uses a valid top-level prefix beginning with f and digits, such as browser_snapshot({refPrefix:"f3"}), this code marks every top-document node as belonging to frame 3. bridgeFrame() then routes subsequent clicks, reads, and other ref-based operations to that child frame, so they fail or act on the wrong document. Reserve this prefix form for internally generated child-frame refs or derive frameId from the actual frame rather than the caller-controlled prefix.
Useful? React with 👍 / 👎.
| domHash, | ||
| takenAt: Date.now(), | ||
| }; | ||
| const domHash = fastHash(`${location.pathname}::${nodes.map((node) => JSON.stringify(node)).join('|')}::${textSnippets.join('|')}`); |
There was a problem hiding this comment.
Hashing the entire node object includes bbox, inViewport, and ref metadata, so scrolling, sticky elements, or harmless layout animation changes domHash even when an action has no semantic effect. The verifier treats any hash difference as successful progress, which can falsely validate no-op clicks and prevent the repeated-action loop guard from firing on animated pages. Hash stable semantic fields instead, and handle scrolling through its dedicated action result.
Useful? React with 👍 / 👎.
MCP clients on main currently discover only 10 tools, while the expanded bridge implementation exists only in the local checkout. Publish the shared 45-tool catalogue through both stdio and HTTP MCP, together with the extension handlers, native-file support, tests and documentation needed to use it.
The six new public tools are browser_image, browser_select_option, browser_handle_dialog, agent_task_pause, agent_task_resume and agent_task_wait. Images use MCP image content instead of a text block of base64. Select supports refs and CSS selectors and rejects hidden/disabled controls and disabled choices. Dialog observation is armed explicitly with status; accept/dismiss never runs automatically. Task waiting uses a transport timeout that covers its requested wait. Existing local tools add tabs, eval/wait, network/console, cookies/storage, downloads/uploads, clipboard and richer snapshots. CDP actions share an attachment, and confirmation state is saved before waiting for a decision.
Validation on an isolated main checkout: scripts/check.sh passed (365 unit tests, fixture/security/native-file/MCP smoke checks, typecheck, lint, dead-code scan, shellcheck and production build); coverage thresholds passed; npm pack --dry-run includes tools.js and local-files.js. MCP smoke drives the real stdio server and HTTP fallback across all 45 tools. Added an opt-in live acceptance script for the new primitives.
Live verification limitation: the installed HTTP MCP endpoint returned all 45 tools. A live CSS-selector selection failure was fixed and covered by a unit regression test. The subsequent browser verification session was stopped by a Computer Use URL restriction, so successful live image/dialog/task-control acceptance is not claimed. The acceptance script is available as node scripts/bridge-tools-live.mjs after rebuilding and reloading the extension.
Unrelated conversation/UI work is excluded. This PR does not publish an npm release; existing MCP clients need reconnection to refresh tools/list.