Reliability + token-efficiency: message-recipient guard, click locator, content-script self-healing, auto-screenshot budget

[gaur-avvv]

Reliability + token-efficiency: message-recipient guard, click locator, content-script self-healing, auto-screenshot budget

Scope: all changes are confined to src/chrome.

Summary

This PR fixes a set of compounding failures that made WebBrain unable to click/send on hardened, heavily dynamic sites (LinkedIn, Gmail, and others), and cuts the two largest avoidable costs in the agent loop — redundant screenshots and repeated full-DOM walks — without changing what the agent can do.

What was broken

1. The message-recipient guard blocked ordinary clicks. Before every click/click_ax/set_field{submit:true}/press_keys(Enter) on guarded pages, a content probe tried to prove the action was not a message send — but it could only prove that in a few narrow ways (navigation link, editable field, conversation row, or a control near a composer). Everything else (Connect, Follow, Like, Save & close, Minimize, feed rows, nav) returned inconclusive, and inconclusive was treated as block, before dispatch. The click never reached the page and the model was told to retry, so it looped.

2. The content script could go permanently silent. After an extension reload/move, the old content script stayed in open tabs with a dead runtime context (it received messages but could never answer), and the double-injection guard (if (window.__webbrain_injected) return;) made re-injection a no-op. The agent saw undefined and reported opaque errors. Separately, the dispatcher had no try/catch on the synchronous path, so a synchronous handler throw closed the channel silently.

3. The text-click locator declared false ambiguity. A label-wrapped button matched itself and its inner text node → "Ambiguous text match" → the model was sent to do coordinate arithmetic. Short contains needles were worse (click({text:"X"}) matched 13 controls), and placeholder/aria-labelled controls were unreachable because the candidate list only considered visible text.

4. Every action paid for a screenshot + a vision call. The default auto-screenshot mode (state_change) captured a shot after every state-changing tool, and each capture triggered a separate vision-model call to describe it — N identical actions cost N captures + N extra LLM round trips.

5. Text-candidate discovery re-walked the whole DOM every time — on every text click, every recipient preflight, and every retry.

What changed, by file

src/chrome/src/content/content.js

• Recipient-guard probe — proven-non-send classifiers so previously-unplaceable clicks are allowed: composer utilities (Save & close, Minimize, Exit full screen/Pop-out, attach/formatting/discard/undo, [role="toolbar"], aria-pressed toggles), far-from-composer non-send controls, and no-composer pages (everything except send-labelled / in-surface controls).
• Gmail typed recipients — collectRecipientInputValues reads To/Cc/Bcc input values (typed, not-yet-chipified addresses) in addition to [email]/[data-hovercard-id] chips, so the guard can name the address and bind a confirmation.
• Generic recipient discovery (every site) — recipient-labelled fields (type=email, autocomplete=email, or a multilingual recipient vocabulary across name/aria-label/placeholder/id/data-testid/<label>) plus address-bearing chips/links ([email], [data-email], mailto:, input[type=email]), so confirm-then-send is not Gmail-specific.
• Click locator — collapse text matches that resolve to the same actionable ancestor; short contains needles require a word-boundary hit; ambiguity is narrowed by Playwright's "receives events" rule and auto-resolved when exactly one survives; _siteInteractionText reads aria-label via getAttribute.
• _clickTextCandidates cache — cached per scope and invalidated by a single attribute-filtered MutationObserver revision counter.
• Self-healing — a generation counter replaces the boolean double-injection guard (fresh install always takes over; only the newest listener answers); the dispatcher always answers (sync throws and empty results become explicit error payloads).

src/chrome/src/agent/agent.js

• Recipient guard — unclassifiable actions pass through (the fail-closed "could not conclusively resolve" block is removed); distinct reasonCodes and loop-proof messages; the mismatch block names the page-observed recipient and prescribes the exact clarify call.
• User-granted authorization — a bound recipient clarify verifies and dispatches the send even when the planner's target differs; it re-points guard.messaging, records run-scoped approvals (guard.approvedRecipients) so multi-email tasks don't re-ask, and an explicit "send all remaining" answer sets guard.messageRecipientApprovedAll for the rest of the task.
• Generic surfaces fail open when there's no recipient to protect — a send-labelled control on a page with no recipient field and no conversation header (a contact/feedback form) is not a mis-addressed message; the guard passes it through and the submit-confirmation gate governs the submission.
• Silent no-answer recovery — an undefined content-action response force-injects the current content scripts and resends once, with a precise staleContentScript error if that still fails.
• Auto-screenshot budget (_shouldAutoScreenshot) — repetition-aware: capture the first action of a burst, then only every 4th; a different action resets the burst; honors opts.domEvidenceProven and opts.turnCaptures against the existing maxScreenshotsPerTurn budget; navigation mode still captures.

src/chrome/src/agent/adapters.js

• Recipient verification is now generic-first (every site). The guard is armed on any URL and lets the content-side recipient extraction decide whether the page is actually sending something — no address and no conversation header ⇒ nothing is a send and no action is blocked. GENERIC_MESSAGING_SURFACE_RE / GENERIC_MESSAGING_MAIL_LIKE_RE (mail/webmail + chat hosts, and generic /mail /messages /chat /dm /direct /inbox /compose /conversations routes) only choose the verification shape; mail-like surfaces get supportsRecipientSets (To/Cc/Bcc as a set), chat-like surfaces keep the strict single-identity rule.

src/chrome/src/ui/sidepanel.js

• Clarification sound — playClarifySound() (same asset/toggle as the completion chime, best-effort) plays when a clarification/permission card is rendered via renderClarifyCard.

Behavior summary (after)

Situation	Before	After
Ordinary control on LinkedIn/Gmail	Blocked loop	Dispatches
Save & close / toolbar / attach in compose	Blocked as a "send"	Dispatches
Send to a different recipient	Blocked, looped asking	Ask once, address pre-filled; approve → send; never re-asks that recipient
Multi-email task	Asked per send	"Send all remaining in this task" covers the rest
click({text:"Send"}) on a label-wrapped button	"Ambiguous text match"	Resolves to the real control
click({text:"X"})	13-match ambiguity	Word-boundary needle
Placeholder/aria-label targets	Not found	Matchable
Extension reload / moved folder	Permanent "no answer"	Self-heals
N identical actions	N screenshots + N vision calls	~N/4, first-of-burst always kept

Testing

Manual, on the built extension (load src/chrome unpacked):

1. Reload the extension and reload the target tab once.
2. LinkedIn: click Connect / Follow / Like / a profile row — dispatch, no guard block.
3. Gmail compose: fill To/Subject/body, click Send — one authorization card with the observed address; approve → send; second email to the same recipient sends without asking.
4. Save & close / Minimize / formatting buttons in compose — work, not treated as sends.
5. click({text:"Send"}) and a label-wrapped button — resolve without ambiguity.
6. A burst of identical clicks — at most ~1 screenshot per 4 actions.
7. Any non-messaging site (e.g. a contact form's Send, or ordinary buttons) — nothing is blocked.

Follow-ups (not in this PR)

• Provider-side prompt caching (Anthropic cache_control / OpenAI implicit prefix caching) is the next big latency lever; it requires a byte-identical prompt prefix per step, so it needs a prefix-stability audit before wiring.
• The clarify-answer → _bindClarifiedMessageRecipient call site should be verified by a live test; the mechanism and its gates were validated by reading, and the failure mode (a typed-but-not-chipified recipient yielding zero observed candidates) is fixed by the input-value extraction.
• Consider scoping the saved-workflow "protected messaging" check to messaging surfaces, since the guard now arms on any URL (message-dispatching saved-workflow steps on non-messaging sites would otherwise refuse to replay).

[Copilot]

Copilot wasn't able to review any files in this pull request.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[vercel]

@gaur-avvv is attempting to deploy a commit to the esokullu's projects Team on Vercel.

A member of the Team first needs to authorize it.

[esokullu]

Heads-up: this is not an image-only PR. In addition to the SVG optimization commit, it changes Chrome behavior in the recipient-send guard, content-script click/recipient probing and recovery, agent screenshot throttling, and clarification notifications.

Blocking findings:

• P1: Generic recipient discovery treats ordinary contact-form input[type=email] fields as message recipients. A contact form with an email field, textarea, and Send/Submit button can be blocked before its normal submit-confirmation gate.
• P1: Generic chat routes enable multi-recipient sets even though chat safety requires one verified conversation identity. This can pin a group chat as an authorized target.
• P1: The runtime changes are Chrome-only; Firefox gets SVG updates but not the matching guard, recovery, notification, or regression coverage.
• P2: The text-click cache includes shadow-root elements but does not invalidate when those roots update, leaving stale controls.
• P2: Screenshot burst state is global and mutated during preflight, so it can leak across tabs/tasks and suppress expected post-action captures.

Please split the SVG-only optimization into a separate PR. The behavioral work should have an accurate title/description, Firefox parity, and regressions covering the cases above before merging.

[gaur-avvv]

Heads-up: this is not an image-only PR. In addition to the SVG optimization commit, it changes Chrome behavior in the recipient-send guard, content-script click/recipient probing and recovery, agent screenshot throttling, and clarification notifications.

Blocking findings:

• P1: Generic recipient discovery treats ordinary contact-form input[type=email] fields as message recipients. A contact form with an email field, textarea, and Send/Submit button can be blocked before its normal submit-confirmation gate.
• P1: Generic chat routes enable multi-recipient sets even though chat safety requires one verified conversation identity. This can pin a group chat as an authorized target.
• P1: The runtime changes are Chrome-only; Firefox gets SVG updates but not the matching guard, recovery, notification, or regression coverage.
• P2: The text-click cache includes shadow-root elements but does not invalidate when those roots update, leaving stale controls.
• P2: Screenshot burst state is global and mutated during preflight, so it can leak across tabs/tasks and suppress expected post-action captures.

Please split the SVG-only optimization into a separate PR. The behavioral work should have an accurate title/description, Firefox parity, and regressions covering the cases above before merging.

everything is working fine now all clicks and also added clarification sound

everything is working well and this img bot only optimized img sizes of country flages only