[Aaryan430]

Why

• In an internal end-to-end test, fetchAccessibilityIssues used about 30 of 68 minutes. A first read of a report that was still finalizing waited up to 5 minutes inside the call. The claude.ai connector aborts a tool call after 305 s without a response, with or without progress notifications, so each of those calls left the agent with nothing. One agent then reported a fix as verified that had added 26 issues.
• In the 4 Nov daily report, 41 of 150 accessibility calls failed (72.7% success). Most are fixed by fix(accessibility): scan fixes from the 2 Nov stability report #541. Of the rest:
  • 8 were refusals only the account can resolve: the 50-scan plan limit 5 times, no External APIs add-on twice, and "user info missing" once. Agents retried them, renamed the scan, or reported them with submitFeedback.
  • 2 were accessibilityExpert questions that the knowledge base refused as over 500 characters.
• Reading a scan run that had failed returned "Retrieved 0 accessibility issues", which reads like a clean page.

What changed

A finalizing report answers at once instead of holding the call

• Before: a first read polled every 10 s, for up to 5 minutes, while the report was finalizing (a web report adding its advanced issues) or rule_processing (an app build).
• Now: one status read. If the first page is still in either state, the tool returns a normal result that says:
  • the report is not ready and no issues were read, so it must not be taken as a clean page or a verified fix;
  • call again with the same arguments (pageUrl and compareWith are kept) in a minute or two;
  • if it still says this after about 10 minutes, tell the user the report is stuck.
• Later pages, and reports whose tests are still running, are read as before, with the "still processing" note.
• REPORT_STATUS_POLL_MS, REPORT_STATUS_WAIT_MS and the progress notifications sent while waiting are removed.

Reading a failed or cancelled scan run says so

• Before: "Retrieved 0 accessibility issues … All issues retrieved".
• Now an error result says:
  • the run failed or was cancelled and has no results, so an empty list does not mean the page is clean;
  • why, using fix(accessibility): scan fixes from the 2 Nov stability report #541's failure details (the run's own reason and each failed page's reason);
  • the dashboard link;
  • to start a new scan.
• The reason lines are shared with startAccessibilityScan's failure result (failureReasons).

Account and plan refusals are explained

The API says	The tool now says
…make over 50 scans…	The plan's limit of saved Website Scanner scans is reached. Delete old scans in the dashboard or upgrade the plan. This is a plan limit: tell the user, and don't retry or rename the scan.
…access to External APIs…	The plan lacks the External APIs add-on that these tools work through. The dashboard still works, and an admin can add the add-on through sales. Don't retry.
user info missing	Website Scanner has not set this user up yet. Open Website Scanner in the dashboard once, then start the scan again.

• The API's own message is kept on a BrowserStack said: line, and telemetry still records the original error.
• These results don't get the "report it with submitFeedback" hint, because the user resolves them on their account. A new NO_FEEDBACK_HINT symbol in feedback-hint.ts marks them. Being a symbol, it is never sent to the client.

Long accessibilityExpert questions are shortened, not refused

• The knowledge base refuses anything over 500 characters ("Query is too long").
• A longer question now keeps its last sentence, where agents put the actual question, plus as much of its opening as fits. A single long sentence is cut at a word break.
• The answer starts with a note giving the original and sent lengths.
• The tool description is unchanged, to stay within the description budget test.

Merging

• Stacked on fix(accessibility): scan fixes from the 2 Nov stability report #541 (base fix/a11y-scan-stability): the failed-run result uses fix(accessibility): scan fixes from the 2 Nov stability report #541's failure details. Merge fix(accessibility): scan fixes from the 2 Nov stability report #541 first, and this PR moves to main.
• Conflicts with chore(accessibility): deprecate the accessibilityExpert tool #542 in executeAccessibilityRAG: both put a note before the expert's answer. Whichever merges second keeps both notes.

Verified

• Lint, prettier, tsc and vitest all pass (131 files, 1892 tests).
  • New: tests/tools/accessibilityAccountRefusals.test.ts (10 tests).
  • Updated: accessibilityScanOutput.test.ts covers finalizing, rule_processing, kept pageUrl/compareWith, later pages, and failed, cancelled and completed runs.
• Live, against a real account:
  • Finalizing: an SDK build read straight after the run returned "still finalizing" in 1.0 s. Read again 26 s later, it returned its issue in 4.0 s. The old code would have held the first call for up to 5 minutes.
  • Agents, in isolated claude -p sessions: an agent with a way to wait came back about 90 s later and read the report's 14 issues. An agent without one told the user to check back, instead of calling the page clean.
  • Failed run: reading scan 4300543 (a domain that doesn't resolve) now says the run failed and gives the page's reason, instead of 0 issues.
  • Expert: a 676-character question was sent as 499 characters and answered in 1.6 s, with the note. Before this change the knowledge base refused it.
• Not tested live: the three account refusals, because the test account hits none of them. Unit tests cover them, using the API messages as logged in telemetry.

🤖 Generated with Claude Code

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: Central YAML (base), Organization UI (inherited), Workspace UI (inherited); Enforced overrides: Organization
• Review profile: ASSERTIVE
• Plan: Enterprise
• Run ID: 0ca79041-0831-4344-a52a-8f8fccda8b56

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• Trigger review

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

[Aaryan430]

Closing in favour of #553: this PR's change is commit ffc614d there, reconciled with #553's failed-run fix (one failed/cancelled-run message instead of two).