Skip to content

fix(accessibility): leave saved false positives out of issue answers - #556

Draft
Aaryan430 wants to merge 1 commit into
ATA-1564-fp-memory-mcpfrom
ATA-1564-fp-memory-leave-out
Draft

Aaryan430 wants to merge 1 commit into
ATA-1564-fp-memory-mcpfrom
ATA-1564-fp-memory-leave-out

Conversation

@Aaryan430

Copy link
Copy Markdown
Collaborator

Stacked on #554. Merge that first; this PR then retargets to main.

Problem

#554 only adds the account's false-positive memory to fetchAccessibilityIssues answers and leaves the agent to apply the entries. startAccessibilityScan never reads the memory. So a scan's first answer lists and counts issues the account has already marked as not real, and says nothing about the memory. The agent has no reason to fetch again.

Seen on https://www.sbb.ch/de/angebote/halbtax (scan 4311473, run 2431733):

Tool Issues Memory in answer
startAccessibilityScan 47 none
fetchAccessibilityIssues (same run) 28 19 left out: role-required entry #4 (12) and color-contrast entries #5–#11 (7)

Change

Both tools now drop, on the server, any issue that a saved entry names by its pattern's tag, tags, tagPrefix or classes, within the entry's site and scope. The answer says how many issues each entry left out. The remaining entries for the page's rules still come with the issues, for the agent to apply with judgment.

  • report-parser: new leaveOut hook. Covered issues are dropped from every page and from every count (summary, duplicates, per-page counts). The full report's order, rule texts and cursor offsets are kept, so a cursor the agent already holds stays valid.
  • paginate: pageOf takes a skip predicate. Skipped rows still move the cursor forward.
  • fp-memory: applyMemory matches entries on pattern fields, never on the predicate's wording. Counter-examples, excludeTags, and tags the predicate's own wording excludes (e.g. "Does not cover sbb-breadcrumb") are never dropped.
  • accessibility: new shared memoryLeaveOut helper, used by both fetchAccessibilityIssues and startAccessibilityScan.
  • web_accessibility capability index: adds pattern.tags and pattern.excludeTags, and explains that filtering uses pattern fields only.

Not in this PR

  • fetchAccessibilityIssues with compareWith (compareScanRuns) still doesn't apply the memory, so saved false positives can still show up as new issues.
  • No test yet for startAccessibilityScan with a memory. The new tests cover the fetch path, the parser and the matcher.

Testing

  • tsc --noEmit passes, and all 140 test files (2,070 tests) pass under Node 22.
  • Checked by hand against the SBB scan above: the fetch answer leaves out the 19 covered issues and keeps sbb-breadcrumb, which the entries exclude.
  • Not yet run by hand: the rebuilt startAccessibilityScan path.

🤖 Generated with Claude Code

fetchAccessibilityIssues only handed the agent the account's false-positive
memory and left it to apply the entries; startAccessibilityScan did not read
the memory at all, so a scan's first answer listed and counted issues the
account had already marked as not real.

Both now leave out, server side, the issues an entry names by its pattern's
tag, tags, tagPrefix or classes (within its site and scope), and say how many
each entry left out. The rest of the memory still comes with the issues for
the agent to apply with judgment.

- report-parser: a leaveOut hook drops covered issues from every page and
  count (summary, duplicates, per-page counts) while keeping the order, rule
  texts and cursor offsets of the full report, so held cursors stay valid.
- paginate: pageOf takes a skip predicate that still advances the cursor.
- fp-memory: applyMemory matches entries by pattern fields, never prose;
  counter-examples, excludeTags and tags a predicate excludes are kept.
- accessibility: shared memoryLeaveOut helper used by both
  fetchAccessibilityIssues and startAccessibilityScan.
- web_accessibility capability index: pattern.tags and pattern.excludeTags.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Nov 6, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Central YAML (base), Organization UI (inherited), Workspace UI (inherited); Enforced overrides: Organization
  • Review profile: ASSERTIVE
  • Plan: Enterprise
  • Run ID: 5236550b-b712-4ab8-b487-e4b83563ac89

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant