Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/TOOLS.md
Original file line number Diff line number Diff line change
Expand Up @@ -344,7 +344,7 @@ searchCapability → describeCapability → invokeCapability
| `accessibilityExpert` | Ask the A11y Expert (WCAG 2.0/2.1/2.2, mobile/web usability, best practices). |
| `createAccessibilityAuthConfig` | Create an authentication configuration (form-based or basic) for accessibility scans behind a login. |
| `getAccessibilityAuthConfig` | Retrieve an existing accessibility authentication configuration by ID. |
| `fetchAccessibilityIssues` | Fetch accessibility issues, with pagination, from a Website Scanner run, an Automated Tests build (web) or App Accessibility build (mobile app, by the build id the SDK prints), a Workflow Analyzer report, an App Accessibility manual report (`appReportId`), or a report's dashboard link. Once the tests or scan have finished, waits while the report is still processing; a run still in progress is read as it is, with a note. For an app build with no scans, lists the likely causes. |
| `fetchAccessibilityIssues` | Fetch accessibility issues, with pagination, from a Website Scanner run, an Automated Tests build (web) or App Accessibility build (mobile app, by the build id the SDK prints), a Workflow Analyzer report, an App Accessibility manual report (`appReportId`), or a report's dashboard link (a Website Scanner link is read by the run's build id, however old the scan; a Workflow Analyzer link by the report id it carries). Once the tests or scan have finished, waits while the report is still processing; a run still in progress is read as it is, with a note. For an app build with no scans, lists the likely causes. |
| `list_app_accessibility_builds` | List an App Accessibility project's builds newest first, with each build's id, status, app and issue counts - use this to find the latest run, a build by name, or one person's builds. |
| `list_app_accessibility_projects` | List the account's App Accessibility projects newest first, or find one by name - use this to get the project id that lists its builds. |
| `list_manual_app_accessibility_reports` | List manual App Accessibility reports (App Live sessions scanned with Workflow Analyzer) newest first, with each report's id, app, status and issue counts - use this to find a report QA shared or the latest manual scan of an app. |
Expand Down
87 changes: 78 additions & 9 deletions src/tools/accessibility.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ import {
AccessibilityReportFetcher,
EXPORT_WAIT_MS,
ExportPendingError,
ScanRunNotFoundError,
scanRunLink,
type IssueSource,
} from "./accessiblity-utils/report-fetcher.js";
import {
Expand Down Expand Up @@ -213,6 +215,11 @@ function sourceArgs(source: IssueSource): string {
switch (source.kind) {
case "scan":
return `scanId: "${source.scanId}", scanRunId: "${source.scanRunId}"`;
// A run read by its build id is named by its link again, or, given as a build id, by that.
case "scanBuild":
return source.scanName !== undefined
? `reportUrl: "${scanRunLink(source)}"`
: `buildId: "${source.thBuildId}"`;
case "build":
case "appBuild":
return `buildId: "${source.buildId}"`;
Expand All @@ -228,6 +235,8 @@ function describeSource(source: IssueSource): string {
switch (source.kind) {
case "scan":
return `Website Scanner scan ${source.scanId}, run ${source.scanRunId}`;
case "scanBuild":
return `Website Scanner run with build id ${source.thBuildId}`;
case "build":
return `Automated Tests build ${source.buildId}`;
case "workflow":
Expand Down Expand Up @@ -955,12 +964,69 @@ async function fetchAccessibilityIssues(
requested.kind === "link"
? await reportFetcher.sourceFromReportUrl(requested.reportUrl)
: requested;
// A build id names a web build or a mobile app build; the APIs tell which.
const source = await reportFetcher.resolveBuild(named);
// A build id names a web build, a mobile app build, or a Website Scanner run (a scanner
// link's thBuildId); the APIs tell which.
let source = await reportFetcher.resolveBuild(named, { orScanRun: true });
// Comparing runs needs the scan's ids, which only the scan list maps a build id to.
if (compareWith !== undefined && source.kind === "scanBuild") {
source = await reportFetcher.findScanRun(source);
}
// Two scan runs are compared through their CSV exports; two app builds, here below.
if (compareWith !== undefined && source.kind === "scan") {
return compareScanRuns(source, compareWith, config, context);
}
const pageArg = pageUrl ? `, pageUrl: "${pageUrl}"` : "";
const notes: string[] = [];
// A run named by its build id is read by it, which works however old the scan is. If the
// API does not know it by that id, the scan list is searched as before.
let reportLink: string | undefined;
if (source.kind === "scanBuild") {
try {
reportLink = await reportFetcher.getReportLink(source, {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: a scanBuild read caches a possibly partial export for the link's whole lifetime.

getReportLink is called here without stillProcessing, because reportStatus returns undefined for scanBuild. So the export goes into reportExports (keyed by JSON.stringify(source)) until the S3 link's Expires minus the margin, which is hours. The first-page note says "a scan that finished in the last few minutes may still be adding issues", but every later call with the same link in this process gets that same snapshot back from the cache, including calls made after the scan has finished. On the hosted server the process lives long, so this is the normal case.

Scenario: user shares a scanner link right after the run ends, the agent reads it (partial), the user asks again 20 minutes later and gets the same partial count with no note that it is stale.

Suggestion: pass stillProcessing: true for scanBuild (neither reuse nor keep the export) until the status can be read by build id. Or give scanBuild entries a short TTL. Once findScanRun has resolved the ids (the fallback path), the normal status check applies, so only the direct route needs this.

exportTaskId,
onWait: exportProgress(context),
});
if (named.kind === "build") {
notes.push(
`Build id ${source.thBuildId} is a Website Scanner run's build id; read it as one.`,
);
}
if (cursor === 0) {
notes.push(
"Read by the run's build id, so its processing status was not checked: a scan that finished in the last few minutes may still be adding issues.",
);
}
} catch (error) {
if (error instanceof ExportPendingError) {
return createSuccessResponse(
exportPendingMessages(source, error, pageArg),
);
}
if (!(error instanceof ScanRunNotFoundError)) throw error;
if (source.scanName === undefined) {
throw new Error(
error.routeMissing
? `No Automated Tests (web) or App Accessibility build with id "${source.thBuildId}". If it is a Website Scanner run's build id, this API cannot read a run by it yet: pass the scan's report link as reportUrl instead`
: `No Automated Tests (web), App Accessibility or Website Scanner build with id "${source.thBuildId}". Pass the id the BrowserStack SDK printed as "testhub id" at the end of the run (40 lowercase letters and digits), or the report's dashboard link as reportUrl`,
);
}
const why = error.routeMissing
? "This API does not read a run by its build id yet"
: "The API could not read this run by the link's build id (not found, or not visible to this account)";
try {
source = await reportFetcher.findScanRun(source);
} catch (searchError) {
throw new Error(
`${why}, and searching the scan list found nothing either: ${
searchError instanceof Error
? searchError.message
: String(searchError)
}`,
);
}
notes.push(`${why}, so it was found in the scan list instead.`);
}
}
if (compareWith !== undefined && source.kind !== "appBuild") {
return createErrorResponse(
`compareWith compares Website Scanner runs and App Accessibility builds; ${describeSource(source)} is neither.`,
Expand All @@ -975,10 +1041,12 @@ async function fetchAccessibilityIssues(
const stillProcessing =
processing.status !== undefined &&
PROCESSING_STATUSES.has(processing.status);
const preface =
requested.kind === "link"
const preface = [
...(requested.kind === "link"
? [`Resolved the report link to ${describeSource(source)}.`]
: [];
: []),
...notes,
];

if (source.kind === "appBuild" && compareWith !== undefined) {
return compareAppBuilds(
Expand Down Expand Up @@ -1015,10 +1083,8 @@ async function fetchAccessibilityIssues(
);
}

const pageArg = pageUrl ? `, pageUrl: "${pageUrl}"` : "";
let reportLink: string;
try {
reportLink = await reportFetcher.getReportLink(source, {
reportLink ??= await reportFetcher.getReportLink(source, {
exportTaskId,
onWait: exportProgress(context),
stillProcessing,
Expand Down Expand Up @@ -1156,10 +1222,13 @@ async function compareScanRuns(
context?: ScanProgressContext,
): Promise<CallToolResult> {
const reportFetcher = await initializeReportFetcher(config);
const source =
const named =
requested.kind === "link"
? await reportFetcher.sourceFromReportUrl(requested.reportUrl)
: requested;
// Runs are compared by their scan's ids, which only the scan list maps a build id to.
const source =
named.kind === "scanBuild" ? await reportFetcher.findScanRun(named) : named;
if (source.kind !== "scan") {
return createErrorResponse(
'compareWith compares Website Scanner runs: pass the newer run (scanId and scanRunId, or its report link) and compareWith: "previous" or the older run\'s scanRunId.',
Expand Down
Loading
Loading