diff --git a/.claude/issues/34-a11y-skill.md b/.claude/issues/34-a11y-skill.md new file mode 100644 index 0000000..41e127a --- /dev/null +++ b/.claude/issues/34-a11y-skill.md @@ -0,0 +1,80 @@ +# Issue #34 — Add accessibility skill and wp-tooling a11y runner (pa11y-ci) + +**Status:** in-progress +**Branch:** `v1.0.0/task/a11y-skill` +**PR:** # +**Assignee:** @Adi-ty + +--- + +## Summary + +WordPress developers — and the coding agents working alongside them — need a fast, local loop to find and fix accessibility (WCAG2AA) problems in a plugin or theme. pa11y is already the org's established accessibility engine (installed and configured by the `setup/pa11y` scaffold), but there was no local, agent-driven command that turns pa11y's output into something an agent can act on and trace back to source. + +This task adds a `wp-tooling a11y` command that runs the consumer-installed `pa11y-ci` and hands back a normalised report, plus a companion Claude Code `accessibility` skill that maps each violation's DOM selector to the template, block, or PHP that rendered it and fixes it with consent. Accessibility is a rendered-DOM concern, so this stays pure Node reusing pa11y — no WordPress ability, no MCP. + +--- + +## Decisions made + +- [2026-07-06] No `--url` flag — the runner is **config-driven only** (`.pa11yci.json` or `--config `). pa11y-ci merges cwd-config URLs with positional CLI URLs, so a flag could not reliably override the config without temp-config patching; a single source of truth keeps the runner clean. +- [2026-07-06] `setup/pa11y` dependency pin corrected `pa11y-ci ^6.0.0` → `^4.1.1` (6.x does not exist on npm; 4.1.1 is latest). The scaffold's dep merge is non-destructive, so only fresh projects were affected. +- [2026-07-06] URL-level load failures (`net::ERR_*`) are classified as `scanError` + `summary.failedUrls`, never as violations; the CLI exits 1 for them (environment problem), keeping exit 3 meaningful. +- [2026-07-06] The `accessibility` skill treats `setup/pa11y` as one way to get a config, not a requirement — a project with its own pa11y setup works as-is. +- [2026-07-06] Skill housed in wp-tooling (`skills/accessibility/`) and distributed via `setup/claude-skills`, rather than wp-dev-tools as the Phase 2 plan sketches, because wp-dev-tools has no public repo yet for remote scaffold sources. Migrate later if the lens suite consolidates there. +- [2026-07-06] `setup/pa11y` template ships `"runners": ["axe", "htmlcs"]` (the engines catch disjoint issues — proven in the evals: axe found a nested-list violation htmlcs missed) per the Phase 2 plan §5.3. +- [2026-07-06] Template URLs changed to project-owned surfaces: front page + `sample_page` (default `/?p=1`) + `search_page` (default `/?s=hello`) + optional `extra_page`, each appended to `base_url`. `wp-admin`/`wp-login` dropped: unauthenticated `/wp-admin/` only audits the login redirect (double-counting it), and `wp-login.php` chrome is core-owned — permanently exit-3 on findings no project can fix. Page paths are scaffold inputs (engine renderer cannot iterate lists, so slot inputs + a falsy-omitted section; further URLs are added directly in the generated file). + +--- + +## Files changed so far + +- `src/a11y/errors.js` — new (`RunnerError`: `EBINMISSING` / `EBINFAIL` / `EBADJSON` / `ENOURLS`) +- `src/a11y/resolve-bin.js` — new (local/hoisted `.bin` walk-up, `npx --no-install` fallback, version probe) +- `src/a11y/urls.js` — new (resolve URLs from the pa11y config) +- `src/a11y/normalize.js` — new (pure normaliser: summary counts, `wcagCriterion` parse, `domHints` extraction, `scanError` classification) +- `src/a11y/run.js` — new (`runA11y()` core + `runCli()` adapter; exit codes 0/1/2/3) +- `src/a11y/index.js` — new (barrel exposed as `@rtcamp/wp-tooling/a11y`) +- `src/cli/commands/a11y.js` — new (dispatcher shim) +- `package.json` — edited (`"./a11y"` exports entry) +- `tests/a11y/*` — new (cli, normalize, resolve-bin, urls specs + fixtures) +- `skills/accessibility/SKILL.md`, `skills/accessibility/evals/evals.json` — new (seven-step lens skill + 3 behavioural evals) +- `scaffolds/setup/claude-skills/**` — edited (manifest description + two `files[]` entries; two new template copies) +- `scaffolds/setup/pa11y/scaffold.json` — edited (dep pin fix) +- `skills/README.md` — edited (What's here + install snippets) +- `CHANGELOG.md` — edited (two Unreleased entries) +- `src/init/index.js`, `tests/ui/selects.test.js` — edited (pre-existing lint-gate errors at HEAD: `no-shadow` on `cap`, prettier wrapping; `npm run check` fails without these fixes) + +--- + +## Verification run + +```bash +$ npm run check # eslint src tests && jest +# ESLint: clean +# Tests: 726 passed, 54 suites +``` + +Tested live end-to-end on a WordPress plugin running under wp-env, with `@rtcamp/wp-tooling` installed as a dev dependency and the skills installed via `setup/claude-skills`: `--dry-run` resolves the local `pa11y-ci` 4.1.1, the config and its URLs; a plugin-rendered alt-less `` seeded on the front page produces H37 alongside WordPress core's F92/ARIA4 on wp-login → exit 3 with `failedUrls: 0`. + +Skill evals ran as subagents from `skills/accessibility/evals/evals.json`, with all assertions passing: eval-1 (audit and fix) 8/8, eval-2 (missing setup) 5/5, eval-3 (upstream/unreachable) 6/6. Outputs and per-eval `grading.json` are kept locally (`skills/*-workspace/` is gitignored). + +--- + +## Open questions + +- _(none blocking)_ + +--- + +## Notes for the reviewer + +- The exit-code contract is the API: 0 clean · 1 run failure or unreachable URL · 2 usage/binary missing · 3 violations. `failedUrls > 0` downgrades an otherwise-clean run to exit 1 so CI never greenlights a scan that silently loaded nothing. +- `pa11y-ci` is never a dependency of `@rtcamp/wp-tooling` (zero-runtime-deps rule) — the runner resolves the consumer's install and `npx --no-install` never fetches from the network. +- `skills/accessibility-workspace/` (eval outputs) is gitignored by the existing `skills/*-workspace/` rule. + +--- + +## Handoff log + +_(no rotations yet — delete this line when the first entry is added)_ diff --git a/node-packages/wp-tooling/CHANGELOG.md b/node-packages/wp-tooling/CHANGELOG.md index 8a77b0d..11f4bb4 100644 --- a/node-packages/wp-tooling/CHANGELOG.md +++ b/node-packages/wp-tooling/CHANGELOG.md @@ -8,6 +8,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ### Added +- `wp-tooling a11y` subcommand + `@rtcamp/wp-tooling/a11y` library (`runA11y()` core, `runCli()` CLI adapter). Runs the consumer-installed `pa11y-ci` — resolved from the project's `node_modules/.bin` (walking up for hoisted installs), else `npx --no-install`; never a network install — against the URLs in the project's pa11y config (`.pa11yci.json` by default, `--config ` to point elsewhere) and normalises the JSON into a stable report: `summary` counts plus per-URL `violations` carrying `id`, parsed `wcagCriterion`, `impact`, `runner`, `message`, `selector`, `context`, and grep-ready `domHints` (`tagName`, `classList`, `idAttr`, `attrs`). A URL that fails to load is a `scanError` counted in `summary.failedUrls`, never a violation. Exit codes: 0 clean · 1 run failure or unreachable URL · 2 usage/binary missing · 3 violations found. Supports `--output text|json` and `--dry-run`. The `setup/pa11y` scaffold's dependency pin is corrected to the published `pa11y-ci ^4.1.1`, and its config template now runs both engines (`"runners": ["axe", "htmlcs"]`) and scans project-owned URLs — the front page plus configurable page paths (`sample_page`, `search_page`, optional `extra_page`, each appended to `base_url`) — instead of `wp-admin`/`wp-login` (unauthenticated admin scans only ever audit the core-owned login chrome). +- `accessibility` Claude Code skill (`skills/accessibility/`) — a find → fix → re-check lens over `wp-tooling a11y`: triages violations by WCAG criterion and impact, maps each one to the theme/plugin source that rendered it via the report's `domHints`, proposes minimal fixes with consent, and re-verifies until clean; core/third-party markup is classified as upstream and reported, never patched. Installed into consumers by `setup/claude-skills` alongside the `scaffold` and `setup` skills. - Remote scaffolds — a scaffold's `scaffold.json` + templates can live in another repo. `scaffolds/sources.json` lists the source repos (pinned `{ repository, ref, path }`); each repo publishes a `scaffolds/index.json` enumerating the scaffolds it offers, which the registry fetches to discover them (one PR in the owning repo adds/changes a scaffold; wp-tooling only changes to onboard a new repo). Manifests + templates are fetched on `add`, cached under `${XDG_CACHE_HOME:-$HOME/.cache}/wp-tooling/remote/` and validated with ETag conditional requests (`304 Not Modified` serves the cache; movable tags refresh when they move). New error code `EFETCHFAIL` (network/HTTP) distinct from `EBADSCAFFOLD` (bad index/manifest). `list` is online-preferred with a cache fallback and reports unreachable sources as warnings; `validate --remote` fetches + schema-validates each index + manifest; `wp-tooling cache clear` empties the cache. Dormant by default — no `sources.json` ships. - Engine-side input discovery (`discover_from`) — an input declaration can say where to source its value from the project, so the engine fills it instead of the caller guessing. Resolves from `composer.json` / `package.json` (dotted paths; `autoload.psr-4` yields the root namespace) and `.wp-tooling.json`, with precedence `supplied → discovered → default`. Fail-safe: a missing or malformed project file falls through to the input's `default`, so a project without those files behaves exactly as before the resolver existed. Adds an optional `transform` step for derived inputs (`json-escape` doubles backslashes for the PSR-4 composer key). The inputs the engine actually rendered with are surfaced on `execute()` as `engine.inputs`. Bundled `setup/psr4` + `wp/*` scaffolds annotated with `discover_from`. - Feature toggle layer — a scaffold may declare an optional `feature` block (`config_key`, `owned_files`, `confirm_remove`, `gitignore`) marking it as a toggleable project feature. New TTY-free `enable` / `disable` / `status` verbs create or remove the owned files idempotently, manage `.gitignore` lines (Mustache-rendered against resolved inputs), prompt before deleting consumer-editable files (`confirm_remove`, overridable with `--force`), and persist on/off state in `.wp-tooling.json`. New `wp-tooling features` command — lists feature status by default, with `--enable` / `--disable` to toggle (plus `--json`, `--force`, `--no-install`, `--dry-run`); `setup/tailwind` ships as the first such feature. Additive — the `feature` block never affects the `add` / `execute` path. diff --git a/node-packages/wp-tooling/package.json b/node-packages/wp-tooling/package.json index 3e14634..a92f471 100644 --- a/node-packages/wp-tooling/package.json +++ b/node-packages/wp-tooling/package.json @@ -27,6 +27,7 @@ "./release": "./src/release/index.js", "./hooks": "./src/hooks/index.js", "./ci": "./src/ci/index.js", + "./a11y": "./src/a11y/index.js", "./version-monitor": "./src/version-monitor/index.js" }, "files": [ diff --git a/node-packages/wp-tooling/scaffolds/setup/claude-skills/scaffold.json b/node-packages/wp-tooling/scaffolds/setup/claude-skills/scaffold.json index b2b920f..3c68524 100644 --- a/node-packages/wp-tooling/scaffolds/setup/claude-skills/scaffold.json +++ b/node-packages/wp-tooling/scaffolds/setup/claude-skills/scaffold.json @@ -2,7 +2,7 @@ "slug": "claude-skills", "category": "setup", "name": "Claude Code skills", - "description": "Installs the rtCamp scaffold and setup Claude Code skills into .claude/skills/. Drop-in copy of the SKILL.md and evals.json that ship with @rtcamp/wp-tooling. Run this once per project to let Claude drive the scaffolding flow.", + "description": "Installs the rtCamp scaffold, setup and accessibility Claude Code skills into .claude/skills/. Drop-in copy of the SKILL.md and evals.json that ship with @rtcamp/wp-tooling. Run this once per project to let Claude drive the scaffolding flow.", "source": "template", "inputs": [ { @@ -31,6 +31,16 @@ "src": "templates/setup-evals.json", "dest": "{{skills_dir}}/setup/evals/evals.json", "raw": true + }, + { + "src": "templates/accessibility-SKILL.md", + "dest": "{{skills_dir}}/accessibility/SKILL.md", + "raw": true + }, + { + "src": "templates/accessibility-evals.json", + "dest": "{{skills_dir}}/accessibility/evals/evals.json", + "raw": true } ] } diff --git a/node-packages/wp-tooling/scaffolds/setup/claude-skills/templates/accessibility-SKILL.md b/node-packages/wp-tooling/scaffolds/setup/claude-skills/templates/accessibility-SKILL.md new file mode 100644 index 0000000..f1e413b --- /dev/null +++ b/node-packages/wp-tooling/scaffolds/setup/claude-skills/templates/accessibility-SKILL.md @@ -0,0 +1,128 @@ +--- +name: accessibility +description: Audit and fix WCAG accessibility violations on the running dev site using `wp-tooling a11y` (pa11y-ci). Runs the scan, triages violations by WCAG criterion and impact, maps each one to the theme/plugin source that rendered it, proposes minimal fixes with consent, and re-verifies until clean. Use whenever the developer mentions accessibility, a11y, WCAG, pa11y, screen readers, contrast, alt text, ARIA, keyboard navigation, or asks to audit or fix accessibility issues — even if they do not name a tool. +--- + +# accessibility + +Find → fix → re-check accessibility violations. Run the project's pa11y scan through `wp-tooling a11y`, turn each violation into a source-mapped fix, apply it with consent, and prove it gone with a re-run. + +## Use for + +- Auditing the dev site for WCAG violations across the URLs in the project's pa11y config. +- Fixing violations the scan reports: missing alt text, unlabelled form controls, ARIA misuse, heading order, contrast, landmark structure. +- Re-verifying accessibility after template, block, or markup changes. + +## Do not use for + +- Static code review without a running site — the runner scans live URLs. +- Performance, SEO, or i18n findings — separate lens skills cover those. +- Rewriting WordPress core or third-party plugin markup — report those instead (§4). + +## The runner + +`npx wp-tooling a11y` shells out to the consumer project's own `pa11y-ci` dev dependency and prints a normalised report. URLs come from the pa11y config only — `.pa11yci.json` at the project root by default, or any config handed over with `--config `. A project that already has its own pa11y setup works as-is; nothing here requires a particular scaffold. + +| Flag | Meaning | +|---|---| +| `--config ` | pa11y config to use (default `.pa11yci.json`) | +| `--output ` | output format — always use `json` in this skill | +| `--dry-run` | print the resolved binary, config and URLs; run nothing | + +| Exit | Meaning | Response | +|---|---|---| +| 0 | clean | report and stop | +| 1 | run failure or unreachable URL | environment problem — see §2, never a code fix | +| 2 | usage error, pa11y-ci missing, or no URLs in the config | close the preflight gap (§1) | +| 3 | violations found | the normal working state — parse and proceed | + +Normalised JSON shape (stdout): + +``` +{ tool: 'pa11y-ci', standard: 'WCAG2AA', + summary: { urls, violations, errors, warnings, notices, passedUrls, failedUrls }, + results: [ { url, scanError, + violations: [ { id, wcagCriterion, impact, runner, message, + selector, context, domHints } ] } ] } +``` + +`domHints` — `{ tagName, classList, idAttr, attrs }`, extracted from the violation's context HTML — is the bridge from finding to source: grep the repo with it (§4). + +## Workflow + +Before any other work, write a TODO list covering steps 1–7 below and keep it updated as you go (exactly one entry `in_progress` at a time). + +### 1. Preflight + +- **Config.** Locate the pa11y config: `.pa11yci.json` at the project root is the default; if the project keeps one elsewhere, pass it with `--config`. Read it and note the `urls` list. +- **Neither config nor pa11y-ci present?** Offer `npx wp-tooling add setup/pa11y --non-interactive --json --base-url=` — it writes `.pa11yci.json` and adds `pa11y-ci` to `devDependencies`. Surface the `npm install` as a developer action; never run it yourself. +- **Both engines.** Check `defaults.runners` in the config. pa11y's default is htmlcs only; axe catches rules htmlcs misses. If absent, offer the one-line edit `"runners": ["axe", "htmlcs"]` with consent. +- **Dev site up.** Probe the first configured URL (`curl -s -o /dev/null -w '%{http_code}'`). If it does not respond, surface how to start it (`npx wp-env start` or the project's own script) as a developer action, or run it with consent. +- **Show the plan.** `npx wp-tooling a11y --dry-run` (plus `--config` if non-default) — confirm the resolved binary, config and URL list with the developer before scanning. + +### 2. Run checks + +```bash +npx wp-tooling a11y --output json # append --config for a non-default config +``` + +- Exit 3 means violations to work on, not an error. Parse stdout as JSON. +- Exit 0 means clean: report `summary` and stop. +- Exit 1 with `summary.failedUrls > 0`: one or more URLs did not load — read each `results[].scanError`, fix the environment (server down, wrong port, wrong base URL in the config), and re-run. A `scanError` is never something to fix in project code. +- Do not invoke `pa11y-ci` directly or re-implement its invocation — the runner owns binary resolution and report normalisation. + +### 3. Triage + +- Group violations by `id` — the same id across many URLs or nodes is usually one underlying template fix. +- Rank groups by impact: `error` first, then `warning`. List `notice` groups (capped at 10) but do not fix them unless asked. +- Present a ranked table: id, wcagCriterion, impact, occurrence count, affected URLs, one example message. +- Confirm with the developer which groups to fix this session. Do not edit anything before that confirmation. + +### 4. Locate source + +For each confirmed group, find the code that renders the failing node: + +- Grep with the strongest hint first: `domHints.idAttr`, then a distinctive `domHints.classList` entry, then attribute values from `domHints.attrs`, then literal text near the node in `context`. +- Search theme/plugin source: PHP templates and template parts, block `render.php` / `render_callback`s, PHP that echoes markup, JS that builds DOM. Never search `vendor/`, `node_modules/`, or build output — a hit in `build/` means trace back to the `src/` file that generates it. +- **Ownership check.** Markup rendered by WordPress core or a third-party plugin (login page chrome, core widgets, embeds) is not fixable in project source. Classify the group as **upstream**, note the available remedies (filter/hook override, template override, upstream report) and move on. +- Read 2–3 nearby render sites so the fix matches house style — escaping helpers, i18n functions, class naming. +- If a group cannot be traced, say so and carry it to the report untouched rather than guessing. + +### 5. Fix (with consent) + +- Propose the minimal source edit that resolves the group: alt text, `label`/`for` association, an ARIA attribute, a heading level, a contrast token. No drive-by refactors. +- Show the diff with file and line range; ask `[apply / edit / skip]`. Apply only on approval. +- Every new user-facing string is translatable (`__()` with the project text domain) and escaped per house style. +- One group at a time: fix → re-check (§6) → next group. + +### 6. Re-check + +- If the fix touched built assets, surface `npm run build` as a developer action first (or run it with consent). +- Re-run `npx wp-tooling a11y --output json`. Confirm the group's `id` + `selector` pairs are gone from the affected URLs and that no new violations appeared. +- Large config? Scope the re-run: copy the config, keep only the affected `urls`, pass the copy with `--config`, delete it afterwards. +- If the same violation survives 3 fix attempts, stop and report: what you tried, what you observed, what is blocking, and 1–3 specific options. Do not keep iterating in silence. + +### 7. Report + +- Before/after `summary` counts: violations, errors, warnings, passedUrls. +- Fixes applied: file:line per group, with its wcagCriterion. +- Upstream findings (core / third-party) with suggested remedies — reported, not fixed. +- Untraced or skipped groups, with reasons. +- Outstanding developer actions: installs, builds, environment commands. + +## Hard rules — never violate + +- Never run `npm install`, `composer require`, or any package-manager command without explicit consent — surface them as developer actions. +- Never apply an edit without showing the diff and receiving consent. +- Never edit `vendor/`, `node_modules/`, WordPress core, or generated build output. +- Never treat a `scanError` (unreachable URL) as a code problem — it is an environment problem. +- Never silence a violation instead of fixing it — no `aria-hidden` on failing content, no pa11y ignore rules — unless the developer explicitly asks for a documented exception. +- Never lower the standard (e.g. `WCAG2AA` → `WCAG2A`) or drop URLs from the config to make a run pass. +- Never commit, push, or open PRs without explicit consent. + +## Reference + +- Runner source: `node_modules/@rtcamp/wp-tooling/src/a11y/` +- Report shape + `domHints` extraction: `node_modules/@rtcamp/wp-tooling/src/a11y/normalize.js` +- Config + dependency scaffold: `npx wp-tooling add setup/pa11y` +- WCAG quick reference: https://www.w3.org/WAI/WCAG22/quickref/ diff --git a/node-packages/wp-tooling/scaffolds/setup/claude-skills/templates/accessibility-evals.json b/node-packages/wp-tooling/scaffolds/setup/claude-skills/templates/accessibility-evals.json new file mode 100644 index 0000000..e9064a7 --- /dev/null +++ b/node-packages/wp-tooling/scaffolds/setup/claude-skills/templates/accessibility-evals.json @@ -0,0 +1,48 @@ +{ + "skill_name": "accessibility", + "evals": [ + { + "id": 1, + "prompt": "Run an accessibility audit on my plugin's dev site and fix whatever you find. The site runs at http://localhost:8765 via wp-env and .pa11yci.json is already set up at the project root.", + "expected_output": "A dry-run shown first (resolved binary, config, URL list), then `npx wp-tooling a11y --output json` executed with exit 3 treated as work-to-do. Violations grouped by id, ranked errors first, and the group list confirmed with the developer before any edit. Each fixed group is traced via domHints to the theme/plugin source, a diff is shown and approved before applying, and a re-run proves the id+selector pairs are gone. Final report includes before/after summary counts.", + "files": [], + "expectations": [ + "The skill runs `wp-tooling a11y --dry-run` and shows the resolved config and URLs before the real scan.", + "The scan uses `--output json` and exit code 3 is treated as violations-to-work-on, not a failure.", + "Violations are grouped by id and ranked by impact (errors before warnings; notices listed but not fixed unasked).", + "The developer confirms which groups to fix before any file is edited.", + "Source location uses domHints (idAttr, classList, attrs) to grep project source, not vendor/, node_modules/, or build output.", + "Every edit is shown as a diff and applied only after consent.", + "After each fix the skill re-runs the scan and verifies the specific id+selector pairs are gone.", + "The final report contains before/after summary counts and file:line for each fix." + ] + }, + { + "id": 2, + "prompt": "Check this project for accessibility issues.", + "expected_output": "The project has no pa11y config and no pa11y-ci installed, so the skill stops at preflight: it offers `npx wp-tooling add setup/pa11y` to scaffold .pa11yci.json and the pa11y-ci dev dependency, surfaces the npm install as a developer action, and does not run any package manager or invoke pa11y-ci by hand.", + "files": [], + "expectations": [ + "The skill detects the missing pa11y config / pa11y-ci during preflight rather than failing mid-run.", + "It offers the setup/pa11y scaffold as one way to get a config, not as a mandatory requirement.", + "It does NOT run npm install or composer require itself; installs are surfaced as developer actions.", + "It does NOT hand-roll a pa11y or pa11y-ci invocation to work around the missing setup.", + "If a config exists but the dev site is down, starting the environment is surfaced as a developer action or run only with consent." + ] + }, + { + "id": 3, + "prompt": "The a11y scan is failing on our WordPress site - can you sort out whatever it's complaining about? Config scans the front page, /wp-admin/ and /wp-login.php.", + "expected_output": "Violations on wp-login.php (WordPress core markup) are classified as upstream findings with suggested remedies (hooks/filters, upstream report) and are not patched in core or vendor code. Any URL with a scanError (e.g. connection refused) is treated as an environment problem - the skill checks the dev server rather than editing code. Project-owned violations follow the normal fix-with-consent loop.", + "files": [], + "expectations": [ + "Violations in markup rendered by WordPress core (wp-login.php) are classified as upstream and NOT fixed by editing core or vendor files.", + "Upstream findings appear in the report with suggested remedies instead of code patches.", + "A scanError entry (unreachable URL) is treated as an environment problem: the skill checks/starts the dev server, never edits code for it.", + "failedUrls > 0 with exit 1 does not get misreported as accessibility violations.", + "Project-owned violations still go through trace -> diff -> consent -> re-check.", + "The skill never adds pa11y ignore rules, drops URLs, or lowers the standard to make the run pass." + ] + } + ] +} diff --git a/node-packages/wp-tooling/scaffolds/setup/pa11y/scaffold.json b/node-packages/wp-tooling/scaffolds/setup/pa11y/scaffold.json index b6ff5df..96b0ed1 100644 --- a/node-packages/wp-tooling/scaffolds/setup/pa11y/scaffold.json +++ b/node-packages/wp-tooling/scaffolds/setup/pa11y/scaffold.json @@ -2,7 +2,7 @@ "slug": "pa11y", "category": "setup", "name": "pa11y-ci", - "description": "Adds .pa11yci.json for WCAG2AA accessibility testing against a running WordPress environment.", + "description": "Adds .pa11yci.json for WCAG2AA accessibility testing (axe + htmlcs) against a running WordPress environment. Scans project-owned URLs — the front page plus configurable page paths (sample_page, search_page, extra_page) — extend urls[] in the generated file for more. Unauthenticated wp-admin/wp-login scans are deliberately excluded: they only audit core-owned login chrome.", "source": "template", "files": [ { @@ -15,10 +15,25 @@ "key": "base_url", "description": "Base URL of the WordPress environment to test against (e.g. http://localhost:8888).", "required": true + }, + { + "key": "sample_page", + "description": "Path of a post or page to scan, appended to base_url (e.g. /hello-world/ or a permalink path).", + "default": "/?p=1" + }, + { + "key": "search_page", + "description": "Path of the search-results page to scan, appended to base_url.", + "default": "/?s=hello" + }, + { + "key": "extra_page", + "description": "Optional path of one more page to scan, appended to base_url. Omitted when empty; add further URLs directly in .pa11yci.json.", + "default": "" } ], "npm_dev_dependencies": { - "pa11y-ci": "^6.0.0" + "pa11y-ci": "^4.1.1" }, "scripts": { "npm": { diff --git a/node-packages/wp-tooling/scaffolds/setup/pa11y/templates/.pa11yci.json.mustache b/node-packages/wp-tooling/scaffolds/setup/pa11y/templates/.pa11yci.json.mustache index 05d9128..50a34df 100644 --- a/node-packages/wp-tooling/scaffolds/setup/pa11y/templates/.pa11yci.json.mustache +++ b/node-packages/wp-tooling/scaffolds/setup/pa11y/templates/.pa11yci.json.mustache @@ -1,6 +1,7 @@ { "defaults": { "standard": "WCAG2AA", + "runners": ["axe", "htmlcs"], "chromeLaunchConfig": { "args": ["--no-sandbox"] }, @@ -8,7 +9,8 @@ }, "urls": [ "{{base_url}}/", - "{{base_url}}/wp-admin/", - "{{base_url}}/wp-login.php" + "{{base_url}}{{sample_page}}", + "{{base_url}}{{search_page}}"{{#extra_page}}, + "{{base_url}}{{extra_page}}"{{/extra_page}} ] } diff --git a/node-packages/wp-tooling/skills/README.md b/node-packages/wp-tooling/skills/README.md index 256e332..e94f819 100644 --- a/node-packages/wp-tooling/skills/README.md +++ b/node-packages/wp-tooling/skills/README.md @@ -6,6 +6,7 @@ Copy-pasteable skill files for AI assistants (Claude Code, Cursor, etc.) that dr - [`scaffold/`](scaffold/SKILL.md) — End-to-end AI skill for `npx wp-tooling add`. Tells the AI how to discover scaffolds, introspect the project, apply naming conventions, invoke the engine, handle adaptive wiring, surface secrets without writing them, drive the TDD loop, and report. - [`setup/`](setup/SKILL.md) — Bootstraps a whole plugin or theme from one natural-language request. Detects existing tooling, plans the right sequence of setup + lint + test + feature scaffolds, confirms the plan with the developer, executes it in two phases, then emits one consolidated report of files written + developer actions outstanding. +- [`accessibility/`](accessibility/SKILL.md) — Find → fix → re-check WCAG violations on the running dev site. Runs `npx wp-tooling a11y` (pa11y-ci), triages violations by criterion and impact, maps each one to the theme/plugin source via the report's `domHints`, proposes minimal fixes with consent, and re-verifies until clean. Each skill is a directory containing a `SKILL.md` file. Same layout as the Claude Code Skills convention. @@ -18,16 +19,18 @@ The simplest path, for Claude Code users: mkdir -p .claude/skills # Option A — copy from the installed npm package (preferred): -cp -r node_modules/@rtcamp/wp-tooling/skills/scaffold .claude/skills/scaffold -cp -r node_modules/@rtcamp/wp-tooling/skills/setup .claude/skills/setup +cp -r node_modules/@rtcamp/wp-tooling/skills/scaffold .claude/skills/scaffold +cp -r node_modules/@rtcamp/wp-tooling/skills/setup .claude/skills/setup +cp -r node_modules/@rtcamp/wp-tooling/skills/accessibility .claude/skills/accessibility # Option B — download directly from GitHub if you can't install the package locally: git clone --depth 1 https://github.com/rtCamp/wp-tooling.git /tmp/wp-tooling -cp -r /tmp/wp-tooling/skills/scaffold .claude/skills/scaffold -cp -r /tmp/wp-tooling/skills/setup .claude/skills/setup +cp -r /tmp/wp-tooling/skills/scaffold .claude/skills/scaffold +cp -r /tmp/wp-tooling/skills/setup .claude/skills/setup +cp -r /tmp/wp-tooling/skills/accessibility .claude/skills/accessibility ``` -Claude Code picks up the skill on next session start. Invoke it with `/scaffold` or `/setup`, or just by describing what you want to add ("add a WP-CLI command to ...", "set up this plugin"). +Claude Code picks up the skill on next session start. Invoke it with `/scaffold`, `/setup` or `/accessibility`, or just by describing what you want ("add a WP-CLI command to ...", "set up this plugin", "fix the a11y failures"). For other AI orchestrators (Cursor, Continue, Aider, custom agents): drop the skill directory wherever your tool reads skill files from. The frontmatter follows the Claude Code convention (`name:`, `description:`); the body is portable Markdown. diff --git a/node-packages/wp-tooling/skills/accessibility/SKILL.md b/node-packages/wp-tooling/skills/accessibility/SKILL.md new file mode 100644 index 0000000..f1e413b --- /dev/null +++ b/node-packages/wp-tooling/skills/accessibility/SKILL.md @@ -0,0 +1,128 @@ +--- +name: accessibility +description: Audit and fix WCAG accessibility violations on the running dev site using `wp-tooling a11y` (pa11y-ci). Runs the scan, triages violations by WCAG criterion and impact, maps each one to the theme/plugin source that rendered it, proposes minimal fixes with consent, and re-verifies until clean. Use whenever the developer mentions accessibility, a11y, WCAG, pa11y, screen readers, contrast, alt text, ARIA, keyboard navigation, or asks to audit or fix accessibility issues — even if they do not name a tool. +--- + +# accessibility + +Find → fix → re-check accessibility violations. Run the project's pa11y scan through `wp-tooling a11y`, turn each violation into a source-mapped fix, apply it with consent, and prove it gone with a re-run. + +## Use for + +- Auditing the dev site for WCAG violations across the URLs in the project's pa11y config. +- Fixing violations the scan reports: missing alt text, unlabelled form controls, ARIA misuse, heading order, contrast, landmark structure. +- Re-verifying accessibility after template, block, or markup changes. + +## Do not use for + +- Static code review without a running site — the runner scans live URLs. +- Performance, SEO, or i18n findings — separate lens skills cover those. +- Rewriting WordPress core or third-party plugin markup — report those instead (§4). + +## The runner + +`npx wp-tooling a11y` shells out to the consumer project's own `pa11y-ci` dev dependency and prints a normalised report. URLs come from the pa11y config only — `.pa11yci.json` at the project root by default, or any config handed over with `--config `. A project that already has its own pa11y setup works as-is; nothing here requires a particular scaffold. + +| Flag | Meaning | +|---|---| +| `--config ` | pa11y config to use (default `.pa11yci.json`) | +| `--output ` | output format — always use `json` in this skill | +| `--dry-run` | print the resolved binary, config and URLs; run nothing | + +| Exit | Meaning | Response | +|---|---|---| +| 0 | clean | report and stop | +| 1 | run failure or unreachable URL | environment problem — see §2, never a code fix | +| 2 | usage error, pa11y-ci missing, or no URLs in the config | close the preflight gap (§1) | +| 3 | violations found | the normal working state — parse and proceed | + +Normalised JSON shape (stdout): + +``` +{ tool: 'pa11y-ci', standard: 'WCAG2AA', + summary: { urls, violations, errors, warnings, notices, passedUrls, failedUrls }, + results: [ { url, scanError, + violations: [ { id, wcagCriterion, impact, runner, message, + selector, context, domHints } ] } ] } +``` + +`domHints` — `{ tagName, classList, idAttr, attrs }`, extracted from the violation's context HTML — is the bridge from finding to source: grep the repo with it (§4). + +## Workflow + +Before any other work, write a TODO list covering steps 1–7 below and keep it updated as you go (exactly one entry `in_progress` at a time). + +### 1. Preflight + +- **Config.** Locate the pa11y config: `.pa11yci.json` at the project root is the default; if the project keeps one elsewhere, pass it with `--config`. Read it and note the `urls` list. +- **Neither config nor pa11y-ci present?** Offer `npx wp-tooling add setup/pa11y --non-interactive --json --base-url=` — it writes `.pa11yci.json` and adds `pa11y-ci` to `devDependencies`. Surface the `npm install` as a developer action; never run it yourself. +- **Both engines.** Check `defaults.runners` in the config. pa11y's default is htmlcs only; axe catches rules htmlcs misses. If absent, offer the one-line edit `"runners": ["axe", "htmlcs"]` with consent. +- **Dev site up.** Probe the first configured URL (`curl -s -o /dev/null -w '%{http_code}'`). If it does not respond, surface how to start it (`npx wp-env start` or the project's own script) as a developer action, or run it with consent. +- **Show the plan.** `npx wp-tooling a11y --dry-run` (plus `--config` if non-default) — confirm the resolved binary, config and URL list with the developer before scanning. + +### 2. Run checks + +```bash +npx wp-tooling a11y --output json # append --config for a non-default config +``` + +- Exit 3 means violations to work on, not an error. Parse stdout as JSON. +- Exit 0 means clean: report `summary` and stop. +- Exit 1 with `summary.failedUrls > 0`: one or more URLs did not load — read each `results[].scanError`, fix the environment (server down, wrong port, wrong base URL in the config), and re-run. A `scanError` is never something to fix in project code. +- Do not invoke `pa11y-ci` directly or re-implement its invocation — the runner owns binary resolution and report normalisation. + +### 3. Triage + +- Group violations by `id` — the same id across many URLs or nodes is usually one underlying template fix. +- Rank groups by impact: `error` first, then `warning`. List `notice` groups (capped at 10) but do not fix them unless asked. +- Present a ranked table: id, wcagCriterion, impact, occurrence count, affected URLs, one example message. +- Confirm with the developer which groups to fix this session. Do not edit anything before that confirmation. + +### 4. Locate source + +For each confirmed group, find the code that renders the failing node: + +- Grep with the strongest hint first: `domHints.idAttr`, then a distinctive `domHints.classList` entry, then attribute values from `domHints.attrs`, then literal text near the node in `context`. +- Search theme/plugin source: PHP templates and template parts, block `render.php` / `render_callback`s, PHP that echoes markup, JS that builds DOM. Never search `vendor/`, `node_modules/`, or build output — a hit in `build/` means trace back to the `src/` file that generates it. +- **Ownership check.** Markup rendered by WordPress core or a third-party plugin (login page chrome, core widgets, embeds) is not fixable in project source. Classify the group as **upstream**, note the available remedies (filter/hook override, template override, upstream report) and move on. +- Read 2–3 nearby render sites so the fix matches house style — escaping helpers, i18n functions, class naming. +- If a group cannot be traced, say so and carry it to the report untouched rather than guessing. + +### 5. Fix (with consent) + +- Propose the minimal source edit that resolves the group: alt text, `label`/`for` association, an ARIA attribute, a heading level, a contrast token. No drive-by refactors. +- Show the diff with file and line range; ask `[apply / edit / skip]`. Apply only on approval. +- Every new user-facing string is translatable (`__()` with the project text domain) and escaped per house style. +- One group at a time: fix → re-check (§6) → next group. + +### 6. Re-check + +- If the fix touched built assets, surface `npm run build` as a developer action first (or run it with consent). +- Re-run `npx wp-tooling a11y --output json`. Confirm the group's `id` + `selector` pairs are gone from the affected URLs and that no new violations appeared. +- Large config? Scope the re-run: copy the config, keep only the affected `urls`, pass the copy with `--config`, delete it afterwards. +- If the same violation survives 3 fix attempts, stop and report: what you tried, what you observed, what is blocking, and 1–3 specific options. Do not keep iterating in silence. + +### 7. Report + +- Before/after `summary` counts: violations, errors, warnings, passedUrls. +- Fixes applied: file:line per group, with its wcagCriterion. +- Upstream findings (core / third-party) with suggested remedies — reported, not fixed. +- Untraced or skipped groups, with reasons. +- Outstanding developer actions: installs, builds, environment commands. + +## Hard rules — never violate + +- Never run `npm install`, `composer require`, or any package-manager command without explicit consent — surface them as developer actions. +- Never apply an edit without showing the diff and receiving consent. +- Never edit `vendor/`, `node_modules/`, WordPress core, or generated build output. +- Never treat a `scanError` (unreachable URL) as a code problem — it is an environment problem. +- Never silence a violation instead of fixing it — no `aria-hidden` on failing content, no pa11y ignore rules — unless the developer explicitly asks for a documented exception. +- Never lower the standard (e.g. `WCAG2AA` → `WCAG2A`) or drop URLs from the config to make a run pass. +- Never commit, push, or open PRs without explicit consent. + +## Reference + +- Runner source: `node_modules/@rtcamp/wp-tooling/src/a11y/` +- Report shape + `domHints` extraction: `node_modules/@rtcamp/wp-tooling/src/a11y/normalize.js` +- Config + dependency scaffold: `npx wp-tooling add setup/pa11y` +- WCAG quick reference: https://www.w3.org/WAI/WCAG22/quickref/ diff --git a/node-packages/wp-tooling/skills/accessibility/evals/evals.json b/node-packages/wp-tooling/skills/accessibility/evals/evals.json new file mode 100644 index 0000000..e9064a7 --- /dev/null +++ b/node-packages/wp-tooling/skills/accessibility/evals/evals.json @@ -0,0 +1,48 @@ +{ + "skill_name": "accessibility", + "evals": [ + { + "id": 1, + "prompt": "Run an accessibility audit on my plugin's dev site and fix whatever you find. The site runs at http://localhost:8765 via wp-env and .pa11yci.json is already set up at the project root.", + "expected_output": "A dry-run shown first (resolved binary, config, URL list), then `npx wp-tooling a11y --output json` executed with exit 3 treated as work-to-do. Violations grouped by id, ranked errors first, and the group list confirmed with the developer before any edit. Each fixed group is traced via domHints to the theme/plugin source, a diff is shown and approved before applying, and a re-run proves the id+selector pairs are gone. Final report includes before/after summary counts.", + "files": [], + "expectations": [ + "The skill runs `wp-tooling a11y --dry-run` and shows the resolved config and URLs before the real scan.", + "The scan uses `--output json` and exit code 3 is treated as violations-to-work-on, not a failure.", + "Violations are grouped by id and ranked by impact (errors before warnings; notices listed but not fixed unasked).", + "The developer confirms which groups to fix before any file is edited.", + "Source location uses domHints (idAttr, classList, attrs) to grep project source, not vendor/, node_modules/, or build output.", + "Every edit is shown as a diff and applied only after consent.", + "After each fix the skill re-runs the scan and verifies the specific id+selector pairs are gone.", + "The final report contains before/after summary counts and file:line for each fix." + ] + }, + { + "id": 2, + "prompt": "Check this project for accessibility issues.", + "expected_output": "The project has no pa11y config and no pa11y-ci installed, so the skill stops at preflight: it offers `npx wp-tooling add setup/pa11y` to scaffold .pa11yci.json and the pa11y-ci dev dependency, surfaces the npm install as a developer action, and does not run any package manager or invoke pa11y-ci by hand.", + "files": [], + "expectations": [ + "The skill detects the missing pa11y config / pa11y-ci during preflight rather than failing mid-run.", + "It offers the setup/pa11y scaffold as one way to get a config, not as a mandatory requirement.", + "It does NOT run npm install or composer require itself; installs are surfaced as developer actions.", + "It does NOT hand-roll a pa11y or pa11y-ci invocation to work around the missing setup.", + "If a config exists but the dev site is down, starting the environment is surfaced as a developer action or run only with consent." + ] + }, + { + "id": 3, + "prompt": "The a11y scan is failing on our WordPress site - can you sort out whatever it's complaining about? Config scans the front page, /wp-admin/ and /wp-login.php.", + "expected_output": "Violations on wp-login.php (WordPress core markup) are classified as upstream findings with suggested remedies (hooks/filters, upstream report) and are not patched in core or vendor code. Any URL with a scanError (e.g. connection refused) is treated as an environment problem - the skill checks the dev server rather than editing code. Project-owned violations follow the normal fix-with-consent loop.", + "files": [], + "expectations": [ + "Violations in markup rendered by WordPress core (wp-login.php) are classified as upstream and NOT fixed by editing core or vendor files.", + "Upstream findings appear in the report with suggested remedies instead of code patches.", + "A scanError entry (unreachable URL) is treated as an environment problem: the skill checks/starts the dev server, never edits code for it.", + "failedUrls > 0 with exit 1 does not get misreported as accessibility violations.", + "Project-owned violations still go through trace -> diff -> consent -> re-check.", + "The skill never adds pa11y ignore rules, drops URLs, or lowers the standard to make the run pass." + ] + } + ] +} diff --git a/node-packages/wp-tooling/src/a11y/errors.js b/node-packages/wp-tooling/src/a11y/errors.js new file mode 100644 index 0000000..29c0d44 --- /dev/null +++ b/node-packages/wp-tooling/src/a11y/errors.js @@ -0,0 +1,27 @@ +/** + * RunnerError: the structured error type thrown by the a11y runner library. + * + * Mirrors `src/scaffolds/errors.js` (ScaffoldError) so callers branch on a + * stable machine-readable `code` while the message stays human-readable. + * Extra fields supplied via `details` are attached verbatim (e.g. `install`, + * `configPath`, `detail`). + * + * Codes: + * EBINMISSING the external binary (pa11y-ci) is not installed + * EBINFAIL the binary ran but failed for a reason other than "found violations" + * EBADJSON the binary produced output that could not be parsed as JSON + * ENOURLS no URLs could be resolved from the pa11y config + */ + +'use strict'; + +class RunnerError extends Error { + constructor(code, message, details = {}) { + super(message); + this.name = 'RunnerError'; + this.code = code; + Object.assign(this, details); + } +} + +module.exports = { RunnerError }; diff --git a/node-packages/wp-tooling/src/a11y/index.js b/node-packages/wp-tooling/src/a11y/index.js new file mode 100644 index 0000000..7b89332 --- /dev/null +++ b/node-packages/wp-tooling/src/a11y/index.js @@ -0,0 +1,15 @@ +/** + * Barrel for the a11y runner library exposed as `@rtcamp/wp-tooling/a11y`. + */ + +'use strict'; + +const { runA11y } = require('./run'); +const { normalizeA11y } = require('./normalize'); +const { resolveUrls } = require('./urls'); + +module.exports = { + runA11y, + normalizeA11y, + resolveUrls, +}; diff --git a/node-packages/wp-tooling/src/a11y/normalize.js b/node-packages/wp-tooling/src/a11y/normalize.js new file mode 100644 index 0000000..a6e250a --- /dev/null +++ b/node-packages/wp-tooling/src/a11y/normalize.js @@ -0,0 +1,235 @@ +/** + * Normalise pa11y-ci `--json` output into a stable shape the a11y skill and + * tests depend on. Pure — no child process, no I/O — so it is unit-testable + * against a fixture. + * + * pa11y-ci raw shape: + * { total, passes, errors, results: { "": [ { code, type, typeCode, + * message, context, selector, runner, runnerExtras } ] } } + * + * Normalised shape: + * { tool, standard, summary: { urls, violations, errors, warnings, notices, + * passedUrls, failedUrls }, results: [ { url, scanError, violations: [ { + * id, wcagCriterion, impact, runner, message, selector, context, + * domHints } ] } ] } + * + * A URL pa11y-ci could not load (e.g. `net::ERR_CONNECTION_REFUSED`) is a + * scan failure, not a violation: its entry carries `scanError` (the load + * error message) with an empty `violations`, and it counts towards + * `summary.failedUrls` rather than the violation totals. + */ + +'use strict'; + +/** pa11y `type` → normalised `impact`. */ +const IMPACT_BY_TYPE = { error: 'error', warning: 'warning', notice: 'notice' }; + +/** Sort order for violations within a URL (lower ranks sort first). */ +const IMPACT_RANK = { error: 0, warning: 1, notice: 2 }; + +/** + * Normalise a full pa11y-ci report. + * + * @param {Object} raw Parsed pa11y-ci `--json` output. + * @param {Object} [options] + * @param {string} [options.standard] Standard label for the report (default WCAG2AA). + * @return {Object} The normalised report. + */ +function normalizeA11y(raw, options = {}) { + const standard = options.standard || 'WCAG2AA'; + const resultsMap = + raw && raw.results && typeof raw.results === 'object' + ? raw.results + : {}; + + const results = []; + let violations = 0; + let errors = 0; + let warnings = 0; + let notices = 0; + let passedUrls = 0; + let failedUrls = 0; + + const urls = Object.keys(resultsMap).sort(); + for (const url of urls) { + const issues = Array.isArray(resultsMap[url]) ? resultsMap[url] : []; + const loadFailures = issues.filter(isScanError); + const normViolations = issues + .filter((issue) => !isScanError(issue)) + .map(normalizeIssue) + .sort(compareViolations); + + const scanError = + loadFailures.length > 0 ? loadFailures[0].message : null; + if (scanError) { + failedUrls++; + } else if (normViolations.length === 0) { + passedUrls++; + } + for (const v of normViolations) { + violations++; + if (v.impact === 'error') { + errors++; + } else if (v.impact === 'warning') { + warnings++; + } else if (v.impact === 'notice') { + notices++; + } + } + + results.push({ url, scanError, violations: normViolations }); + } + + return { + tool: 'pa11y-ci', + standard, + summary: { + urls: urls.length, + violations, + errors, + warnings, + notices, + passedUrls, + failedUrls, + }, + results, + }; +} + +/** + * Detect a URL-level load failure. pa11y-ci reports one as a bare + * `{ message }` entry with none of the fields a real issue carries + * (`code`, `type`, `runner`). + * + * @param {Object} issue Raw pa11y results entry. + * @return {boolean} True when the entry is a load failure, not an issue. + */ +function isScanError(issue) { + return ( + issue !== null && + typeof issue === 'object' && + issue.code === undefined && + issue.type === undefined && + issue.runner === undefined && + typeof issue.message === 'string' + ); +} + +/** + * Normalise a single pa11y issue. + * + * @param {Object} issue Raw pa11y issue. + * @return {Object} Normalised violation. + */ +function normalizeIssue(issue) { + const code = typeof issue.code === 'string' ? issue.code : ''; + const type = typeof issue.type === 'string' ? issue.type : 'error'; + const context = typeof issue.context === 'string' ? issue.context : ''; + const selector = typeof issue.selector === 'string' ? issue.selector : ''; + + return { + id: code, + wcagCriterion: parseWcagCriterion(code), + impact: IMPACT_BY_TYPE[type] || 'error', + runner: typeof issue.runner === 'string' ? issue.runner : null, + message: typeof issue.message === 'string' ? issue.message : '', + selector, + context, + domHints: extractDomHints(context, selector), + }; +} + +/** + * Extract the WCAG success-criterion number from an HTMLCS code such as + * `WCAG2AA.Principle1.Guideline1_1.1_1_1.H37` → `1.1.1`. Returns `null` for + * codes that carry no criterion (e.g. axe rule ids like `image-alt`). + * + * @param {string} code pa11y issue code. + * @return {string|null} Dotted criterion, or null. + */ +function parseWcagCriterion(code) { + const m = /(\d+)_(\d+)_(\d+)/.exec(code || ''); + if (!m) { + return null; + } + return `${m[1]}.${m[2]}.${m[3]}`; +} + +/** + * Regex-extract identifying tokens from the issue's context HTML + selector so + * the skill can grep the repo for the source that rendered the node without + * re-parsing HTML itself. + * + * @param {string} context Issue context HTML snippet. + * @param {string} selector Issue CSS selector. + * @return {{tagName: string|null, classList: string[], idAttr: string|null, + * attrs: Object}} Extracted hints. + */ +function extractDomHints(context, selector) { + const hints = { tagName: null, classList: [], idAttr: null, attrs: {} }; + + const tagFromContext = /^\s*<\s*([a-zA-Z][\w-]*)/.exec(context || ''); + if (tagFromContext) { + hints.tagName = tagFromContext[1].toLowerCase(); + } else { + const segments = (selector || '') + .split('>') + .map((s) => s.trim()) + .filter(Boolean); + const last = segments[segments.length - 1] || ''; + const tagFromSelector = /^([a-zA-Z][\w-]*)/.exec(last); + if (tagFromSelector) { + hints.tagName = tagFromSelector[1].toLowerCase(); + } + } + + const openTag = /<[^>]*>/.exec(context || ''); + if (openTag) { + const attrRe = + /([a-zA-Z_:][-\w:.]*)\s*=\s*"([^"]*)"|([a-zA-Z_:][-\w:.]*)\s*=\s*'([^']*)'/g; + let m; + while ((m = attrRe.exec(openTag[0])) !== null) { + const name = (m[1] || m[3]).toLowerCase(); + const value = m[2] !== undefined ? m[2] : m[4]; + if (name === 'class') { + hints.classList = value.split(/\s+/).filter(Boolean); + } else if (name === 'id') { + hints.idAttr = value; + } else { + hints.attrs[name] = value; + } + } + } + + return hints; +} + +/** + * Deterministic ordering: by impact, then id, then selector. + * + * @param {Object} a First violation. + * @param {Object} b Second violation. + * @return {number} Comparator result. + */ +function compareViolations(a, b) { + const ra = a.impact in IMPACT_RANK ? IMPACT_RANK[a.impact] : 9; + const rb = b.impact in IMPACT_RANK ? IMPACT_RANK[b.impact] : 9; + if (ra !== rb) { + return ra - rb; + } + if (a.id !== b.id) { + return a.id < b.id ? -1 : 1; + } + if (a.selector !== b.selector) { + return a.selector < b.selector ? -1 : 1; + } + return 0; +} + +module.exports = { + normalizeA11y, + normalizeIssue, + isScanError, + parseWcagCriterion, + extractDomHints, +}; diff --git a/node-packages/wp-tooling/src/a11y/resolve-bin.js b/node-packages/wp-tooling/src/a11y/resolve-bin.js new file mode 100644 index 0000000..9f20878 --- /dev/null +++ b/node-packages/wp-tooling/src/a11y/resolve-bin.js @@ -0,0 +1,105 @@ +/** + * Consumer binary resolution for the a11y runner. + * + * `@rtcamp/wp-tooling` has zero runtime dependencies, so `pa11y-ci` is never + * a dependency here — it lives in the CONSUMER project's own dev + * dependencies. These helpers locate that consumer-installed binary + * (a direct or workspace-hoisted `node_modules/.bin/`), falling back to + * `npx --no-install` so we never silently fetch it from the network. + */ + +'use strict'; + +const fs = require('fs'); +const path = require('path'); +const { execFileSync } = require('child_process'); + +const VERSION_PROBE_TIMEOUT_MS = 20000; + +/** + * Walk up from `cwd` looking for `node_modules/.bin/`. + * + * @param {string} binName Binary name (e.g. `pa11y-ci`). + * @param {string} cwd Directory to start the search from. + * @return {{command: string, source: 'local'|'hoisted'}|null} The resolved + * binary, or `null` when no installed copy is found. + */ +function findInNodeModules(binName, cwd) { + const start = path.resolve(cwd); + let dir = start; + for (;;) { + const candidate = path.join(dir, 'node_modules', '.bin', binName); + if (fs.existsSync(candidate)) { + return { + command: candidate, + source: dir === start ? 'local' : 'hoisted', + }; + } + const parent = path.dirname(dir); + if (parent === dir) { + return null; + } + dir = parent; + } +} + +/** + * Resolve how to invoke a consumer-installed binary. + * + * @param {string} binName Binary name. + * @param {Object} [options] + * @param {string} [options.cwd] Directory to resolve from. + * @return {{command: string, args: string[], source: 'local'|'hoisted'|'npx'}} + * Command + leading args + how it was resolved. + */ +function resolveBin(binName, options = {}) { + const cwd = options.cwd || process.cwd(); + const found = findInNodeModules(binName, cwd); + if (found) { + return { command: found.command, args: [], source: found.source }; + } + // `--no-install` keeps npx from fetching the package: if the consumer has + // not installed it, the probe below simply reports it unavailable and the + // caller surfaces the install hint. + return { command: 'npx', args: ['--no-install', binName], source: 'npx' }; +} + +/** + * Probe a binary's `--version` to confirm it is actually runnable. + * + * @param {string} binName Binary name. + * @param {Object} [options] + * @param {string} [options.cwd] Directory to run in. + * @return {{available: boolean, version: string|null, command: string, + * args: string[], source: string, error?: string}} Probe result. + */ +function detectBin(binName, options = {}) { + const cwd = options.cwd || process.cwd(); + const { command, args, source } = resolveBin(binName, { cwd }); + try { + const out = execFileSync(command, [...args, '--version'], { + cwd, + encoding: 'utf8', + stdio: ['ignore', 'pipe', 'pipe'], + timeout: VERSION_PROBE_TIMEOUT_MS, + }); + return { + available: true, + version: out.toString().trim(), + command, + args, + source, + }; + } catch (err) { + return { + available: false, + version: null, + command, + args, + source, + error: (err.stderr || err.message || '').toString().trim(), + }; + } +} + +module.exports = { resolveBin, detectBin, findInNodeModules }; diff --git a/node-packages/wp-tooling/src/a11y/run.js b/node-packages/wp-tooling/src/a11y/run.js new file mode 100644 index 0000000..ada31d5 --- /dev/null +++ b/node-packages/wp-tooling/src/a11y/run.js @@ -0,0 +1,345 @@ +/** + * a11y -- run pa11y-ci and emit normalized accessibility violations. + * + * Library API: + * const { runA11y } = require( '@rtcamp/wp-tooling/a11y' ); + * + * CLI: + * wp-tooling a11y [options] + * + * URLs and scan defaults come from the project's pa11y config + * (`.pa11yci.json`, or an explicit `--config` path) — the config is the + * single source of truth. Zero runtime dependencies: Node built-ins plus + * the project-installed `pa11y-ci` binary (`wp-tooling add setup/pa11y` + * scaffolds a config and the dev dependency for projects that need one). + */ + +'use strict'; + +const { execFileSync } = require('child_process'); +const { RunnerError } = require('./errors'); +const { resolveUrls } = require('./urls'); +const { detectBin } = require('./resolve-bin'); +const { normalizeA11y } = require('./normalize'); + +const BIN = 'pa11y-ci'; +const INSTALL_HINT = 'wp-tooling add setup/pa11y'; +const MAX_BUFFER = 64 * 1024 * 1024; + +/** + * Run pa11y-ci against the config's URLs and return the normalized report. + * + * @param {Object} [options] + * @param {string} [options.configPath] Path to the pa11y config (default `.pa11yci.json`). + * @param {string} [options.cwd] Project root. + * @return {Object} Normalized report (see normalize.js). + * @throws {RunnerError} EBINMISSING / EBINFAIL / EBADJSON / ENOURLS. + */ +function runA11y(options = {}) { + const cwd = options.cwd || process.cwd(); + const { configPath } = resolveUrls(options); + + const bin = detectBin(BIN, { cwd }); + if (!bin.available) { + throw new RunnerError( + 'EBINMISSING', + `${BIN} not found. Install it in the project (\`${INSTALL_HINT}\` sets it up).`, + { bin: BIN, install: INSTALL_HINT } + ); + } + + const raw = execPa11y(bin.command, buildArgs(bin, configPath), cwd); + return normalizeA11y(raw); +} + +/** + * Build the pa11y-ci argument vector. The config path is always passed + * explicitly so the run uses exactly the config the runner resolved. + * + * @param {Object} bin Resolved binary ({ command, args }). + * @param {string} configPath Config path to hand to pa11y-ci. + * @return {string[]} Argument vector. + */ +function buildArgs(bin, configPath) { + return [...bin.args, '--json', '--config', configPath]; +} + +/** + * Invoke pa11y-ci and return its parsed JSON report. + * + * pa11y-ci exits non-zero (typically 2) WHEN it finds violations -- that is a + * successful run for us, and the report is still on stdout. Only a run that + * yields no parseable report is a genuine failure. + * + * @param {string} command Binary command. + * @param {string[]} args Argument vector. + * @param {string} cwd Working directory. + * @return {Object} Parsed pa11y-ci report. + * @throws {RunnerError} EBINFAIL / EBADJSON. + */ +function execPa11y(command, args, cwd) { + let stdout; + try { + stdout = execFileSync(command, args, { + cwd, + encoding: 'utf8', + stdio: ['ignore', 'pipe', 'pipe'], + maxBuffer: MAX_BUFFER, + }); + } catch (err) { + const parsed = tryParse((err.stdout || '').toString()); + if (parsed && parsed.results) { + return parsed; + } + const detail = (err.stderr || err.message || '').toString().trim(); + throw new RunnerError('EBINFAIL', `${BIN} failed to run: ${detail}`, { + detail, + }); + } + + const parsed = tryParse(stdout); + if (!parsed || !parsed.results) { + throw new RunnerError( + 'EBADJSON', + `${BIN} produced output that could not be parsed as a JSON report.` + ); + } + return parsed; +} + +/** + * Parse pa11y-ci JSON, tolerating a leading non-JSON preamble line. + * + * @param {string} text Raw stdout. + * @return {Object|null} Parsed object, or null when not parseable. + */ +function tryParse(text) { + if (!text) { + return null; + } + const start = text.indexOf('{'); + if (start === -1) { + return null; + } + try { + return JSON.parse(text.slice(start)); + } catch { + return null; + } +} + +const VALID_OUTPUTS = ['text', 'json']; + +/** + * Consume the argv slot at `index` as a value for `flag`. + * + * @param {string[]} argv Argument vector. + * @param {number} index Position of the value. + * @param {string} flag Flag name, for the error message. + * @return {string} The validated value. + */ +function takeValue(argv, index, flag) { + const value = argv[index]; + if (value === undefined || value.startsWith('-')) { + throw new Error(`missing value for ${flag}`); + } + return value; +} + +/** + * Parse argv (without leading `node` and script path). + * + * @param {string[]} argv Argument vector. + * @return {Object} Parsed options. + */ +function parseArgs(argv) { + const opts = { output: 'text' }; + let i = 0; + while (i < argv.length) { + const arg = argv[i]; + switch (arg) { + case '--config': + opts.configPath = takeValue(argv, ++i, '--config'); + break; + case '--output': + opts.output = takeValue(argv, ++i, '--output'); + break; + case '--dry-run': + opts.dryRun = true; + break; + case '--help': + case '-h': + opts.help = true; + break; + default: + throw new Error(`unknown argument: ${arg}`); + } + i++; + } + return opts; +} + +/** + * Emit the normalized report in the requested output mode. + * + * @param {Object} report Normalized report. + * @param {string} mode 'text' | 'json'. + * @return {void} + */ +function emit(report, mode) { + if (mode === 'json') { + process.stdout.write(JSON.stringify(report) + '\n'); + return; + } + const s = report.summary; + const failed = s.failedUrls > 0 ? `, ${s.failedUrls} failed to load` : ''; + const lines = [ + `${report.tool} (${report.standard}): ${s.violations} violation(s) across ${s.urls} URL(s) — ${s.errors} error, ${s.warnings} warning, ${s.notices} notice; ${s.passedUrls} clean${failed}.`, + ]; + for (const r of report.results) { + lines.push(''); + if (r.scanError) { + lines.push(`${r.url} — scan failed`); + lines.push(` ${r.scanError}`); + continue; + } + lines.push(`${r.url} — ${r.violations.length} violation(s)`); + for (const v of r.violations) { + const crit = v.wcagCriterion ? ` [${v.wcagCriterion}]` : ''; + lines.push(` ${v.impact}${crit} ${v.selector}`); + lines.push(` ${v.message}`); + } + } + lines.push(''); + process.stdout.write(lines.join('\n')); +} + +/** + * Print CLI usage. + * + * @return {void} + */ +function printUsage() { + process.stdout.write( + [ + 'Usage: a11y [options]', + '', + ' Runs pa11y-ci against the URLs in the project pa11y config and', + ' prints normalized accessibility violations. Requires a pa11y config', + ' and the pa11y-ci dev dependency (`wp-tooling add setup/pa11y` sets', + ' both up for projects that have neither).', + '', + ' --config Path to the pa11y config (default: .pa11yci.json).', + ' --output Output format (default: text).', + ' --dry-run Print the resolved binary, URLs and command; run nothing.', + ' --help, -h Print this help.', + '', + 'Exit codes: 0 clean · 1 run failure or unreachable URL · 2 usage or binary missing · 3 violations found.', + '', + ].join('\n') + ); +} + +/** + * Print the dry-run plan (resolved binary, config, URLs, command) without + * running. + * + * @param {Object} opts Parsed options. + * @param {string} cwd Working directory. + * @return {number} Exit code. + */ +function runDryRun(opts, cwd) { + let urlInfo; + try { + urlInfo = resolveUrls({ configPath: opts.configPath, cwd }); + } catch (err) { + return handleError(err); + } + + const bin = detectBin(BIN, { cwd }); + const args = buildArgs(bin, urlInfo.configPath); + const binState = bin.available ? bin.version : 'NOT FOUND'; + + process.stdout.write( + [ + '[dry-run] a11y would run:', + ` binary: ${bin.command} (${bin.source}, ${binState})`, + ` config: ${urlInfo.configPath}`, + ` urls: ${urlInfo.urls.join(', ')}`, + ` command: ${bin.command} ${args.join(' ')}`, + '', + ].join('\n') + ); + return 0; +} + +/** + * Map a thrown error to an exit code and a stderr message. + * + * @param {Error} err The error. + * @return {number} Exit code: 2 (usage / binary missing), 1 (run failure). + */ +function handleError(err) { + process.stderr.write(`a11y: ${err.message}\n`); + if ( + err instanceof RunnerError && + (err.code === 'EBINMISSING' || err.code === 'ENOURLS') + ) { + return 2; + } + return 1; +} + +/** + * Run the CLI. Returns the intended exit code. + * + * @param {string[]} argv argv slice (without `node` and script path). + * @return {number} 0 clean · 1 run failure or unreachable URL · 2 usage/binary-missing · 3 violations found. + */ +function runCli(argv) { + let opts; + try { + opts = parseArgs(argv); + } catch (err) { + process.stderr.write(`a11y: ${err.message}\n`); + return 2; + } + + if (opts.help) { + printUsage(); + return 0; + } + + if (!VALID_OUTPUTS.includes(opts.output)) { + process.stderr.write( + `a11y: invalid --output "${opts.output}" (expected one of: ${VALID_OUTPUTS.join( + ', ' + )})\n` + ); + return 2; + } + + const cwd = process.cwd(); + + if (opts.dryRun) { + return runDryRun(opts, cwd); + } + + let report; + try { + report = runA11y({ configPath: opts.configPath, cwd }); + } catch (err) { + return handleError(err); + } + + emit(report, opts.output); + if (report.summary.failedUrls > 0) { + process.stderr.write( + `a11y: ${report.summary.failedUrls} URL(s) failed to load — treating as a run failure.\n` + ); + return 1; + } + return report.summary.violations > 0 ? 3 : 0; +} + +module.exports = { runA11y, runCli }; diff --git a/node-packages/wp-tooling/src/a11y/urls.js b/node-packages/wp-tooling/src/a11y/urls.js new file mode 100644 index 0000000..befc68c --- /dev/null +++ b/node-packages/wp-tooling/src/a11y/urls.js @@ -0,0 +1,95 @@ +/** + * Resolve the URL list the a11y runner should scan. + * + * URLs come from the project's pa11y config — `.pa11yci.json` by default, + * or an explicit `--config` path. Read-only — never mutates the config. + */ + +'use strict'; + +const fs = require('fs'); +const path = require('path'); +const { RunnerError } = require('./errors'); + +/** Default pa11y-ci config filename, relative to the project root. */ +const DEFAULT_CONFIG = '.pa11yci.json'; + +/** + * Resolve the URLs to scan from the pa11y config. + * + * @param {Object} [options] + * @param {string} [options.configPath] Path to the pa11y config (default `.pa11yci.json`). + * @param {string} [options.cwd] Project root. + * @return {{urls: string[], configPath: string}} Resolved URLs and the config path read. + * @throws {RunnerError} `ENOURLS` when no URLs are available; `EBADJSON` when the config is malformed. + */ +function resolveUrls(options = {}) { + const cwd = options.cwd || process.cwd(); + const configPath = options.configPath + ? path.resolve(cwd, options.configPath) + : path.join(cwd, DEFAULT_CONFIG); + + let raw; + try { + raw = fs.readFileSync(configPath, 'utf8'); + } catch (err) { + throw new RunnerError( + 'ENOURLS', + `no URLs to scan: could not read ${configPath} (${( + err.message || '' + ).toString()}). Add a "${DEFAULT_CONFIG}" with a "urls" array — \`wp-tooling add setup/pa11y\` can scaffold one.`, + { configPath } + ); + } + + let parsed; + try { + parsed = JSON.parse(raw); + } catch (err) { + throw new RunnerError( + 'EBADJSON', + `invalid JSON in ${configPath}: ${err.message}`, + { configPath } + ); + } + + const urls = extractUrls(parsed); + if (urls.length === 0) { + throw new RunnerError( + 'ENOURLS', + `no "urls" entries found in ${configPath}. Add the URLs to scan there.`, + { configPath } + ); + } + + return { urls, configPath }; +} + +/** + * Pull the URL strings out of a parsed pa11y config. pa11y-ci accepts both + * bare strings and `{ url, ... }` objects in `urls[]`. + * + * @param {Object} config Parsed pa11y config. + * @return {string[]} URL strings. + */ +function extractUrls(config) { + if (!config || !Array.isArray(config.urls)) { + return []; + } + const out = []; + for (const entry of config.urls) { + if (typeof entry === 'string' && entry.length > 0) { + out.push(entry); + } else if ( + entry && + typeof entry === 'object' && + typeof entry.url === 'string' && + entry.url.length > 0 + ) { + out.push(entry.url); + } + } + return out; +} + +module.exports = { resolveUrls, extractUrls, DEFAULT_CONFIG }; diff --git a/node-packages/wp-tooling/src/cli/commands/a11y.js b/node-packages/wp-tooling/src/cli/commands/a11y.js new file mode 100644 index 0000000..893c97b --- /dev/null +++ b/node-packages/wp-tooling/src/cli/commands/a11y.js @@ -0,0 +1,16 @@ +/** + * a11y subcommand registration. + * + * The dispatcher (`src/cli/index.js`) auto-discovers every `*.js` file in + * this directory. Each module must export `{ name, summary, run }`. + * `run` is required lazily so cold-start cost stays close to a single + * subcommand's footprint. + */ + +'use strict'; + +module.exports = { + name: 'a11y', + summary: 'Run pa11y-ci and emit normalized accessibility violations', + run: (argv) => require('../../a11y/run').runCli(argv), +}; diff --git a/node-packages/wp-tooling/src/init/index.js b/node-packages/wp-tooling/src/init/index.js index b952384..7b4bdec 100644 --- a/node-packages/wp-tooling/src/init/index.js +++ b/node-packages/wp-tooling/src/init/index.js @@ -347,7 +347,9 @@ const setupSteps = (config, root, flags) => { skip: (c) => c.cancelled || (!(config.features || []).length && - !(config.examples && (config.examples.groups || []).length)), + !( + config.examples && (config.examples.groups || []).length + )), async run(c) { const features = config.features || []; const groups = @@ -387,18 +389,18 @@ const setupSteps = (config, root, flags) => { ]; const order = []; const byCat = new Map(); - for (const cap of caps) { - if (!byCat.has(cap.category)) { - byCat.set(cap.category, []); - order.push(cap.category); + for (const entry of caps) { + if (!byCat.has(entry.category)) { + byCat.set(entry.category, []); + order.push(entry.category); } - byCat.get(cap.category).push(cap); + byCat.get(entry.category).push(entry); } const treeGroups = order.map((category) => ({ label: category, - items: byCat.get(category).map((cap) => ({ - label: cap.label, - checked: cap.checked, + items: byCat.get(category).map((entry) => ({ + label: entry.label, + checked: entry.checked, })), })); const checked = new Set( diff --git a/node-packages/wp-tooling/tests/a11y/cli.test.js b/node-packages/wp-tooling/tests/a11y/cli.test.js new file mode 100644 index 0000000..a821f50 --- /dev/null +++ b/node-packages/wp-tooling/tests/a11y/cli.test.js @@ -0,0 +1,233 @@ +'use strict'; + +jest.mock('child_process'); + +const path = require('path'); +const { execFileSync } = require('child_process'); +const { runCli } = require('../../src/a11y/run'); + +const FIXTURE_CONFIG = path.join(__dirname, 'fixtures', '.pa11yci.json'); + +const REPORT = { + total: 1, + passes: 0, + errors: 1, + results: { + 'http://localhost:8888/': [ + { + code: 'WCAG2AA.Principle1.Guideline1_1.1_1_1.H37', + type: 'error', + typeCode: 1, + message: 'Img element missing an alt attribute.', + context: '', + selector: 'html > body > img', + runner: 'htmlcs', + }, + ], + }, +}; + +const CLEAN_REPORT = { + total: 0, + passes: 1, + errors: 0, + results: { 'http://localhost:8888/': [] }, +}; + +const FAILED_REPORT = { + total: 1, + passes: 0, + errors: 0, + results: { + 'http://localhost:8888/': [ + { + message: + 'net::ERR_CONNECTION_REFUSED at http://localhost:8888/', + }, + ], + }, +}; + +/** + * Drive the mocked pa11y-ci binary. + * + * @param {Object} [o] + * @param {boolean} [o.available=true] Whether the --version probe succeeds. + * @param {*} [o.report=REPORT] Report returned by the run. + * @param {boolean} [o.runThrows] Whether the run throws (violations / failure). + * @param {string} [o.runStdout] stdout attached to a thrown run error. + * @param {string} [o.runStderr] stderr attached to a thrown run error. + * @param {string} [o.runReturn] Raw stdout returned by a non-throwing run. + */ +function mockBin(o = {}) { + const available = o.available !== false; + execFileSync.mockImplementation((cmd, args) => { + if (args.includes('--version')) { + if (!available) { + const err = new Error('command not found'); + err.stderr = 'command not found'; + throw err; + } + return '3.1.0\n'; + } + if (o.runThrows) { + const err = new Error('exited non-zero'); + err.status = 2; + err.stdout = o.runStdout !== undefined ? o.runStdout : ''; + err.stderr = o.runStderr !== undefined ? o.runStderr : ''; + throw err; + } + if (o.runReturn !== undefined) { + return o.runReturn; + } + return JSON.stringify(o.report !== undefined ? o.report : REPORT); + }); +} + +describe('a11y runCli', () => { + let stdout; + let stderr; + let outSpy; + let errSpy; + + beforeEach(() => { + stdout = []; + stderr = []; + outSpy = jest.spyOn(process.stdout, 'write').mockImplementation((c) => { + stdout.push(c.toString()); + return true; + }); + errSpy = jest.spyOn(process.stderr, 'write').mockImplementation((c) => { + stderr.push(c.toString()); + return true; + }); + }); + + afterEach(() => { + outSpy.mockRestore(); + errSpy.mockRestore(); + }); + + test('--help prints usage and exits 0', () => { + expect(runCli(['--help'])).toBe(0); + expect(stdout.join('')).toMatch(/Usage: a11y/); + }); + + test('unknown flag exits 2', () => { + expect(runCli(['--bogus'])).toBe(2); + expect(stderr.join('')).toMatch(/unknown argument/); + }); + + test('invalid --output exits 2', () => { + expect(runCli(['--output', 'xml', '--config', FIXTURE_CONFIG])).toBe(2); + expect(stderr.join('')).toMatch(/invalid --output/); + }); + + test('--config with no value exits 2', () => { + expect(runCli(['--config'])).toBe(2); + expect(stderr.join('')).toMatch(/missing value for --config/); + }); + + test('an unreadable config exits 2 (ENOURLS)', () => { + expect( + runCli(['--config', path.join(__dirname, 'nope.pa11yci.json')]) + ).toBe(2); + expect(stderr.join('')).toMatch(/no URLs to scan/); + }); + + test('missing pa11y-ci exits 2 with the install hint', () => { + mockBin({ available: false }); + expect(runCli(['--config', FIXTURE_CONFIG])).toBe(2); + expect(stderr.join('')).toMatch(/pa11y-ci not found/); + expect(stderr.join('')).toMatch(/wp-tooling add setup\/pa11y/); + }); + + test('the resolved config path is handed to pa11y-ci via --config', () => { + let runArgs; + execFileSync.mockImplementation((cmd, args) => { + if (args.includes('--version')) { + return '3.1.0\n'; + } + runArgs = [...args]; + return JSON.stringify(CLEAN_REPORT); + }); + expect(runCli(['--config', FIXTURE_CONFIG])).toBe(0); + // Leading args depend on how the binary resolved (direct vs npx); + // the runner's own contribution is the tail. + expect(runArgs.slice(-3)).toEqual([ + '--json', + '--config', + FIXTURE_CONFIG, + ]); + }); + + test('violations found: exits 3 with a parseable JSON report', () => { + mockBin({ runThrows: true, runStdout: JSON.stringify(REPORT) }); + const code = runCli(['--config', FIXTURE_CONFIG, '--output', 'json']); + expect(code).toBe(3); + const parsed = JSON.parse(stdout.join('')); + expect(parsed.summary.violations).toBe(1); + expect(parsed.results[0].violations[0].wcagCriterion).toBe('1.1.1'); + }); + + test('clean run exits 0', () => { + mockBin({ report: CLEAN_REPORT }); + const code = runCli(['--config', FIXTURE_CONFIG, '--output', 'json']); + expect(code).toBe(0); + expect(JSON.parse(stdout.join('')).summary.violations).toBe(0); + }); + + test('text mode prints a human summary', () => { + mockBin({ report: CLEAN_REPORT }); + expect(runCli(['--config', FIXTURE_CONFIG])).toBe(0); + expect(stdout.join('')).toMatch(/pa11y-ci \(WCAG2AA\)/); + }); + + test('unparseable output exits 1 (EBADJSON)', () => { + mockBin({ runReturn: 'not json at all' }); + expect(runCli(['--config', FIXTURE_CONFIG])).toBe(1); + expect(stderr.join('')).toMatch(/could not be parsed/); + }); + + test('a genuine run failure exits 1 (EBINFAIL)', () => { + mockBin({ + runThrows: true, + runStdout: '', + runStderr: 'Chrome crashed', + }); + expect(runCli(['--config', FIXTURE_CONFIG])).toBe(1); + expect(stderr.join('')).toMatch(/failed to run/); + }); + + test('an unreachable URL is a run failure (exit 1), not a violation', () => { + mockBin({ report: FAILED_REPORT }); + const code = runCli(['--config', FIXTURE_CONFIG, '--output', 'json']); + expect(code).toBe(1); + const parsed = JSON.parse(stdout.join('')); + expect(parsed.summary.failedUrls).toBe(1); + expect(parsed.summary.violations).toBe(0); + expect(stderr.join('')).toMatch(/failed to load/); + }); + + test('a scan failure is reported in text mode', () => { + mockBin({ report: FAILED_REPORT }); + expect(runCli(['--config', FIXTURE_CONFIG])).toBe(1); + const out = stdout.join(''); + expect(out).toMatch(/scan failed/); + expect(out).toMatch(/1 failed to load/); + }); + + test('--dry-run prints the plan and runs pa11y-ci not at all', () => { + mockBin(); + const code = runCli(['--dry-run', '--config', FIXTURE_CONFIG]); + expect(code).toBe(0); + const out = stdout.join(''); + expect(out).toMatch(/\[dry-run\] a11y would run:/); + expect(out).toMatch(/http:\/\/localhost:8888\//); + expect(out).toContain(FIXTURE_CONFIG); + expect(out).toMatch(/pa11y-ci/); + // Only the --version probe ran; pa11y-ci itself was never invoked. + expect(execFileSync.mock.calls).toHaveLength(1); + expect(execFileSync.mock.calls[0][1]).toContain('--version'); + }); +}); diff --git a/node-packages/wp-tooling/tests/a11y/fixtures/.pa11yci.json b/node-packages/wp-tooling/tests/a11y/fixtures/.pa11yci.json new file mode 100644 index 0000000..882007b --- /dev/null +++ b/node-packages/wp-tooling/tests/a11y/fixtures/.pa11yci.json @@ -0,0 +1,10 @@ +{ + "defaults": { + "standard": "WCAG2AA", + "timeout": 60000 + }, + "urls": [ + "http://localhost:8888/", + { "url": "http://localhost:8888/about", "timeout": 30000 } + ] +} diff --git a/node-packages/wp-tooling/tests/a11y/fixtures/empty-urls.pa11yci.json b/node-packages/wp-tooling/tests/a11y/fixtures/empty-urls.pa11yci.json new file mode 100644 index 0000000..592ade4 --- /dev/null +++ b/node-packages/wp-tooling/tests/a11y/fixtures/empty-urls.pa11yci.json @@ -0,0 +1,4 @@ +{ + "defaults": { "standard": "WCAG2AA" }, + "urls": [] +} diff --git a/node-packages/wp-tooling/tests/a11y/fixtures/pa11y-ci.json b/node-packages/wp-tooling/tests/a11y/fixtures/pa11y-ci.json new file mode 100644 index 0000000..c80c5f4 --- /dev/null +++ b/node-packages/wp-tooling/tests/a11y/fixtures/pa11y-ci.json @@ -0,0 +1,46 @@ +{ + "total": 4, + "passes": 1, + "errors": 2, + "results": { + "http://localhost:8888/about": [], + "http://localhost:8888/": [ + { + "code": "WCAG2AA.Principle1.Guideline1_1.1_1_1.H37", + "type": "error", + "typeCode": 1, + "message": "Img element missing an alt attribute.", + "context": "", + "selector": "html > body > main > img:nth-child(2)", + "runner": "htmlcs" + }, + { + "code": "color-contrast", + "type": "error", + "typeCode": 1, + "message": "Elements must have sufficient color contrast.", + "context": "Buy", + "selector": "#cta", + "runner": "axe" + }, + { + "code": "WCAG2AA.Principle1.Guideline1_3.1_3_1.H42.2", + "type": "warning", + "typeCode": 2, + "message": "Heading markup should be used if this content is intended as a heading.", + "context": "Section", + "selector": "html > body > b", + "runner": "htmlcs" + }, + { + "code": "WCAG2AA.Principle4.Guideline4_1.4_1_2.H91.A.NoContent", + "type": "notice", + "typeCode": 3, + "message": "Anchor element found with no link content.", + "context": "", + "selector": "html > body > a", + "runner": "htmlcs" + } + ] + } +} diff --git a/node-packages/wp-tooling/tests/a11y/normalize.test.js b/node-packages/wp-tooling/tests/a11y/normalize.test.js new file mode 100644 index 0000000..9a74343 --- /dev/null +++ b/node-packages/wp-tooling/tests/a11y/normalize.test.js @@ -0,0 +1,172 @@ +'use strict'; + +const fs = require('fs'); +const path = require('path'); + +const { + normalizeA11y, + parseWcagCriterion, + extractDomHints, +} = require('../../src/a11y/normalize'); + +const RAW = JSON.parse( + fs.readFileSync(path.join(__dirname, 'fixtures', 'pa11y-ci.json'), 'utf8') +); + +describe('normalizeA11y', () => { + const report = normalizeA11y(RAW); + + test('tags the tool and default standard', () => { + expect(report.tool).toBe('pa11y-ci'); + expect(report.standard).toBe('WCAG2AA'); + }); + + test('summary counts violations by impact across URLs', () => { + expect(report.summary).toEqual({ + urls: 2, + violations: 4, + errors: 2, + warnings: 1, + notices: 1, + passedUrls: 1, + failedUrls: 0, + }); + }); + + test('results are sorted by URL', () => { + expect(report.results.map((r) => r.url)).toEqual([ + 'http://localhost:8888/', + 'http://localhost:8888/about', + ]); + }); + + test('a URL with no issues is reported as clean', () => { + const about = report.results.find( + (r) => r.url === 'http://localhost:8888/about' + ); + expect(about.violations).toEqual([]); + }); + + test('violations sort by impact, then id, then selector', () => { + const home = report.results.find( + (r) => r.url === 'http://localhost:8888/' + ); + expect(home.violations.map((v) => v.id)).toEqual([ + 'WCAG2AA.Principle1.Guideline1_1.1_1_1.H37', + 'color-contrast', + 'WCAG2AA.Principle1.Guideline1_3.1_3_1.H42.2', + 'WCAG2AA.Principle4.Guideline4_1.4_1_2.H91.A.NoContent', + ]); + expect(home.violations.map((v) => v.impact)).toEqual([ + 'error', + 'error', + 'warning', + 'notice', + ]); + }); + + test('maps HTMLCS codes to a WCAG criterion and axe codes to null', () => { + const home = report.results.find( + (r) => r.url === 'http://localhost:8888/' + ); + const byId = Object.fromEntries(home.violations.map((v) => [v.id, v])); + expect( + byId['WCAG2AA.Principle1.Guideline1_1.1_1_1.H37'].wcagCriterion + ).toBe('1.1.1'); + expect(byId['color-contrast'].wcagCriterion).toBeNull(); + expect( + byId['WCAG2AA.Principle1.Guideline1_3.1_3_1.H42.2'].wcagCriterion + ).toBe('1.3.1'); + expect( + byId['WCAG2AA.Principle4.Guideline4_1.4_1_2.H91.A.NoContent'] + .wcagCriterion + ).toBe('4.1.2'); + }); + + test('extracts DOM hints from the context for the grep-to-source step', () => { + const home = report.results.find( + (r) => r.url === 'http://localhost:8888/' + ); + const img = home.violations.find((v) => v.id.endsWith('H37')); + expect(img.domHints.tagName).toBe('img'); + expect(img.domHints.classList).toEqual(['card__media', 'hero']); + expect(img.domHints.idAttr).toBeNull(); + expect(img.domHints.attrs.src).toBe('/hero.jpg'); + + const cta = home.violations.find((v) => v.id === 'color-contrast'); + expect(cta.domHints.tagName).toBe('a'); + expect(cta.domHints.idAttr).toBe('cta'); + expect(cta.domHints.classList).toEqual(['wp-block-acme-cta']); + expect(cta.domHints.attrs.href).toBe('/buy'); + }); + + test('honours a custom standard label', () => { + expect(normalizeA11y(RAW, { standard: 'WCAG2AAA' }).standard).toBe( + 'WCAG2AAA' + ); + }); + + test('tolerates missing or malformed results', () => { + expect(normalizeA11y({}).summary.urls).toBe(0); + expect(normalizeA11y(null).results).toEqual([]); + expect(normalizeA11y({ results: { '/x': 'nope' } }).results).toEqual([ + { url: '/x', scanError: null, violations: [] }, + ]); + }); + + test('a load failure becomes scanError, not a violation', () => { + const rep = normalizeA11y({ + results: { + 'http://localhost:8888/': [ + { + message: + 'net::ERR_CONNECTION_REFUSED at http://localhost:8888/', + }, + ], + }, + }); + expect(rep.summary).toEqual({ + urls: 1, + violations: 0, + errors: 0, + warnings: 0, + notices: 0, + passedUrls: 0, + failedUrls: 1, + }); + expect(rep.results[0].scanError).toMatch(/ERR_CONNECTION_REFUSED/); + expect(rep.results[0].violations).toEqual([]); + }); +}); + +describe('parseWcagCriterion', () => { + test('pulls the dotted criterion from an HTMLCS code', () => { + expect( + parseWcagCriterion('WCAG2AA.Principle2.Guideline2_4.2_4_4.H77') + ).toBe('2.4.4'); + }); + + test('returns null when there is no criterion', () => { + expect(parseWcagCriterion('image-alt')).toBeNull(); + expect(parseWcagCriterion('')).toBeNull(); + expect(parseWcagCriterion(undefined)).toBeNull(); + }); +}); + +describe('extractDomHints', () => { + test('falls back to the selector tag when context has no opening tag', () => { + const hints = extractDomHints('', 'html > body > main > button.cta'); + expect(hints.tagName).toBe('button'); + expect(hints.classList).toEqual([]); + }); + + test('captures aria attributes', () => { + const hints = extractDomHints( + '
', + 'div' + ); + expect(hints.tagName).toBe('div'); + expect(hints.attrs.role).toBe('button'); + expect(hints.attrs['aria-label']).toBe('Close'); + }); +}); diff --git a/node-packages/wp-tooling/tests/a11y/resolve-bin.test.js b/node-packages/wp-tooling/tests/a11y/resolve-bin.test.js new file mode 100644 index 0000000..baa8647 --- /dev/null +++ b/node-packages/wp-tooling/tests/a11y/resolve-bin.test.js @@ -0,0 +1,113 @@ +'use strict'; + +jest.mock('child_process'); + +const fs = require('fs'); +const os = require('os'); +const path = require('path'); +const { execFileSync } = require('child_process'); + +const { + resolveBin, + detectBin, + findInNodeModules, +} = require('../../src/a11y/resolve-bin'); + +const BIN = 'pa11y-ci'; + +function tmpTree() { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'a11y-bin-')); + return root; +} + +function makeBin(dir, binName) { + const binDir = path.join(dir, 'node_modules', '.bin'); + fs.mkdirSync(binDir, { recursive: true }); + const p = path.join(binDir, binName); + fs.writeFileSync(p, '#!/bin/sh\n'); + return p; +} + +describe('findInNodeModules', () => { + let root; + + afterEach(() => { + if (root) { + fs.rmSync(root, { recursive: true, force: true }); + root = null; + } + }); + + test('finds a directly installed binary as local', () => { + root = tmpTree(); + const p = makeBin(root, BIN); + const found = findInNodeModules(BIN, root); + expect(found).toEqual({ command: p, source: 'local' }); + }); + + test('finds a hoisted binary in an ancestor as hoisted', () => { + root = tmpTree(); + const p = makeBin(root, BIN); + const child = path.join(root, 'packages', 'app'); + fs.mkdirSync(child, { recursive: true }); + const found = findInNodeModules(BIN, child); + expect(found).toEqual({ command: p, source: 'hoisted' }); + }); + + test('returns null when no installed copy exists', () => { + root = tmpTree(); + expect( + findInNodeModules('definitely-not-installed-xyz', root) + ).toBeNull(); + }); +}); + +describe('resolveBin', () => { + test('falls back to npx --no-install when nothing is installed', () => { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'a11y-bin-')); + try { + const r = resolveBin('definitely-not-installed-xyz', { cwd: root }); + expect(r).toEqual({ + command: 'npx', + args: ['--no-install', 'definitely-not-installed-xyz'], + source: 'npx', + }); + } finally { + fs.rmSync(root, { recursive: true, force: true }); + } + }); +}); + +describe('detectBin', () => { + test('reports available with a trimmed version when the probe succeeds', () => { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'a11y-bin-')); + try { + execFileSync.mockReturnValue('3.1.0\n'); + const r = detectBin('definitely-not-installed-xyz', { cwd: root }); + expect(r.available).toBe(true); + expect(r.version).toBe('3.1.0'); + expect(r.source).toBe('npx'); + const call = execFileSync.mock.calls[0]; + expect(call[1]).toContain('--version'); + } finally { + fs.rmSync(root, { recursive: true, force: true }); + } + }); + + test('reports unavailable when the probe throws', () => { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'a11y-bin-')); + try { + execFileSync.mockImplementation(() => { + const err = new Error('not found'); + err.stderr = 'command not found'; + throw err; + }); + const r = detectBin('definitely-not-installed-xyz', { cwd: root }); + expect(r.available).toBe(false); + expect(r.version).toBeNull(); + expect(r.error).toMatch(/command not found/); + } finally { + fs.rmSync(root, { recursive: true, force: true }); + } + }); +}); diff --git a/node-packages/wp-tooling/tests/a11y/urls.test.js b/node-packages/wp-tooling/tests/a11y/urls.test.js new file mode 100644 index 0000000..a589da7 --- /dev/null +++ b/node-packages/wp-tooling/tests/a11y/urls.test.js @@ -0,0 +1,76 @@ +'use strict'; + +const path = require('path'); + +const { resolveUrls, extractUrls } = require('../../src/a11y/urls'); +const { RunnerError } = require('../../src/a11y/errors'); + +const FIXTURES = path.join(__dirname, 'fixtures'); + +/** + * Run `fn` and return whatever it throws (or null). Keeps assertions out of a + * catch block, which `jest/no-conditional-expect` forbids. + * + * @param {Function} fn Function expected to throw. + * @return {Error|null} The thrown error, or null if it did not throw. + */ +function grab(fn) { + try { + fn(); + } catch (err) { + return err; + } + return null; +} + +describe('resolveUrls', () => { + test('reads string and { url } entries from the default config', () => { + const r = resolveUrls({ cwd: FIXTURES }); + expect(r.urls).toEqual([ + 'http://localhost:8888/', + 'http://localhost:8888/about', + ]); + expect(r.configPath).toBe(path.join(FIXTURES, '.pa11yci.json')); + }); + + test('a custom --config path is resolved relative to cwd (empty urls -> ENOURLS)', () => { + const err = grab(() => + resolveUrls({ + cwd: FIXTURES, + configPath: 'empty-urls.pa11yci.json', + }) + ); + expect(err).toBeInstanceOf(RunnerError); + expect(err.code).toBe('ENOURLS'); + }); + + test('a missing config throws ENOURLS with the install hint', () => { + const err = grab(() => + resolveUrls({ cwd: FIXTURES, configPath: 'does-not-exist.json' }) + ); + expect(err.code).toBe('ENOURLS'); + expect(err.message).toMatch(/wp-tooling add setup\/pa11y/); + }); +}); + +describe('extractUrls', () => { + test('handles strings, objects, and skips junk', () => { + expect( + extractUrls({ + urls: [ + 'http://a/', + { url: 'http://b/' }, + { nope: true }, + '', + 42, + ], + }) + ).toEqual(['http://a/', 'http://b/']); + }); + + test('returns [] when urls is absent or not an array', () => { + expect(extractUrls({})).toEqual([]); + expect(extractUrls({ urls: 'x' })).toEqual([]); + expect(extractUrls(null)).toEqual([]); + }); +}); diff --git a/node-packages/wp-tooling/tests/scaffolds/bundled-manifests.test.js b/node-packages/wp-tooling/tests/scaffolds/bundled-manifests.test.js index 2c44225..3cb0a02 100644 --- a/node-packages/wp-tooling/tests/scaffolds/bundled-manifests.test.js +++ b/node-packages/wp-tooling/tests/scaffolds/bundled-manifests.test.js @@ -88,3 +88,49 @@ describe('wiring targetFile normalisation', () => { expect(target).not.toContain('..'); }); }); + +describe('setup/pa11y rendered config', () => { + it('renders valid JSON with default page paths (extra_page omitted)', async () => { + const r = registry; + const target = makeTmpDir(); + await r.execute( + 'setup/pa11y', + { base_url: 'http://localhost:8888' }, + { cwd: target } + ); + const config = JSON.parse( + fs.readFileSync(path.join(target, '.pa11yci.json'), 'utf8') + ); + expect(config.defaults.standard).toBe('WCAG2AA'); + expect(config.defaults.runners).toEqual(['axe', 'htmlcs']); + expect(config.urls).toEqual([ + 'http://localhost:8888/', + 'http://localhost:8888/?p=1', + 'http://localhost:8888/?s=hello', + ]); + }); + + it('renders custom page paths and appends extra_page when given', async () => { + const r = registry; + const target = makeTmpDir(); + await r.execute( + 'setup/pa11y', + { + base_url: 'http://localhost:8765', + sample_page: '/hello-world/', + search_page: '/?s=wordpress', + extra_page: '/about/', + }, + { cwd: target } + ); + const config = JSON.parse( + fs.readFileSync(path.join(target, '.pa11yci.json'), 'utf8') + ); + expect(config.urls).toEqual([ + 'http://localhost:8765/', + 'http://localhost:8765/hello-world/', + 'http://localhost:8765/?s=wordpress', + 'http://localhost:8765/about/', + ]); + }); +}); diff --git a/node-packages/wp-tooling/tests/ui/selects.test.js b/node-packages/wp-tooling/tests/ui/selects.test.js index 54edfb5..7078216 100644 --- a/node-packages/wp-tooling/tests/ui/selects.test.js +++ b/node-packages/wp-tooling/tests/ui/selects.test.js @@ -205,7 +205,10 @@ describe('checkboxTree (non-TTY)', () => { { label: 'Taxonomies', checked: true }, ], }, - { label: 'Dev', items: [{ label: 'Tailwind', checked: false }] }, + { + label: 'Dev', + items: [{ label: 'Tailwind', checked: false }], + }, ], }); @@ -227,7 +230,10 @@ describe('checkboxTree (non-TTY)', () => { { label: 'Taxonomies', checked: true }, ], }, - { label: 'Dev', items: [{ label: 'Tailwind', checked: false }] }, + { + label: 'Dev', + items: [{ label: 'Tailwind', checked: false }], + }, ], });