Skip to content
Merged

Dev #2860

Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
---
name: validate-cypress-selectors
description: Extract and audit Cypress selectors against Unity Grant Manager application markup, and detect when a branch's app-code changes have broken a selector that worked on main. Use when changing Razor/C#/JS markup, Cypress specs, page objects, shared Cypress commands, element IDs, or CSS selectors — or when investigating why a Cypress test started failing after unrelated app changes.
---

# Validate Cypress Selectors

Maintain the selector contract between `applications/Unity.AutoUI` and `applications/Unity.GrantManager`. Run everything from `applications/Unity.AutoUI`.

## Two modes

**Full snapshot** — classify every selector in the repo right now:
```bash
npm run selectors:report
```
Writes `cypress/selectors/registry.json`. Use this to get a baseline understanding, or before/after a change to eyeball the raw counts.

**Baseline diff (the useful one for "did this branch break something")**:
```bash
npm run selectors:diff # report only
npm run selectors:fix # + dry-run patch preview for fixable regressions
npm run selectors:apply # + write the unambiguous fixes to disk
```
Compares the current working tree against the **committed baseline** — `cypress/selectors/registry.json` as it exists at `origin/main` (override with `--base <ref>`, e.g. `npm run selectors:diff -- --base origin/develop`). It does **not** re-parse git diffs to guess what changed; it re-runs the full scanner on the current tree and compares the resulting per-selector `status` against the baseline's. Anything that got *worse* (`matched` → `missing`, `matched` → `unverified`, etc.) is a regression this branch likely introduced.

## Interpreting a full-scan entry

- `matched` — every identifying token (`id`, `data-cy`, `data-testid`) has static evidence in application source.
- `missing` — at least one identifying token has no static evidence. Investigate.
- `unverified` — a structural/class-based CSS selector that can't be proven reliably by static scanning (no identifying token to search for).
- `exempt` — matches a configured ownership rule (`selector-contract.config.json`): external identity-provider markup, framework-generated selectors (Bootstrap-select, Select2, DataTables, SweetAlert2), or Form.io/CHEFS dynamic fields. Outside the static contract by design.
- `syntaxKind` — `css` or `xpath`. Everything today is `css`; `xpath` exists to catch it early if `cy.xpath(...)`-style selectors are ever introduced (they're harder to keep in sync with markup and generally discouraged).

`matched` only proves the token exists *somewhere* in source — it says nothing about visibility, permission-gating, or runtime reachability. It's a fast static sanity check, not a substitute for actually running the Cypress spec.

## Diff-mode regression auto-fix — how it decides what's safe to touch

Only `matched` (baseline) → `missing` (current) regressions are eligible for auto-fix — that's the one transition backed by concrete proof the selector worked before. Pre-existing `missing`/`unverified` entries are left alone; they aren't this branch's fault and guessing at them is out of scope.

For each eligible regression:
1. Find where the token was matched at baseline (`applicationMatches` in the baseline entry) and the exact line via `git show <base>:<file>`.
2. In the *current* version of that file, score every line for similarity to the old line (word-overlap, with a small proximity bonus toward the original line number — an attribute rename overwhelmingly stays in place rather than the element relocating).
3. **Exactly one clear winner** → propose restoring the missing attribute onto it (inserted right after the tag name, alongside whatever else is there — never replaces existing attributes).
4. **Zero or ambiguous candidates** (e.g. several structurally-identical sibling elements) → reported as "NEEDS REVIEW" with the old-baseline context shown. Never guessed.

`--apply` only ever writes the unambiguous fixes from step 3. It never touches an ambiguous case, never invents a token value it doesn't have baseline evidence for, and never commits or pushes — it stops at "working tree modified, go review and commit like any other change." Treat an applied fix the same as any other diff: read it, and actually run the relevant Cypress spec before trusting it, since restoring the identifying token proves the selector *resolves* again, not that the element behaves correctly.

## Workflow

1. Before editing UI selectors or markup: `npm run selectors:report` to capture a baseline mentally (or diff against origin/main if you want the machine to do it).
2. Prefer a unique `data-cy` attribute for application-owned interactive elements over relying on a plain `#id`.
3. Update application markup and the corresponding Cypress selector together.
4. `npm run selectors:diff` to see what changed relative to `origin/main`. If something regressed, `npm run selectors:fix` to preview a proposed patch, then `npm run selectors:apply` if it looks right — followed by actually running the affected Cypress spec.
5. For anything reported "NEEDS REVIEW," check the rendered page or Cypress scenario by hand — conditional, permission-gated, or JavaScript-generated elements won't resolve automatically.
6. Report unresolved findings; don't hide them by widening `ownershipRules` exemptions in `selector-contract.config.json` just to make a `missing` entry disappear.

## Keeping the baseline current

`cypress/selectors/registry.json` is a committed file that's only meaningful if `origin/main`'s copy is kept up to date — regenerate and commit it on `main` after selector-affecting changes land, otherwise `selectors:diff` will compare against a stale snapshot.
61 changes: 61 additions & 0 deletions applications/Unity.AutoUI/cypress/selectors/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# Cypress Selector Contract

Keeps every selector used by the Cypress suite (`applications/Unity.AutoUI/cypress`) cross-checked against the actual application markup (`applications/Unity.GrantManager/src` + `modules`), so a Razor/C#/JS change that quietly removes or renames an `id`/`data-cy`/`data-testid` a test depends on gets caught before it breaks a run.

All commands below run from `applications/Unity.AutoUI`.

## What's in this directory

- **`registry.json`** — generated output, one entry per distinct selector string found in Cypress code. **This file is meant to be committed** on `main`; it's the baseline every branch's `npm run selectors:diff` compares itself against. Regenerate and commit it after selector-affecting changes land on `main`, or the baseline drifts stale.

## The two tools

| Command | What it does |
|---|---|
| `npm run selectors:report` | Full snapshot: re-scans everything, writes `registry.json`, prints a summary. No comparison to anything — just "here's the state of the world right now." |
| `npm run selectors:diff` | Compares the current working tree against `registry.json` as it exists on `origin/main`, and reports any selector whose status got *worse* (a regression this branch likely introduced). |
| `npm run selectors:fix` | Same as `selectors:diff`, plus a dry-run preview of the patch it would apply for any regression it can confidently fix. Writes nothing. |
| `npm run selectors:apply` | Same as `selectors:fix`, but actually writes the unambiguous fixes to disk. Never commits, never pushes, never touches an ambiguous case. |

Pass `-- --base <ref>` to any diff/fix/apply command to compare against something other than `origin/main`.

## How the full scan works (`scripts/selector-contract.mjs`)

1. Walks `cypressRoots` (from `selector-contract.config.json`) and parses every `.ts`/`.tsx`/`.js`/`.jsx` file with the TypeScript compiler API, pulling out every string literal passed as the first argument to a selector-shaped call (`cy.get`, `.find`, `.contains`, `.xpath`, etc.) that looks like a selector.
2. Walks `applicationRoots` for `.cshtml`/`.razor`/`.html`/`.cs`/`.js`/`.ts` source.
3. For each selector, extracts identifying tokens (`#id`, `[data-cy=...]`, `[data-testid=...]`) and checks whether that literal token appears anywhere in the app source (`id="..."`, `asp-for="@Model...."` with underscores mapped to dots, etc.).
4. Classifies each selector:
- **status**: `matched` (evidence found) / `missing` (identifying token, no evidence) / `unverified` (no identifying token to check — a structural CSS selector) / `exempt` (matches an `ownershipRules` pattern in `selector-contract.config.json`: identity-provider markup, framework-generated selectors like Bootstrap-select/Select2/DataTables/SweetAlert2, or Form.io/CHEFS dynamic fields).
- **syntaxKind**: `css` or `xpath`. Nothing in this repo uses XPath today — this exists to flag it immediately if it ever shows up, since XPath selectors are harder to keep in sync with markup than `id`/`data-cy`.

`matched` proves the token exists somewhere in source. It does **not** prove the element is visible, enabled, permission-gated correctly, or on the route the test expects — it's a fast static check, not a substitute for running the spec.

## How the diff/fix works (`scripts/selector-diff-report.mjs`)

No git-diff parsing, no guessing at what changed. It just runs the same full scan against the current tree, fetches the baseline `registry.json` via `git show <base>:.../registry.json`, and compares `status` per selector by exact string match.

- **Regression** = a selector's status is worse now than at baseline. Only `matched → missing` regressions are eligible for auto-fix — that's the one case with concrete proof the selector used to work.
- **Fix candidate search**: for each missing token, look up which file(s) had it at baseline (`applicationMatches` in the baseline entry), pull the exact old line via `git show`, then score every line in the *current* version of that file by word-overlap similarity to the old line (with a small bonus for being near the original line number — renames overwhelmingly stay in place). One clear winner → propose restoring the attribute there. Zero or multiple close-scoring candidates → reported as "NEEDS REVIEW," nothing is touched.
- **`--apply`** only ever writes the unambiguous fixes. It inserts the missing attribute next to whatever's already on that element — it never removes or rewrites existing attributes, never invents a token value without baseline evidence, and never auto-commits.

Treat an applied fix like any other code change: read the diff, and run the actual Cypress spec that uses the selector before trusting it — restoring the token proves the selector *resolves* again, not that the underlying behavior is correct.

## As a Claude Code skill

`applications/Unity.AutoUI/.claude/skills/validate-cypress-selectors/SKILL.md` documents this same system for on-demand use inside a Claude Code session — invoke it by name when working on Cypress selectors, page objects, or Razor/C#/JS markup that backs them.

## Configuration

- **`selector-contract.config.json`** (repo root of `Unity.AutoUI`) — `cypressRoots`, `applicationRoots`, and `ownershipRules` (regex patterns + reason strings for what's exempt from the static contract).

## Files that need to be committed for this to work for everyone

| File | Why |
|---|---|
| `scripts/selector-contract.mjs` | The scanner/classifier — also exports the functions `selector-diff-report.mjs` reuses. |
| `scripts/selector-diff-report.mjs` | The baseline-diff + fix logic. |
| `selector-contract.config.json` | Scan roots and ownership exemptions. |
| `package.json` | `selectors:report` / `selectors:diff` / `selectors:fix` / `selectors:apply` npm scripts. |
| `.claude/skills/validate-cypress-selectors/SKILL.md` | The on-demand skill definition. |
| `cypress/selectors/registry.json` | **The baseline itself.** Without this committed on `main`, `selectors:diff`/`fix`/`apply` have nothing to compare against and will just print a "no baseline found" message. |
| `cypress/selectors/README.md` | This file. |
Loading
Loading