Skip to content

feat: transform complexity-verifier into @makerx/verify (multi-check verifier)#5

Merged
robdmoore merged 34 commits into
mainfrom
feat/multi-check-verify
Jul 9, 2026
Merged

feat: transform complexity-verifier into @makerx/verify (multi-check verifier)#5
robdmoore merged 34 commits into
mainfrom
feat/multi-check-verify

Conversation

@robdmoore

Copy link
Copy Markdown
Member

Summary

Generalises the tool from a single complexity gate into @makerx/verify — a growing collection of code verifications that give AI coding agents back-pressure against writing hard-to-maintain code. Complexity is now one of several checks. The package ships both the verify CLI and the agent commands/skills that steer assistants to run it.

How it works

  • Convention-based orchestration. Bare verify with no verify:* scripts runs the built-in default set in-process; with verify:* scripts it runs those in parallel, buffering output and flushing only on failure ($CLAUDECODE-aware, --measure/--all/--verbose/diff-filter support).
  • Check registry (src/checks/): native complexity, comment-block, block-comments, hardcoded-colors, forbidden-strings; external adapters for knip, circular-deps (skott), duplicate-code (jscpd), lint (oxlint) that skip gracefully when the tool isn't installed.
  • Scaffolding (src/scaffold/): interactive verify init (checks + agent targets, writes verify:* scripts, installs opted-in tools) and idempotent verify upgrade-docs, emitting agent files to .claude/ and .agent-skills/.

Ported from staff0rd/assist (verify runner + the three new native checks) and the MakerX data-streams CLI (upgrade-docs), reimplemented on the TypeScript compiler API + native fs to avoid ts-morph/grep/yaml deps (cross-platform).

Packaging & CI

  • Renamed to @makerx/verify (bin verify), commander + enquirer deps, external tools as optional peerDependencies.
  • Both workflows run the non-editing verify via the shared workflow's lint-script (lint/format checks, complexity, comment-block, types). Local auto-fix is npm run fix.
  • Dependencies refreshed to latest.

Verification

  • npm run verify (dogfoods itself, non-editing) — passes
  • npm run check-types, npm test (48 tests), npm run build — all green
  • Manually exercised verify list, single checks, the default-run vs verify:* paths, verify init (both paths), and verify upgrade-docs idempotency

🤖 Generated with Claude Code

robdmoore and others added 6 commits July 8, 2026 19:29
Generalise the tool from a single complexity gate into a collection of
verifications that give AI coding agents back-pressure. The `verify` CLI
(commander) runs checks by convention: with no `verify:*` scripts it runs
the built-in default set in-process; with `verify:*` scripts it runs those
in parallel, suppressing output unless a check fails.

- Check registry (src/checks): native complexity, comment-block,
  block-comments, hardcoded-colors, forbidden-strings; external adapters
  for knip, circular-deps (skott), duplicate-code (jscpd), lint (oxlint)
  that skip gracefully when the tool is absent.
- Orchestrator (src/orchestrator): resolveEntries + parallel npm-script
  runner with buffered/flush-on-fail output, --measure, --all, diff filters;
  in-process default runner.
- Scaffolding (src/scaffold): interactive `verify init` and idempotent
  `verify upgrade-docs`, emitting agent commands/skills to .claude and
  .agent-skills. Templates shipped under templates/.
- block-comments/hardcoded-colors/forbidden-strings and the verify runner
  are ported from staff0rd/assist; agent-file writing follows the MakerX
  data-streams CLI. Reimplemented on the TS compiler API and native fs to
  avoid ts-morph/grep/yaml deps (cross-platform).
- Package renamed to @makerx/verify: bin `verify`, commander + enquirer
  deps, optional peerDependencies for the external tools, dogfoods itself
  via verify:* scripts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Point the shared node-ci workflow's lint-script at `npm run verify` in both
pr.yml and publish.yml so CI runs the verifications. Make every `verify:*`
script non-editing so the same command is safe in CI:

- verify:lint -> `oxlint .` (was `--fix`)
- verify:format -> `oxfmt --check .` (was rewriting)
- add verify:check-types (`tsc --noEmit`)
- add a local `fix` script (`lint:fix` + `format`) for auto-fixing

Also refresh dependencies to latest: commander ^15, @types/node ^26,
oxfmt ^0.58; peer ranges knip ^6, jscpd ^5, skott ^0.35.11; drop the unused
@jscpd/finder peer (the check shells out to the jscpd bin).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rewrite the README as full user docs for the multi-check tool: install as a
pinned dev dependency, the run-by-convention model, the built-in check table,
`verify init` / `verify upgrade-docs`, per-repo config, CI/CD usage, and the
programmatic API. Update CLAUDE.md to the new architecture, the non-editing
verify + `fix` workflow, and the console/lint conventions.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…y:* scripts

Correct the run model so the person or AI agent running `verify` locally gets
auto-fixes (no token churn hand-fixing lint/format), while CI fails instead of
rewriting:

- Fixable checks (lint, format) fix by default and switch to check-only when
  CI is set (or `--check`). `--fix`/`--check` on the root force a mode; it
  propagates to spawned verify:* scripts via the VERIFY_MODE env.
- Add `format` (oxfmt) and `check-types` (tsc, skipped without tsconfig) as
  built-in checks; `lint` now fixes locally / checks in CI.
- Scaffold every check (native and external) as `verify <name>` so the
  fix-vs-check logic lives in the tool, not in hand-rolled raw commands; the
  repo's own verify:* scripts now call `node src/cli.ts <check>`.
- Resolve external tools from the local node_modules/.bin (prepended to PATH)
  so they run however verify was invoked, not only via npm scripts.
- Drop the redundant per-subcommand --check/--fix and init's --check <name>
  (renamed to --select): duplicating the root option blanks commander's parsed
  options.
- Simplify package.json scripts (prepublishOnly -> npm run build, drop the
  now-redundant fix script).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add unit tests for the run-mode logic that previously only had manual
verification:

- resolveMode: fix locally by default, check under CI, VERIFY_MODE overrides
  CI (both directions), unrecognised VERIFY_MODE ignored.
- configureMode: --check/--fix set VERIFY_MODE (and propagate), no flag leaves
  the env untouched.
- selectCommand: fixable checks pick the fix command only in fix mode;
  non-fixable checks always run the check command.

Extract selectCommand as a pure helper in external.ts to make the
fix-vs-check choice directly testable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@robdmoore robdmoore marked this pull request as ready for review July 8, 2026 15:37
robdmoore and others added 23 commits July 8, 2026 23:41
… script

`verify` is a Windows cmd builtin, so `npx verify` and `npm run` script bodies
that call bare `verify` run the builtin instead of this tool. Rename the
executable to `verifyx` (verify + fix) so every invocation works cross-platform.
The package stays `@makerx/verify` and the npm script stays named `verify`
(a script lookup, not command resolution), so `npm run verify` works everywhere.

- bin, commander program name, scaffold script bodies, init/skip messages,
  agent templates, tests, README (with a "why verifyx" callout) and CLAUDE.md
  all updated to `verifyx`.
- Add a dev-only `prepare` shim (scripts/dev-verifyx-bin.mjs) that links
  node_modules/.bin/verifyx -> node src/cli.ts, so the repo's own verify:*
  scripts read `verifyx <check>` exactly like a consumer's. `prepare` never runs
  for registry consumers.
- Deny @parcel/watcher's native install script via npm's allowScripts
  (skott's watch dep; the prebuilt binary is used, so no node-gyp build runs).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make the gate explicit: bare `verifyx` runs only the project's own `verify:*`
scripts (nothing when none are defined), and a new `verifyx all` runs every
built-in check. A `verify:<name>` script overrides its matching built-in, so
you can swap one check's implementation without redefining the rest.

- Add orchestrator `runAll` (per-check override via verify:<name>); bare
  `orchestrate` no longer falls back to a built-in default run. Rename the
  filter-bypass flag to `--no-filter` to avoid clashing with the `all` command.
- Name checks for their function, not the tool: `knip` -> `unused-code`, and
  drop tool names from descriptions (the tool is a bin/devDeps detail).
- Replace the `inDefaultRun` flag with `recommended` (only affects what
  `verifyx init` preselects); defaults-only init now wires `verify` to
  `verifyx all`.
- Un-export internal-only symbols the unused-code check flagged, and add a
  knip.json (ignore the runtime-invoked jscpd/skott, ignore the verifyx bin).
- Drop `verify:block-comments` from this repo's own gate — it fails on any
  comment on a changed line, which conflicts with the repo's deliberate
  (brief, why-focused) comment style. It stays available via `verifyx all`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When overriding a check with a `verify:<name>` script, a `verify:<name>:fix`
variant can be defined too. `verifyx` runs the `:fix` variant in fix mode
(locally) and the base script in check mode (CI), and never both — so an
override backed by a non-mode-aware tool still fixes locally and only checks
in CI (e.g. `verify:lint` = `eslint .`, `verify:lint:fix` = `eslint . --fix`).

- Add `selectEntries` (bare verifyx: collapse verify:<name>/<name>:fix pairs to
  the variant for the current mode) and `resolveOverride` (verifyx all: pick the
  per-check override variant), both pure and unit-tested.
- Wire them into `orchestrate` and `runAll`.
- Document the fix/check override variants in the README.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…slash command

Replace the /verify slash command with a skill, so the integration is identical
across Claude and other agents and nothing has to be invoked manually:

- init / upgrade-docs write the same terse `SKILL.md` to `.claude/skills/verify/`
  (Claude) and `.agent-skills/verify/` (cross-vendor), and append a one-line
  pointer to `CLAUDE.md` / `AGENTS.md`.
- The pointer is only appended when the file doesn't already mention
  `npm run verify` (idempotent); existing content is never rewritten
  (`ensurePointer`). Symlinks / non-regular files are refused.
- Trim the shipped skill to ~50 words (assist-style density); the verbose
  command/skill copy is gone.
- Dedupe writeManaged/ensurePointer boilerplate (createFile, assertRegularFile)
  and share the report ACTION_MARK, so verify:duplicate-code stays clean.
- Dogfood: this repo now ships `.claude/skills/verify/SKILL.md`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Checks are named for their function, which hides the underlying tool. On
failure an external check now prints a one-line hint — the tool it used, the
exact command it ran, and a docs link — so an agent can set up the tool's
config (e.g. knip.json) without guessing. Emitted only on failure; passing
runs stay silent to save tokens.

- Add `docs` to ExternalCheckSpec and `externalFailureHint` (pure, tested);
  populate docs for oxlint/oxfmt/tsc/knip/skott/jscpd.
- `verifyx all` override failures show the `npm run verify:<name>` that ran
  (not the built-in tool).
- Document the failure-hint + buffer-and-flush-on-failure behaviour in README.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…--measure

A clean run now prints nothing (just exit 0), instead of a "Running N…"
preamble and an "All passed" footer. Failures are always reported.

- Extract a shared `report.ts` (`chatty`, `reportOutcomes`) used by both bare
  `verifyx` and `verifyx all`, so the preamble + success footer only print when
  `--verbose` or `--measure` is set. (Also dedups the reporter across the two
  orchestrators.)
- External checks now buffer their tool output and flush only on failure
  (streamed under --verbose), so `verifyx all` isn't noisy on success either.
- README: document that a clean run is silent.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The per-script diff filter (ported from assist) was off by default, required
hand-written `verify.filters` globs no one set, and added the confusing
`--no-filter` flag. Our checks scan the whole tree anyway, so skip-unless-a-
matching-file-changed was low value. Remove it entirely:

- delete filterByChangedFiles + getChangedFiles; drop the `filter` field on
  VerifyEntry and the `verify.filters` config parsing.
- remove the `--no-filter` flag; bare `verifyx` now simply runs your verify:*
  scripts (via selectEntries for :fix pairing).
- drop the filter docs from the README.
- dedupe the shared run options (--measure/--verbose/--check/--fix) across the
  root and `all` commands so duplicate-code stays clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…them

Remove the standalone `check-types`, `lint`, `lint:fix`, `format`,
`format:check` scripts. They duplicated the `verify:*` scripts (`verifyx lint`,
`verifyx format`, `verifyx check-types`); ad-hoc use is `verifyx <check>` with
--fix/--check. Deleting `check-types` also drops CI's default
`npm run check-types --if-present` to a no-op, removing the double tsc run
(verify already type-checks via verify:check-types).

Add oxlint + oxfmt to knip.json ignoreDependencies: with the scripts gone they
are only invoked by verifyx at runtime (via node_modules/.bin), which knip
can't see — the same reason jscpd/skott are already ignored. The list is now
the honest set of tools verifyx shells out to.

Every script now routes through verifyx.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… setup)

When `unused-code` (knip) is selected, `verifyx init` now ensures the other
external tools verifyx invokes at runtime (oxlint/oxfmt/skott/jscpd) are in
knip's ignoreDependencies — knip can't see runtime node_modules/.bin calls and
would otherwise flag them as unused. So consumers get it for free instead of
hand-editing knip config.

- ensureKnipIgnores: merge into knip.json or package.json#knip (create knip.json
  if neither exists), adding only missing entries, never rewriting other content;
  code-based knip.ts/js configs are left untouched. Idempotent.
- Wired into applyInit (only when unused-code is selected; ignores the other
  selected external checks' devDeps, minus typescript).
- Tests for create/merge/unchanged/package.json/code-config + the init wiring.
- README documents it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`verifyx init` now asks how verify should run before anything else: run
all built-in checks (verifyx all, no verify:* scripts) or hand-pick
checks to wire up. The "defaults" path routes to the existing
defaultsOnly scaffolding and passes every check so all external devDeps
get installed. --defaults-only remains the non-interactive equivalent.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The long-comment-block detector (comment-block) and the diff-based
"no comments on changed lines" gate (block-comments) are now one native
`comments` check. Default behaviour is long-block detection; the
diff-based gate is opt-in via --block-new-comments. Existing flags
(--max-lines, --pushback, --warn, --ignore, [pattern]) are preserved,
as are the JSDoc/context: and machine-directive exemptions.

The recommended default runs with --max-lines 2 --pushback (both the
`verifyx all` built-in and the scaffolded verify:comments script). The
verify config key blockComments is renamed to comments.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ormula

Restructure the run-selection section around the three invocation modes
(verifyx / verifyx all / verifyx <check>) with a lead-in table, and
order the details gate → all → fix-vs-check so the :fix variant lands
next to the fix/check explanation it belongs with.

Expand the complexity docs with the metrics it computes, the
maintainability-index formula, the clamp, and rough interpretation
bands (ported from the pre-transformation README). Also fix the
built-in checks table: the unused-code check was mislabelled `knip`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The table referenced `verify:*` scripts before the concept was
introduced. Lead with a plain-language definition (any npm script named
verify:<something>, each a check, run in parallel to form the gate) and
frame the modes as "curated checks" rather than a single "gate".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
What you run day-to-day is `npm run verify`, which maps to whatever the
`verify` script points at — not inherent to the subcommand-less
invocation. Reframe around the mode a `"verify": "verifyx"` script uses.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The `verifyx <check>` row referenced "built-in" without defining it.
Add an explicit built-in (ships with verifyx, invoked as a subcommand)
vs custom (any command wired as a verify:* script) breakdown, and
clarify that `verifyx <check>` runs a single built-in by name.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Rewrite the top-of-README intro to lead with the concrete selling
  points: auto-fix locally / fail in CI on one command, silent-on-green
  for cheap agent loops, agent-actionable failure output, and
  convention-over-configuration verify:* scripts.
- Correct the `verifyx all` description: it honours matching
  verify:<name> overrides rather than ignoring the verify:* list; note
  that custom-only scripts run via bare verifyx.
- Soften the install guidance from "never globally" to a positive
  recommendation to pin it as a dev dependency for CI parity.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Move "verify ships both" above the selling-points list and drop the
  "Complexity was the first check" line.
- Convert the "Why is the command verifyx?" aside into a GitHub
  [!NOTE] alert box.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Strip all em-dashes from the README (colons, commas, parentheses).
- verifyx all: document that it also runs custom verify:* scripts, not
  just built-ins (matches the behaviour change in the runAll rework).
- Note the verify:<name>:fix split works for custom scripts too, not
  only built-in overrides.
- Rewrite the `comments` built-in row to be concrete: name --max-lines
  (default 2) instead of "the limit".
- Fix stale Programmatic API export names (recommendedChecks / runAll,
  not defaultChecks / runDefaults); drop the duplicate standalone
  formula section now that the complexity docs cover it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Open with why long comments are a problem (capable models, Opus 4.8
  included, narrate changes with low-value comment blocks that rot and
  add maintenance load) and what the default check does about it.
- Give --pushback the explanation it deserves: it is prompt-engineering
  baked into the lint failure, telling the agent the only way to keep a
  comment pages a human, which stops agents silencing the check.
- Promote --block-new-comments to its own paragraph, led by the
  override-and-opt-in instruction, and spell out the exemptions.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two behaviour changes to `verifyx all`:
- Silent on green: in-process (native + external) check output is now
  buffered and flushed only on failure, matching bare verifyx. A clean
  `verifyx all` prints nothing; use --verbose/--measure for the
  roll-call. Overrides already ran quietly; now they respect the same
  buffering when not loud.
- Runs everything: custom verify:* scripts (those with no matching
  built-in) now run too, so `all` is a true superset of the curated
  gate rather than built-ins only.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Document each native check config where the check is described
  (hardcoded-colors and forbidden-strings gain their own sections with
  CLI flags + verify-config keys); reframe the Configuration section as
  a reference and note external checks use their own tool config.
- List each external check config file with a docs link.
- Pull the silent-on-success note up as a general property of both
  orchestration modes (now that verifyx all shares it), and note single
  built-in runs still print their report.
- Expand CI/CD: explain CI-mode switch, give a full GitHub Actions job,
  and describe the non-zero exit + flushed output.
- Rewrite Programmatic API: clarify external checks run via the registry
  (getCheck().runDefault()), and list entry points by area.
- Drop the data-streams upgrade-docs attribution line.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
verify now runs the project test suite as part of a run, without
wiring a check:
- locally: verify:test if present, else the standard test script;
- on CI (CI env set): only test:ci, if present (a plain test never
  runs on CI, which usually needs a different invocation for junit
  output). No test:ci means no tests on CI.
The test run is buffered like any other check (silent on success,
streamed with --verbose) and is a normal gate entry (shows in
--measure, fails the run). It applies to both bare verifyx and
verifyx all.

--no-tests skips the step, for when CI runs tests in a dedicated step.
Wire the PR/publish workflows to `npm run verify -- --no-tests` (the
`--` matters, or npm eats the flag) since they run test:ci separately.

The tests step owns the `test` check name: an explicit verify:test is
run by the step, not the normal batch, so --no-tests governs it and it
never double-runs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Cut the CLAUDE.md detail that now lives in README (mode propagation,
orchestrator internals) down to what an agent working in this repo
actually needs: the verifyx-not-verify naming + dev shim, the
--max-lines 1 comments gotcha, and the duplicate-code helper gotcha.

Add the same "run npm run verify and fix what it reports" pointer that
`verifyx init` scaffolds into a consumer, so the repo dogfoods it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Comment thread src/shared/git.ts Outdated
Comment thread src/commands/registerChecks.ts
robdmoore and others added 4 commits July 9, 2026 11:21
The shared run options (--measure, --verbose, --check, --fix,
--no-tests) are declared on the root command, which commander treats as
global. Passed after `all` they bound to the root, not the `all`
subcommand, so runAll only saw the subcommand opts and ignored them:
`verifyx all --measure` printed no table, and `verifyx all --no-tests`
did not skip tests (now relevant since `verify` is `verifyx all` and CI
passes --no-tests).

Read command.optsWithGlobals() in the `all` action so globals passed
after the subcommand are picked up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
verifyx all now runs every check concurrently (like bare verifyx),
cutting wall-clock from the sum of durations to roughly the slowest
check. To keep parallel output clean, each check runs inside an
AsyncLocalStorage-scoped capture (src/shared/output.ts): console and
subprocess output route to a per-check buffer, printed in registry
order afterwards (only failures normally, every check under --verbose),
so concurrent checks never interleave. runCommand emits into that
buffer when capturing, else flushes on failure as before, so bare
verifyx is unchanged.

Also decouple --measure from verbose: chatty() now tracks --verbose
only, so `verifyx all --measure` (and bare `verifyx --measure`) print
just the status/duration table, not the full roll-call. Failures still
print so a measured run can not fail silently.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- comments --block-new-comments diffed `git diff HEAD`, a no-op in CI
  where a clean checkout has nothing uncommitted (the gate silently
  passed). Resolve the diff base per environment: HEAD locally, and in
  CI the merge base with the PR base branch (GITHUB_BASE_REF, or
  VERIFY_DIFF_BASE to override), falling back to HEAD if unresolved.
  Renamed gitDiffHead -> gitDiffAgainstBase.
- CLI --ignore defaulted to [] (via `collect, []`), so
  `opts.ignore ?? config.ignore ?? []` short-circuited on the empty
  array and dropped configured hardcodedColors.ignore / comments.ignore
  when run through the CLI, contradicting the README. Treat an empty
  array as "not provided" so config ignores apply.
- Document the per-environment diff base (and the fetch-depth: 0 caveat).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- PR/publish workflows run `verify -- --no-tests --verbose` so CI logs
  show each check, not just failures.
- CLAUDE.md: describe `npm run verify` as `verifyx all` (every built-in
  + overrides + tests), matching the package.json verify script.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@robdmoore robdmoore force-pushed the feat/multi-check-verify branch from 5974e67 to f4c9d9a Compare July 9, 2026 04:31
@robdmoore robdmoore merged commit 5d1385d into main Jul 9, 2026
1 check passed
@robdmoore robdmoore deleted the feat/multi-check-verify branch July 9, 2026 04:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants