feat: transform complexity-verifier into @makerx/verify (multi-check verifier)#5
Merged
Conversation
Generalise the tool from a single complexity gate into a collection of verifications that give AI coding agents back-pressure. The `verify` CLI (commander) runs checks by convention: with no `verify:*` scripts it runs the built-in default set in-process; with `verify:*` scripts it runs those in parallel, suppressing output unless a check fails. - Check registry (src/checks): native complexity, comment-block, block-comments, hardcoded-colors, forbidden-strings; external adapters for knip, circular-deps (skott), duplicate-code (jscpd), lint (oxlint) that skip gracefully when the tool is absent. - Orchestrator (src/orchestrator): resolveEntries + parallel npm-script runner with buffered/flush-on-fail output, --measure, --all, diff filters; in-process default runner. - Scaffolding (src/scaffold): interactive `verify init` and idempotent `verify upgrade-docs`, emitting agent commands/skills to .claude and .agent-skills. Templates shipped under templates/. - block-comments/hardcoded-colors/forbidden-strings and the verify runner are ported from staff0rd/assist; agent-file writing follows the MakerX data-streams CLI. Reimplemented on the TS compiler API and native fs to avoid ts-morph/grep/yaml deps (cross-platform). - Package renamed to @makerx/verify: bin `verify`, commander + enquirer deps, optional peerDependencies for the external tools, dogfoods itself via verify:* scripts. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Point the shared node-ci workflow's lint-script at `npm run verify` in both pr.yml and publish.yml so CI runs the verifications. Make every `verify:*` script non-editing so the same command is safe in CI: - verify:lint -> `oxlint .` (was `--fix`) - verify:format -> `oxfmt --check .` (was rewriting) - add verify:check-types (`tsc --noEmit`) - add a local `fix` script (`lint:fix` + `format`) for auto-fixing Also refresh dependencies to latest: commander ^15, @types/node ^26, oxfmt ^0.58; peer ranges knip ^6, jscpd ^5, skott ^0.35.11; drop the unused @jscpd/finder peer (the check shells out to the jscpd bin). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rewrite the README as full user docs for the multi-check tool: install as a pinned dev dependency, the run-by-convention model, the built-in check table, `verify init` / `verify upgrade-docs`, per-repo config, CI/CD usage, and the programmatic API. Update CLAUDE.md to the new architecture, the non-editing verify + `fix` workflow, and the console/lint conventions. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…y:* scripts Correct the run model so the person or AI agent running `verify` locally gets auto-fixes (no token churn hand-fixing lint/format), while CI fails instead of rewriting: - Fixable checks (lint, format) fix by default and switch to check-only when CI is set (or `--check`). `--fix`/`--check` on the root force a mode; it propagates to spawned verify:* scripts via the VERIFY_MODE env. - Add `format` (oxfmt) and `check-types` (tsc, skipped without tsconfig) as built-in checks; `lint` now fixes locally / checks in CI. - Scaffold every check (native and external) as `verify <name>` so the fix-vs-check logic lives in the tool, not in hand-rolled raw commands; the repo's own verify:* scripts now call `node src/cli.ts <check>`. - Resolve external tools from the local node_modules/.bin (prepended to PATH) so they run however verify was invoked, not only via npm scripts. - Drop the redundant per-subcommand --check/--fix and init's --check <name> (renamed to --select): duplicating the root option blanks commander's parsed options. - Simplify package.json scripts (prepublishOnly -> npm run build, drop the now-redundant fix script). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add unit tests for the run-mode logic that previously only had manual verification: - resolveMode: fix locally by default, check under CI, VERIFY_MODE overrides CI (both directions), unrecognised VERIFY_MODE ignored. - configureMode: --check/--fix set VERIFY_MODE (and propagate), no flag leaves the env untouched. - selectCommand: fixable checks pick the fix command only in fix mode; non-fixable checks always run the check command. Extract selectCommand as a pure helper in external.ts to make the fix-vs-check choice directly testable. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… script `verify` is a Windows cmd builtin, so `npx verify` and `npm run` script bodies that call bare `verify` run the builtin instead of this tool. Rename the executable to `verifyx` (verify + fix) so every invocation works cross-platform. The package stays `@makerx/verify` and the npm script stays named `verify` (a script lookup, not command resolution), so `npm run verify` works everywhere. - bin, commander program name, scaffold script bodies, init/skip messages, agent templates, tests, README (with a "why verifyx" callout) and CLAUDE.md all updated to `verifyx`. - Add a dev-only `prepare` shim (scripts/dev-verifyx-bin.mjs) that links node_modules/.bin/verifyx -> node src/cli.ts, so the repo's own verify:* scripts read `verifyx <check>` exactly like a consumer's. `prepare` never runs for registry consumers. - Deny @parcel/watcher's native install script via npm's allowScripts (skott's watch dep; the prebuilt binary is used, so no node-gyp build runs). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make the gate explicit: bare `verifyx` runs only the project's own `verify:*` scripts (nothing when none are defined), and a new `verifyx all` runs every built-in check. A `verify:<name>` script overrides its matching built-in, so you can swap one check's implementation without redefining the rest. - Add orchestrator `runAll` (per-check override via verify:<name>); bare `orchestrate` no longer falls back to a built-in default run. Rename the filter-bypass flag to `--no-filter` to avoid clashing with the `all` command. - Name checks for their function, not the tool: `knip` -> `unused-code`, and drop tool names from descriptions (the tool is a bin/devDeps detail). - Replace the `inDefaultRun` flag with `recommended` (only affects what `verifyx init` preselects); defaults-only init now wires `verify` to `verifyx all`. - Un-export internal-only symbols the unused-code check flagged, and add a knip.json (ignore the runtime-invoked jscpd/skott, ignore the verifyx bin). - Drop `verify:block-comments` from this repo's own gate — it fails on any comment on a changed line, which conflicts with the repo's deliberate (brief, why-focused) comment style. It stays available via `verifyx all`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When overriding a check with a `verify:<name>` script, a `verify:<name>:fix` variant can be defined too. `verifyx` runs the `:fix` variant in fix mode (locally) and the base script in check mode (CI), and never both — so an override backed by a non-mode-aware tool still fixes locally and only checks in CI (e.g. `verify:lint` = `eslint .`, `verify:lint:fix` = `eslint . --fix`). - Add `selectEntries` (bare verifyx: collapse verify:<name>/<name>:fix pairs to the variant for the current mode) and `resolveOverride` (verifyx all: pick the per-check override variant), both pure and unit-tested. - Wire them into `orchestrate` and `runAll`. - Document the fix/check override variants in the README. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…slash command Replace the /verify slash command with a skill, so the integration is identical across Claude and other agents and nothing has to be invoked manually: - init / upgrade-docs write the same terse `SKILL.md` to `.claude/skills/verify/` (Claude) and `.agent-skills/verify/` (cross-vendor), and append a one-line pointer to `CLAUDE.md` / `AGENTS.md`. - The pointer is only appended when the file doesn't already mention `npm run verify` (idempotent); existing content is never rewritten (`ensurePointer`). Symlinks / non-regular files are refused. - Trim the shipped skill to ~50 words (assist-style density); the verbose command/skill copy is gone. - Dedupe writeManaged/ensurePointer boilerplate (createFile, assertRegularFile) and share the report ACTION_MARK, so verify:duplicate-code stays clean. - Dogfood: this repo now ships `.claude/skills/verify/SKILL.md`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Checks are named for their function, which hides the underlying tool. On failure an external check now prints a one-line hint — the tool it used, the exact command it ran, and a docs link — so an agent can set up the tool's config (e.g. knip.json) without guessing. Emitted only on failure; passing runs stay silent to save tokens. - Add `docs` to ExternalCheckSpec and `externalFailureHint` (pure, tested); populate docs for oxlint/oxfmt/tsc/knip/skott/jscpd. - `verifyx all` override failures show the `npm run verify:<name>` that ran (not the built-in tool). - Document the failure-hint + buffer-and-flush-on-failure behaviour in README. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…--measure A clean run now prints nothing (just exit 0), instead of a "Running N…" preamble and an "All passed" footer. Failures are always reported. - Extract a shared `report.ts` (`chatty`, `reportOutcomes`) used by both bare `verifyx` and `verifyx all`, so the preamble + success footer only print when `--verbose` or `--measure` is set. (Also dedups the reporter across the two orchestrators.) - External checks now buffer their tool output and flush only on failure (streamed under --verbose), so `verifyx all` isn't noisy on success either. - README: document that a clean run is silent. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The per-script diff filter (ported from assist) was off by default, required hand-written `verify.filters` globs no one set, and added the confusing `--no-filter` flag. Our checks scan the whole tree anyway, so skip-unless-a- matching-file-changed was low value. Remove it entirely: - delete filterByChangedFiles + getChangedFiles; drop the `filter` field on VerifyEntry and the `verify.filters` config parsing. - remove the `--no-filter` flag; bare `verifyx` now simply runs your verify:* scripts (via selectEntries for :fix pairing). - drop the filter docs from the README. - dedupe the shared run options (--measure/--verbose/--check/--fix) across the root and `all` commands so duplicate-code stays clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…them Remove the standalone `check-types`, `lint`, `lint:fix`, `format`, `format:check` scripts. They duplicated the `verify:*` scripts (`verifyx lint`, `verifyx format`, `verifyx check-types`); ad-hoc use is `verifyx <check>` with --fix/--check. Deleting `check-types` also drops CI's default `npm run check-types --if-present` to a no-op, removing the double tsc run (verify already type-checks via verify:check-types). Add oxlint + oxfmt to knip.json ignoreDependencies: with the scripts gone they are only invoked by verifyx at runtime (via node_modules/.bin), which knip can't see — the same reason jscpd/skott are already ignored. The list is now the honest set of tools verifyx shells out to. Every script now routes through verifyx. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… setup) When `unused-code` (knip) is selected, `verifyx init` now ensures the other external tools verifyx invokes at runtime (oxlint/oxfmt/skott/jscpd) are in knip's ignoreDependencies — knip can't see runtime node_modules/.bin calls and would otherwise flag them as unused. So consumers get it for free instead of hand-editing knip config. - ensureKnipIgnores: merge into knip.json or package.json#knip (create knip.json if neither exists), adding only missing entries, never rewriting other content; code-based knip.ts/js configs are left untouched. Idempotent. - Wired into applyInit (only when unused-code is selected; ignores the other selected external checks' devDeps, minus typescript). - Tests for create/merge/unchanged/package.json/code-config + the init wiring. - README documents it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`verifyx init` now asks how verify should run before anything else: run all built-in checks (verifyx all, no verify:* scripts) or hand-pick checks to wire up. The "defaults" path routes to the existing defaultsOnly scaffolding and passes every check so all external devDeps get installed. --defaults-only remains the non-interactive equivalent. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The long-comment-block detector (comment-block) and the diff-based "no comments on changed lines" gate (block-comments) are now one native `comments` check. Default behaviour is long-block detection; the diff-based gate is opt-in via --block-new-comments. Existing flags (--max-lines, --pushback, --warn, --ignore, [pattern]) are preserved, as are the JSDoc/context: and machine-directive exemptions. The recommended default runs with --max-lines 2 --pushback (both the `verifyx all` built-in and the scaffolded verify:comments script). The verify config key blockComments is renamed to comments. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ormula Restructure the run-selection section around the three invocation modes (verifyx / verifyx all / verifyx <check>) with a lead-in table, and order the details gate → all → fix-vs-check so the :fix variant lands next to the fix/check explanation it belongs with. Expand the complexity docs with the metrics it computes, the maintainability-index formula, the clamp, and rough interpretation bands (ported from the pre-transformation README). Also fix the built-in checks table: the unused-code check was mislabelled `knip`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The table referenced `verify:*` scripts before the concept was introduced. Lead with a plain-language definition (any npm script named verify:<something>, each a check, run in parallel to form the gate) and frame the modes as "curated checks" rather than a single "gate". Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
What you run day-to-day is `npm run verify`, which maps to whatever the `verify` script points at — not inherent to the subcommand-less invocation. Reframe around the mode a `"verify": "verifyx"` script uses. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The `verifyx <check>` row referenced "built-in" without defining it. Add an explicit built-in (ships with verifyx, invoked as a subcommand) vs custom (any command wired as a verify:* script) breakdown, and clarify that `verifyx <check>` runs a single built-in by name. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Rewrite the top-of-README intro to lead with the concrete selling points: auto-fix locally / fail in CI on one command, silent-on-green for cheap agent loops, agent-actionable failure output, and convention-over-configuration verify:* scripts. - Correct the `verifyx all` description: it honours matching verify:<name> overrides rather than ignoring the verify:* list; note that custom-only scripts run via bare verifyx. - Soften the install guidance from "never globally" to a positive recommendation to pin it as a dev dependency for CI parity. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Move "verify ships both" above the selling-points list and drop the "Complexity was the first check" line. - Convert the "Why is the command verifyx?" aside into a GitHub [!NOTE] alert box. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Strip all em-dashes from the README (colons, commas, parentheses). - verifyx all: document that it also runs custom verify:* scripts, not just built-ins (matches the behaviour change in the runAll rework). - Note the verify:<name>:fix split works for custom scripts too, not only built-in overrides. - Rewrite the `comments` built-in row to be concrete: name --max-lines (default 2) instead of "the limit". - Fix stale Programmatic API export names (recommendedChecks / runAll, not defaultChecks / runDefaults); drop the duplicate standalone formula section now that the complexity docs cover it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Open with why long comments are a problem (capable models, Opus 4.8 included, narrate changes with low-value comment blocks that rot and add maintenance load) and what the default check does about it. - Give --pushback the explanation it deserves: it is prompt-engineering baked into the lint failure, telling the agent the only way to keep a comment pages a human, which stops agents silencing the check. - Promote --block-new-comments to its own paragraph, led by the override-and-opt-in instruction, and spell out the exemptions. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two behaviour changes to `verifyx all`: - Silent on green: in-process (native + external) check output is now buffered and flushed only on failure, matching bare verifyx. A clean `verifyx all` prints nothing; use --verbose/--measure for the roll-call. Overrides already ran quietly; now they respect the same buffering when not loud. - Runs everything: custom verify:* scripts (those with no matching built-in) now run too, so `all` is a true superset of the curated gate rather than built-ins only. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Document each native check config where the check is described (hardcoded-colors and forbidden-strings gain their own sections with CLI flags + verify-config keys); reframe the Configuration section as a reference and note external checks use their own tool config. - List each external check config file with a docs link. - Pull the silent-on-success note up as a general property of both orchestration modes (now that verifyx all shares it), and note single built-in runs still print their report. - Expand CI/CD: explain CI-mode switch, give a full GitHub Actions job, and describe the non-zero exit + flushed output. - Rewrite Programmatic API: clarify external checks run via the registry (getCheck().runDefault()), and list entry points by area. - Drop the data-streams upgrade-docs attribution line. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
verify now runs the project test suite as part of a run, without wiring a check: - locally: verify:test if present, else the standard test script; - on CI (CI env set): only test:ci, if present (a plain test never runs on CI, which usually needs a different invocation for junit output). No test:ci means no tests on CI. The test run is buffered like any other check (silent on success, streamed with --verbose) and is a normal gate entry (shows in --measure, fails the run). It applies to both bare verifyx and verifyx all. --no-tests skips the step, for when CI runs tests in a dedicated step. Wire the PR/publish workflows to `npm run verify -- --no-tests` (the `--` matters, or npm eats the flag) since they run test:ci separately. The tests step owns the `test` check name: an explicit verify:test is run by the step, not the normal batch, so --no-tests governs it and it never double-runs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Cut the CLAUDE.md detail that now lives in README (mode propagation, orchestrator internals) down to what an agent working in this repo actually needs: the verifyx-not-verify naming + dev shim, the --max-lines 1 comments gotcha, and the duplicate-code helper gotcha. Add the same "run npm run verify and fix what it reports" pointer that `verifyx init` scaffolds into a consumer, so the repo dogfoods it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
staff0rd
approved these changes
Jul 9, 2026
The shared run options (--measure, --verbose, --check, --fix, --no-tests) are declared on the root command, which commander treats as global. Passed after `all` they bound to the root, not the `all` subcommand, so runAll only saw the subcommand opts and ignored them: `verifyx all --measure` printed no table, and `verifyx all --no-tests` did not skip tests (now relevant since `verify` is `verifyx all` and CI passes --no-tests). Read command.optsWithGlobals() in the `all` action so globals passed after the subcommand are picked up. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
verifyx all now runs every check concurrently (like bare verifyx), cutting wall-clock from the sum of durations to roughly the slowest check. To keep parallel output clean, each check runs inside an AsyncLocalStorage-scoped capture (src/shared/output.ts): console and subprocess output route to a per-check buffer, printed in registry order afterwards (only failures normally, every check under --verbose), so concurrent checks never interleave. runCommand emits into that buffer when capturing, else flushes on failure as before, so bare verifyx is unchanged. Also decouple --measure from verbose: chatty() now tracks --verbose only, so `verifyx all --measure` (and bare `verifyx --measure`) print just the status/duration table, not the full roll-call. Failures still print so a measured run can not fail silently. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- comments --block-new-comments diffed `git diff HEAD`, a no-op in CI where a clean checkout has nothing uncommitted (the gate silently passed). Resolve the diff base per environment: HEAD locally, and in CI the merge base with the PR base branch (GITHUB_BASE_REF, or VERIFY_DIFF_BASE to override), falling back to HEAD if unresolved. Renamed gitDiffHead -> gitDiffAgainstBase. - CLI --ignore defaulted to [] (via `collect, []`), so `opts.ignore ?? config.ignore ?? []` short-circuited on the empty array and dropped configured hardcodedColors.ignore / comments.ignore when run through the CLI, contradicting the README. Treat an empty array as "not provided" so config ignores apply. - Document the per-environment diff base (and the fetch-depth: 0 caveat). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- PR/publish workflows run `verify -- --no-tests --verbose` so CI logs show each check, not just failures. - CLAUDE.md: describe `npm run verify` as `verifyx all` (every built-in + overrides + tests), matching the package.json verify script. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
5974e67 to
f4c9d9a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Generalises the tool from a single complexity gate into
@makerx/verify— a growing collection of code verifications that give AI coding agents back-pressure against writing hard-to-maintain code. Complexity is now one of several checks. The package ships both theverifyCLI and the agent commands/skills that steer assistants to run it.How it works
verifywith noverify:*scripts runs the built-in default set in-process; withverify:*scripts it runs those in parallel, buffering output and flushing only on failure ($CLAUDECODE-aware,--measure/--all/--verbose/diff-filter support).src/checks/): nativecomplexity,comment-block,block-comments,hardcoded-colors,forbidden-strings; external adapters forknip,circular-deps(skott),duplicate-code(jscpd),lint(oxlint) that skip gracefully when the tool isn't installed.src/scaffold/): interactiveverify init(checks + agent targets, writesverify:*scripts, installs opted-in tools) and idempotentverify upgrade-docs, emitting agent files to.claude/and.agent-skills/.Ported from staff0rd/assist (verify runner + the three new native checks) and the MakerX data-streams CLI (
upgrade-docs), reimplemented on the TypeScript compiler API + native fs to avoidts-morph/grep/yamldeps (cross-platform).Packaging & CI
@makerx/verify(binverify),commander+enquirerdeps, external tools as optionalpeerDependencies.verifyvia the shared workflow'slint-script(lint/format checks, complexity, comment-block, types). Local auto-fix isnpm run fix.Verification
npm run verify(dogfoods itself, non-editing) — passesnpm run check-types,npm test(48 tests),npm run build— all greenverify list, single checks, the default-run vsverify:*paths,verify init(both paths), andverify upgrade-docsidempotency🤖 Generated with Claude Code