diff --git a/README.md b/README.md index 32df561..ba318ad 100644 --- a/README.md +++ b/README.md @@ -1,16 +1,15 @@ # comment-checker -> **comment-checker is a Claude Code hook — an alternative to blunt flag-everything checkers — that blocks only the comments that don't earn their place.** +[![CI](https://github.com/systemfsoftware/claude-code-comment-checker/actions/workflows/ci.yml/badge.svg)](https://github.com/systemfsoftware/claude-code-comment-checker/actions/workflows/ci.yml) +[![License: Apache-2.0](https://img.shields.io/badge/License-Apache--2.0-blue.svg)](LICENSE) -Every flag names the specific reason — restates the code, TODO without a tracked ticket, dead code left in a comment, change-log memo — so an agent cannot hand-wave it away. Comments that earn their place are spared: license headers, linter and type-checker directives, public-API docs, and non-obvious intent. +> A Claude Code `PostToolUse` hook that blocks unnecessary code comments with checkable, cited reasons — while sparing earned API documentation, directives, and non-obvious intent. ```bash -cargo install --git https://github.com/systemfsoftware/comment-checker --package claude-code-comment-checker +pnpm install -g @systemfsoftware/claude-code-comment-checker ``` -Wired as a `PostToolUse` hook, it runs on every `Write`, `Edit`, and `MultiEdit`, checks the comments in the written code, and blocks the change when any are unnecessary: - -```bash +``` $ echo '{"tool_name":"Write","tool_input":{"file_path":"src/load_config.py","content":"import json\n\ndef load_config(path):\n # Parse the config file\n data = json.load(open(path))\n # TODO: fix this later\n # print(data)\n return data\n"}}' | comment-checker An automated reviewer flagged 3 comment(s) in src/load_config.py as unnecessary. @@ -18,78 +17,63 @@ Each is stated with the specific reason it should be removed. Do not dismiss these as "justified" — the reason is given so the claim can be checked, not argued away. - line 4 — # Parse the config file — restates what the code already says + line 4 — # Parse the config file — restates what the code already says (shares config, file, parse) line 6 — # TODO: fix this later — a TODO with no tracked reference — file a ticket or delete it line 7 — # print(data) — dead code left in a comment Action: delete the flagged comments. If the code is unclear without one, make the code self-explanatory instead — better names, extraction, a clearer type — and do not re-add the comment. -exit 2 ``` -## Install +--- + +## Why -**Status: pre-release.** The npm distribution is built but the first release has not been published yet. Once it lands, npm is the recommended install: +Most comment linters rely on blunt allowlists: they flag every comment that lacks a specific annotation, or blindly permit any text placed inside a docstring block. This creates high false-positive noise that trains agents and engineers to dismiss warnings entirely. + +`comment-checker` uses tree-sitter AST extraction across 37 programming languages and evaluates comments against prioritized classification rules. When a comment is flagged, the hook provides concrete citations — such as token overlap percentages or verb-to-operator mappings — allowing the agent to verify why a comment failed and fix the underlying code rather than arguing with the tool. + +| Capability | Flag-everything linters | comment-checker | +|---|---|---| +| **Classification model** | Blunt allowlist or regex scan | Prioritized rule tables with syntactic AST context | +| **Public API docstrings** | Flagged or blindly permitted | Spared when containing structured contract tags (`@param`, `Args:`, `Returns:`) | +| **Flag feedback** | Generic warning message | Specific, checkable reason citing token overlap and operator evidence | +| **Incremental edits** | Re-evaluates entire source file | Evaluates only newly added comments; skips fragment restatements | +| **Evaluation standard** | Ad-hoc heuristics | F1 ≥ 0.85 on 60-case multi-language benchmark (`eval/corpus.json`) | + +--- + +## Quick Start + +### 1. Install the binary + +**Recommended (npm / pnpm / yarn / bun):** ```bash pnpm install -g @systemfsoftware/claude-code-comment-checker ``` -The package ships prebuilt binaries for Linux (x64, arm64), macOS (x64, arm64), and Windows (x64) as optional dependencies — npm installs only the one for your platform. No postinstall script runs, so `--ignore-scripts` and strict package managers work. Packages are published with OIDC trusted publishing and provenance. +The package distributes prebuilt native binaries for Linux (x64, arm64), macOS (x64, arm64), and Windows (x64) via `optionalDependencies`. Package managers install only the single binary target required for your operating system. No postinstall lifecycle scripts run during installation, ensuring full compatibility with `--ignore-scripts`. -Until then, install from source (works today): +
+Other install methods (Cargo, direct binary download) + +**Install via Cargo (requires Rust 1.85+):** ```bash cargo install --git https://github.com/systemfsoftware/comment-checker --package claude-code-comment-checker ``` -Requires Rust 1.85+. Each [GitHub release](https://github.com/systemfsoftware/comment-checker/releases) also attaches `comment-checker-.tar.gz` tarballs for direct download. - -## Publishing - -Releases are tag-triggered: pushing a tag `vX.Y.Z` to `main` runs -[`.github/workflows/release.yml`](.github/workflows/release.yml), which builds -all five target binaries, publishes the five platform packages and then the -root launcher — all with npm OIDC trusted publishing and provenance, no static -tokens in CI. The exact step sequence and per-package trusted-publisher bindings are documented -in the release plan (`docs/plans/2026-08-17-001-feat-npm-distribution-release-plan.md`) -and automated via `cd scripts && deno task publish:unpublished`. -To release: - -1. Create the six [npm trusted-publisher - entries](https://docs.npmjs.com/generating-provenance-statements) with the - bindings below (one time). -2. Push a semver tag: `git tag v0.1.0 && git push origin v0.1.0`. -3. Watch CI; the root package is published only after every platform package - exists and its published binary matches the recorded sha256. -4. Verify post-publish: a fresh `pnpm dlx`/`npm i -g` install of the root - package, run the binary on Linux and on one non-Linux platform, and confirm - `npm view @systemfsoftware/claude-code-comment-checker provenance` shows - provenance. - -The npm trusted-publisher records bind the publishing identity to this -workflow — the registry-side record has no tag-pattern field, so -`refs/tags/v*` is enforced by the workflow's `on: push: tags` filter, never by -the registry-side record: - -| Package | Trusted publisher binding | -|---|---| -| `@systemfsoftware/claude-code-comment-checker` | Org: `systemfsoftware`, repo: `comment-checker`, workflow: `.github/workflows/release.yml` | -| `@systemfsoftware/claude-code-comment-checker-linux-x64` | same | -| `@systemfsoftware/claude-code-comment-checker-linux-arm64` | same | -| `@systemfsoftware/claude-code-comment-checker-darwin-x64` | same | -| `@systemfsoftware/claude-code-comment-checker-darwin-arm64` | same | -| `@systemfsoftware/claude-code-comment-checker-win32-x64` | same | +**Direct download:** -The version pinned in the committed launcher manifest (`0.1.0`) may lag behind -releases by design — the git tag is the single version source; the release -workflow rewrites the published manifest. +Prebuilt tarballs (`comment-checker-.tar.gz`) for all supported platforms are attached to every [GitHub Release](https://github.com/systemfsoftware/comment-checker/releases). -## Quick Start +
+ +### 2. Configure Claude Code hook -1. Install (above). -2. Add the hook to `~/.claude/settings.json` (or `.claude/settings.json` in a project): +Add `comment-checker` as a `PostToolUse` hook in your user configuration (`~/.claude/settings.json`) or project configuration (`.claude/settings.json`): ```json { @@ -106,114 +90,130 @@ workflow rewrites the published manifest. } ``` -3. Done. A clean change exits 0: +### 3. Verify execution + +When an agent writes code containing justified comments or clean documentation, the tool exits cleanly with status code `0`: ```bash $ echo '{"tool_name":"Write","tool_input":{"file_path":"src/client.py","content":"# SPDX-License-Identifier: Apache-2.0\ndef load(path):\n return open(path).read()\n"}}' | comment-checker [check-comments] Skipping: No unnecessary comments found -exit 0 ``` -## What it flags +--- -Five kinds of comments, each with a stated reason: +## What It Flags -| Kind | Reason given | Example | +`comment-checker` identifies unnecessary comments across five distinct categories: + +| Category | Reason cited | Example | |---|---|---| -| Restates the code | `restates what the code already says` (shares tokens, or an operator match) | `// adds one to one` next to `x := 1 + 1` | -| Narrates the flow | `narrates the for construct the code already shows` | `// loop over each item` next to `for item in items:` | -| Change-log memo | `describes what changed, not why — git history already records this` | `// Changed from old_value to new_value` | -| Dead code in a comment | `dead code left in a comment` | `// fmt.Println("debug")` | -| TODO without a ticket | `a TODO with no tracked reference — file a ticket or delete it` | `// TODO: fix this later` | +| **Restates the code** | `restates what the code already says ()` | `// adds one to one` adjacent to `x += 1` | +| **Narrates control flow** | `narrates the construct the code already shows` | `// loop over each item` adjacent to `for item in items:` | +| **Change-log memo** | `describes what changed, not why — git history already records this` | `// Changed from old_value to new_value` | +| **Dead code** | `dead code left in a comment` | `// fmt.Println("debug")` | +| **Untracked TODO** | `a TODO with no tracked reference — file a ticket or delete it` | `// TODO: fix this later` | -Restatement flags cite the evidence: `shares counter; increment ↔ +=` — the -overlap or verb-to-operator match the verdict was built on, so the reason is -checkable against the code. +--- -## What it spares +## What It Spares -Comments that earn their place are classified as justified and pass: +Comments that provide non-redundant intent or satisfy interface documentation standards are classified as justified and pass without warnings: -- **License and provenance** — SPDX identifiers, copyright lines, generated-file notices (`// SPDX-License-Identifier: Apache-2.0`, `// THIS FILE IS AUTO-GENERATED - DO NOT EDIT`) -- **Directives** — `# noqa: E501`, `// @ts-ignore`, `// eslint-disable-next-line`, `# shellcheck disable=SC2086`, `// clippy::too_many_arguments`, `/* istanbul ignore next */` -- **BDD steps** — `# given`, `# when`, `// then` -- **Public-API docs** — docstrings with `@param`, `@returns`, `Args:`, `Returns:`, `# panics`, `# safety`; likewise line/block comments whose text *leads* with a contract tag (`# Returns: …`, `// @param …`) at a contract position (head of a declaration) -- **Non-obvious intent** — `// workaround:`, `# because …`, `// to avoid the TOCTOU race`, `Why:`, `!NOTE:`, `1-based`/`0-based` conventions -- **Attribution and references** — `// @author`, `// ref: https://…`, `adapted from`, `ported from` -- **Shebangs** — `#!/usr/bin/env python` +- **License & generated headers** — SPDX identifiers, copyright lines, and generated-file notices (`// SPDX-License-Identifier: Apache-2.0`, `/* Copyright (c) 2026 ... */`) +- **Compiler & linter directives** — `# noqa: E501`, `// @ts-ignore`, `// eslint-disable-next-line`, `# shellcheck disable=SC2086`, `// clippy::too_many_arguments`, `/* istanbul ignore next */` +- **BDD test steps** — `# given`, `# when`, `// then` +- **Structured API docstrings & contract tags** — Docstrings containing `@param`, `@returns`, `Args:`, `Returns:`, `# panics`, or `# safety`, as well as leading contract tags on declarations +- **Non-obvious intent & rationale** — Comments explaining *why* something is done (`// workaround: SDK panics on empty input`, `# because SQLite locks during write`, `// to avoid TOCTOU race`, `Why: 1-based index`) +- **Attribution & references** — `// @author Jane Doe`, `// ref: https://...`, `// adapted from ...` +- **Executable shebangs** — `#!/usr/bin/env python3`, `#!/bin/bash` -On `Edit` and `MultiEdit`, only **newly added** comments are checked — pre-existing comments in the file never block a change. The fragment may cut off the surrounding code, so restatement detection is disabled on edits: explicit rules still block, but a comment the hook cannot verify against reliable context passes. +--- -## Why it's different +## Supported Languages -Blunt comment checkers flag every comment that isn't on a small allowlist — docstrings included. Most flags are false, and the agent learns to dismiss the warning entirely. +Tree-sitter AST parsers are compiled directly into the binary across 37 programming languages and formats: -| | Flag-everything checkers | comment-checker | -|---|---|---| -| Decision | Flag everything not on an allowlist | Classify each comment against ordered rule tables | -| Docstrings | Flagged | Spared when they document an API | -| Reason per flag | Generic warning | One of five specific, checkable reasons | -| Edits | Whole file | Only newly-added comments; restatement disabled on fragments | -| Precision bar | None | F1 ≥ 0.85 on the context-bearing corpus (60 cases, per-kind floors) - -Built in Rust on tree-sitter: 37 languages (Python, TypeScript, JavaScript, Rust, Go, Java, C/C++, C#, Kotlin, Scala, Ruby, PHP, Swift, Elixir, Bash, Lua, SQL, JSON, YAML, TOML, HTML, CSS, Dockerfile, HCL, Markdown, R, Dart, Zig, Haskell, OCaml, Svelte, Elm, Groovy, CUE, Protocol Buffers), statically linked — no runtime network, no dynamic loading. The classifier core is mutation-tested to 100%. +| Category | Languages | +|---|---| +| **Systems & Native** | Rust (`.rs`), C (`.c`, `.h`), C++ (`.cpp`, `.cc`, `.cxx`, `.hpp`), Zig (`.zig`) | +| **Web & Applications** | TypeScript (`.ts`, `.tsx`), JavaScript (`.js`, `.jsx`, `.mjs`, `.cjs`), Python (`.py`, `.pyi`), Go (`.go`), Java (`.java`), C# (`.cs`), Kotlin (`.kt`), Scala (`.scala`), Swift (`.swift`), Dart (`.dart`), PHP (`.php`), Ruby (`.rb`), Elixir (`.ex`, `.exs`), Svelte (`.svelte`), Elm (`.elm`), Lua (`.lua`), Groovy (`.groovy`, `.gradle`), OCaml (`.ml`, `.mli`), Haskell (`.hs`), R (`.r`, `.rmd`) | +| **Shell & Config** | Bash / Shell (`.sh`, `.bash`, `.zsh`), SQL (`.sql`), JSON (`.json`), YAML (`.yaml`, `.yml`), TOML (`.toml`), HTML (`.html`), CSS (`.css`), Dockerfile (`Dockerfile`), HCL / Terraform (`.tf`, `.hcl`), CUE (`.cue`), Protocol Buffers (`.proto`), Markdown (`.md`) | -## Exit codes +Files written in unsupported extensions or non-code formats are skipped automatically, allowing standard tool execution to proceed without interruptions. -| Code | Meaning | -|---|---| -| 0 | Pass — no unnecessary comments found (also for skipped input: malformed payload, no file path, unsupported language) | -| 2 | Block — unnecessary comments found; the report is printed to stdout | +--- -Malformed input never blocks — a hook must not fail the user's work on bad input. +## Usage & Configuration -## Custom prompt +### Custom Prompt Formatting -The default report can be replaced; `{{comments}}` inserts it: +The `--prompt` command-line option allows teams to override the default notification text delivered to Claude Code. The `{{comments}}` placeholder is replaced with the structured list of flagged comments and citations: ```bash -comment-checker --prompt "Your changes contain: {{comments}}" +comment-checker --prompt "Review feedback:\n\n{{comments}}\n\nPlease revise the code." ``` -To wire it into the hook command in `settings.json`: +You can configure this flag directly in your `.claude/settings.json` file: ```json -{ "type": "command", "command": "comment-checker --prompt \"Your changes contain: {{comments}}\"" } +{ + "hooks": { + "PostToolUse": [ + { + "matcher": "Write|Edit|MultiEdit", + "hooks": [ + { + "type": "command", + "command": "comment-checker --prompt \"Violations detected:\n\n{{comments}}\"" + } + ] + } + ] + } +} ``` -The default prompt is tuned for precision. Override only with a tested alternative. +### Exit Codes + +`comment-checker` returns deterministic status codes suitable for shell scripts and automated editor integrations: -## Non-goals +| Exit Code | Status | Description | +|---|---|---| +| `0` | **Pass** | No unnecessary comments found, clean payload, or unparseable input (hook never blocks on invalid input) | +| `2` | **Block** | One or more unnecessary comments detected; diagnostic report emitted to stdout | -- **No rewriting.** The hook blocks and reports; it does not edit the code or auto-delete comments. -- **Not a linter.** It judges comments only — not style, naming, or architecture. -- **Rule-based, not learned.** A justified comment matching no justification rule can still be flagged, and a noisy one matching a justification pattern can pass. Edge cases belong in the issue tracker. -- **37 languages.** Files in other languages are skipped and exit 0. +--- ## FAQ -**Q: `command not found: comment-checker` after install.** -A: The binary isn't on your shell's PATH. `cargo install` puts it in `~/.cargo/bin`; a global npm install puts it in the npm global bin directory. Restart the shell or add the directory to PATH, then re-check. +**Q: `command not found: comment-checker` after installation.** +A: Verify that your global package bin directory is included in your shell `PATH` environment variable. For global npm or pnpm installations, you can check active bin paths with `npm bin -g` or `pnpm root -g`. + +**Q: Why are comments in `Edit` or `MultiEdit` tool calls treated differently than `Write`?** +A: When Claude Code uses `Edit` or `MultiEdit`, only the newly added comments in the diff are evaluated; pre-existing comments in the file are ignored. Furthermore, since code fragments may lack surrounding AST context, restatement detection is disabled on fragments to prevent false positives. + +**Q: Does `comment-checker` modify my source files?** +A: No. `comment-checker` is purely diagnostic. It emits a report to stdout and exits with code 2 to inform the agent of the required correction. -**Q: The hook errors with `BinaryNotFound`.** -A: The npm launcher could not find its platform package. This is expected while the npm package is unpublished, or on an unsupported platform (only Linux/macOS x64 + arm64 and Windows x64 exist). Until the first npm release, install with `cargo install --git https://github.com/systemfsoftware/comment-checker --package claude-code-comment-checker`. +**Q: Does `comment-checker` transmit code over the network?** +A: No. The binary runs entirely locally, processes JSON over stdin, and makes no network requests. -**Q: The hook blocked my write. Do I really have to delete the comment?** -A: Only the flagged ones. Each line names the reason, so verify it against the code. If the code is unclear without the comment, make the code self-explanatory — better names, extraction, a clearer type — and don't re-add the comment. +--- -**Q: It flagged a comment that is genuinely useful.** -A: The classifier is rule-based and precision-oriented. Open an issue with the comment and the language; the rule tables are the maintained surface for exactly this feedback. +## Maintenance & Releases -**Q: It didn't flag a comment I expected it to.** -A: Only the five kinds above are flagged. Also, on `Edit` and `MultiEdit`, only newly-added comments are checked — pre-existing comments pass by design, and restatement detection is disabled because the fragment cannot vouch for the surrounding code. +Releases are triggered by semantic tags pushed to `main` (such as `v0.1.0`), which automatically executes [`.github/workflows/release.yml`](.github/workflows/release.yml). The pipeline compiles release binaries across all matrix targets, validates cryptographic checksums, and publishes each platform package followed by the root launcher with npm OIDC provenance. -**Q: Does it send my code anywhere?** -A: No. It is a local binary that reads the hook payload from stdin and writes to stdout — no network, no dynamic loading, no telemetry. +Detailed release specifications, matrix bindings, and trusted publisher instructions are documented in [docs/plans/2026-08-17-001-feat-npm-distribution-release-plan.md](docs/plans/2026-08-17-001-feat-npm-distribution-release-plan.md). + +--- ## Contributing -Development setup and workflow: [AGENTS.md](AGENTS.md). +Development setup, test execution, and mutation testing guidelines are maintained in [AGENTS.md](AGENTS.md). + +--- ## License -Apache License 2.0. See [LICENSE](LICENSE). +Distributed under the [Apache-2.0 License](LICENSE). © [System F Software](https://github.com/systemfsoftware)