Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 26 additions & 6 deletions README-zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,17 +14,36 @@ skill 就是纯 markdown,任何能遵循指令的 coding agent 都能用:Cla

## 安装

安装 `old-coder`:

```sh
npx skills add https://github.com/amazingang/old-coder
npx skills add https://github.com/amazingang/old-coder --skill old-coder
```

也可以手动安装:

- **Claude Code**——把 skill 拷进 skills 文件夹,然后用 `/old-coder` 调用,或在"证明它能用"这类请求时让它自动触发
- **Claude Code**——把 skill 拷进 skills 文件夹,然后用 `/old-coder` 调用,或让它在高可靠性任务中自动触发
```sh
cp -r skills/old-coder ~/.claude/skills/ # 或 <project>/.claude/skills/
cp -r skills/old-coder ~/.claude/skills/
# 或拷贝到 <project>/.claude/skills/
```
- **其他 agent**——把 `skills/old-coder/SKILL.md` 加进你的 `AGENTS.md`、规则文件或 system prompt,并把 `references/gauntlet.md` 放在旁边备查。
- **其他 agent**——把 `skills/old-coder/SKILL.md` 加进你的 `AGENTS.md`、规则文件或 system prompt,并将它的 `references/` 目录放在旁边备查。

### 可选配套 skill:`old-coder-api`

仓库还包含一个专门用于 HTTP/JSON API 设计与评审的 skill,在需要兼容性、授权、幂等、分页、限流与可运维性闸门时安装:

```sh
npx skills add https://github.com/amazingang/old-coder --skill old-coder-api
```

同时安装两个 skill:

```sh
npx skills add https://github.com/amazingang/old-coder --skill old-coder --skill old-coder-api
```

两者同时适用时,`old-coder` 负责流程、批准与证据,`old-coder-api` 负责 API 契约;API 闸门结论进入 SPEC,并由 gauntlet 验证。



Expand Down Expand Up @@ -84,8 +103,9 @@ agent 是在给自己的作业打分,所以规则很严:不许为通过而
## 仓库里有什么

```
skills/old-coder/ skill 本体(SKILL.md + references/gauntlet.md)
demo-rate-limiter/ 按此 skill 端到端做出来的限流器示例
skills/old-coder/ 可靠编码流程(SKILL.md + references/)
skills/old-coder-api/ HTTP/JSON API 设计与评审(SKILL.md + references/)
demo-rate-limiter/ 按 old-coder 端到端做出来的限流器示例
```

demo 的 `evidence.md` 就是重点:41 个测试、100% 覆盖率(49/49 个语句、20/20 个分支),22/22 个埋入的 bug 全部被抓。更重要的是,对此前绿色状态进行的 fresh-context verification 仍发现了真实的行为缺陷和一个不可靠的 mutation runner——这恰好说明,关卡全绿并不能自证其可信。当前报告同时披露了修复情况和最终源码状态的验证状态。整份报告可以重跑:
Expand Down
34 changes: 27 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,23 +8,42 @@

**An old coder's strategy for the agent era: don't read the code — make it run the gauntlet.**

A skill that makes coding agents **prove their work**. Instead of you reading every line the agent writes, the agent must push its code through a gauntlet of checks and hand you a test plan before coding and an evidence report after. You review those two documents, not the code.
A skill that makes coding agents **prove their work**. Instead of you reading every line, the agent pushes its code through a gauntlet of checks and hands you a test plan before coding and an evidence report after. You review those two documents, not the code.

It's plain markdown, so it works with any coding agent that follows instructions: Claude Code, Codex CLI, Cursor, Aider, or your own agent loop.

## Installation

Install `old-coder`:

```sh
npx skills add https://github.com/amazingang/old-coder
npx skills add https://github.com/amazingang/old-coder --skill old-coder
```

Or manually:

- **Claude Code** — copy the skill into a skills folder, then invoke `/old-coder` or let it trigger on "prove it works"-style requests:
- **Claude Code** — copy the skill into a skills folder, then invoke `/old-coder` or let it trigger on high-assurance requests:
```sh
cp -r skills/old-coder ~/.claude/skills/ # or <project>/.claude/skills/
cp -r skills/old-coder ~/.claude/skills/
# or copy it to <project>/.claude/skills/
```
- **Other agents** — add `skills/old-coder/SKILL.md` to your `AGENTS.md`, rules file, or system prompt, and keep `references/gauntlet.md` alongside it.
- **Other agents** — add `skills/old-coder/SKILL.md` to your `AGENTS.md`, rules file, or system prompt, and keep its `references/` directory alongside it.

### Optional companion: `old-coder-api`

This repository also includes a focused HTTP/JSON API design and review skill. Install it when you want compatibility, authorization, idempotency, pagination, rate-limit, and operability gates:

```sh
npx skills add https://github.com/amazingang/old-coder --skill old-coder-api
```

To install both skills:

```sh
npx skills add https://github.com/amazingang/old-coder --skill old-coder --skill old-coder-api
```

When both apply, `old-coder` owns workflow, approval, and evidence; `old-coder-api` owns the API contract, and its gate decisions become SPEC constraints and gauntlet checks.

## The idea

Expand Down Expand Up @@ -79,8 +98,9 @@ And one limit stated plainly: the gauntlet turns the constraints expressed in th
## What's in the repo

```
skills/old-coder/ the skill (SKILL.md + references/gauntlet.md)
demo-rate-limiter/ a rate limiter built end-to-end under the skill
skills/old-coder/ reliable coding workflow (SKILL.md + references/)
skills/old-coder-api/ HTTP/JSON API design and review (SKILL.md + references/)
demo-rate-limiter/ a rate limiter built end-to-end under old-coder
```

The demo's `evidence.md` is the point of the exercise: 41 tests, 100% coverage (49/49 statements and 20/20 branches), and 22/22 planted bugs caught. More importantly, fresh-context verification of earlier green states still found real behavioral defects and an unsound mutation runner — evidence that a green gauntlet is not self-authenticating. The current report discloses both the fixes and the final state's verification status. Rerun the whole report:
Expand Down
62 changes: 40 additions & 22 deletions demo-rate-limiter/evidence.md
Original file line number Diff line number Diff line change
@@ -1,34 +1,34 @@
# Evidence Report — Sliding-Window Rate Limiter (Tier 3)

- Spec approval: **obtained** for REVISION 4 (2026-08-09) — the human approved
each contract change item by item before implementation. Earlier revisions
(2026-07-25, 2026-07-27) were autonomous and are still unapproved; treat
them as the weaker part of the spec.
- Spec approval: **obtained** for REVISION 4 (2026-08-09) and REVISION 5
(2026-08-18) — the human approved each contract change before
implementation. Earlier revisions (2026-07-25, 2026-07-27) were autonomous
and are still unapproved; treat them as the weaker part of the spec.
- Independent verification: **not performed against the final source state
`8b88bda`.** Six earlier rounds were performed; the last verified state
`d45cc2f`.** Six earlier rounds were performed; the last verified state
`d0b506c` returned `failed`, and the fixes made since — one of them
behavioural — are disclosed below as unverified. This report is finalized as
a **declared downgrade**, not on the strength of a passing verdict. A
verdict attaches to the state a verifier actually saw, and no verifier has
seen this one.
- Source state: git commit `8b88bda`; sha256 tree hash `c80e8cccf0a1ed3a` —
reproduce both with `./tools/source_state.sh` (works from any directory;
now includes `.github/workflows`, which decides whether the gauntlet runs
in CI at all). Commits after `8b88bda` touch only paths outside the hashed
tree — `skills/`, the READMEs, `CONTRIBUTING.md`, and this report itself —
hence the same hash at a later HEAD, not a stale binding.
- Source state: source commit `d45cc2f`; sha256 tree hash
`76389992f4e342e2` — reproduce both with `./tools/source_state.sh` from any
directory. The script separately reports current HEAD; commits after
`d45cc2f` that touch only this report or other out-of-scope paths preserve
the source commit and tree hash. The manifest includes `.github/workflows`,
which decides whether the gauntlet runs in CI at all.
- Toolchain: pinned in `requirements-dev.txt` (local run: Python 3.14.3;
CI runs the same gauntlet on 3.12 via `.github/workflows/gauntlet.yml`).
- Entry point: `./tools/gauntlet.sh` reruns every layer below.

All numbers are from one final fresh run of the entry point, executed
2026-08-10 after the last code edit.
2026-08-18 at source commit `d45cc2f` after the last code edit.

`spec.md` was deliberately pruned back to a contract afterwards (339 → 255
lines). Every clause, invariant, obligation and failure-model row survives;
what was removed is the per-revision forensics, which lives in the honest
notes below and in git. The spec is the artifact a human reads before any
code exists, and it had stopped being readable as one.
`spec.md` was deliberately pruned back to a contract before REVISION 5
(339 → 255 lines). Every clause, invariant, obligation and failure-model row
survived; what was removed is the per-revision forensics, which lives in the
honest notes below and in git. REVISION 5 adds the approved source-binding
contract and tests without changing rate-limiter behaviour.

## Spec → Test mapping

Expand Down Expand Up @@ -64,24 +64,27 @@ Status legend: pass / fail / unverified / n-a.
| Must NOT: denials store nothing (no memory growth) | test_ratelimiter.py::test_must_not_denials_store_nothing + M8 | pass |
| Must NOT: the limiter is never driven by a real clock | layer: must-not scan in `tools/gauntlet.sh` over tests/ → no matches | pass |
| failure-model row: allow() is atomic | test_ratelimiter.py::test_allow_is_atomic_a_second_caller_cannot_interleave + M13 | pass |
| REVISION 5: source binding is reproducible and fail-closed | test_source_state.py (ignored artifacts, staged/unstaged/untracked/deleted inputs, clean clone, no-Git archive, arbitrary cwd, evidence-only commit) | pass |

## Gauntlet (final fresh run: `./tools/gauntlet.sh`)

| Layer | Command | Result |
|---|---|---|
| Checker self-test | `sh tools/test_gauntlet_checks.sh` (first layer; asserts the must-not scan fails on a planted pattern, passes on a clean tree, and fails closed with a distinct rc 2 when the scan itself breaks) | 3/3 expectations ok |
| Source-state self-test | `pytest -q tests/test_source_state.py` (negative controls for ambient ignored artifacts and every fail-closed branch; clean clone and no-Git archive comparison) | 6/6 passed |
| Mutation harness negative control | `python tools/mutants.py --negative-control` (a killer and a strictly-equivalent mutant of identical size under one pinned mtime) | C1 KILLED, C2 SURVIVED — ok |
| Tests | `pytest -q --cov=ratelimiter` | 41 passed, 0 failed |
| Types | `mypy src tests examples tools` (strict) | 0 errors in 6 files |
| Lint + format + complexity | `ruff check . && ruff format --check .` (mccabe ≤ 8) | 0 warnings, 8 files formatted |
| Tests | `pytest -q --cov=ratelimiter` | 47 passed, 0 failed |
| Types | `mypy src tests examples tools` (strict) | 0 errors in 8 files |
| Lint + format + complexity | `ruff check . && ruff format --check .` (mccabe ≤ 8) | 0 warnings, 10 files formatted |
| Changed-line coverage | `pytest --cov … --cov-fail-under=100` | 49/49 statements, 20/20 branches (100%). **This layer is a gate**; before 2026-08-09 it printed a percentage and exited 0 no matter how far coverage fell |
| Mutation | `python tools/mutants.py` (manual, scripted; only pytest exit 1 counts as a kill; `__pycache__` cleared and `PYTHONDONTWRITEBYTECODE` set per mutant) | 22/22 killed |
| Property-based | hypothesis, 2 properties | 100 examples each, 0 falsified |
| Real execution | `python examples/demo.py` (real `time.monotonic`) | burst of 5 → `[True, True, True, False, False]`; other key unaffected; allowed again after window |
| Supply chain | `pip-audit -r requirements-dev.txt` | no known vulnerabilities; runtime dependencies: **none** (stdlib only; `threading` is stdlib) |
| Secret scan | must-not scan in `tools/gauntlet.sh` over src, tests, tools, examples, spec.md, pyproject.toml, requirements-dev.txt and `../.github` | clean, no matches |
| Source binding | `tools/source_state.sh` (last gauntlet layer) | HEAD/source commit `d45cc2f`; tree `76389992f4e342e2` |
| License check | — | n-a: zero runtime dependencies, nothing redistributed beyond this repo's own MIT code |
| Suite health | pytest-randomly (order shuffled every run) | 41 passed in randomized order, 10/10 consecutive runs |
| Suite health | pytest-randomly (order shuffled every run) | 47 passed in randomized order, 10/10 consecutive runs |

## Layer attribution

Expand Down Expand Up @@ -148,10 +151,25 @@ independently verified**:
- the six prose corrections listed in commit `66df5cd`;
- the prune of `spec.md` from 339 to 255 lines in commit `8b88bda`. No clause
was changed, but it is a large edit to the document a verifier attacks
hardest, and it was made after the last verified state.
hardest, and it was made after the last verified state;
- REVISION 5 and the reproducible, fail-closed source-state mechanism in
commits `86bfcf4` and `d45cc2f`.

## Honest notes

- **The previous source binding was invalid.** The reported tree
`c80e8cccf0a1ed3a` included four Git-ignored `*.egg-info` files created by
`pip install -e .`; a clean checkout at the cited commit instead produced
`939188446f61289c`. Because those generated files can also vary with the
setuptools version, neither hash was a trustworthy Git-source binding. The
old `find | sort | xargs | shasum` pipeline could additionally print success
after a missing input. REVISION 5 replaces it with a canonical tracked-file
manifest, explicit dirty/untracked rejection, structured path+content
hashing, a deterministic no-Git fallback, six negative controls, and a
second clean-state check around hashing. The corrected source binding is
`d45cc2f` / `76389992f4e342e2`; it is gauntlet-tested but not independently
verified.

- **The A/B experiment that started this failed.** The design was to plant a
defect in one copy and verify a clean copy as a false-positive control. The
"clean" arm was not clean: it independently invented the exact mutation that
Expand Down
41 changes: 41 additions & 0 deletions demo-rate-limiter/spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -245,6 +245,47 @@ table. A row whose catcher cannot be shown to fail is a defect, not a mapping.
the gap is visible rather than absent.
- **Distributed / multi-process limiting.** In-process state only.

## REVISION 5 — reproducible source-state binding (Tier 3)

Approved 2026-08-18. This revision repairs the evidence mechanism; it does
not change rate-limiter runtime behaviour or its public API.

### Behaviour

- In a Git checkout, `tools/source_state.sh` hashes only version-controlled
files in the declared source scope. Ignored build products such as
`*.egg-info`, bytecode caches and coverage output cannot change the hash.
- The same tracked content produces the same tree hash in the working tree, a
clean checkout and the no-Git archive fallback, regardless of current
working directory.
- In Git, relevant staged changes, unstaged changes, deletions or non-ignored
untracked files make the command fail closed instead of emitting a binding.
- The command reports both current HEAD and the most recent commit that
changed the source scope. A later evidence-only commit may change HEAD while
preserving the source commit and tree hash.
- Missing or unreadable manifest inputs make the command fail non-zero; no
partial hash may be reported.
- The gauntlet runs a negative-control self-test for these properties and then
emits the source-state binding only after every other layer has passed.

### Must NOT do

- Do not derive a Git binding from ambient ignored files on disk.
- Do not silently omit a new, non-ignored file inside the source scope.
- Do not use a hashing pipeline whose intermediate read failure can be hidden
by the exit status of its final command.
- Do not add a runtime or development dependency for this repair.

### Setup plan

- Modify `tools/source_state.sh`; add its implementation and regression tests
under `tools/` and `tests/`; connect the self-test and binding to
`tools/gauntlet.sh`; clarify the reusable rule in the old-coder evidence
template; update `evidence.md` after the implementation commit is clean.
- Commit cadence: this approved SPEC first; tests plus implementation second;
evidence rebinding third. Independent verification remains `not performed`
unless a separate verifier actually inspects the final source state.

## Revision history

Revisions 1–3 (2026-07-25 → 07-27) were made autonomously during the original
Expand Down
Loading
Loading