RE-CUT: Split the templates out of gauntlet.md into templates.md

demo-rate-limiter/evidence.md

@@ -92,13 +92,17 @@ Status legend: pass / fail / unverified / n-a.
 - Scenario suite alone: **22/22**. The headline mutation score is carried
   entirely by the scenario tests.
 
-## Skipped layers
-
-- Tool-based mutation (mutmut): unverified compatibility with Python 3.14;
-  replaced with the scripted manual procedure (`tools/mutants.py`, 22 mutants).
-- Shell lint (shellcheck) for the four scripts that implement half the gates:
-  **not run**, no tool installed. Every Python file gets three static layers
-  and the shell gets none. Known gap, raised by verification round 4.
+## Layers not run as specified
+
+- **SUBSTITUTED — tool-based mutation (mutmut):** unverified compatibility with
+  Python 3.14; the scripted manual procedure ran instead (`tools/mutants.py`,
+  22 mutants). What it cannot detect: the mutant list is hand-written, so
+  unlike a tool generating mutants from the syntax tree it can only test
+  weaknesses somebody thought of in advance.
+- **UNAVAILABLE — shell lint (shellcheck)** for the four scripts that implement
+  half the gates: no tool installed, and nothing ran in its place. Every Python
+  file gets three static layers and the shell gets none. Known gap, raised by
+  verification round 4.
 
 ## Independent verification
 

skills/old-coder/SKILL.md

@@ -71,7 +71,8 @@ implementation files:
   wrong, say so explicitly and revise it visibly — never silently drift.
 - **Write the spec to a file and name it by absolute path.** A relative path is
   not clickable in a terminal, so the human cannot open the one artifact they
-  are being asked to approve. Same for EVIDENCE when you get there.
+  are being asked to approve. Same for EVIDENCE when you get there. The SPEC
+  and Gherkin templates are in `references/templates.md`.
 
 ### 2. RED — prove each test can fail
 
@@ -175,7 +176,7 @@ such excuse: you chose them, so choose real bugs.
 ### 6. EVIDENCE — the only thing the human reads after code
 
 End with a report the human can trust without opening a single source file
-(template in `references/gauntlet.md`):
+(template in `references/templates.md`):
 
 - The approved spec, with each behavior mapped to the test that verifies it.
 - Each gauntlet layer: the command run, and its actual result (pasted numbers,

skills/old-coder/references/gauntlet.md

@@ -183,77 +183,9 @@ into a vacuous pass. Prove each home-grown check can fail with a one-off
 negative control (feed it a known-bad fixture; make its input unreadable) and
 record the control in EVIDENCE's honest notes.
 
-## Gherkin scenario template (for the SPEC step)
-
-```gherkin
-Feature: <capability in user language>
-  Scenario: <one concrete behavior>
-    Given <concrete starting state>
-    When  <concrete action with concrete input>
-    Then  <concrete observable outcome, exact values>
-
-  Scenario: <the error case>
-    Given ...
-    When  <invalid/hostile input>
-    Then  <exact error type/message/status, and what state must NOT change>
-```
-
-Each scenario maps 1:1 to at least one automated test; name the test after the
-scenario so the evidence report's spec→test mapping is mechanical.
-
-## Evidence report template (for the EVIDENCE step)
-
-```markdown
-## Evidence Report — <task name> (Tier <1|2|3>)
-
-- Spec approval: <obtained from user | not obtained (autonomous run) —
-  confidence downgraded; spec is the artifact to review after the fact>
-- Source state: <commit SHA | no git: sha256 tree hash> — persist the
-  computation as a script (e.g. tools/source_state.sh); a hash recipe written
-  in prose is working-directory-sensitive and will fail to reproduce
-- Toolchain: <pinned versions file, e.g. requirements-dev.txt>
-- Entry point: <single command that reruns every layer>
-- Independent verification: <not performed | passed | failed | blocked>
-  **against the final source state** — a state no verifier saw is
-  `not performed` however many rounds preceded it (Tier 3; protocol in
-  `verifier.md`)
-
-### Spec → Test mapping
-Status is one of: **pass / fail / unverified / n-a**. A row mapped to
-"skipped: <reason>" must carry unverified or n-a — never pass.
-
-| Scenario | Test | Status |
-|---|---|---|
-| <scenario name> | <test file>::<test name> | pass |
-| Must NOT: <negative constraint> | <test / layer / skipped: reason> | pass \| unverified |
+## Templates
 
-### Gauntlet (final fresh run)
-| Layer | Command | Result |
-|---|---|---|
-| Tests | <cmd> | <N> passed, 0 failed |
-| Types | <cmd> | 0 errors |
-| Lint | <cmd> | 0 warnings |
-| Changed-line coverage | <cmd> | <covered>/<total> changed lines (list any misses) |
-| Mutation | <tool or "manual"> | <killed>/<total> killed |
-| Property-based | <cmd> | <N> properties, <examples/property> examples each |
-| Real execution | <cmd> | <observed output> |
-| Supply chain | <cmd> | 0 known vulns; new deps: none (or list, each ↔ SPEC justification) |
-| Suite health | <cmd> | randomized order (seed <n>), all passed |
-
-### Independent verification (never omit; see verifier.md)
-- Verifier: <host / model family>; fresh context; which inputs it received;
-  what correlation that breaks and what it does not.
-- Rounds: <n> (cap <m>); verdict per round, each against the state it saw.
-- Grading: who classified each finding behavioural vs description, and who
-  approved stopping.
-- Attacked: <what was tried, not only what was found>.
-- Findings: behavioural (fixed, then re-verified in a new context) vs
-  description/mapping (fixed and disclosed, no new round).
-- Fixed after the last verified state, therefore unverified: <list | none>.
-
-### Skipped layers
-- <layer>: <reason>  (or "none")
-
-### Honest notes
-- <failures hit during the task and how they were resolved; spec revisions; anything reducing confidence>
-```
+The Gherkin scenario template, the SPEC template, the EVIDENCE report template,
+and the tracker roll-up live in `references/templates.md`. This file is read
+while building the gauntlet; that one is read while writing the two artifacts
+the human reads.

skills/old-coder/references/templates.md

@@ -0,0 +1,195 @@
+# SPEC and EVIDENCE Templates
+
+The fill-in forms for the two artifacts the human actually reads. The layers
+they reference are defined in `gauntlet.md`.
+
+## SPEC template (for the SPEC step)
+
+Written before any implementation file is touched.
+
+```markdown
+# SPEC — <task name>
+
+- Tier: <1|2|3>
+- Issue: <tracker id, or "none"> <!-- optional; no dependency on any tracker -->
+- Setup plan:
+  - Tools to install: <or "none">
+  - Git: <init? checkpoint commit cadence? commit flags the repo mandates>
+  - Files the gauntlet will add, **by path**: `tools/mutants.py` (mutation
+    layer), `tools/gauntlet.sh` (entry point) — mark either "already exists,
+    reused" — plus any fixture or harness file
+  - New dependencies: <each with a one-line justification, or "none">
+
+## Scenarios
+<Gherkin below>
+
+## Must NOT
+- <negative constraint / invariant that must survive>
+
+## Revisions
+- <appended only; each entry says what changed and why>
+```
+
+`## Revisions` is where "the spec is append-only, revise it visibly" stops being
+a promise. Without somewhere to write the revision, a spec that changed
+mid-task and a spec that never changed are the same document, and a reader
+cannot tell which one they are holding.
+
+Commit `SPEC.md` at approval where the repo's git conventions allow it — the
+setup plan is where that was authorized. Once the approved spec is a commit,
+later drift is literally a `git diff`. Without a durable spec, a compaction
+loses the approved contract while the code it authorized remains, and nobody
+can check whether a scenario was quietly dropped from the EVIDENCE mapping.
+
+**If the human rejects the spec**, revise `SPEC.md` in place, add the reason to
+`## Revisions`, and re-request approval. Do not delete it and start clean —
+what the human turned down, and why, is the most useful thing in the file.
+
+### Gherkin scenario template
+
+```gherkin
+Feature: <capability in user language>
+  Scenario: <one concrete behavior>
+    Given <concrete starting state>
+    When  <concrete action with concrete input>
+    Then  <concrete observable outcome, exact values>
+
+  Scenario: <the error case>
+    Given ...
+    When  <invalid/hostile input>
+    Then  <exact error type/message/status, and what state must NOT change>
+```
+
+Each scenario maps 1:1 to at least one automated test; name the test after the
+scenario so the evidence report's spec→test mapping is mechanical.
+
+## Evidence report template (for the EVIDENCE step)
+
+```markdown
+## Evidence Report — <task name> (Tier <1|2|3>)
+
+- Spec approval: <obtained from user | not obtained (autonomous run) —
+  confidence downgraded; spec is the artifact to review after the fact>
+- Source state: <commit SHA | no git: sha256 tree hash> — persist the
+  computation as a script (e.g. tools/source_state.sh); a hash recipe written
+  in prose is working-directory-sensitive and will fail to reproduce
+- Toolchain: <pinned versions file, e.g. requirements-dev.txt>
+- Entry point: <single command that reruns every layer>
+- Independent verification: <not performed | passed | failed | blocked>
+  **against the final source state** — a state no verifier saw is
+  `not performed` however many rounds preceded it (Tier 3; protocol in
+  `verifier.md`)
+- Tracker: <issue id — roll-up posted | roll-up written to ROLLUP.md, not
+  posted (nobody present to approve it) | none: SPEC named no issue>
+
+### Spec → Test mapping
+Status is one of: **pass / fail / unverified / n-a**. A row mapped to
+"skipped: <reason>" must carry unverified or n-a — never pass.
+
+| Scenario | Test | Status |
+|---|---|---|
+| <scenario name> | <test file>::<test name> | pass |
+| Must NOT: <negative constraint> | <test / layer / skipped: reason> | pass \| unverified |
+
+### Gauntlet (final fresh run)
+| Layer | Command | Result |
+|---|---|---|
+| Tests | <cmd> | <N> passed, 0 failed |
+| Types | <cmd> | 0 errors |
+| Lint | <cmd> | 0 warnings |
+| Changed-line coverage | <cmd> | <covered>/<total> changed lines (list any misses) |
+| Mutation | <tool or "manual"> | <killed>/<total> killed |
+| Property-based | <cmd> | <N> properties, <examples/property> examples each |
+| Real execution | <cmd> | <observed output> |
+| Supply chain | <cmd> | 0 known vulns; new deps: none (or list, each ↔ SPEC justification) |
+| Suite health | <cmd> | randomized order (seed <n>), all passed |
+
+### Independent verification (never omit; see verifier.md)
+- Verifier: <host / model family>; fresh context; which inputs it received;
+  what correlation that breaks and what it does not.
+- Rounds: <n> (cap <m>); verdict per round, each against the state it saw.
+- Grading: who classified each finding behavioural vs description, and who
+  approved stopping.
+- Attacked: <what was tried, not only what was found>.
+- Findings: behavioural (fixed, then re-verified in a new context) vs
+  description/mapping (fixed and disclosed, no new round).
+- Fixed after the last verified state, therefore unverified: <list | none>.
+
+### Layers not run as specified
+Split by status, because they mean different things to a reader:
+- **N-A (this project has no such surface):** <layer — why it does not exist here>
+- **UNAVAILABLE (tool missing):** <layer — which tool, nothing run in its place>
+- **SUBSTITUTED:** <layer — what ran instead, and what that cannot detect>
+- (or "none")
+
+### Dismissed review findings
+Fixes are self-evidencing; dismissals are not. One line each:
+- <finding> — dismissed because <the command / file:line / test that disproves
+  it>. <If the argument is "no alternative exists": which call sites it covers,
+  and which it does not.>
+- (or "none — every finding was fixed or accepted as a known limit")
+
+### Structural blind spot
+- <the layer this project cannot run at all, e.g. "the suite never exercises the
+  container runtime, so nothing here is evidence about deployment behavior">
+
+### Honest notes
+- <failures hit during the task and how they were resolved; spec revisions; anything reducing confidence>
+```
+
+**Why "layers not run" is split three ways.** One "skipped" list collapses
+three states a reader has to tell apart: there is no such surface in this
+project, versus the surface exists but the tool was missing and nothing ran,
+versus something else ran and here is what it cannot detect. Those are very
+different confidence claims and they read identically as "skipped". The third
+is the dangerous one: `SUBSTITUTED` may never be written as a pass. Two repeat
+runs in place of randomized order is not "suite health: stable" — it is
+`SUBSTITUTED (2 repeat runs — cannot detect whole-suite order dependence)`. A
+reader who cannot tell a substitute from the real layer reads "found nothing"
+where the truth is "did not look with that instrument". `N-A` is not a
+degraded run at all: three `N-A` layers describe the project, and EVIDENCE
+should say so rather than leaving a reader to count absences.
+
+**Why dismissals need a line each.** A fix carries its own evidence — the test
+that now passes. A dismissal carries none: "not a real problem" is
+indistinguishable from "did not check". Naming the command, `file:line`, or
+test that disproves the finding is the same rule the verification protocol
+already applies to attack lists — say what you tried, not only what you found.
+
+**Why name the blind spot.** A layer a project cannot run at all otherwise
+reads as absent rather than accepted. Stating it converts a silent gap into a
+known limit the reader can price in.
+
+## Tracker roll-up (only when the SPEC names an issue)
+
+Tracker linkage is **bidirectional and tracker-agnostic**: the SPEC header
+carries an issue ID, and on completion a short roll-up goes back to that issue.
+No hard dependency on any particular tracker — if the SPEC's `Issue` field says
+`none`, this step does not exist.
+
+Write it to `ROLLUP.md` alongside the spec and evidence files. Post it to the
+issue only if an approver says so; with nobody present, leave it on disk and
+say so in EVIDENCE. A hosted tracker notifies people and cannot be un-sent, so
+it gets the same gate as a commit.
+
+```markdown
+- Built: <one or two lines — what now exists that did not before>
+- Left undone: <deliberate omissions, and why they were deliberate>
+- Traps for the next task: <the thing that will bite whoever picks this up>
+- Evidence: <path to SPEC.md, EVIDENCE.md, logs/>
+```
+
+**Why it is short, and why it is not a copy of EVIDENCE.** The two serve
+different readers, and keeping them distinct is what stops them drifting into
+rival sources of truth:
+
+| | Reader | Obligation |
+|---|---|---|
+| EVIDENCE | the human reviewing **this** change | complete — every layer, every number, every limit |
+| Tracker roll-up | whoever picks up the **next** task | short — what changed the ground under them, and where to find the rest |
+
+A tracker whose notes are append-only by API gives the spec's no-silent-drift
+property for free: an earlier note cannot be quietly rewritten to match a later
+story. Where the tracker permits editing, that property is not there and the
+git-commit-at-SPEC mechanism remains the enforcement — the roll-up is a
+convenience for the next reader, never the authoritative record.