Fail closed when a gauntlet layer never runs

[AmazingAng]

Closes #13.

The defect

tools/gauntlet.sh printed a layer heading and ran that layer's command on two separate lines. Delete the command, keep the heading, commit, and on a clean tree the entry point printed the heading, did no work for the layer, exited zero and announced all layers green. Reproduced by deleting the committed mutation invocation: no mutant ran, and the gauntlet still reported success. Because evidence.md is written by hand, the previous run's 22/22 would then be copied forward.

This is the failure class the repo exists to eliminate — a mechanism reporting success while doing nothing — sitting in the orchestration layer that every other layer's result is reported through. Nothing guarded it: of the six scripts in tools/, four already carried negative controls and one is a six-line wrapper. gauntlet.sh was the one with none.

The repair

tools/gauntlet_layers.sh holds a fixed expected-layer manifest, an execution wrapper and a final audit:

• run_layer prints the heading and runs the command in one call, so the two cannot be separated by deleting a line.
• A layer is recorded complete only after its commands succeed. A failing command preserves its status, names its layer, and stops later work.
• Unknown and duplicate layer names fail, so a typo cannot invent a layer and a cheap layer cannot be run twice to stand in for a missing one.
• all layers green is emitted only by the final audit, after every expected layer completed exactly once.

The manifest and the invocations live in different files, so an accidental one-file deletion is caught. A coordinated edit to both is still possible; that residual is stated in evidence.md rather than papered over.

Assurance boundary

Recorded in spec.md REVISION 7 and generalised into skills/old-coder/references/gauntlet.md:

Application coverage and mutation target the subject under test. Do not widen them across orchestration scripts by default. Protect home-grown tools in the gauntlet's trust chain with targeted negative controls for identified fail-open modes, and pin the failure reason, not merely a non-zero status. A control proves only its named known-bad case, not the whole tool.

Coverage and mutation were deliberately not extended to tools/. The decisive argument is in evidence.md: the shallow-history defect fixed in REVISION 6 would have survived 100% coverage and a full mutation run, because it was a missing scenario, not an unkilled mutant. The control that found it was a negative control.

Controls

tools/test_gauntlet_orchestration.sh, five scenarios / 13 expectations, run as the gauntlet's first layer and therefore accounted for by the mechanism it tests:

1. omitted layer → final audit fails, names it, no green
2. failing command → exact status preserved, layer named, later work stopped, no green — explicitly guarding against relying on set -e through a conditional context
3. unknown layer name → fails
4. duplicate layer → fails
5. complete manifest → reaches green, so the controls are not a script that can only fail

The RED step was performed against a deliberately fail-open helper: 8 of 13 expectations failed, including an omitted mutation layer returning zero and printing green.

Verification

• End-to-end regression against the original defect: layer deleted, committed, clean tree → rc=1, FAIL: missing layer 'mutation', all layers green absent.
• Fail-open mutation battery against gauntlet_layers.sh (always-zero audit, skip the missing check, record failed layers as complete, drop the duplicate guard, drop the unknown guard, swallow the exit status): 6/6 turn the control red.
• Full gauntlet green at 42528d9; binding 42528d9 / 5aa96ec5487c957c reproduces from a clean clone.
• The 50-test randomized suite was rerun 10/10 at this source state rather than inherited.

Commits

905e989	SPEC REVISION 7
5b8dc1b	implementation, controls, skill guidance
9f4fae3	evidence rebind
42528d9	review fix: scenario 5 captured stderr outside the substitution
5c9d48a	evidence rebind

Two hardening items found in review and deliberately deferred are in #15: unprefixed shell globals in gauntlet_layers.sh (verified unreachable today), and the absence of a control for the plural half of "names every missing layer".