The gauntlet applies its two strongest layers to src/ratelimiter only:
| Layer |
Covers tools/ |
types (mypy src tests examples tools) |
yes |
lint (ruff check .) |
yes |
changed-line coverage (--cov=ratelimiter --cov-fail-under=100) |
no |
mutation (TARGET = src/ratelimiter/__init__.py) |
no |
That boundary — "src is the subject, tools is the instrument" — was coherent
while tools/ held only the harness. It is less coherent now that
tools/source_state.py is what certifies every other number in evidence.md.
A silent fault there does not fail a layer; it mislabels the evidence that all
the layers report into. REVISION 6 is one instance: a real provenance defect
that shipped, ran green in CI twice, and was found by hand rather than by a
layer.
The mutation layer exists because tests that assert nothing still pass. The
REVISION 5 source-state controls were exactly that shape — three asserted only
a non-zero exit, and the wrapper returns the same status when the
implementation is missing entirely. They were repaired by review, which is the
mechanism this project is trying to make unnecessary.
This issue is to decide the boundary, not to widen it. Specifically not
proposing "100% coverage for everything under tools/": most of that code is
scaffolding where the cost is real and the payoff is not.
Worth deciding:
- Which
tools/ paths are load-bearing for the evidence — source_state.py is
the clear candidate; mutants.py and must_not_match.sh are arguable, since
each already carries its own negative control or self-test.
- Whether targeted mutants are worth writing for those paths, and which ones
would have caught a defect that actually occurred, rather than mutants chosen
because they are easy to generate.
- Whether
mutants.py should support more than one target at all, given
TARGET is currently a single module-level constant.
- Whether the coverage gate should extend, or whether the negative controls are
the right instrument for tooling and coverage is the wrong one.
Deferred deliberately from the REVISION 6 repair (#12 follow-up) so a boundary
decision is made on its own rather than carried along by a technical fix.
The gauntlet applies its two strongest layers to
src/ratelimiteronly:tools/mypy src tests examples tools)ruff check .)--cov=ratelimiter --cov-fail-under=100)TARGET = src/ratelimiter/__init__.py)That boundary — "
srcis the subject,toolsis the instrument" — was coherentwhile
tools/held only the harness. It is less coherent now thattools/source_state.pyis what certifies every other number inevidence.md.A silent fault there does not fail a layer; it mislabels the evidence that all
the layers report into. REVISION 6 is one instance: a real provenance defect
that shipped, ran green in CI twice, and was found by hand rather than by a
layer.
The mutation layer exists because tests that assert nothing still pass. The
REVISION 5 source-state controls were exactly that shape — three asserted only
a non-zero exit, and the wrapper returns the same status when the
implementation is missing entirely. They were repaired by review, which is the
mechanism this project is trying to make unnecessary.
This issue is to decide the boundary, not to widen it. Specifically not
proposing "100% coverage for everything under
tools/": most of that code isscaffolding where the cost is real and the payoff is not.
Worth deciding:
tools/paths are load-bearing for the evidence —source_state.pyisthe clear candidate;
mutants.pyandmust_not_match.share arguable, sinceeach already carries its own negative control or self-test.
would have caught a defect that actually occurred, rather than mutants chosen
because they are easy to generate.
mutants.pyshould support more than one target at all, givenTARGETis currently a single module-level constant.the right instrument for tooling and coverage is the wrong one.
Deferred deliberately from the REVISION 6 repair (#12 follow-up) so a boundary
decision is made on its own rather than carried along by a technical fix.