ROADMAP.md already records the static half of this as a known non-goal, with
its reason: detecting that an installed rule never fires needs firing counts from
the tiers themselves, and needs "clean tree" separated from "dead rule" before
the signal is worth anything. That entry is not being re-filed.
This is the other half, which is recorded nowhere: the same defect in the
review tier, where it is both harder to see and cheaper to introduce.
Why the review tier is worse off than the static one
A static rule that has been switched off leaves a trace. It is absent from a
config file, and uphold check reconciles claims against configuration and
refuses a claim whose rule is no longer there. The failure is loud by
construction, and that loudness is the point of policy/upheld.toml existing at
all.
A review_questions entry that no review has ever acted on leaves no trace of
any kind:
- it is textually present, so nothing looks missing
- it passes
--check, because the compiled document is generated from the
record and is therefore never stale -- staleness is the only failure mode the
emit gate can see, and this is not one
- it is indistinguishable from a question guarding a clean tree, which is the
same ambiguity the roadmap names, arriving here without even a config file to
inspect
There is no reconcile for the review tier because there is nothing on the other
side of it to reconcile against. upheld.toml names a rule; nothing names a
question that worked.
And here it displaces, rather than merely sitting there
The static case is inert: a dead rule costs a line in a config file. The review
case is not, because [review] max_lines is a hard ceiling that fails the build.
Dead weight in a length-capped document is not neutral. It consumes budget the
ceiling then denies to something a reviewer would have used. The ceiling was set
to stop the tier degrading into guidance long enough to be skimmed, and it does
that job honestly -- but it cannot tell which 900 lines were worth keeping, and
right now neither can anyone else.
What is decidable, and what is not
Not proposing model telemetry, and not proposing a ranking heuristic --
scripts/review.py already refuses the latter for length, on the ground that a
reader cannot tell a short document from a truncated one, and the same objection
applies to a scoreboard that quietly drops the tail.
Something narrower is observable: the record a finding cites. The compiled
document is already organised by record, each block headed by the id, so a
reviewer filing a finding can name the record it came from at no extra cost --
and a record with zero citations across N reviews is a candidate for having said
nothing anyone could use.
The limit, stated rather than papered over, is the roadmap's own: zero citations
is not zero applicability. A record can be silent because nothing violated it.
Separating the two needs a denominator -- how many reviewed changes did this
record's applies_when actually describe -- and that is a second measurement,
not a free one. Until it exists the count is a hint, and must be reported as a
hint.
Why this is worth an issue now, and not later
#64 proposes emitting one unit per record instead of one document. That converts
this from a hint into a requirement.
A record inside the compiled document has, at minimum, the property that it was
read: everything under the ceiling reached the reviewer. A conditionally-loaded
unit does not have that property. It can fail to be selected, which is a
failure one step earlier than failing to fire, and it leaves even less
evidence -- no citation, and no record that it was ever considered and rejected.
So #64 should not land before there is some answer here, or the tier acquires a
silent failure mode in exchange for removing a loud one.
ROADMAP.mdalready records the static half of this as a known non-goal, withits reason: detecting that an installed rule never fires needs firing counts from
the tiers themselves, and needs "clean tree" separated from "dead rule" before
the signal is worth anything. That entry is not being re-filed.
This is the other half, which is recorded nowhere: the same defect in the
review tier, where it is both harder to see and cheaper to introduce.
Why the review tier is worse off than the static one
A static rule that has been switched off leaves a trace. It is absent from a
config file, and
uphold checkreconciles claims against configuration andrefuses a claim whose rule is no longer there. The failure is loud by
construction, and that loudness is the point of
policy/upheld.tomlexisting atall.
A
review_questionsentry that no review has ever acted on leaves no trace ofany kind:
--check, because the compiled document is generated from therecord and is therefore never stale -- staleness is the only failure mode the
emit gate can see, and this is not one
same ambiguity the roadmap names, arriving here without even a config file to
inspect
There is no reconcile for the review tier because there is nothing on the other
side of it to reconcile against.
upheld.tomlnames a rule; nothing names aquestion that worked.
And here it displaces, rather than merely sitting there
The static case is inert: a dead rule costs a line in a config file. The review
case is not, because
[review] max_linesis a hard ceiling that fails the build.Dead weight in a length-capped document is not neutral. It consumes budget the
ceiling then denies to something a reviewer would have used. The ceiling was set
to stop the tier degrading into guidance long enough to be skimmed, and it does
that job honestly -- but it cannot tell which 900 lines were worth keeping, and
right now neither can anyone else.
What is decidable, and what is not
Not proposing model telemetry, and not proposing a ranking heuristic --
scripts/review.pyalready refuses the latter for length, on the ground that areader cannot tell a short document from a truncated one, and the same objection
applies to a scoreboard that quietly drops the tail.
Something narrower is observable: the record a finding cites. The compiled
document is already organised by record, each block headed by the id, so a
reviewer filing a finding can name the record it came from at no extra cost --
and a record with zero citations across N reviews is a candidate for having said
nothing anyone could use.
The limit, stated rather than papered over, is the roadmap's own: zero citations
is not zero applicability. A record can be silent because nothing violated it.
Separating the two needs a denominator -- how many reviewed changes did this
record's
applies_whenactually describe -- and that is a second measurement,not a free one. Until it exists the count is a hint, and must be reported as a
hint.
Why this is worth an issue now, and not later
#64 proposes emitting one unit per record instead of one document. That converts
this from a hint into a requirement.
A record inside the compiled document has, at minimum, the property that it was
read: everything under the ceiling reached the reviewer. A conditionally-loaded
unit does not have that property. It can fail to be selected, which is a
failure one step earlier than failing to fire, and it leaves even less
evidence -- no citation, and no record that it was ever considered and rejected.
So #64 should not land before there is some answer here, or the tier acquires a
silent failure mode in exchange for removing a loud one.