Skip to content

docs: re-base the policy audit matrix on the 44-policy corpus - #310

Merged
piyushmakhija28 merged 2 commits into
mainfrom
fix/policy-audit-matrix-consolidation
Aug 10, 2026
Merged

piyushmakhija28 merged 2 commits into
mainfrom
fix/policy-audit-matrix-consolidation

Conversation

@piyushmakhija28

Copy link
Copy Markdown
Collaborator

Why

PR #308 consolidated three session policies into one, taking docs/policies/ from 46 files to 44. It was docs-only, so CI's path filter skipped the test matrix and the breakage landed on main unnoticed — surfacing on the next unrelated PR (#309).

scripts/verify_policy_audit_matrix.py AC1 asserts row-set identity between the matrix and the live corpus:

FAIL: in corpus but missing from matrix: session-management-policy.md
FAIL: in matrix but absent from corpus: session-chaining-policy.md
FAIL: in matrix but absent from corpus: session-memory-policy.md
FAIL: in matrix but absent from corpus: session-pruning-policy.md

What changed

Rows 34/35/36 collapse into one row for session-management-policy.md, and every derived count that AC6/AC7 recompute was updated with it:

before after
rows 46 44
verification split 9 MEASURED / 37 CITED 8 / 36
delete disposition 11 9
CONTRADICTED 8 6
Rule 1 keep-as-is 18 (12E/6P) 17 (12E/5P)
OAQ-2 provenance 15 13
orphans 14 of 46 12 of 44

Rows 37-46 renumbered to 35-44 with ~15 cross-references updated — including row 40's evidence, which said "shares row 41's registration" and would otherwise have pointed at the wrong policy.

The historical findings are preserved, not erased

The three retired rows recorded real, measured defects. Consolidation does not make them false. New §1.4 transcribes all three verbatim (status, plan, verification, finding) and records that V2-034 removed the spawn sites on 2026-08-05 — three days after the audit. §3.1 and §5 are deliberately left unedited as dated findings whose cited line numbers no longer hold the code.

Row 34's Basis carries a new CONSOLIDATION provenance class, explicitly not folded into Rule 1 (hooks/session_context.py gives it residual hook coupling).

Also left behind by #308, fixed here

  • docs/README.md linked to the three deleted files and said "46 file(s)". It also asserted that docs/policies/ mirrors ~/.claude/policies/ — false, and this repo's own CLAUDE.md says so. Corrected.
  • tests/test_verify_policy_orphan_dispositions.py pinned SAMPLE_ORPHAN to session-pruning-policy.md and hardcoded "(14 of 46)". That would have failed CI too; it now reads the count out of the heading.

Verification

python scripts/verify_policy_audit_matrix.py         -> EXIT 0   PASS (AC1-AC7)
python scripts/verify_policy_orphan_dispositions.py  -> EXIT 0   PASS (AC1, AC2, SUP1)
python scripts/verify_policy_capability_dispositions.py -> EXIT 0
pytest tests/test_verify_policy_{orphan,capability}_dispositions.py -> 24 passed

Follow-ups this exposed (not fixed here)

  1. The CI path filter should trigger these gates on docs/ changes. All three are stdlib-only and run in seconds, and they read docs/. As configured, a docs-only PR can break a hard gate and merge clean, and the cost lands on an unrelated author — exactly what happened.
  2. Treating a docs/reports/ point-in-time audit as a live registry is not sustainable. AC1 binds a document stamped "MEASURED 2026-08-02" to a directory that changes whenever anyone edits a policy. Every corpus edit forces recomputation of ~15 interlocking derived counts by hand, and the gates check only 7 of them — the rest can drift while CI still exits 0. The registry should be a generated artifact the gate reads; the audit should be frozen prose that cites it.

🤖 Generated with Claude Code

piyushmakhija28 and others added 2 commits August 10, 2026 17:57
PR #308 consolidated session-chaining, session-memory and session-pruning
into one session-management-policy.md. That PR was docs-only, so CI's path
filter skipped the test matrix and the breakage surfaced on later PRs:
verify_policy_audit_matrix.py failed AC1 with one corpus file missing from
the matrix and three matrix rows absent from the corpus.

Replace the three rows with one row 34 for session-management-policy.md
(PARTIAL, keep-as-is, MEASURED against three verified path:line citations)
and renumber rows 37-46 to 35-44.

The three rows' findings are preserved, not erased. New section 1.4
transcribes them verbatim and records what has since happened to each: the
Stop-hook spawn sites they measured as dead were removed by V2-034 on
2026-08-05, three days after the audit, and sections 3.1 and 5 are left
unedited as dated 2026-08-02 findings.

Recomputed every count that reasons about those rows, each stated with what
it changed from: rows 46->44; verification split 9/37 -> 8 MEASURED / 36
CITED; real citations 37 -> 35 (37/46 -> 35/44 density); CONTRADICTED 8 ->
6; delete 11 -> 9; OAQ-2 provenance 15 -> 13 and its 5/5/5 split -> 5/5/3;
Rule 1's keep-as-is 18 -> 17 (row 34 is excluded from Rule 1, not silently
folded into it); hook-coupled rows 15 -> 13 with the PreToolUse breakdown
8+1+1+1+2; anchored-OAQ grep 19/15 -> 17/13.

as-built-prd section 4: session-management inherits session-memory's NFR-3
mapping, so 4.1 stays at 32 and 4.2 falls to 12; orphan count 14 of 46 ->
12 of 44, and six-of-14-beyond-SRS-fix -> five of 12.

Also fixes two things PR #308 left behind: docs/README.md linked to three
deleted files and claimed 46, and its "mirror ~/.claude/policies/" claim is
false. The orphan-gate test pinned SAMPLE_ORPHAN to a deleted policy and
hardcoded "(14 of 46)"; it now reads the declared count out of the heading,
so a future corpus change cannot fail a test whose gate is still correct.

Both gates exit 0 (AC1-AC7; AC1, AC2, SUP1), as does the capability gate.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…dation

Two advertised counts still described the 46-policy corpus and failed the
count-vs-tree gates:

  README.md:478        docs/ tree said 255 files, tree holds 253
  docs/README.md:22    policies row said 46, corpus holds 44

Both are the same arithmetic as the matrix re-base -- three policy files
removed, one added -- reached through a different assertion. The earlier
pass fixed docs/README.md's collapsible summary and missed the table row
two lines above it, which is the kind of duplicate-count-in-one-file that
only a gate finds.

Counts follow the gates' own definition, git-tracked files, not a
filesystem walk: find(1) reports 256 for docs/ because three files are
untracked.

Verified: tests/test_doc_counts_match_the_tree.py and
tests/test_docs_index_matches_the_tree.py pass, and the full unit suite
is green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@piyushmakhija28
piyushmakhija28 merged commit 1030325 into main Aug 10, 2026
6 checks passed
@piyushmakhija28
piyushmakhija28 deleted the fix/policy-audit-matrix-consolidation branch August 10, 2026 13:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant