Record v3.12.0, and why the audit method missed all twelve findings

[amrali-eg]

The docs/SAFETY-AUDIT.md record for v3.12.0, which the release page already links to. One file, 43 insertions, no deletions.

It arrives after its own tag for the reason this file already documents: committing it changes the assembly through the PDB checksum, so recording it first would publish artifacts naming a hash they do not contain.

What it records

The exemption this release cannot claim. v3.11.2 could say it changed neither detection nor conversion policy. This one changed ConversionPolicy, so the checklist asked for a corpus run and none was made. Recorded as a gap rather than filled with the v3.11.0 figures.

Archive digests, verified rather than quoted. Both archives were downloaded and hashed locally; both match GitHub's reported digests.

Why the audit method missed all twelve findings. The comfortable answer — "the corpora only measure detection" — is false: the harness converts, and compares decoded source text against strict output. The true answer is that the three defects that touched files sat outside its reach for three different reasons:

• the alias defect preserves text perfectly, rewriting a file to identical bytes, so every comparison the harness makes passes while a modification time is silently lost;
• the 64 KiB defect needs a file valid for 64 KiB and invalid afterwards, a shape no corpus contains, because corpora hold files with authoritative metadata;
• the preview defect lives in -Plan, a mode the harness does not run.

A corpus run at full marks would have said nothing about any of the three.

That the review over-rated four of its own findings, and that the reader caught each one rather than the review catching itself — one mechanism proved with an input built to prove it, then described for significance without checking what the product does with such an input. Two were downgraded after measurement against realistic files, one withdrawn once its fix was shown to break four existing tests, one withdrawn once what EC actually reports was checked.

Limits it records

No corpus run against a checklist that asked for one; unsigned archives, signing skipped for the second release running; the preview fix being decode-only; unreadable directories still exiting 0; the four findings left open, including the BOM-less UTF-16 file that can be detected as UTF-32 and converted silently; and no accessibility spot check on file.

Verification

Documentation only — no code, no tests, no behaviour. CRLF and the file's no-BOM choice preserved; the diff is a pure insertion between the v3.11.2 record and ## Known limits. The heading generates exactly the anchor the release body links to.

🤖 Generated with Claude Code