story-014: Verify the suite in a clean clone before the story commits

[jerodw]

Summary

The verifier runs the suite in the working tree — the one environment where the story's own commit does not yet exist. _complete then commits that tree, after the documenter and after every check the workflow performs. The state the code actually ships in is created after the last thing that could object to it, and nothing ran the suite there.

Three failures across two stories lived in that gap, and every one reported green to the verifier:

• story-011's differential test resolved its baseline as git show HEAD:orchestration/story_coordinator.py — the pre-story coordinator only while uncommitted. 419 passed uncommitted against 394 passed, 25 errors committed.
• story-011 then failed in CI because actions/checkout defaults to a one-commit clone, so the baseline walk had no history to walk.
• story-013 shipped a test asserting git status --porcelain -- tests/ lists the tester's new files — true mid-run, false once _complete commits them — on a run that reported 460/460 and passed verification on its first iteration.

This story gives the coordinator a second run of the suite, in a fresh clone with the story committed into it, executed after the verifier passes and before the documenter runs. A failure reroutes to the implementer as a retry. The result is written to the run directory as clean-clone-result.json, so a reader can tell the check ran rather than inferring it from a pass.

This is not verifier leniency. The suite genuinely passes where the verifier stands. The bug is the absent check, not a missed one. It does not replace CI, which stays the final word — it moves discovery earlier, so a story is not reported complete and committed before the failure is known.

Changes

• orchestration/story_coordinator.py — clean_clone_check is the last thing the verifier branch does on a passing verdict. It builds a scratch clone with tempfile.mkdtemp, runs the suite, removes the scratch directory in a finally whatever the result, and writes CleanCloneResult.as_record() under the declared artifact name.
  • _build_clone does git clone --no-hardlinks from the target's filesystem path — never the network — applies the target's tracked edits as git diff --binary HEAD piped to git apply, copies untracked-but-not-ignored files from git ls-files --others --exclude-standard, then commits inside the clone. The target repository is only read: no commit, no branch, no index change, no stash.
  • _link_interpreter_roots links each configured interpreter's top-level directory into the clone and appends those names to the clone's .git/info/exclude — a virtualenv is gitignored and therefore absent from a fresh clone, and a .gitignore entry for a directory does not cover a symlink standing in its place.
  • Zero exit appends the pass event and falls through to the existing advance. Non-zero takes the retry path the verification-failed branch already takes — archive_attempt above the increment, then increment, save, event, and reroute to on_failure.retry_stage — or the existing escalation path at the ceiling.
• workflows/story-workflow.json — the verifier stage declares clean_clone: "clean-clone-result.json". That key is what turns the check on: the coordinator reads stage.get("clean_clone") and does nothing when absent, so removing the declaration disables the check with no orchestration change, and the artifact name never appears in code.
• schemas/clean-clone-result.schema.json (new) — records whether the check ran, the command, the interpreter and its version, the exit code and an output tail. Coordinator-written like execution-history, so it appears in no stage's schemas map. Optional fields are expressed by absence, not null — a check that refused to run has no exit code to report.
• orchestration/context_assembler.py, prompts/implementer.md — a {{clean_clone_result}} placeholder carries the evidence into a retried implementer's prompt. This exists because a clean-clone retry has no verifier finding to carry: the verifier passed, and the coordinator must not fabricate an agent's judgement by writing retry-guidance.json itself.
• schemas/execution-history.schema.json, tests/test_schema_validator.py, tests/test_story_004_validation.py — the new event kinds, and the two schema-inventory assertions updated. Both still assert exact set equality.
• .harness/docs/ARCHITECTURE.md — the check, its placement in the post-verifier order, the clone construction and why it is a clone rather than a copy, the reroute-rather-than-escalate decision, and the note that story-014's own run is not governed by the check it adds.

Testing

549 passed — on both Python 3.14 (the harness interpreter) and Python 3.10 (the version the check now targets).

New validation is in tests/test_story_014_validation.py, written by the tester stage: 1156 lines covering the clone construction, the gitignore exclusion, scratch removal after a failing run, target immutability, the retry routing, the ceiling escalation, the omitted-declaration case, and the HEAD-baseline fixture.

Notes for review

• The check targets Python 3.10, not the harness's own interpreter. clean_clone_python in .harness/config.yaml names it. The .venv here is 3.14 while CI tests 3.10–3.12, so before this nothing local exercised a version CI runs — a 3.10 incompatibility would have passed the verifier, passed the clean-clone check, and failed only in CI. That is the same class of gap as the terminal-width bug that reached CI on story-016, on a different axis. The suite passes on 3.10, so nothing was hiding.
• This story's own run is not governed by the check it adds, and correctly so: the coordinator loaded the workflow at run start, before clean_clone existed in it. The run directory therefore contains no clean-clone-result.json. The verifier recorded this rather than treating it as a defect.
• The events are module-level helpers rather than inline calls, for a reason worth preserving. tests/test_story_011_validation.py proves its own non-vacuity by deleting the first retry_decision="retry", line at the verification-failed branch's indentation. An inline clean-clone branch nests deeper and sits earlier in the file, so its line would contain that indented text and the mutation would land there instead of where it was aimed.
• Two test files outside the task list were repaired. tests/test_story_009_validation.py and tests/test_story_010_validation.py asserted git diff HEAD -- <paths> over workflows/, schemas/ and prompts/ — all of which this story edits, so they went red while the tree was dirty. The implementer repointed both at their own story's commit range. This is story-015's work arriving early by necessity rather than by choice; the verifier confirmed the markers resolve to ff80849 and 2239a23, so both assertions compare a real commit against its parent and can still fail.
• One path the verifier could not exercise: _build_clone raises RuntimeError if git clone, git apply or the clone-side commit fails, which would propagate out without writing the artifact or escalating. No case could be constructed where git apply of git diff --binary HEAD fails against a fresh clone of the same HEAD, so there is no evidence the path is reachable.

🤖 Generated with Claude Code