Repository navigation
fix(follow): announce each bench outcome once across restarts - #57
Open
ancelin-code wants to merge 2 commits into
Open
ancelin-code wants to merge 2 commits into
ancelin-code wants to merge 2 commits into
Conversation
Re-arming the loop printed every PUBLISHED again, and old BENCH-FAILED lines whose failure had long been handled by hand. The announced file kept beside the logs was only written with --exit-on-human; every run now records each outcome there, keyed by task, and skips what is already in it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
Re-arming
follow.shafter its watcher expired printed everyPUBLISHEDagain, plus the oldBENCH-FAILED skill-001, although that failure had been fixed by hand and PR #39 was open (feedback item 6). A watcher reads those lines as new pull requests to review and new failures.The loop already kept
<log-dir>/announcedso that--exit-on-humanwould not stop twice on one bench, but it wrote the file only in that mode. Now every run records each outcome there and announces nothing it already holds.How to review
All of it is in the outcome block of
etabli-orchestrator/scripts/follow.sh:<id><TAB><last log line>. AnERRORline carries no task id, so two benches with the same error would otherwise mask each other.publishedis still set for a silent outcome, so the task keeps its place inWAITING review=.Not done: the alternative the feedback suggested, comparing a log's time with the loop's start. It would drop the outcome of a bench that finished while no loop was running.
How to try it
Run
follow.sh --max-polls 1twice over the same logs: the second run prints nothing. Overwrite a log with a new outcome, as a relaunch does, and the next run announces it. The tests inTestBenchOutcomesdo both.Does it earn its place?
This reuses a rule that already existed, in every mode instead of one. The keyed format makes the first run after upgrading announce once more what
--exit-on-humanhad already announced; this is recorded inCHANGELOG.md.🤖 Generated with Claude Code