Repository navigation
fix(publish): retry gh, keep its error, and report a pushed bench apart - #53
Open
ancelin-code wants to merge 1 commit into
Open
ancelin-code wants to merge 1 commit into
ancelin-code wants to merge 1 commit into
Conversation
A proxy timeout on gh pr create failed a finished bench: the label warning blamed the label, because stderr was discarded and any failure was read as a missing label, and follow.sh reported BENCH-FAILED although the work was pushed. gh is now tried again after 5 and 15 seconds, the label warning needs an error about the label, and a lasting failure after the push prints PUBLISH-FAILED with the error and the command that re-runs the publisher. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
In the tiroir
retrievalcut,skill-001finished and was pushed, thengh pr createhit a proxy timeout (feedback item 2). Three things went wrong:ai-assistedlabel could not be applied.create_pr --label …ran with2>/dev/null, so any failure was read as a missing label.follow.shreportedBENCH-FAILED, although the work had succeeded.Now:
publish-bench.shtries again after 5 and 15 seconds, looking for an already open pull request before each new try.ETABLI_GH_RETRY_DELAYSsets the waits.gh's stderr is kept. The label is dropped, with a warning that quotes the error, only when that error mentions the label.PUBLISH-FAILED <id> <error> <re-run command>, andgh not founddoes the same.follow.shshows it as its own event, and--exit-on-humanstill stops on it.How to review
Start with the loop at the end of
publish-bench.sh, thenbench_endand thecaseon the outcome infollow.sh. The exit code stays 1 for a failedghcall, so nothing that checks it changes.Left out on purpose: retrying the push. A failed push is not this bug, and it can fail for reasons a retry does not fix, such as a hook.
How to try it
tests/unit/test_publish_bench_script.pyhas a fakeghthat times out on the first N creates: two failures then a labelled pull request, or every try failing and ending inPUBLISH-FAILEDwith the exact re-run command. In a real bench, re-running that command from the line opens the pull request.Does it earn its place?
Without the retry, a passing network glitch costs a person a manual re-run. Without
PUBLISH-FAILED, a finished bench reads as a failure, and the label warning sends them after the wrong cause.PUBLISH-FAILEDis a new output line from the publisher and a new event fromfollow.sh, recorded inCHANGELOG.md. An agent watching the loop should treat it as "open this pull request", not "fix this bench".🤖 Generated with Claude Code