Re-arm the readiness deadline after a gate resolves - #156
Merged
Conversation
wait_for_readiness's overall deadline was computed once at function
entry and never touched again. The Gate branch correctly parks with no
deadline of its own -- a human answering a first-run gate can take as
long as they need -- but the Undecided branch still measured its
remaining time against that same original, never-advanced deadline.
Concretely: a run parks at a gate, a human takes real time to notice
and answer it (the entire point of escalating), the screen advances to
something not yet recognized -- and because the original entry-time
deadline had long since elapsed while the run was correctly parked,
that very next Undecided tick failed the run instantly, with a message
("no recognizable prompt or first-run gate appeared before the
readiness cap elapsed") that reads as though nothing had happened at
all. The human who did exactly what the escalation asked of them was
the one punished for it.
No shipped vendor classifier walks this path today: claude and codex's
gates both resolve straight to a screen their own classifiers recognize
as the composer, so Gate never hands off to Undecided in production.
Planning a vendor whose first-run flow has more than one screen is what
surfaced this before it shipped.
Fixed with one new bool tracking whether the previous classification was
a Gate, checked and cleared on the next Undecided tick to re-arm the
deadline exactly once per Gate-to-Undecided transition -- never on every
Undecided tick, which would make fail-closed stop meaning anything for a
screen that is genuinely, permanently unrecognizable after a gate.
escalated_gate (the per-gate escalation dedup) is untouched, a separate
concern. The failure text also now distinguishes the two cases: "no
recognizable prompt appeared within a fresh readiness window after a
first-run gate was left" when a gate was seen and left, versus the
original wording when nothing was ever recognized at all.
Two tests: parking on a gate long enough for the original deadline to
elapse, then transitioning to a briefly-unrecognized screen, must still
succeed (the positive control -- verified by temporarily disabling the
re-arm and confirming this test, and only this test, starts failing);
and a screen that stays unrecognized forever after a gate must still
fail once the fresh window elapses, bounded well inside an outer
timeout, proving the re-arm fires once per transition rather than on
every tick.
cargo fmt --all, cargo clippy --all-targets --all-features -D warnings,
CREW_DISABLE_VENDOR_CLI=1 cargo test --workspace (69 test binaries, all
passing), and a full bun run check all verified clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
wait_for_readiness's overall deadline was computed once at function entry and never touchedagain. The Gate branch correctly parks with no deadline of its own -- a human answering a first-run
gate can take as long as they need -- but the Undecided branch still measured its remaining time
against that same original, never-advanced deadline.
Concretely: a run parks at a gate, a human takes real time to notice and answer it (the entire point
of escalating), the screen advances to something not yet recognized -- and because the original
entry-time deadline had long since elapsed while the run was correctly parked, that very next
Undecided tick failed the run instantly, with a message that reads as though nothing had happened at
all. The human who did exactly what the escalation asked of them was the one punished for it.
No shipped vendor classifier walks this path today: claude and codex's gates both resolve straight to
a screen their own classifiers recognize as the composer, so Gate never hands off to Undecided in
production. Planning a vendor whose first-run flow has more than one screen is what surfaced this
before it shipped.
Fixed with one new bool tracking whether the previous classification was a Gate, checked and cleared
on the next Undecided tick to re-arm the deadline exactly once per Gate-to-Undecided transition --
never on every Undecided tick, which would make fail-closed stop meaning anything for a screen that
is genuinely, permanently unrecognizable after a gate. The per-gate escalation dedup is untouched, a
separate concern. The failure text also now distinguishes the two cases: a gate that was seen and
left gets a message saying so, rather than one implying nothing was ever recognized.
Testing: parking on a gate long enough for the original deadline to elapse, then transitioning to a
briefly-unrecognized screen, must still succeed -- verified as a genuine regression control by
temporarily disabling the re-arm and confirming this test, and only this test, starts failing. A
screen that stays unrecognized forever after a gate must still fail once the fresh window elapses,
bounded well inside an outer timeout, proving the re-arm fires once per transition rather than on
every tick.