feat: fix Baton adapter retries and publish English results - #16
Open
chenghao-mou wants to merge 3 commits into
Open
feat: fix Baton adapter retries and publish English results#16chenghao-mou wants to merge 3 commits into
chenghao-mou wants to merge 3 commits into
Conversation
chenghao-mou
changed the base branch from
main
to
feat/baton-turn-detection
September 13, 2026 15:39
chenghao-mou
changed the base branch from
feat/baton-turn-detection
to
main
September 13, 2026 15:40
Retry classification matched digits like 429 anywhere in the error text, which included up to 400 chars of the response body, so a permanent 422 could be retried for ten minutes. Raise a typed BatonHTTPError and classify on its status. The capacity and transient budgets are now disjoint. An empty or malformed events payload now raises instead of scoring every grid point as p_eot=0.0, matching the other streaming adapters. Audio decoding goes through the shared helpers so 16 kHz stereo is down-mixed, and the adapter exposes concurrency (default 8) like its siblings.
Reproduces the README row (12.3%, 4.8%, 577 ms, 350 ms) from a fresh run of all 400 English turns at concurrency 8 and commits the artifacts under output/, so the numbers are auditable like every other row. Adds Baton to the site model config and README adapter docs.
The results table names vendor plus product wherever the product name alone does not identify the vendor. Adapter ID and run directory are unchanged.
chenghao-mou
force-pushed
the
chenghao/feat/AGT-3504-baton-benchmark
branch
from
September 13, 2026 15:53
feb94c6 to
d9910fe
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follows #13, now merged. Fixes correctness issues in the Baton adapter and publishes the reproduced English results.
Correctness fixes in the adapter: retries classify on a typed HTTP status error instead of digits found in the response body, the capacity and transient budgets are disjoint, empty or malformed
eventspayloads raise instead of scoring every grid point as zero, 16 kHz stereo is down-mixed via the shared audio helpers, and the adapter exposesconcurrencylike its siblings.Reproduced results: a fresh run of all 400 English turns gives 12.3%, 4.8%, 577 ms, 350 ms, identical to the README row in #13. Artifacts, comparison report, and site data are committed. The other 13 languages are a follow-up.
Thanks to @Ted-Johnson-JoinIn-AI for the adapter.
Addresses AGT-3504
Initial prompt and agent context
Model: Claude Fable 5.1
🤖 Generated with Claude Code