Skip to content

fix: give the attempt ceiling its own retryable kind and adapter error - #26

Merged
AlexeyShalaev merged 1 commit into
masterfrom
fix/attempt-timeout-kind
Sep 6, 2026
Merged

fix: give the attempt ceiling its own retryable kind and adapter error#26
AlexeyShalaev merged 1 commit into
masterfrom
fix/attempt-timeout-kind

Conversation

@AlexeyShalaev

Copy link
Copy Markdown
Member

TimeoutConfig(attempt=0.4) under total=1.0 was meant to cut a hung attempt and try again. Instead the first ceiling escaped client.get() as a bare stdlib TimeoutError at 0.4 s, unretried and invisible to except httpx.TimeoutException: the engine classified anything raised as a TimeoutError as total_timeout, which is not retryable, and only an expired deadline ever became a CallError for the adapter translator. The same except TimeoutError had a second victim. aiohttp's timeout family subclasses asyncio.TimeoutError (the builtin on 3.11+), so a sock_read timeout never reached the adapter's classifier either: one attempt, total_timeout, no retry.

The engine now asks the cancel scope whether it fired (asyncio.Timeout.expired()) instead of matching on the exception type. An SDK exception, TimeoutError subclass or not, goes through classify_error; a fired ceiling is total_timeout when the deadline is gone and the new FailureKind.ATTEMPT_TIMEOUT otherwise. attempt_timeout is retryable and trips the breaker by default, next to read_timeout. When the retry gate declines and the last outcome is an attempt timeout, _attempts raises AttemptTimeoutError(CallError) carrying the ceiling, chained from the TimeoutError, the way it already raises DeadlineExceededError, so it leaves run() through the translator as HttpxAttemptTimeoutError (an httpx.TimeoutException) or AiohttpAttemptTimeoutError (an aiohttp.ServerTimeoutError). The adapter packages export those two and the root exports AttemptTimeoutError.

To have one place where an outcome becomes final, the retry gate (decide, skip counter, budget spend) moved into _retry_delay. The sync engine got the identical move so the two loops still read the same; nothing else changes there, since it has no ceiling to fire, and requests/urllib3 get no new error class.

Adding a default retryable kind the sync adapters can never emit would have made dead_retryable_kinds flag every default config on requests and urllib3, and strict would have failed those builds. They now declare attempt_timeout collapsed into read_timeout: on a sync runtime the stall a ceiling would have caught arrives as the clamped read phase, the same reasoning as the existing write_timeout -> total_timeout collapse. The httpx family record is shared by both flavours, so it lists attempt_timeout in emits with a note that only the async client produces it, as deadline_hard does. A new test compiles the default config against every registered capability record and asserts there are no dead kinds, so this cannot regress quietly.

Rejected: reusing DeadlineExceededError for the ceiling (it is not the deadline, and the label that says which knob fired is the point); leaving attempt_timeout out of the retry defaults (a ceiling that is never retried is not worth setting next to retry); keeping except TimeoutError and only relabelling it (leaves aiohttp's read timeouts misclassified).

Docs: timeouts guide, retries guide, agents page (TimeoutConfig, RetryConfig, FailureKind, errors), adapters reference, writing-an-adapter. The changelog belongs to release-please.

The repro from #24 (its exact script), before and after:

$ python repro24.py        # master
escaped: builtins TimeoutError 0.40s

$ python repro24.py        # this branch
httpx family: HttpxDeadlineExceededError 1.00s

Three requests reach the origin on the branch: two 0.4 s ceilings, then the third clamped to the 0.17 s the total had left (attempt_timeout, attempt_timeout, total_timeout in the attempt metrics; the integration test pins that sequence). The lab's 02_retry_multiplies.py prints the same three lines before and after, as its cases hit the total, not the ceiling:

hand-rolled loop: 3 attempts, timeout=1.0 each       -> ReadTimeout after 3.04s, 3 requests reached the server
clientwright: total=1.0, 3 attempts allowed          -> HttpxDeadlineExceededError after 1.00s, 1 requests reached the server
clientwright: total=3.0, read=1.0, 3 attempts allowed -> HttpxDeadlineExceededError after 3.00s, 3 requests reached the server

The aiohttp side, read=0.2 against an origin that answers in 1 s with three attempts allowed:

# master
raised: aiohttp.client_exceptions SocketTimeoutError
attempt outcomes: ['total_timeout']
requests reaching origin: 1

# this branch
raised: aiohttp.client_exceptions SocketTimeoutError
attempt outcomes: ['read_timeout', 'read_timeout', 'read_timeout']
requests reaching origin: 3

Negative control: with master's engine/aio.py and the sync capability records swapped back in, the ten new behavioural tests fail (10 failed, 120 deselected); with the branch they pass. Gate: make check clean (ruff, ruff format, mypy, import-linter); make test 819 passed, 42 skipped, coverage 99.60 % against the 97 % floor; uv.lock untouched.

Closes #24

A TimeoutConfig.attempt ceiling firing before the total was classified
total_timeout, which is not retryable, and escaped client.get() as a
bare stdlib TimeoutError because only an expired deadline became a
CallError. The same except-TimeoutError branch also swallowed aiohttp's
phase timeouts (its whole timeout family subclasses TimeoutError), so a
sock_read timeout was labelled total_timeout and never retried.

The engine now asks the cancel scope whether it fired instead of
matching on the exception type. An SDK exception goes through the
adapter's classifier; a fired ceiling is total_timeout when the deadline
is gone and the new attempt_timeout otherwise. attempt_timeout is
retryable and trips the breaker by default, like read_timeout, and a
call whose last attempt died on the ceiling raises AttemptTimeoutError
through the adapter translator: HttpxAttemptTimeoutError is an
httpx.TimeoutException, AiohttpAttemptTimeoutError an
aiohttp.ServerTimeoutError. The retry gate moved into _retry_delay in
both engines so an outcome becomes final in one place. Sync adapters
declare attempt_timeout collapsed into read_timeout, so the default
retry config has no dead kinds under strict.

Closes #24
@AlexeyShalaev
AlexeyShalaev merged commit e36beeb into master Sep 6, 2026
8 checks passed
@AlexeyShalaev
AlexeyShalaev deleted the fix/attempt-timeout-kind branch September 6, 2026 21:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

attempt timeout escapes as a bare TimeoutError and is never retried

1 participant