Skip to content

fix: bound the response body by the total on the httpx family - #28

Merged
AlexeyShalaev merged 1 commit into
masterfrom
fix/total-covers-body
Sep 6, 2026
Merged

fix: bound the response body by the total on the httpx family#28
AlexeyShalaev merged 1 commit into
masterfrom
fix/total-covers-body

Conversation

@AlexeyShalaev

Copy link
Copy Markdown
Member

Branched from master; independent of #26 (both touch engine/aio.py, in different places).

TimeoutConfig.total stopped at the response headers on the httpx family. The engine sits in the transport and returns when handle_async_request does, so a body that kept dripping after the headers was outside the wall clock: total=1.0 against a server sending one byte every 0.5 s returned a full body after 4 s, same as bare httpx with timeout=1.0, while the same config against a server that stalls before the headers correctly raised at 1.0 s. The one thing a caller cannot tell from the outside is exactly the thing the total did not cover.

The response stream already passes through the family's AsyncTimedStreamMixin / SyncTimedStreamMixin for the body-duration metric, so the deadline now travels with it: run() builds the Deadline (it was built a few microseconds later in _admitted) and hands it to wrap_stream(response, on_done, deadline). The async wrapper bounds every chunk with asyncio.timeout(deadline.remaining()) and, when the scope fires or the budget is already gone before the next read, raises the adapter's DeadlineExceededError through the adapter's translator (bound into the normalizer, so _httpx_shared stays SDK-free); httpcore closes the connection on the cancellation and httpx's send() closes the response on the way out. The sync wrapper cannot interrupt a blocked read, so it re-checks the deadline before starting the next one, the way the sync engine re-checks at attempt boundaries: a chunk that would start after the deadline is refused, late by at most one chunk or one read timeout. Both hand on_done an Outcome(kind=total_timeout).

Where this diverges from the issue is the declaration. DurationBoundary is documented (agents page, Prometheus help text) as where the call duration closes, and call_end still fires at the headers after this change; an honest FULL would move requests_total, request_duration and the in-flight gauge to the end of the body for every httpx user, and an abandoned stream would leak in flight. That is a different change. So boundary stays headers and the coverage is a new Capability.DEADLINE_COVERS_BODY: emulated on the httpx family (hard on async, soft on sync, in the note) and listed in the build report's emulated; absent on aiohttp, requests and urllib3 with notes saying where the body streams instead. Consequence stated plainly, in the docs too: a body-phase deadline shows in the exception, in body_duration and in the outcome the wrapper reports, not in requests_total{outcome}, because that metric closed at the headers.

aiohttp stays at headers, declared. The middleware hands back a ClientResponse whose body the caller reads through aiohttp's StreamReader, which aiohttp relies on for release, EOF and connection reuse; wrapping it is private surgery, and aiohttp's own total timer would cover the body but wraps the whole middleware chain including our retries, which is why the adapter zeroes it. An integration test pins the declared divergence the way the sync read_timeout parity test does.

Two knock-ons. wrap_stream on the normalizer protocols gains the deadline argument; the protocols live below clientwright.core, which the reference page marks internal, so no semver break, but a third-party adapter implementing AsyncNormalizer/SyncNormalizer needs the extra parameter (none exist in-tree). And OriginServer gets a /drip/{count}/{interval} route, documented in the testing guide and on the agents page, so the scenario is a real integration test.

Rejected: boundary=FULL (above); wrapping aiohttp's StreamReader; leaving the sync client's body unbounded when a check between chunks costs nothing and matches how the sync total already behaves.

Docs: timeouts guide (new paragraph on the body), sync-and-async, the httpx and aiohttp adapter pages (the aiohttp page also claimed body duration was measured via trace hooks, which the capability record and the agents page both deny; corrected while there), agents page (Capability row, rule 10, adapter bullets, origin routes), testing guide. The changelog belongs to release-please.

The reporter's 01_read_timeout.py from the blog lab, before and after (the last line is the one that changes):

# master
httpx timeout=1.0, body drips                -> 200, 8 bytes, 4.02s
httpx timeout=1.0, headers stall             -> ReadTimeout, 1.01s
httpx + asyncio.timeout(1.0), body drips     -> TimeoutError, 1.00s
clientwright total=1.0, headers stall        -> HttpxDeadlineExceededError, 1.00s
clientwright total=1.0, body drips           -> 200, 8 bytes, 4.02s

# this branch
httpx timeout=1.0, body drips                -> 200, 8 bytes, 4.02s
httpx timeout=1.0, headers stall             -> ReadTimeout, 1.01s
httpx + asyncio.timeout(1.0), body drips     -> TimeoutError, 1.00s
clientwright total=1.0, headers stall        -> HttpxDeadlineExceededError, 1.00s
clientwright total=1.0, body drips           -> HttpxDeadlineExceededError, 1.00s

Negative control: with the two wrapper loops put back to their unbounded form (signatures kept), the dripping-body tests fail (5 failed, 2 passed, the two passing ones being the bodies that finish inside the budget, which must not change); with the branch they pass. Gate: make check clean (ruff, ruff format, mypy, import-linter); make test 818 passed, 42 skipped, coverage 99.60 % against the 97 % floor; uv.lock untouched. The branch merges cleanly with #26 (git merge-tree reports no conflicts), so either can land first.

Closes #25

TimeoutConfig.total stopped at the response headers on httpx and httpx2:
the engine returns at handle_async_request, so a body that kept dripping
after the headers escaped the wall clock entirely and a 1 s total let a
4 s body through. The response stream already passes through the
family's timed-stream wrapper for the body-duration metric, so the
deadline now travels with it. The async wrapper bounds every chunk by
the remaining budget and raises the adapter's DeadlineExceededError when
the budget is gone; the sync wrapper cannot interrupt a blocked read and
instead refuses to start the next one, late by at most one chunk or one
read timeout. Both hand on_done an Outcome(kind=total_timeout).

The declaration is a new Capability.DEADLINE_COVERS_BODY rather than
boundary=FULL: DurationBoundary says where the call duration metric
closes, and that is still the headers. The httpx family declares it
emulated and reports it in the build; aiohttp, requests and urllib3
declare it absent with notes. The normalizer contract's wrap_stream
gains the deadline argument, and OriginServer gets a
/drip/{count}/{interval} route for the scenario.

Closes #25
@AlexeyShalaev
AlexeyShalaev merged commit 5767b89 into master Sep 6, 2026
8 checks passed
@AlexeyShalaev
AlexeyShalaev deleted the fix/total-covers-body branch September 6, 2026 21:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

total deadline stops at the response headers on httpx, a dripping body escapes it

1 participant