Repository navigation
Fix streaming on multi-worker ASGI with Redis shared storage - #4054
Conversation
Redis subscriptions ran a blocking XREAD in the loop's default executor, one thread per open downlink, and aget/aset/apublish used the same executor. At a dozen downlinks per worker every publish queued behind the parked reads for 5-15 s, so on uvicorn --workers 4 a quarter to half of streams failed at 25 browsers. The Redis backend now has a native redis.asyncio path: one client per event loop, loop-native key/value and publish, and one reader task per loop that serves every async subscription with a single multi-stream XREAD (woken for new subscriptions through a private wake stream). Diskcache's async subscriptions poll without blocking and sleep on the loop. The ASGI downlink writes its connection record with the async calls, and its closed record from a task of its own so a client disconnect can't skip it.
Dash performance benchmarks
growth = late-third / early-third per-op time; ~1 is flat, a large value means the per-op cost scales with accumulated state. machine scale vs baseline: 1.01x - divided out of the baseline ratios so they compare like for like (the absolute warn/fail ceilings are left un-scaled); calibrated on |
Split the reader's dispatch and failure paths out of its run loop, drop copies of subscriber sets that are never mutated while iterated, keep one possible raise per pytest.raises block, and silence pylint's import-error on the redis.asyncio import, which CI's lint environment can't resolve.
Port the topic ttl from #4010 to the async Redis path: apublish takes ttl, each stream entry carries the topic's lifetime, and the reader renews the topics it reads before they could expire, as a sync poll does.
KoolADE85
left a comment
KoolADE85
left a comment
There was a problem hiding this comment.
I saw alot of exceptions in my server stdout when running this PR. It seems related to a Redis timeout.
I'm attaching a repro app that Claude wrote for me: a callback that yields a frame every 10s. It will stop sending data after ~3 ticks, and you'll find a number of exceptions in your server terminal.
It seems to break with the latest redis-py but works on older versions (redis-py<8)
redis-py 8 defaults socket_timeout to 5 s, as long as the XREAD block, so a stream quiet for that long timed out and killed its downlink. Cap each block at half the client's timeout. A downlink whose storage fails now ends its body with a warning instead of raising into the server mid-response.
|
|
@KoolADE85 Thanks, good catch. redis-py 8 defaults |
2 participants
On uvicorn
--workers 4withRedisSharedStorage, FastAPI and Quart failed 24-57% of streaming callbacks at only 25 browsers, with p95 frame latency of 10-35 s. The same apps were fine on one worker with local storage, and Flask/gunicorn was fine on the same Redis.Cause
Redis subscriptions used
PollingSubscription, whose async iterator ran a blockingXREAD(5 s block) in the loop's default executor: one thread per open downlink, out of 12 on an 8-core machine.aget/aset/apublishused the same executor, so at a dozen downlinks per worker every publish queued behind the parked reads for 5-15 s and streams ran into the client's timeout. Instrumenting the workers showed nothing else: no stream was cancelled by the cross-worker connection-record check.Changes
redis.asynciopath, one client per event loop.aget/aset/adelete/apublishare loop-native. One reader task per loop serves every async subscription with a single multi-streamXREAD, so it is one Redis connection per worker, not one per browser. A new subscription wakes the read through a private wake stream. Gaps are detected by sequence contiguity; a stream reset under a subscriber is caught on quiet cycles. The sync path is unchanged.Shared storage and streaming are unreleased (#3930, #3931), so no CHANGELOG entry.
Numbers
benchmarks/streamingload sweep (#4055), Redis, 4 workers, 8-core laptop:Before: 24-57% errors at 25 browsers.
Tests
tests/streaming/test_stream_asgi_redis.py: uvicorn--workers 2on Redis, 80 idle downlinks open, then a real browser stream must finish in under 4 s (FastAPI and Quart). Before the change it was stuck at the first frame after 15 s.tests/shared_storage/test_redis_backend.py: async subscriptions hold no executor threads (fails before), replay, gap, close from another thread, a passed client, a subscriber joining behind a read in flight (no false gap), per-loop clients dropped once their loop closes.tests/shared_storage/test_diskcache_backend.py: async subscriptions hold no executor threads (fails before).tests/streaming/test_stream_transport.py: a cancelled async downlink still records itself closed.Not in this PR
--workers N, uvicorn binds its socket with protocol 0, so asyncio never setsTCP_NODELAYand every first frame waits ~40 ms on Nagle plus delayed ACK (3 ms on one worker). That is uvicorn, not Dash.