Skip to content

fix(chat): expose stream_batch content for every response - #202

Open
latent-9 wants to merge 1 commit into
xai-org:mainfrom
latent-9:fix/stream-batch-shared-proto
Open

fix(chat): expose stream_batch content for every response#202
latent-9 wants to merge 1 commit into
xai-org:mainfrom
latent-9:fix/stream-batch-shared-proto

Conversation

@latent-9

Copy link
Copy Markdown

Description

stream_batch(n) wraps all n options in Response objects that share one underlying proto, but only responses[0].process_chunk(chunk) is called for every incoming chunk. Chunk content is buffered per output index inside the first response, and only materialized into the shared proto when responses[0].content is read, or when the telemetry span closes.

A caller that reads responses[i].content for i > 0 during streaming, or before responses[0], gets an empty string for the whole stream. This is the natural way to stream per-option content:

async for responses, _ in chat.stream_batch(2):
    print(responses[1].content)

With this code responses[1].content is "" on every iteration, even though the server streams the content. The fix materializes the buffered content into the shared proto after every chunk, so each response reflects the streamed content regardless of read order.

Changes

  • src/xai_sdk/aio/chat.py: sync content buffers to the shared proto after each chunk in stream_batch
  • src/xai_sdk/sync/chat.py: same fix for the sync client
  • Regression tests in tests/aio/chat_test.py and tests/sync/chat_test.py that read responses[1].content during the stream. They fail on the current code and pass with this change.
  • Changelog entry under Unreleased.

Type of Change

  • Bug fix

Related Issue

N/A

stream_batch(n) wraps all n options in Response objects sharing one proto,
but only responses[0].process_chunk is called for each chunk. Content is
buffered per output index in the first response and only materialized into
the shared proto when responses[0].content is read or the telemetry span
closes. Reading responses[i].content for i > 0 during streaming returns an
empty string for the whole stream.

Materialize the first response's buffers into the shared proto after each
chunk so every response reflects streamed content regardless of read order.
@latent-9
latent-9 requested a review from a team as a code owner August 22, 2026 01:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant