fix(openai): support opt-in streaming chat completions for long-running turns - #671
Open
BlueX888 wants to merge 2 commits into
Open
fix(openai): support opt-in streaming chat completions for long-running turns#671BlueX888 wants to merge 2 commits into
BlueX888 wants to merge 2 commits into
Conversation
added 2 commits
September 1, 2026 21:35
…ng turns Non-streaming chat completions fail against Anthropic-backed OpenAI-compatible endpoints once a turn exceeds the upstream 10-minute cap (HTTP 400 'Streaming is required'). Add an opt-in params.stream flag on the model node: the provider issues the request with stream=True and stream_options include_usage, then reassembles the chunks (content, tool calls, usage) into the classic chat.completion shape so the rest of the pipeline is unchanged. Gateways that reject stream_options get one retry without it. Fixes OpenBMB#653
Add tests for the params.stream opt-in: non-streaming default preserved, streaming request flags, chunk reassembly into the classic response shape, streamed tool-call accumulation, stream_options rejection fallback, and call_model end-to-end.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Add an opt-in
params.stream: trueon the model node. When set,OpenAIProviderissues Chat Completions withstream=True(+stream_options: {"include_usage": true}) and reassembles the chunks — content, tool calls, and usage — into the classicchat.completionshape, so token tracking, timeline recording, and deserialization are unchanged. Gateways that rejectstream_optionsget one retry without it. Default (non-streaming) behavior is untouched.Why
Anthropic-backed OpenAI-compatible endpoints (Anthropic API, Bedrock, enterprise LLM gateways) reject non-streaming operations that may exceed 10 minutes with HTTP 400 "Streaming is required". Any long single-turn output currently fails there, and the node's
final_messagebecomes the error text.How tested
7 new tests in
tests/test_openai_provider_streaming.py: non-streaming default preserved, streaming opt-in flags, chunk reassembly into the classic response shape, streamed tool-call accumulation,stream_optionsrejection fallback, andcall_modelend-to-end. Fulltests/suite passes on Python 3.12.Fixes #653