You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
OpenAI Chat Completions: tool rounds drop the model's reasoning #326
OpenAILanguageModel's Chat Completions tool loop drops the model's reasoning between tool rounds. Thinking models return it alongside their tool calls, in reasoning_content (DeepSeek's API) or reasoning and reasoning_details (OpenRouter, Vercel AI Gateway). The next request in the same response rebuilds the assistant message from the content and tool calls only, so the reasoning that planned those calls is lost.
Through a gateway it fails silently. The Vercel AI Gateway accepts the requests, so nothing errors. But each round, the model sees its earlier tool calls without the reasoning behind them.
Reproduction
Run a session with tools against deepseek/deepseek-v4.1-flash-fast or zai/glm-5.3-fast on the Vercel AI Gateway (https://ai-gateway.vercel.sh/v1), and record the request bodies. Each response's choices[0].message has reasoning and reasoning_details next to tool_calls. In the next request, the matching assistant message has only content and tool_calls. Across one 8-round response, 0 of the 8 assistant messages carried the reasoning back.
PR #290 added reasoning transcript entries and replay for Anthropic. Chat Completions doesn't read or send these fields yet (checked on main at 80d50b0).
A possible fix
In the tool loop, decode the three fields from the response message and send them back unchanged on that round's assistant message. Keeping them within the one response matches DeepSeek's rule, so the transcript doesn't need to change. I did this for the nonstreaming loop in a fork, with a test: gtokman/AnyLanguageModel@daf184b. The commit is on 0.16.0. The streaming loop would need the same change, gathering the reasoning deltas.
A fuller version would store the reasoning as Transcript.Reasoning entries, as #290 does for Anthropic, so it could also be replayed after a transcript is restored. I'm happy to open a PR for either approach.
Thanks, @gtokman. #325 now builds the tool call message from the transcript, both for the round in flight and for the replay on the next turn. If the reasoning goes only into the in-flight message, the two would differ again, and we'd lose the prompt caching fix. So I think the fuller version fits better: store the reasoning as Transcript.Reasoning entries, as #290 does for Anthropic, and have toolCallsMessage send them back, in both the streaming and nonstreaming loops.
Could you open that on top of #325? I'd be happy to review it. One question first: does DeepSeek accept reasoning_content from earlier turns, or does it need to be dropped after the turn ends?
Activity
mattt commentedon Oct 14, 2026
Thanks, @gtokman. #325 now builds the tool call message from the transcript, both for the round in flight and for the replay on the next turn. If the reasoning goes only into the in-flight message, the two would differ again, and we'd lose the prompt caching fix. So I think the fuller version fits better: store the reasoning as
Transcript.Reasoningentries, as #290 does for Anthropic, and havetoolCallsMessagesend them back, in both the streaming and nonstreaming loops.Could you open that on top of #325? I'd be happy to review it. One question first: does DeepSeek accept
reasoning_contentfrom earlier turns, or does it need to be dropped after the turn ends?