fix(litellm): avoid duplicate streamed tool-call content - #6670
Open
thesynapses wants to merge 1 commit into
Open
fix(litellm): avoid duplicate streamed tool-call content#6670thesynapses wants to merge 1 commit into
thesynapses wants to merge 1 commit into
Conversation
- Omit already-streamed text from the final aggregated tool-call response. - Give the streaming tool-call test a unique name. - Assert the final response contains only the function-call part. Fixes google#3697
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Link to Issue or Description of Change
1. Link to an existing issue:
Problem:
LiteLLM streaming responses emit text as partial events, then repeat the same text in the final aggregated tool-call response. This duplicates planning/reasoning content in clients and conversation history.
Solution:
Set
content=Noneon the final aggregated tool-call message. The text remains available through the already-emitted partial responses, while the final response contains only the function call.The branch has been rebuilt as one commit directly on current
main; the unrelated changes and merge commits from #3698 are no longer present.Testing Plan
Unit Tests:
The regression test verifies that streamed text is emitted through partial responses and the final aggregated response contains only the function-call part.
Formatting and import-order checks pass with the repository-pinned Pyink and Isort versions.
Manual End-to-End (E2E) Tests:
Not run; the adapter behavior is covered by the existing mocked LiteLLM streaming fixture and the focused regression assertion.
Checklist
Additional context
A maintainer reconfirmed the diagnosis on #3697 and requested that #3698 be rebased. GitHub returned HTTP 422 when reopening #3698, including after temporarily restoring its recorded head, so this PR carries the requested clean rebase.