tool calling alignment with openai's spec#4585
Open
lvhan028 wants to merge 10 commits into
Open
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates LMDeploy’s OpenAI-compatible tool-calling support to better align with OpenAI’s spec, including request/response schema additions and stricter handling of tool-call parsing and rendering.
Changes:
- Extend OpenAI protocol models with
Function.strict,AllowedToolChoice,parallel_tool_calls, and makeDeltaToolCall.idoptional. - Update response/tool parsing to (a) omit
idon argument-only tool-call deltas, (b) optionally truncate/skip parallel tool calls, and (c) surface invalid tool-call payloads as an error message inparse_complete. - Update
BaseChatTemplate.messages2prompt()to rendertool_callsandtool_call_idin prompts; add targeted unit tests for the new behaviors.
Reviewed changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_lmdeploy/serve/parsers/test_gpt_oss_parser.py | Updates expected dumped tool schema to include strict. |
| tests/test_lmdeploy/serve/openai/test_protocol_strict.py | Adds coverage for Function.strict passthrough and dumping. |
| tests/test_lmdeploy/serve/openai/test_parallel_tool_calls.py | Adds coverage for parallel_tool_calls default and truncation behavior. |
| tests/test_lmdeploy/serve/openai/test_messages2prompt_tool_calls.py | Adds coverage for rendering tool_calls / tool_call_id into prompts. |
| tests/test_lmdeploy/serve/openai/test_delta_tool_call_id.py | Adds coverage that streaming tool-call id only appears on the first chunk. |
| tests/test_lmdeploy/serve/openai/test_arguments_validation.py | Adds coverage for tool-call argument JSON validation/normalization and error surfacing. |
| tests/test_lmdeploy/serve/openai/test_allowed_tool_choice.py | Adds coverage for AllowedToolChoice and tool filtering in dump_tools(). |
| lmdeploy/serve/parsers/tool_parser/tool_parser.py | Adjusts JSON incremental tool-call delta emission so arg deltas omit id. |
| lmdeploy/serve/parsers/tool_parser/qwen3coder_tool_parser.py | Tightens argument JSON parsing (raise on invalid JSON) and skips normalization on errors. |
| lmdeploy/serve/parsers/response_parser.py | Adds allowed-tool filtering in dump_tools(), truncation helper, streaming skipping for non-parallel tool calls, and error return on tool parse failure. |
| lmdeploy/serve/openai/protocol.py | Adds protocol fields/models (strict, AllowedToolChoice, parallel_tool_calls) and makes DeltaToolCall.id optional. |
| lmdeploy/model.py | Renders assistant tool_calls and tool tool_call_id in messages2prompt(). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…bility Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Per the OpenAI streaming spec, the tool call id should appear only on the name-delta chunk and be None on subsequent argument-delta chunks. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…dumps Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…2prompt Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
lvhan028
force-pushed
the
feat/openai-tool-calling-alignment
branch
from
69c476f to
66477eb
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.