Skip to content

Python: Use the tool call index for streamed Azure AI Inference tool calls - #14751

Open
Lanre Shittu (Shizoqua) wants to merge 2 commits into
microsoft:mainfrom
Shizoqua:fix-azure-ai-inference-streaming-tool-call-index
Open

Lanre Shittu (Shizoqua) wants to merge 2 commits into
microsoft:mainfrom
Shizoqua:fix-azure-ai-inference-streaming-tool-call-index

Conversation

@Shizoqua

Copy link
Copy Markdown

Motivation and Context

Fixes #14750

When the Azure AI Inference connector streams parallel tool calls, it set the index of every FunctionCallContent to the choice index, so all tool calls got index 0. Argument deltas after the first one carry only the tool call index, so when the chunks were added together the arguments of the second call were appended to the first call. The first function was then called with invalid JSON arguments such as {"city": "Paris"}{"zone": "Tokyo"}.

Description

_create_streaming_chat_message_content now reads the index the service sends for each tool call with tool_call.get("index"). The SDK model StreamingChatResponseToolCallUpdate has no index attribute, but it keeps the raw field. When the index is missing, it falls back to the position of the tool call in the delta. For a single tool call this gives 0, the same as before.

Added test_azure_ai_inference_streaming_parallel_tool_calls_keep_their_index, which streams two parallel tool calls and adds the chunks together. It fails on main with ('call_1', 0, 'Weather-get_weather', '{"city": "Paris"}{"zone": "Tokyo"}') != ('call_1', 0, 'Weather-get_weather', '{"city": "Paris"}') and passes with the fix.

Contribution Checklist

…calls

The Azure AI Inference connector set the index of each streamed tool
call to the choice index, so every parallel tool call in a stream got
index 0. Argument deltas after the first one carry only the index, so
when the chunks were added together the arguments of the second call
were appended to the first call, which then had invalid JSON arguments.

Read the index the service sends for each tool call, and fall back to
the position in the delta when it is missing.
@Shizoqua
Lanre Shittu (Shizoqua) requested a review from a team as a code owner November 8, 2026 15:32
Copilot AI balanced review requested due to automatic review settings November 8, 2026 15:32
@semantic-kernel-automation semantic-kernel-automation Bot added the python Pull requests for the Python Semantic Kernel label Nov 8, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are reported for the change preserving streamed tool-call indexes.

No open issues 🎉

What changed in this PR

Fixes #14750 by preserving tool-call indexes in the Python Azure AI Inference connector, preventing streamed parallel calls from merging arguments into the wrong function call.

Changes:

  • Uses the service-provided tool-call index, falling back to the call’s position in the delta when absent.
  • Adds a regression test verifying that accumulated parallel tool calls retain separate indexes and arguments.
File Description
python/​tests/​unit/​connectors/​ai/​azure_ai_inference/​services/​test_azure_ai_inference_chat_completion.py Adds regression coverage for streamed parallel tool-call accumulation.
python/​semantic_kernel/​connectors/​ai/​azure_ai_inference/​services/​azure_ai_inference_chat_completion.py Uses individual tool-call indexes instead of the choice index.

Review effort: Balanced

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks correct. Here's what I checked.

  • Why the old index broke parallel calls. FunctionCallContent.__add__ (function_call_content.py:121) raises ContentAdditionException when the indexes differ. _merge_items_lists (streaming_content_mixin.py:28) uses that exception to decide whether to append a new item or merge into an existing one. With every call at choice.index (0), the second call's argument-only delta had no id to tell it apart, so it merged into the first call. That's the {"city": "Paris"}{"zone": "Tokyo"} in #14750.
  • Reading the index. StreamingChatResponseToolCallUpdate is an azure-core Model, which is a MutableMapping over the raw JSON. So tool_call.get("index") reads the service's field even though the typed model doesn't declare it.
  • The fallback. idx from enumerate(choice.delta.tool_calls) gives 0 for the usual one-call delta, which is what main produced. So a service that omits index behaves as before.
  • choice_index is unchanged. The message keeps choice_index=choice.index, so n > 1 choices are still kept apart at the message level (streaming_chat_message_content.py:186).
  • git merge-tree against main is clean.

I didn't run the tests. The new test feeds the argument-only deltas without id, which is the case that failed. It would also be worth one case where both calls arrive in a single delta (tool_calls with two entries and no index). That pins the enumerate fallback, so a later change can't quietly put both back at 0.

Reviewed with help from Claude Code.

Cover the fallback where the service sends several tool calls in one
delta without an index. They should be indexed by their position.
@Shizoqua

Copy link
Copy Markdown
Author

Thanks for the review. I added a test for two tool calls arriving in a single delta without an index.

This branch was successfully deployed

1 active deployment
github-app-auth — cf9fdf47 Deployed Nov 14, 2026 by Shizoqua via add_label #29690
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Pull requests for the Python Semantic Kernel

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python: Bug: Azure AI Inference streaming merges parallel tool call arguments into the first call

3 participants