Repository navigation
Python: do not run a tool call whose arguments were never written when the reply was cut off - #14724
Open
Simin Yuan (simin-yuan) wants to merge 6 commits into
Open
Simin Yuan (simin-yuan) wants to merge 6 commits into
Simin Yuan (simin-yuan) wants to merge 6 commits into
Conversation
A reply cut off at the token limit right after the tool name arrives with an empty argument string. When every parameter of the function is optional, the auto-invoke loops run it with the defaults, because they never read finish_reason - although the model was stopped before it wrote the arguments that decide what the tool does. Refuse such a call when finish_reason is LENGTH and the argument string is empty, record a result explaining why, and let the model retry within maximum_auto_invoke_attempts. Arguments that exist, complete or truncated, keep the parsing they have today, and a call whose arguments were finished still runs even if the reply is marked length afterwards. Adds regression tests through the real OpenAIChatCompletion connector, streaming and non-streaming, including both compatibility cases.
Simin Yuan (simin-yuan)
temporarily deployed
to
github-app-auth
November 5, 2026 03:03 — with
GitHub Actions
Inactive
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Tool-call argument handling can execute incomplete calls and reject valid empty calls, while refusal results are not yielded and may be omitted from agent threads.
3 open issues
What changed in this PR
This PR adds safeguards to Python’s automatic tool invocation so replies cut off before tool arguments arrive do not execute functions with unintended defaults.
Changes:
- Filters calls with missing arguments and a token-limit finish reason in both invocation loops.
- Records refusal results in chat history so the model can retry.
- Adds eight streaming and non-streaming test cases covering refusal and compatibility behavior.
| File | Description |
|---|---|
python/tests/unit/connectors/ai/test_cut_off_tool_call.py |
Adds mock-transport tests for cut-off and completed tool calls. |
python/semantic_kernel/connectors/ai/chat_completion_client_base.py |
Adds argument-presence checks and refusal recording to automatic invocation. |
Review effort: Balanced
Three gaps in the previous commit:
- Streaming aggregation combines empty argument fragments into "{}", so a
reply that stopped right after the tool name looked as if the model had
written an empty object and ran. Track the fragments before aggregation:
a reply that produced only empty ones is still a reply that wrote nothing,
while a model that really wrote {} produces non-empty fragments and keeps
running.
- The refused calls were recorded in the history but never reached the
streaming caller. They are now merged and yielded like any other tool
result, and the model sees the explanation on the next auto-invoke
attempt.
- A connector that reports no argument text at all (None, as the Anthropic
connector does for a tool without input) was refused; it is now treated
as "nothing was written", which is what it means for a cut-off reply.
Adds tests for all three, plus one through ChatCompletionAgent.invoke_stream
that checks the refusal reaches the model on the next request.
Simin Yuan (simin-yuan)
temporarily deployed
to
github-app-auth
November 5, 2026 03:20 — with
GitHub Actions
Inactive
Author
|
@microsoft-github-policy-service agree |
The first streamed tool-call chunk carries the call id, every later chunk carries an empty id and only
the index. Tracking argument fragments by id filed the text the model did write under a different key
than the one looked up when the reply ended, so a complete call that still reported finish_reason=length
was refused, and so was a tool whose input really was {}.
Key fragments by index, record only the calls that carried argument text, and refuse a call at
finish_reason=length when it is not in that set. The Anthropic streaming connector now keeps a complete
empty input ({}) instead of reporting it as no arguments at all.
Simin Yuan (simin-yuan)
temporarily deployed
to
github-app-auth
November 5, 2026 04:11 — with
GitHub Actions
Inactive
Bedrock's non-streamed reply hands the tool input over as a parsed object
instead of argument text, so a call cut off right after the tool name
reaches the auto-invoke loop as `{}` rather than as an empty string. The
cut-off check only recognised a string and `None`, so that shape still ran
the tool on its parameter defaults, while the streamed shape was already
refused.
An empty mapping under finish_reason=length means nothing was written: a
deliberate no-argument call ends with a tool_use stop reason, not with the
length limit. The condition and the Bedrock evidence come from @kimnamu in
microsoft#14723, who measured both shapes live on Bedrock with Claude Haiku 4.5.
Adds the cut-off case and its compatibility counterpart through the
Anthropic connector, which reports the same parsed object shape.
Simin Yuan (simin-yuan)
temporarily deployed
to
github-app-auth
November 7, 2026 06:06 — with
GitHub Actions
Inactive
…itten
The Anthropic connector serializes the tool input it receives, so a call
cut off before its arguments reaches the streaming loop as the text "{}" —
the same emptiness the non-streamed shape reports as an empty mapping, and
just as unreadable as a call whose arguments were written. The pre-
aggregation evidence treated any non-empty text as written, so that shape
still ran the tool on its defaults.
Argument text that holds only an empty object is now not counted as
written. Nothing changes for a reply that was not cut off, where the
finish reason is not the limit, and a call the model wrote itself still
counts as written however it was chunked.
Adds both cases through the streaming Anthropic connector; the cut-off one
ran the tool on the previous commit.
Simin Yuan (simin-yuan)
temporarily deployed
to
github-app-auth
November 7, 2026 06:37 — with
GitHub Actions
Inactive
Simin Yuan (simin-yuan)
deployed
to
github-app-auth
November 10, 2026 20:15 — with
GitHub Actions
Active
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Motivation and Context
A reply cut off at the token limit right after the tool name carries no arguments. When every parameter of the function is optional,
ChatCompletionClientBaseruns it with the defaults, because the auto-invoke loops never readfinish_reason. The model was stopped before it wrote the arguments that decide what the tool does, so the call it actually intended is unknown.The shape is real, captured from an OpenAI-compatible endpoint:
max_tokens=3returnsfinish_reason=length, the tool name and one streamed delta with"arguments": "".Streamed replies put a second trap in the way. The first tool-call chunk carries
idandname; every later chunk carries an emptyidand no name, and onlyindexidentifies the call:So any per-call bookkeeping across chunks has to key on
index; keying onidfiles the arguments the model did write under a different key than the one looked up when the reply ends.Description
python/semantic_kernel/connectors/ai/chat_completion_client_base.py_note_written_arguments()records, before the streaming aggregation runs, which calls already carried argument text._fragment_key()keys those onindex(falling back toidonly when a connector sets no index), which is what makes the chunks above one call._arguments_never_written()is the single decision used by both loops: the reply was cut off at the limit and this call never wrote argument text. Streaming passes the pre-aggregation evidence; non-streaming judges the finished arguments directly."{}", so the pre-aggregation evidence is what separates "the model wrote nothing" from "the model wrote{}". Connectors that hand the parsed input over instead of argument text report a call cut off before its arguments as an empty object, in the streamed shape as the text"{}"; both are treated as nothing written._record_refused_calls()appends oneFunctionResultContentper refused call explaining the cut-off; the streaming loop merges and yields those results like any other tool result.python/semantic_kernel/connectors/ai/anthropic/services/anthropic_chat_completion.py{}) as no arguments at all, because it tested the input's truthiness. It now only reportsNonewhen the input really isNone, so a tool the model called with{}keeps running unless the reply was cut off.Behaviour:
maximum_auto_invoke_attempts.lengthafter the arguments are complete, including when the arguments arrive in several chunks.{}still runs when the model writes it as fragments, and when a connector hands it over as an empty input object in a reply that was not cut off. Under a cut-off reply an empty object counts as nothing written, in both the streamed and the non-streamed shapes.Size: two files, +110 / −10.
Trade-off
The rule reads the stop reason; it does not read the data. A connector that hands the parsed input over reports a complete
{}and an input abandoned at the limit identically, and a streamed call serialized to"{}"looks the same whether the model wrote it or was stopped first. So in the rare reply where the model finishes a no-argument tool call and only then reaches the limit, that call is refused once — the model receives the result explaining the cut-off and can retry.The previous behaviour was the opposite error: running a call whose arguments never existed. Nothing in these connectors separates the two cases, and the measurements on Bedrock are what tipped the choice towards refusing.
Tests
python/tests/unit/connectors/ai/test_cut_off_tool_call.py— 17 cases: 13 through the realOpenAIChatCompletionconnector with anhttpx.MockTransport, and four through the realAnthropicChatCompletion, the connector that hands the parsed input over instead of argument text. The streamed fixtures follow the wire format above (id and name on the first chunk only), so the multi-chunk cases are the ones that would catch an id-keyed lookup."{}"){}written as fragments{}instead of argument text (Anthropic shape)"{}"(Anthropic shape){}in a reply that was not cut off (Anthropic shape, streamed and non-streamed)None)ChatCompletionAgent.invoke_streamwith a cut-off reply{}None"{}"What was run locally: every case above fails on the commit it fixes and passes after it — the string shapes on
main, the mapping case on612d72c, the serialized stream on081d926.pytestovertests/unit/connectors/ai/{open_ai,anthropic,bedrock},tests/unit/agents/chat_completionand the new file, Python 3.11 on Windows: 461 tests, 0 failed, 4 skipped.ruff check/ruff formatclean (0.9.6, the version this repository pins). The same workflow runs green on my fork across all 22 matrix jobs.Contribution Checklist
AI disclosure
The patch and tests were drafted with AI assistance and executed locally: the new cases fail without the change and pass with it, and the suites above were run on that same machine. I reviewed the change and will respond to review feedback on this PR.