Skip to content

Python: do not run a tool call whose arguments were never written when the reply was cut off - #14724

Open
Simin Yuan (simin-yuan) wants to merge 6 commits into
microsoft:mainfrom
simin-yuan:fix/cut-off-empty-tool-arguments
Open

Simin Yuan (simin-yuan) wants to merge 6 commits into
microsoft:mainfrom
simin-yuan:fix/cut-off-empty-tool-arguments

Conversation

@simin-yuan

@simin-yuan Simin Yuan (simin-yuan) commented Nov 5, 2026 •

Copy link
Copy Markdown

Motivation and Context

A reply cut off at the token limit right after the tool name carries no arguments. When every parameter of the function is optional, ChatCompletionClientBase runs it with the defaults, because the auto-invoke loops never read finish_reason. The model was stopped before it wrote the arguments that decide what the tool does, so the call it actually intended is unknown.

The shape is real, captured from an OpenAI-compatible endpoint: max_tokens=3 returns finish_reason=length, the tool name and one streamed delta with "arguments": "".

Streamed replies put a second trap in the way. The first tool-call chunk carries id and name; every later chunk carries an empty id and no name, and only index identifies the call:

chunk 1: {"index": 0, "id": "call_7d0c...", "function": {"name": "set_mode", "arguments": ""}}
chunk 2: {"index": 0, "id": "",            "function": {"arguments": "{\""}}
chunk 3: {"index": 0, "id": "",            "function": {"arguments": "mode\": \""}}
chunk 4: {"index": 0, "id": "",            "function": {"arguments": "silent\"}"}}
chunk 5: finish_reason=length

So any per-call bookkeeping across chunks has to key on index; keying on id files the arguments the model did write under a different key than the one looked up when the reply ends.

Description

python/semantic_kernel/connectors/ai/chat_completion_client_base.py

  • _note_written_arguments() records, before the streaming aggregation runs, which calls already carried argument text. _fragment_key() keys those on index (falling back to id only when a connector sets no index), which is what makes the chunks above one call.
  • _arguments_never_written() is the single decision used by both loops: the reply was cut off at the limit and this call never wrote argument text. Streaming passes the pre-aggregation evidence; non-streaming judges the finished arguments directly.
  • Aggregation turns empty fragments into "{}", so the pre-aggregation evidence is what separates "the model wrote nothing" from "the model wrote {}". Connectors that hand the parsed input over instead of argument text report a call cut off before its arguments as an empty object, in the streamed shape as the text "{}"; both are treated as nothing written.
  • _record_refused_calls() appends one FunctionResultContent per refused call explaining the cut-off; the streaming loop merges and yields those results like any other tool result.

python/semantic_kernel/connectors/ai/anthropic/services/anthropic_chat_completion.py

  • The streaming connector reported a complete empty input ({}) as no arguments at all, because it tested the input's truthiness. It now only reports None when the input really is None, so a tool the model called with {} keeps running unless the reply was cut off.

Behaviour:

  • A tool whose reply was cut off before any argument text was written is not executed; the model may retry within maximum_auto_invoke_attempts.
  • A call with complete arguments still runs, including when the reply is marked length after the arguments are complete, including when the arguments arrive in several chunks.
  • An explicit {} still runs when the model writes it as fragments, and when a connector hands it over as an empty input object in a reply that was not cut off. Under a cut-off reply an empty object counts as nothing written, in both the streamed and the non-streamed shapes.
  • A tool with required parameters is still not executed; only the message the model receives differs (it now explains the cut-off).
  • Calls whose reply was not cut off are untouched.

Size: two files, +110 / −10.

Trade-off

The rule reads the stop reason; it does not read the data. A connector that hands the parsed input over reports a complete {} and an input abandoned at the limit identically, and a streamed call serialized to "{}" looks the same whether the model wrote it or was stopped first. So in the rare reply where the model finishes a no-argument tool call and only then reaches the limit, that call is refused once — the model receives the result explaining the cut-off and can retry.

The previous behaviour was the opposite error: running a call whose arguments never existed. Nothing in these connectors separates the two cases, and the measurements on Bedrock are what tipped the choice towards refusing.

Tests

python/tests/unit/connectors/ai/test_cut_off_tool_call.py — 17 cases: 13 through the real OpenAIChatCompletion connector with an httpx.MockTransport, and four through the real AnthropicChatCompletion, the connector that hands the parsed input over instead of argument text. The streamed fixtures follow the wire format above (id and name on the first chunk only), so the multi-chunk cases are the ones that would catch an id-keyed lookup.

case before after
cut off, one empty fragment, optional-only tool (stream / non-stream) runs with defaults not run
cut off, several empty fragments (aggregate to "{}") runs with defaults not run
cut off, explicit {} written as fragments runs runs
cut off, non-streamed reply carrying {} instead of argument text (Anthropic shape) runs with defaults not run
cut off, streamed reply serialized as "{}" (Anthropic shape) runs with defaults not run
{} in a reply that was not cut off (Anthropic shape, streamed and non-streamed) runs runs
cut off, complete arguments in one chunk runs runs
cut off, complete arguments spread over several chunks with empty ids not run (wrongly refused) runs
cut off, no argument text at all (None) not run not run
cut off, empty fragments, tool with a required parameter not run not run
complete reply, empty arguments (zero-argument tool) runs runs
refused streamed call result not yielded result yielded to the caller
ChatCompletionAgent.invoke_stream with a cut-off reply — model sees the refusal on the next request
Anthropic streaming, tool input {} handed over as None handed over as "{}"

What was run locally: every case above fails on the commit it fixes and passes after it — the string shapes on main, the mapping case on 612d72c, the serialized stream on 081d926. pytest over tests/unit/connectors/ai/{open_ai,anthropic,bedrock}, tests/unit/agents/chat_completion and the new file, Python 3.11 on Windows: 461 tests, 0 failed, 4 skipped. ruff check / ruff format clean (0.9.6, the version this repository pins). The same workflow runs green on my fork across all 22 matrix jobs.

Contribution Checklist

AI disclosure

The patch and tests were drafted with AI assistance and executed locally: the new cases fail without the change and pass with it, and the suites above were run on that same machine. I reviewed the change and will respond to review feedback on this PR.

A reply cut off at the token limit right after the tool name arrives with
an empty argument string. When every parameter of the function is
optional, the auto-invoke loops run it with the defaults, because they
never read finish_reason - although the model was stopped before it
wrote the arguments that decide what the tool does.

Refuse such a call when finish_reason is LENGTH and the argument string
is empty, record a result explaining why, and let the model retry within
maximum_auto_invoke_attempts. Arguments that exist, complete or
truncated, keep the parsing they have today, and a call whose arguments
were finished still runs even if the reply is marked length afterwards.

Adds regression tests through the real OpenAIChatCompletion connector,
streaming and non-streaming, including both compatibility cases.
@simin-yuan
Simin Yuan (simin-yuan) requested a review from a team as a code owner November 5, 2026 03:03
Copilot AI balanced review requested due to automatic review settings November 5, 2026 03:03
@semantic-kernel-automation semantic-kernel-automation Bot added the python Pull requests for the Python Semantic Kernel label Nov 5, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Tool-call argument handling can execute incomplete calls and reject valid empty calls, while refusal results are not yielded and may be omitted from agent threads.

3 open issues
What changed in this PR

This PR adds safeguards to Python’s automatic tool invocation so replies cut off before tool arguments arrive do not execute functions with unintended defaults.

Changes:

  • Filters calls with missing arguments and a token-limit finish reason in both invocation loops.
  • Records refusal results in chat history so the model can retry.
  • Adds eight streaming and non-streaming test cases covering refusal and compatibility behavior.
File Description
python/​tests/​unit/​connectors/​ai/​test_cut_off_tool_call.py Adds mock-transport tests for cut-off and completed tool calls.
python/​semantic_kernel/​connectors/​ai/​chat_completion_client_base.py Adds argument-presence checks and refusal recording to automatic invocation.

Review effort: Balanced

Three gaps in the previous commit:

- Streaming aggregation combines empty argument fragments into "{}", so a
  reply that stopped right after the tool name looked as if the model had
  written an empty object and ran. Track the fragments before aggregation:
  a reply that produced only empty ones is still a reply that wrote nothing,
  while a model that really wrote {} produces non-empty fragments and keeps
  running.
- The refused calls were recorded in the history but never reached the
  streaming caller. They are now merged and yielded like any other tool
  result, and the model sees the explanation on the next auto-invoke
  attempt.
- A connector that reports no argument text at all (None, as the Anthropic
  connector does for a tool without input) was refused; it is now treated
  as "nothing was written", which is what it means for a cut-off reply.

Adds tests for all three, plus one through ChatCompletionAgent.invoke_stream
that checks the refusal reaches the model on the next request.
@simin-yuan

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree

The first streamed tool-call chunk carries the call id, every later chunk carries an empty id and only
the index. Tracking argument fragments by id filed the text the model did write under a different key
than the one looked up when the reply ended, so a complete call that still reported finish_reason=length
was refused, and so was a tool whose input really was {}.

Key fragments by index, record only the calls that carried argument text, and refuse a call at
finish_reason=length when it is not in that set. The Anthropic streaming connector now keeps a complete
empty input ({}) instead of reporting it as no arguments at all.
Bedrock's non-streamed reply hands the tool input over as a parsed object
instead of argument text, so a call cut off right after the tool name
reaches the auto-invoke loop as `{}` rather than as an empty string. The
cut-off check only recognised a string and `None`, so that shape still ran
the tool on its parameter defaults, while the streamed shape was already
refused.

An empty mapping under finish_reason=length means nothing was written: a
deliberate no-argument call ends with a tool_use stop reason, not with the
length limit. The condition and the Bedrock evidence come from @kimnamu in
microsoft#14723, who measured both shapes live on Bedrock with Claude Haiku 4.5.

Adds the cut-off case and its compatibility counterpart through the
Anthropic connector, which reports the same parsed object shape.
…itten

The Anthropic connector serializes the tool input it receives, so a call
cut off before its arguments reaches the streaming loop as the text "{}" —
the same emptiness the non-streamed shape reports as an empty mapping, and
just as unreadable as a call whose arguments were written. The pre-
aggregation evidence treated any non-empty text as written, so that shape
still ran the tool on its defaults.

Argument text that holds only an empty object is now not counted as
written. Nothing changes for a reply that was not cut off, where the
finish reason is not the limit, and a call the model wrote itself still
counts as written however it was chunked.

Adds both cases through the streaming Anthropic connector; the cut-off one
ran the tool on the previous commit.

This branch was successfully deployed

1 active deployment
github-app-auth — 4ec267f5 Deployed Nov 10, 2026 by simin-yuan via add_label #29659
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Pull requests for the Python Semantic Kernel

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants