Skip to content

Python: fix background=True + tools infinite-retrieve loop#5462

Open
he-yufeng wants to merge 1 commit intomicrosoft:mainfrom
he-yufeng:fix/background-tool-loop-continuation-leak
Open

Python: fix background=True + tools infinite-retrieve loop#5462
he-yufeng wants to merge 1 commit intomicrosoft:mainfrom
he-yufeng:fix/background-tool-loop-continuation-leak

Conversation

@he-yufeng
Copy link
Copy Markdown

Fixes #5394.

The bug

When a caller combines background=True with local function tools, the agent gets stuck retrieving the same completed response on every tool-loop iteration and tool results are never POSTed. After max_iterations the loop exits with empty text.

Root cause

FunctionInvocationLayer.get_response() builds a single mutable_options = dict(options) dict and threads it through every iteration of the tool loop via super_get_response(options=mutable_options). RawOpenAIChatClient._inner_get_response reads continuation_token from that dict and, when present, takes the responses.retrieve(response_id) branch.

After the very first retrieve returns a completed background response, continuation_token is still sitting in mutable_options. Every subsequent iteration therefore re-enters the retrieve branch and hits the same completed response again — never the responses.create/parse path that would POST the tool results.

The HTTP trace from the issue confirms it: 1 POST followed by ~40 GETs of the same response_id, and no tool-result POSTs.

Fix

In _inner_get_response, after the non-streaming retrieve completes, check whether the returned ChatResponse still carries a continuation_token. If it doesn't (i.e. the background operation is no longer in progress), pop continuation_token and background from the caller's options dict in place. FunctionInvocationLayer passes the same dict reference across iterations, so the mutation makes the next iteration fall through to _prepare_requestresponses.create(...) and the tool results flow normally.

This is the same change the reporter (@Laende) verified as a runtime monkeypatch, lifted into _chat_client.py.

Scope

  • Non-streaming path only. The streaming continuation branch (responses.retrieve(stream=True)) yields chunks inside an async generator and isn't exercised by the tool loop in the same way; leaving that out until someone sees a real reproduction.
  • No public API change; the dict being mutated is the options argument that's already owned and re-used by the caller.

Test plan

  • Reporter's repro in Python: [Bug]: background=True causes infinite tool-call loop, tool-result submissions inherit background mode #5394 (agent with background=True + function tools): after this fix the HTTP trace should match the monkeypatched trace in the issue — 1 POST, a few GETs while the background response is in progress, then the tool-result POSTs that let the model produce a final answer.
  • Existing agent/chat-client tests pass with ruff check / ruff format green locally; happy to add a regression test if a reviewer points me at the right mock scaffolding for client.responses.retrieve.

# of POSTing tool results (issue #5394).
if chat_response.continuation_token is None and isinstance(options, dict):
options.pop("continuation_token", None)
options.pop("background", None)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it should still ask for a background response, but you are right in that the continuation_token needs to be reset

… tool loop

Fixes microsoft#5394.

When `background=True` is combined with local function tools,
`FunctionInvocationLayer` calls `_inner_get_response(options=mutable_options)`
repeatedly with the same dict reference across loop iterations. Once the
first poll retrieves a completed background response, `continuation_token`
stays in `mutable_options`, so every subsequent iteration takes the
`continuation_token is not None` branch and `GET`s the same completed
response instead of `POST`ing the tool results. The loop exits after
`max_iterations` with empty text and the model never sees any tool output.

After the retrieve, if the returned `ChatResponse.continuation_token` is
`None` (the background response is no longer in progress), pop
`continuation_token` and `background` from the shared options dict in
place. The next loop iteration then falls through to the normal
`responses.create`/`parse` path and posts tool results.

The diagnosis and a verified runtime monkeypatch are in the issue; this
is the same fix moved in-tree.
@he-yufeng he-yufeng force-pushed the fix/background-tool-loop-continuation-leak branch from 1ed0f6a to 8bfa009 Compare April 25, 2026 06:00
@he-yufeng
Copy link
Copy Markdown
Author

@eavanvalkenburg good catch — kept background so subsequent iterations still POST as background. Pushed 8bfa0098 with that change and updated the comment to make the rationale clearer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python: [Bug]: background=True causes infinite tool-call loop, tool-result submissions inherit background mode

3 participants