Skip to content

Python: [Bug]: A failed run with ToolApprovalMiddleware leaves the tool-loop budget in the session, and the retry gets fewer or no tool calls #9677

Description

Observed Behavior

ToolApprovalMiddleware keeps the function invocation budget (attempt_count, total_function_calls, start_time) in session.state["_function_invocation_budget_state"] and passes that same dict to the tool loop, so a run paused for approval resumes with the same counts (_harness/_tool_approval.py:436-437).

The loop removes that key only when a run returns (_clear_budget_state_from_session, _tools.py:4776). The one error path that cleans it is provider invalidation (_response_invalidation_cleanup, _tools.py:4800). Any other exception, such as a provider error, a timeout or a cancelled task, leaves the counts in the session.

The next run on that session picks them up: attempt_start comes from attempt_count (_tools.py:5749 and 5975), total_function_calls counts toward max_function_calls, and start_time is kept by setdefault (_tools.py:6404), so max_duration_seconds also measures from the failed run. The retry gets fewer iterations, or tools are turned off on its first request and the user gets "Function invocation limit reached before a final answer could be produced."

start_time is an absolute perf_counter() value, so with max_duration_seconds the wait between the failed run and the retry also counts. With function_invocation_configuration={"max_duration_seconds": 1} (default max_iterations) and the same script, a retry 1.5 s after the failed run gets tool_choice="none" on its first request and returns the limit text. With the fix it runs both tools.

create_harness_agent adds ToolApprovalMiddleware by default (_harness/_agent.py:667-668), so harness agents hit this after any failed turn. I checked the same repro with create_harness_agent and got the same output. Without the middleware the loop uses a fresh budget per call and the retry works.

Streaming and non-streaming behave the same.

Expected Behavior

A run that fails releases the session budget the way a completed run does, so retrying on the same session starts with the full max_iterations, max_function_calls and max_duration_seconds. A run that is paused with pending approvals should keep its budget, as it does today.

The invalidation path already does this for one kind of failure: test_response_invalidation_short_circuits_current_iteration asserts the key is gone from session.state after the run raises.

Steps to Reproduce

On main at 67ba500:

  1. Create an Agent with a tool and middleware=[ToolApprovalMiddleware()] (or use create_harness_agent), and one AgentSession.
  2. Run 1: the model calls the tool once, then the provider raises.
  3. Run 2 on the same session: the budget from run 1 is still in session.state, and the loop starts at the old attempt_count.

Minimal Reproduction

A scripted client, no network. max_iterations=2 keeps it short; with the default of 40 the counts add up over several failed turns.

import asyncio
from collections.abc import AsyncIterable

from agent_framework import (
    Agent,
    AgentSession,
    BaseChatClient,
    ChatMiddlewareLayer,
    ChatResponse,
    ChatResponseUpdate,
    Content,
    FunctionInvocationLayer,
    Message,
    ResponseStream,
    ToolApprovalMiddleware,
    tool,
)

executed: list[str] = []


@tool
def lookup(query: str) -> str:
    executed.append(query)
    return f"result for {query}"


class ScriptedClient(FunctionInvocationLayer, ChatMiddlewareLayer, BaseChatClient):
    def __init__(self, turns, **kwargs):
        super().__init__(**kwargs)
        self.turns = list(turns)
        self.tool_choices = []

    def _next(self, options):
        self.tool_choices.append(options.get("tool_choice"))
        turn = self.turns.pop(0)
        if isinstance(turn, Exception):
            raise turn
        return turn

    def _inner_get_response(self, *, messages, stream, options, **kwargs):
        if stream:

            async def updates() -> AsyncIterable[ChatResponseUpdate]:
                yield ChatResponseUpdate(role="assistant", contents=[self._next(options)])

            return ResponseStream(updates(), finalizer=ChatResponse.from_updates)

        async def get() -> ChatResponse:
            return ChatResponse(messages=[Message(role="assistant", contents=[self._next(options)])])

        return get()


async def main(stream: bool) -> None:
    executed.clear()
    client = ScriptedClient(
        [
            Content.from_function_call(call_id="c1", name="lookup", arguments='{"query": "first"}'),
            RuntimeError("transient provider error"),
            Content.from_function_call(call_id="c2", name="lookup", arguments='{"query": "retry"}'),
            Content.from_text("final answer"),
        ],
        function_invocation_configuration={"max_iterations": 2},
    )
    agent = Agent(client=client, tools=[lookup], middleware=[ToolApprovalMiddleware()])
    session = AgentSession()

    async def run(text: str):
        if stream:
            response_stream = agent.run(text, session=session, stream=True)
            async for _ in response_stream:
                pass
            return await response_stream.get_final_response()
        return await agent.run(text, session=session)

    print(f"stream={stream}")
    try:
        await run("look it up")
    except RuntimeError as exc:
        print("  run 1 raised:", exc)
    budget = session.state.get("_function_invocation_budget_state") or {}
    print("  budget left in session:", {k: budget[k] for k in ("attempt_count", "total_function_calls") if k in budget})
    calls_before = len(client.tool_choices)
    response = await run("look it up again")
    print("  run 2 tool_choice per provider call:", client.tool_choices[calls_before:])
    print("  tools executed:", executed)
    print("  run 2 text:", repr(response.text))


asyncio.run(main(stream=False))
asyncio.run(main(stream=True))

Output on main (67ba500):

stream=False
  run 1 raised: transient provider error
  budget left in session: {'attempt_count': 2, 'total_function_calls': 1}
  run 2 tool_choice per provider call: ['none']
  tools executed: ['first']
  run 2 text: 'Function invocation limit reached before a final answer could be produced.'
stream=True
  run 1 raised: transient provider error
  budget left in session: {'attempt_count': 2, 'total_function_calls': 1}
  run 2 tool_choice per provider call: ['none']
  tools executed: ['first']
  run 2 text: 'Function invocation limit reached before a final answer could be produced.'

Expected for run 2: tool_choice ['auto', 'auto'], tools executed ['first', 'retry'], text 'final answer'.

Error Messages and Stack Traces

No error on the retry. Run 2 gets tool_choice="none" and returns the limit fallback text.

Package Versions

agent-framework-core 1.23.0 (main at 67ba500)

Python Version

3.14.5

Operating System

macOS 26.6.2

Regression

Not checked against older releases.

Additional Context

A streamed run that the caller stops reading early (breaks out of the loop without an error) also leaves the budget in the session. That case is different: nothing tells the loop whether the run was abandoned or will be resumed, so I left it out here.

I have a fix and regression tests ready locally (clear the session budget with _clear_budget_state_from_session when the non-streaming loop raises or the stream fails, skipping ResponseInvalidatedException, which keeps its own cleanup). I'll wait for agreement before opening a PR.

This issue was prepared with AI assistance. I reviewed the analysis and ran the reproduction myself.

Acknowledgements

  • I searched existing issues and did not find a duplicate.
  • I personally verified this behavior and the reproduction details are authentic.
  • I will wait for explicit maintainer agreement before starting implementation of a non-trivial change.

Activity

  1. added
    pythonUsage: [Issues, PRs], Target: Python
    triageUsage: [Issues], Target: All issues that still need to be triaged
    on Oct 25, 2026
  2. added
    reproducedUsage: [Issues], Target: all issues that can be reproduced by the triage workflow
    on Oct 25, 2026
  3. github-actions commented on Oct 25, 2026

    @github-actions
    Contributor

    🤖 Automated triage reproduction notes (agent-authored — trust but verify)

    Agent analysis

    Repro: python/packages/core/agent_framework/_tools.py::_get_response_with_function_invocation and _stream_response_with_function_invocation leave the session budget populated when an ordinary provider exception escapes. This occurs when ToolApprovalMiddleware shares its session budget with a run that executes a tool and then fails. Minimal repro: use the scripted client in python/packages/core/tests/core/test_issue_9677.py, fail the first run after one tool call, then retry with the same AgentSession.

    • Failing test: python/packages/core/tests/core/test_issue_9677.py
    • Files examined: python/packages/core/agent_framework/_tools.py, python/packages/core/agent_framework/_harness/_tool_approval.py, python/packages/core/tests/core/test_function_invocation_logic.py, python/packages/core/pyproject.toml
    • Tests run: test_response_invalidation_short_circuits_current_iteration (passed), python/packages/core/tests/core/test_issue_967.py (2 failed)
    • Reported version: 1.23.0
    • Current version: 1.23.0
  4. added
    agentsUsage: [Issues, PRs], Target: Single agent
    middlewareUsage: [Issues, PRs], Target: middleware related features
    and removed
    triageUsage: [Issues], Target: All issues that still need to be triaged
    on Oct 25, 2026
  5. eavanvalkenburg commented on Oct 26, 2026

    @eavanvalkenburg
    Member

    /plan Define cleanup of per-attempt function-invocation budget state after provider errors and cancellation without resetting budgets that must survive resumable approvals. Cover same-session retries and streaming/non-streaming parity.

  6. eavanvalkenburg commented on Nov 12, 2026

    @eavanvalkenburg
    Member

    /plan Define cleanup of per-attempt function-invocation budget state after provider errors and cancellation without resetting budgets that must survive resumable approvals. Cover same-session retries and streaming/non-streaming parity.

  7. github-actions commented on Nov 12, 2026

    @github-actions
    Contributor

    Summary

    When ToolApprovalMiddleware is used, it puts the tool-loop budget (attempt_count, total_function_calls, start_time) into session.state and passes the same dict to the loop. The loop only removes it on normal return or on provider invalidation. If a provider error, timeout or cancellation escapes, the counts stay in the session. A retry on the same session then starts with fewer iterations, or with tools disabled. The issue says streaming and non-streaming behave the same.

    What I confirmed in the code (read only, nothing run)

    • The middleware shares the session budget at _tool_approval.py ~L454.
    • The non-streaming loop calls _clear_budget_state_from_session only on its return paths. The one error path with cleanup is _response_invalidation_cleanup, which handles only ResponseInvalidatedException (_tools.py ~L4880-4925). Other exceptions escape without cleanup.
    • The retry reads attempt_count and total_function_calls back from the leftover dict. start_time is kept with setdefault, so max_duration_seconds also measures from the failed run.
    • _clear_budget_state_from_session already keeps the budget when approvals are pending or a mixed-pause batch is deferred. It saves a portable snapshot in the second case. That is the guard the request asks for, so a failure cleanup can reuse it.
    • python/packages/core/AGENTS.md and spec 004 say loop changes need streaming/non-streaming parity and core-team agreement. Spec 004 already lists provider invalidation as clearing budget state.
    • I did not read the streaming loop body in full, so the streaming wiring below is inferred. I checked the session-state bookkeeping only. I did not look at the middleware's own error handling.

    Recommendation

    Code change, and it is warranted. This is a framework bug with a clear contract: a failed attempt should not eat into the next attempt's limits. It hits every harness agent, because create_harness_agent adds the middleware by default. The only workaround is for the caller to delete session.state["_function_invocation_budget_state"] after a failure, which depends on a private key. The scope is small, since the existing helper already knows how to preserve approval state. The issue is already labelled ready for implementation. An open PR exists (#10146). I did not review it and formed this plan independently.

    Proposed plan (a proposal only; no changes have been made)

    1. Define the rule. Any exception that ends a loop run, including Exception and asyncio.CancelledError, releases the session budget through _clear_budget_state_from_session. Because that helper keeps the budget when approvals are pending or a mixed-pause batch is deferred, a run that is really paused for approval keeps it. A run that fails while approvals are still pending also keeps it. Treat this as intended and test it.

    2. Non-streaming loop. Wrap the body of _get_response_with_function_invocation so that a raised exception clears the budget before propagating. Use except BaseException or try/finally with a success flag, so cancellation is covered. CancelledError is not an Exception.

    3. Streaming loop. Apply the same wrapper around _stream_response_with_function_invocation. Cover errors raised while iterating the provider stream and during finalization.

      • A caller who stops reading early (generator close) is a separate case. The issue deliberately leaves it out. I suggest keeping it out of scope, since the loop cannot tell abandoned from resumable. Call this out as a known limitation.
    4. Invalidation. ResponseInvalidatedException already restores its own continuation state, and the clear is idempotent. The new handler should not alter that restore.

    5. Check three edges first.

      • Failure during approval resume (Phase 1).
      • Failure after a mixed-pause snapshot was written.
      • The shared dict held in client_kwargs. After the session key is popped, the next run's setdefault must create a fresh dict, not reuse the stale one. Clearing the shared dict in place may be needed.
    6. Tests, using the issue's scripted client in both modes:

      • Provider error after one tool call, then a retry on the same session. Expect tool_choice ['auto', 'auto'] and both tools executed.
      • Same for max_duration_seconds, with a gap between failure and retry.
      • Cancellation mid-run, in both modes.
      • Failure while approvals are pending keeps the budget, and the approved resume still shares the max_function_calls budget.
      • Existing test_response_invalidation_short_circuits_current_iteration and the mixed-pause budget tests still pass.
    7. Docs. Add one line to spec 004 (the scenario table, with test names) saying failed or cancelled runs release the request budget except while approvals are pending. No other docs are needed.

    How we'd know it's fixed: the issue's reproduction prints ['auto', 'auto'], ['first', 'retry'] and 'final answer' for both modes, and the approval-resume budget tests are unchanged.

    Open point: whether the abandoned-stream case should get a separate follow-up issue.

    Generated by Issue Planning for #9677 · copilot · auto · ~33 AIC · ⌖ ~1 AIC · ⊞ 4.8K · ◷

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

agentsUsage: [Issues, PRs], Target: Single agentmiddlewareUsage: [Issues, PRs], Target: middleware related featurespythonUsage: [Issues, PRs], Target: Pythonready-for-implementationMaintainers agree the problem and direction are ready for implementation.reproducedUsage: [Issues], Target: all issues that can be reproduced by the triage workflow

Type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions