Repository navigation
Description
Observed Behavior
ToolApprovalMiddleware keeps the function invocation budget (attempt_count, total_function_calls, start_time) in session.state["_function_invocation_budget_state"] and passes that same dict to the tool loop, so a run paused for approval resumes with the same counts (_harness/_tool_approval.py:436-437).
The loop removes that key only when a run returns (_clear_budget_state_from_session, _tools.py:4776). The one error path that cleans it is provider invalidation (_response_invalidation_cleanup, _tools.py:4800). Any other exception, such as a provider error, a timeout or a cancelled task, leaves the counts in the session.
The next run on that session picks them up: attempt_start comes from attempt_count (_tools.py:5749 and 5975), total_function_calls counts toward max_function_calls, and start_time is kept by setdefault (_tools.py:6404), so max_duration_seconds also measures from the failed run. The retry gets fewer iterations, or tools are turned off on its first request and the user gets "Function invocation limit reached before a final answer could be produced."
start_time is an absolute perf_counter() value, so with max_duration_seconds the wait between the failed run and the retry also counts. With function_invocation_configuration={"max_duration_seconds": 1} (default max_iterations) and the same script, a retry 1.5 s after the failed run gets tool_choice="none" on its first request and returns the limit text. With the fix it runs both tools.
create_harness_agent adds ToolApprovalMiddleware by default (_harness/_agent.py:667-668), so harness agents hit this after any failed turn. I checked the same repro with create_harness_agent and got the same output. Without the middleware the loop uses a fresh budget per call and the retry works.
Streaming and non-streaming behave the same.
Expected Behavior
A run that fails releases the session budget the way a completed run does, so retrying on the same session starts with the full max_iterations, max_function_calls and max_duration_seconds. A run that is paused with pending approvals should keep its budget, as it does today.
The invalidation path already does this for one kind of failure: test_response_invalidation_short_circuits_current_iteration asserts the key is gone from session.state after the run raises.
Steps to Reproduce
On main at 67ba500:
- Create an
Agentwith a tool andmiddleware=[ToolApprovalMiddleware()](or usecreate_harness_agent), and oneAgentSession. - Run 1: the model calls the tool once, then the provider raises.
- Run 2 on the same session: the budget from run 1 is still in
session.state, and the loop starts at the oldattempt_count.
Minimal Reproduction
A scripted client, no network. max_iterations=2 keeps it short; with the default of 40 the counts add up over several failed turns.
import asyncio
from collections.abc import AsyncIterable
from agent_framework import (
Agent,
AgentSession,
BaseChatClient,
ChatMiddlewareLayer,
ChatResponse,
ChatResponseUpdate,
Content,
FunctionInvocationLayer,
Message,
ResponseStream,
ToolApprovalMiddleware,
tool,
)
executed: list[str] = []
@tool
def lookup(query: str) -> str:
executed.append(query)
return f"result for {query}"
class ScriptedClient(FunctionInvocationLayer, ChatMiddlewareLayer, BaseChatClient):
def __init__(self, turns, **kwargs):
super().__init__(**kwargs)
self.turns = list(turns)
self.tool_choices = []
def _next(self, options):
self.tool_choices.append(options.get("tool_choice"))
turn = self.turns.pop(0)
if isinstance(turn, Exception):
raise turn
return turn
def _inner_get_response(self, *, messages, stream, options, **kwargs):
if stream:
async def updates() -> AsyncIterable[ChatResponseUpdate]:
yield ChatResponseUpdate(role="assistant", contents=[self._next(options)])
return ResponseStream(updates(), finalizer=ChatResponse.from_updates)
async def get() -> ChatResponse:
return ChatResponse(messages=[Message(role="assistant", contents=[self._next(options)])])
return get()
async def main(stream: bool) -> None:
executed.clear()
client = ScriptedClient(
[
Content.from_function_call(call_id="c1", name="lookup", arguments='{"query": "first"}'),
RuntimeError("transient provider error"),
Content.from_function_call(call_id="c2", name="lookup", arguments='{"query": "retry"}'),
Content.from_text("final answer"),
],
function_invocation_configuration={"max_iterations": 2},
)
agent = Agent(client=client, tools=[lookup], middleware=[ToolApprovalMiddleware()])
session = AgentSession()
async def run(text: str):
if stream:
response_stream = agent.run(text, session=session, stream=True)
async for _ in response_stream:
pass
return await response_stream.get_final_response()
return await agent.run(text, session=session)
print(f"stream={stream}")
try:
await run("look it up")
except RuntimeError as exc:
print(" run 1 raised:", exc)
budget = session.state.get("_function_invocation_budget_state") or {}
print(" budget left in session:", {k: budget[k] for k in ("attempt_count", "total_function_calls") if k in budget})
calls_before = len(client.tool_choices)
response = await run("look it up again")
print(" run 2 tool_choice per provider call:", client.tool_choices[calls_before:])
print(" tools executed:", executed)
print(" run 2 text:", repr(response.text))
asyncio.run(main(stream=False))
asyncio.run(main(stream=True))Output on main (67ba500):
stream=False
run 1 raised: transient provider error
budget left in session: {'attempt_count': 2, 'total_function_calls': 1}
run 2 tool_choice per provider call: ['none']
tools executed: ['first']
run 2 text: 'Function invocation limit reached before a final answer could be produced.'
stream=True
run 1 raised: transient provider error
budget left in session: {'attempt_count': 2, 'total_function_calls': 1}
run 2 tool_choice per provider call: ['none']
tools executed: ['first']
run 2 text: 'Function invocation limit reached before a final answer could be produced.'
Expected for run 2: tool_choice ['auto', 'auto'], tools executed ['first', 'retry'], text 'final answer'.
Error Messages and Stack Traces
No error on the retry. Run 2 gets tool_choice="none" and returns the limit fallback text.
Package Versions
agent-framework-core 1.23.0 (main at 67ba500)
Python Version
3.14.5
Operating System
macOS 26.6.2
Regression
Not checked against older releases.
Additional Context
A streamed run that the caller stops reading early (breaks out of the loop without an error) also leaves the budget in the session. That case is different: nothing tells the loop whether the run was abandoned or will be resumed, so I left it out here.
I have a fix and regression tests ready locally (clear the session budget with _clear_budget_state_from_session when the non-streaming loop raises or the stream fails, skipping ResponseInvalidatedException, which keeps its own cleanup). I'll wait for agreement before opening a PR.
This issue was prepared with AI assistance. I reviewed the analysis and ran the reproduction myself.
Acknowledgements
- I searched existing issues and did not find a duplicate.
- I personally verified this behavior and the reproduction details are authentic.
- I will wait for explicit maintainer agreement before starting implementation of a non-trivial change.
Metadata
Metadata
Labels
Type
Projects
- StatusShow more project fieldsNo status
Activity
github-actions commentedon Oct 25, 2026
🤖 Automated triage reproduction notes (agent-authored — trust but verify)
Agent analysis
Repro: python/packages/core/agent_framework/_tools.py::_get_response_with_function_invocation and _stream_response_with_function_invocation leave the session budget populated when an ordinary provider exception escapes. This occurs when ToolApprovalMiddleware shares its session budget with a run that executes a tool and then fails. Minimal repro: use the scripted client in python/packages/core/tests/core/test_issue_9677.py, fail the first run after one tool call, then retry with the same AgentSession.
python/packages/core/tests/core/test_issue_9677.py1.23.01.23.0eavanvalkenburg commentedon Oct 26, 2026
/plan Define cleanup of per-attempt function-invocation budget state after provider errors and cancellation without resetting budgets that must survive resumable approvals. Cover same-session retries and streaming/non-streaming parity.
eavanvalkenburg commentedon Nov 12, 2026
/plan Define cleanup of per-attempt function-invocation budget state after provider errors and cancellation without resetting budgets that must survive resumable approvals. Cover same-session retries and streaming/non-streaming parity.
github-actions commentedon Nov 12, 2026
Summary
When
ToolApprovalMiddlewareis used, it puts the tool-loop budget (attempt_count,total_function_calls,start_time) intosession.stateand passes the same dict to the loop. The loop only removes it on normal return or on provider invalidation. If a provider error, timeout or cancellation escapes, the counts stay in the session. A retry on the same session then starts with fewer iterations, or with tools disabled. The issue says streaming and non-streaming behave the same.What I confirmed in the code (read only, nothing run)
_tool_approval.py~L454._clear_budget_state_from_sessiononly on its return paths. The one error path with cleanup is_response_invalidation_cleanup, which handles onlyResponseInvalidatedException(_tools.py~L4880-4925). Other exceptions escape without cleanup.attempt_countandtotal_function_callsback from the leftover dict.start_timeis kept withsetdefault, somax_duration_secondsalso measures from the failed run._clear_budget_state_from_sessionalready keeps the budget when approvals are pending or a mixed-pause batch is deferred. It saves a portable snapshot in the second case. That is the guard the request asks for, so a failure cleanup can reuse it.python/packages/core/AGENTS.mdand spec 004 say loop changes need streaming/non-streaming parity and core-team agreement. Spec 004 already lists provider invalidation as clearing budget state.Recommendation
Code change, and it is warranted. This is a framework bug with a clear contract: a failed attempt should not eat into the next attempt's limits. It hits every harness agent, because
create_harness_agentadds the middleware by default. The only workaround is for the caller to deletesession.state["_function_invocation_budget_state"]after a failure, which depends on a private key. The scope is small, since the existing helper already knows how to preserve approval state. The issue is already labelled ready for implementation. An open PR exists (#10146). I did not review it and formed this plan independently.Proposed plan (a proposal only; no changes have been made)
Define the rule. Any exception that ends a loop run, including
Exceptionandasyncio.CancelledError, releases the session budget through_clear_budget_state_from_session. Because that helper keeps the budget when approvals are pending or a mixed-pause batch is deferred, a run that is really paused for approval keeps it. A run that fails while approvals are still pending also keeps it. Treat this as intended and test it.Non-streaming loop. Wrap the body of
_get_response_with_function_invocationso that a raised exception clears the budget before propagating. Useexcept BaseExceptionortry/finallywith a success flag, so cancellation is covered.CancelledErroris not anException.Streaming loop. Apply the same wrapper around
_stream_response_with_function_invocation. Cover errors raised while iterating the provider stream and during finalization.Invalidation.
ResponseInvalidatedExceptionalready restores its own continuation state, and the clear is idempotent. The new handler should not alter that restore.Check three edges first.
client_kwargs. After the session key is popped, the next run'ssetdefaultmust create a fresh dict, not reuse the stale one. Clearing the shared dict in place may be needed.Tests, using the issue's scripted client in both modes:
tool_choice['auto', 'auto']and both tools executed.max_duration_seconds, with a gap between failure and retry.max_function_callsbudget.test_response_invalidation_short_circuits_current_iterationand the mixed-pause budget tests still pass.Docs. Add one line to spec 004 (the scenario table, with test names) saying failed or cancelled runs release the request budget except while approvals are pending. No other docs are needed.
How we'd know it's fixed: the issue's reproduction prints
['auto', 'auto'],['first', 'retry']and'final answer'for both modes, and the approval-resume budget tests are unchanged.Open point: whether the abandoned-stream case should get a separate follow-up issue.