Repository navigation
Description
Observed Behavior
python/samples/02-agents/auto_retry.py is the sample linked for rate-limit handling (#1362, #6942), but none of its three approaches retries a 429.
All three retry on retry_if_exception_type(openai.RateLimitError). The chat clients raise ChatClientException and keep the RateLimitError on __cause__, so the predicate never matches and the first 429 goes straight to the caller.
The OpenAI SDK's own retries (max_retries=2 by default) still run underneath. It is the sample's own retry layer that never fires.
Expected Behavior
Each approach retries the 429, up to RETRY_ATTEMPTS tries, and returns the response once the service answers.
Steps to Reproduce
- Build a
FoundryChatClientwhose OpenAI client sends every request to anhttpx.MockTransportthat answers 429 ("code": "rate_limit_exceeded"). SDK retries are off (max_retries=0), so only the sample's layer is measured. - Add the sample's class-based middleware (approach 2a) and run the agent.
- Count the requests: 1, then
ChatClientExceptionis raised. With a predicate that checks__cause__, the same setup makes 3 requests.
Minimal Reproduction
import asyncio
import sys
import httpx
from agent_framework import Agent, ChatContext, ChatMiddleware
from agent_framework.foundry import FoundryChatClient
from openai import AsyncOpenAI, RateLimitError
from tenacity import AsyncRetrying, retry_if_exception, retry_if_exception_type, stop_after_attempt
requests = 0
def always_429(request: httpx.Request) -> httpx.Response:
global requests
requests += 1
error = {"message": "Rate limit reached", "type": "requests", "param": None, "code": "rate_limit_exceeded"}
return httpx.Response(429, json={"error": error})
class FakeProjectClient: # stands in for AIProjectClient
def get_openai_client(self, **kwargs):
return AsyncOpenAI(
api_key="unused",
base_url="https://example.invalid/openai/v1",
max_retries=0, # measure only the sample's retry layer
http_client=httpx.AsyncClient(transport=httpx.MockTransport(always_429)),
)
async def close(self):
pass
# The predicate auto_retry.py uses today, and one that checks the cause.
SAMPLE = retry_if_exception_type(RateLimitError)
CAUSE = retry_if_exception(lambda e: isinstance(e.__cause__, RateLimitError))
class RateLimitRetryMiddleware(ChatMiddleware): # approach 2a from auto_retry.py, minus the back-off
def __init__(self, predicate):
self.predicate = predicate
async def process(self, context: ChatContext, call_next) -> None:
async for attempt in AsyncRetrying(stop=stop_after_attempt(3), retry=self.predicate, reraise=True):
with attempt:
await call_next()
async def main(predicate) -> None:
client = FoundryChatClient(project_client=FakeProjectClient(), model="m")
agent = Agent(client=client, middleware=[RateLimitRetryMiddleware(predicate)])
try:
await agent.run("Say hello!")
except Exception as exc:
print(f"{type(exc).__name__} (cause: {type(exc.__cause__).__name__}), requests={requests}")
asyncio.run(main(CAUSE if "--cause" in sys.argv else SAMPLE))
# no flag: ChatClientException (cause: RateLimitError), requests=1
# --cause: ChatClientException (cause: RateLimitError), requests=3Error Messages and Stack Traces
From the reproduction with the PyPI packages, print(f"{type(exc).__name__}: {exc}"):
ChatClientException: ("<class 'agent_framework_foundry._chat_client.FoundryChatClient'> service failed to complete the prompt: Error code: 429 - Rate limit reached", RateLimitError('Error code: 429 - Rate limit reached'))
Package Versions
agent-framework-core 1.23.0, agent-framework-foundry 1.15.0, agent-framework-openai 1.17.0, openai 3.41.0, tenacity 9.2.1 (PyPI); also main at b7884ad8
Python Version
Python 3.13 (PyPI packages), Python 3.11 (main)
Operating System
macOS and Linux
Regression
No. As far as I can tell from the code, the OpenAI client already wrapped SDK errors in ChatClientException when the sample was added in #4223, so I don't think it ever retried. I did not run that older version.
Additional Context
I also ran the sample file itself, importing its three approaches unchanged, on FoundryChatClient with the same mock transport: first request 429, second 200, SDK retries off.
| Approach | main |
Predicate that checks __cause__ |
|---|---|---|
| 1, class decorator | 1 request, ChatClientException |
2 requests, response returned |
| 2a, class-based middleware | 1 request, ChatClientException |
2 requests, response returned |
| 2b, function-based middleware | 1 request, ChatClientException |
2 requests, response returned |
| any approach, first answer 400 | 1 request | 1 request (not retried) |
The fix I'd suggest stays in the sample: one predicate, used by all three approaches.
def is_rate_limit_error(exc: BaseException) -> bool:
"""Return True when a chat client call failed with a rate limit (429) error."""
return isinstance(exc, ChatClientException) and isinstance(exc.__cause__, RateLimitError)
# retry=retry_if_exception(is_rate_limit_error)This reads the provider error from __cause__, the way #9616 keeps provider details for callers. A typed rate-limit exception in the framework would be a separate API change; this only fixes the sample. The change is about 20 lines in the one file and is ready on my fork: main...sohom-cs:agent-framework:fix/auto-retry-sample-429. I'd like to take this and will open the PR once you agree with the direction.
Not tested: a real Foundry, Azure OpenAI or OpenAI rate limit (no credentials here), and streaming (approach 1 passes streaming through without retry by design).
AI assistance: I used Claude Code to trace the exception path, write the reproduction and draft this report. I ran the reproduction and checked the output myself.
Acknowledgements
- I searched existing issues and did not find a duplicate.
- I personally verified this behavior and the reproduction details are authentic.
- I will wait for explicit maintainer agreement before starting implementation of a non-trivial change.
Metadata
Metadata
Labels
Type
Projects
- StatusShow more project fieldsDone
Activity
github-actions commentedon Oct 31, 2026
🤖 Automated triage reproduction notes (agent-authored — trust but verify)
Agent analysis
Repro:
python/samples/02-agents/auto_retry.py::RateLimitRetryMiddleware.processmakes only one attempt when Foundry wraps an OpenAI 429RateLimitErrorinChatClientException; the same raw-exception predicate appears in all three sample approaches at lines 90, 137, and 166. Minimal repro: use aFoundryChatClientwith anhttpx.MockTransportthat always returns 429, disable SDK retries, attach the sample middleware with three attempts, and run the agent. The failure occurs withrequest_count == 1while the raisedChatClientException.__cause__isRateLimitError.python/packages/foundry/tests/foundry/test_issue_9904.pyagent-framework-foundry 1.15.0agent-framework-foundry 1.16.0Aaravkhanal commentedon Nov 2, 2026
I’d like to take this focused sample fix. I plan to make the retry examples recognize rate-limit errors wrapped by ChatClientException, and add an offline regression showing the configured retry count is honored. AI assistance is helping with issue analysis and implementation; I will review and validate the change. Could a maintainer confirm this scope and mark the issue ready-for-implementation before I begin?
sohom-cs commentedon Nov 2, 2026
Thanks Aarav Khanal (@Aaravkhanal). Just to avoid duplicating effort, I opened this issue after reproducing the bug and already have a fix prepared and tested across all three retry approaches, with the implementation available on my fork.
As noted in the original report, I’d like to take this through to a PR and have been holding off in accordance with the repository’s requirement for explicit maintainer agreement.
Eduard van Valkenburg (@eavanvalkenburg), could you confirm whether I can proceed with the existing implementation? I’m happy to incorporate any additional regression coverage or adjustments you’d recommend.