Skip to content

Python: [Bug]: auto_retry.py sample never retries a 429: the client raises ChatClientException, not RateLimitError #9904

Closed

Description

Observed Behavior

python/samples/02-agents/auto_retry.py is the sample linked for rate-limit handling (#1362, #6942), but none of its three approaches retries a 429.

All three retry on retry_if_exception_type(openai.RateLimitError). The chat clients raise ChatClientException and keep the RateLimitError on __cause__, so the predicate never matches and the first 429 goes straight to the caller.

The OpenAI SDK's own retries (max_retries=2 by default) still run underneath. It is the sample's own retry layer that never fires.

Expected Behavior

Each approach retries the 429, up to RETRY_ATTEMPTS tries, and returns the response once the service answers.

Steps to Reproduce

  1. Build a FoundryChatClient whose OpenAI client sends every request to an httpx.MockTransport that answers 429 ("code": "rate_limit_exceeded"). SDK retries are off (max_retries=0), so only the sample's layer is measured.
  2. Add the sample's class-based middleware (approach 2a) and run the agent.
  3. Count the requests: 1, then ChatClientException is raised. With a predicate that checks __cause__, the same setup makes 3 requests.

Minimal Reproduction

import asyncio
import sys

import httpx
from agent_framework import Agent, ChatContext, ChatMiddleware
from agent_framework.foundry import FoundryChatClient
from openai import AsyncOpenAI, RateLimitError
from tenacity import AsyncRetrying, retry_if_exception, retry_if_exception_type, stop_after_attempt

requests = 0


def always_429(request: httpx.Request) -> httpx.Response:
    global requests
    requests += 1
    error = {"message": "Rate limit reached", "type": "requests", "param": None, "code": "rate_limit_exceeded"}
    return httpx.Response(429, json={"error": error})


class FakeProjectClient:  # stands in for AIProjectClient
    def get_openai_client(self, **kwargs):
        return AsyncOpenAI(
            api_key="unused",
            base_url="https://example.invalid/openai/v1",
            max_retries=0,  # measure only the sample's retry layer
            http_client=httpx.AsyncClient(transport=httpx.MockTransport(always_429)),
        )

    async def close(self):
        pass


# The predicate auto_retry.py uses today, and one that checks the cause.
SAMPLE = retry_if_exception_type(RateLimitError)
CAUSE = retry_if_exception(lambda e: isinstance(e.__cause__, RateLimitError))


class RateLimitRetryMiddleware(ChatMiddleware):  # approach 2a from auto_retry.py, minus the back-off
    def __init__(self, predicate):
        self.predicate = predicate

    async def process(self, context: ChatContext, call_next) -> None:
        async for attempt in AsyncRetrying(stop=stop_after_attempt(3), retry=self.predicate, reraise=True):
            with attempt:
                await call_next()


async def main(predicate) -> None:
    client = FoundryChatClient(project_client=FakeProjectClient(), model="m")
    agent = Agent(client=client, middleware=[RateLimitRetryMiddleware(predicate)])
    try:
        await agent.run("Say hello!")
    except Exception as exc:
        print(f"{type(exc).__name__} (cause: {type(exc.__cause__).__name__}), requests={requests}")


asyncio.run(main(CAUSE if "--cause" in sys.argv else SAMPLE))
# no flag:  ChatClientException (cause: RateLimitError), requests=1
# --cause:  ChatClientException (cause: RateLimitError), requests=3

Error Messages and Stack Traces

From the reproduction with the PyPI packages, print(f"{type(exc).__name__}: {exc}"):

ChatClientException: ("<class 'agent_framework_foundry._chat_client.FoundryChatClient'> service failed to complete the prompt: Error code: 429 - Rate limit reached", RateLimitError('Error code: 429 - Rate limit reached'))

Package Versions

agent-framework-core 1.23.0, agent-framework-foundry 1.15.0, agent-framework-openai 1.17.0, openai 3.41.0, tenacity 9.2.1 (PyPI); also main at b7884ad8

Python Version

Python 3.13 (PyPI packages), Python 3.11 (main)

Operating System

macOS and Linux

Regression

No. As far as I can tell from the code, the OpenAI client already wrapped SDK errors in ChatClientException when the sample was added in #4223, so I don't think it ever retried. I did not run that older version.

Additional Context

I also ran the sample file itself, importing its three approaches unchanged, on FoundryChatClient with the same mock transport: first request 429, second 200, SDK retries off.

Approach main Predicate that checks __cause__
1, class decorator 1 request, ChatClientException 2 requests, response returned
2a, class-based middleware 1 request, ChatClientException 2 requests, response returned
2b, function-based middleware 1 request, ChatClientException 2 requests, response returned
any approach, first answer 400 1 request 1 request (not retried)

The fix I'd suggest stays in the sample: one predicate, used by all three approaches.

def is_rate_limit_error(exc: BaseException) -> bool:
    """Return True when a chat client call failed with a rate limit (429) error."""
    return isinstance(exc, ChatClientException) and isinstance(exc.__cause__, RateLimitError)

# retry=retry_if_exception(is_rate_limit_error)

This reads the provider error from __cause__, the way #9616 keeps provider details for callers. A typed rate-limit exception in the framework would be a separate API change; this only fixes the sample. The change is about 20 lines in the one file and is ready on my fork: main...sohom-cs:agent-framework:fix/auto-retry-sample-429. I'd like to take this and will open the PR once you agree with the direction.

Not tested: a real Foundry, Azure OpenAI or OpenAI rate limit (no credentials here), and streaming (approach 1 passes streaming through without retry by design).

AI assistance: I used Claude Code to trace the exception path, write the reproduction and draft this report. I ran the reproduction and checked the output myself.

Acknowledgements

  • I searched existing issues and did not find a duplicate.
  • I personally verified this behavior and the reproduction details are authentic.
  • I will wait for explicit maintainer agreement before starting implementation of a non-trivial change.

Activity

  1. added
    pythonUsage: [Issues, PRs], Target: Python
    triageUsage: [Issues], Target: All issues that still need to be triaged
    on Oct 31, 2026
  2. added
    reproducedUsage: [Issues], Target: all issues that can be reproduced by the triage workflow
    on Oct 31, 2026
  3. github-actions commented on Oct 31, 2026

    @github-actions
    Contributor

    🤖 Automated triage reproduction notes (agent-authored — trust but verify)

    Agent analysis

    Repro: python/samples/02-agents/auto_retry.py::RateLimitRetryMiddleware.process makes only one attempt when Foundry wraps an OpenAI 429 RateLimitError in ChatClientException; the same raw-exception predicate appears in all three sample approaches at lines 90, 137, and 166. Minimal repro: use a FoundryChatClient with an httpx.MockTransport that always returns 429, disable SDK retries, attach the sample middleware with three attempts, and run the agent. The failure occurs with request_count == 1 while the raised ChatClientException.__cause__ is RateLimitError.

    • Failing test: python/packages/foundry/tests/foundry/test_issue_9904.py
    • Files examined: python/samples/02-agents/auto_retry.py, python/packages/foundry/agent_framework_foundry/_chat_client.py, python/packages/openai/agent_framework_openai/_chat_client.py, python/packages/foundry/tests/foundry/test_foundry_chat_client.py, python/packages/foundry/pyproject.toml
    • Tests run: python/packages/foundry/tests/foundry/test_issue_9904.py
    • Reported version: agent-framework-foundry 1.15.0
    • Current version: agent-framework-foundry 1.16.0
  4. removed
    triageUsage: [Issues], Target: All issues that still need to be triaged
    on Oct 31, 2026
  5. Aaravkhanal commented on Nov 2, 2026

    @Aaravkhanal

    I’d like to take this focused sample fix. I plan to make the retry examples recognize rate-limit errors wrapped by ChatClientException, and add an offline regression showing the configured retry count is honored. AI assistance is helping with issue analysis and implementation; I will review and validate the change. Could a maintainer confirm this scope and mark the issue ready-for-implementation before I begin?

  6. sohom-cs commented on Nov 2, 2026

    @sohom-cs
    Author

    Thanks Aarav Khanal (@Aaravkhanal). Just to avoid duplicating effort, I opened this issue after reproducing the bug and already have a fix prepared and tested across all three retry approaches, with the implementation available on my fork.

    As noted in the original report, I’d like to take this through to a PR and have been holding off in accordance with the repository’s requirement for explicit maintainer agreement.

    Eduard van Valkenburg (@eavanvalkenburg), could you confirm whether I can proceed with the existing implementation? I’m happy to incorporate any additional regression coverage or adjustments you’d recommend.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

pythonUsage: [Issues, PRs], Target: Pythonready-for-implementationMaintainers agree the problem and direction are ready for implementation.reproducedUsage: [Issues], Target: all issues that can be reproduced by the triage workflow

Type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions