Skip to content

Python: [Feature]: Make timeout configurable on framework OpenAI clients (chat, chat completion, embedding) #9923

Description

Problem

Description

Add a timeout: float | None = None parameter to the three high-level OpenAI client classes:

  • OpenAIChatClient
  • OpenAIChatCompletionClient
  • OpenAIEmbeddingClient

This provides consistent HTTP timeout control across all OpenAI integrations in the Agent Framework.

Related: #9698 added timeout to FoundryChatClient.

Proposal

Expose the existing timeout capability from the raw clients (RawOpenAIChatClient, RawOpenAIChatCompletionClient, RawOpenAIEmbeddingClient) on the developer-facing client classes.

Behavior:

  • Single timeout value applies to all request phases (connect, read, write, pool)
  • Default None preserves OpenAI SDK defaults (connect: 5s, read/write/pool: 600s)
  • Ignored when async_client is provided — configure timeout on that client instead
  • Works for both OpenAI and Azure OpenAI routing

Example usage:

from agent_framework.openai import OpenAIChatClient

client = OpenAIChatClient(
    model="gpt-4o",
    api_key="sk-...",
    timeout=30.0,
)

Language

Python

Context

The raw OpenAI client classes already support a timeout parameter, but the high-level classes exposed to developers (OpenAIChatClient, OpenAIChatCompletionClient, OpenAIEmbeddingClient) do not. This forces developers to either accept the SDK default timeout or manually construct an async_client just to set a custom timeout, an inconsistent experience compared to using the OpenAI package directly.

This is a small quality-of-life improvement (not a blocker) that makes the framework’s OpenAI clients feel more familiar to developers adopting them from the OpenAI package.

Use Case and Evidence

Example usage:

from agent_framework.openai import OpenAIChatClient

client = OpenAIChatClient(
    model="gpt-4o",
    api_key="sk-...",
    timeout=30.0,
)

Desired Outcome

The desired outcome is to give developers consistent, easy‑to‑use control over HTTP timeouts for all OpenAI API requests (chat, chat completions, and embeddings) by exposing a timeout parameter on the framework’s high‑level client classes, so they can set a single timeout value that applies to connect, read, write, and pool phases matching the experience they already have with the raw OpenAI clients.

Alternatives Considered

Currently the workaround to define custom timeout is by constructing the classes with async_client.

Proposed Direction

To be consistent with OpenAI package. I think to a certain level the compatibility is met in the RawOpenAI*Clients however it is not exposed to the agentic framework developers. I also found this is not available in .NET package too.

Scope and Non-Goals

No response

Language

Python

Contribution Intent

I am willing to implement this

Acknowledgements

  • I searched existing issues and did not find a duplicate.
  • I will wait for explicit maintainer agreement before starting implementation of a non-trivial change.

Activity

  1. added
    pythonUsage: [Issues, PRs], Target: Python
    triageUsage: [Issues], Target: All issues that still need to be triaged
    on Nov 1, 2026
  2. removed
    triageUsage: [Issues], Target: All issues that still need to be triaged
    on Nov 1, 2026
  3. eavanvalkenburg commented on Nov 2, 2026

    @eavanvalkenburg
    Member

    Thanks for the proposal. Consistent timeout control on these high-level OpenAI clients is in scope for MAF, and the team will consider it. This is not a commitment to implementation or timing. We first need to plan the public parameter contract across the three clients, OpenAI/Azure routing, SDK defaults, and caller-supplied async_client precedence. The supported configured-client alternative remains available. We will account for #9924 in planning; accepting the issue scope does not approve that implementation or authorize further implementation before explicit agreement on the plan.

  4. eavanvalkenburg commented on Nov 2, 2026

    @eavanvalkenburg
    Member

    /plan Plan explicit timeout control on OpenAIChatClient, OpenAIChatCompletionClient, and OpenAIEmbeddingClient as an accepted core feature, coordinating existing #9924 rather than starting a parallel implementation. Reuse the raw-client/settings mechanism and establish exact omitted/None/value behavior from the actual SDK and existing raw-client contract instead of assuming None preserves SDK defaults. Keep caller-supplied async_client as-is with its transport/auth/timeout configuration authoritative. Cover OpenAI and Azure routing, overload/runtime/docstring consistency, supported timeout types, default preservation, and focused constructor/forwarding tests for all three clients. Resolve the maintainer question in #9924 (comment): determine whether invalid/non-positive values should be rejected by the framework or delegated to the SDK, and avoid duplicate inconsistent validation. Keep scope limited to this timeout request, not retries or a general transport abstraction. This planning request is not implementation approval for the existing PR.

  5. IshamMohamed commented on Nov 2, 2026

    @IshamMohamed
    Author

    Thank you.

  6. eavanvalkenburg commented on Nov 12, 2026

    @eavanvalkenburg
    Member

    /plan Plan explicit timeout control on OpenAIChatClient, OpenAIChatCompletionClient, and OpenAIEmbeddingClient as an accepted core feature, coordinating existing #9924 rather than starting a parallel implementation. Reuse the raw-client/settings mechanism and establish exact omitted/None/value behavior from the actual SDK and existing raw-client contract instead of assuming None preserves SDK defaults. Keep caller-supplied async_client as-is with its transport/auth/timeout configuration authoritative. Cover OpenAI and Azure routing, overload/runtime/docstring consistency, supported timeout types, default preservation, and focused constructor/forwarding tests for all three clients. Resolve the maintainer question in #9924 (comment): determine whether invalid/non-positive values should be rejected by the framework or delegated to the SDK, and avoid duplicate inconsistent validation. Keep scope limited to this timeout request, not retries or a general transport abstraction. This planning request is not implementation approval for the existing PR.

  7. github-actions commented on Nov 12, 2026

    @github-actions
    Contributor

    Summary

    You asked for a plan to add a timeout setting to OpenAIChatClient, OpenAIChatCompletionClient and OpenAIEmbeddingClient. It should reuse the existing raw-client mechanism, work with both OpenAI and Azure, leave a caller-supplied async_client untouched, and settle the validation question from the PR thread. I did not look at the open PR #9924 for solution evidence. I could not open the PR thread you linked, so the validation answer below comes from the code and is a proposal for you to confirm. I also could not read the installed openai SDK in this run. SDK behavior below is from my general knowledge of it and needs a check against the pinned version.

    What the code shows today (confirmed)

    • The issue says all three raw clients already support timeout. That is only partly true. Only RawOpenAIChatClient has it (_chat_client.py around line 755 and 826). I found no timeout in RawOpenAIChatCompletionClient, RawOpenAIEmbeddingClient, or any of the three high-level classes.

    • The shared helper load_openai_service_settings in _shared.py already takes timeout. It sets it on the SDK client only when timeout is not None, in three places:

      • the OpenAI client
      • the Azure /openai/v1 bridge client
      • the AsyncAzureOpenAI client
    • When a client is injected (async_client), the helper returns it before any timeout is applied, in both the OpenAI and Azure branches. So an injected client's own timeout, transport and auth are already left alone.

    • The timeout docstring on the raw chat client says "SDK default is used", but only the chat raw client documents it, and the high-level classes do not.

    Recommendation

    Code changes are warranted. This is a small feature that finishes work the shared helper already does, and it is already accepted as in scope. The workaround (build your own async_client) works but loses framework routing, header and env handling. I recommend one shared contract for all three clients.

    Omitted, None and value behavior

    • Omitted: do not pass timeout to the SDK, so the SDK default applies. I believe that default is about 600s total with a 5s connect time.
    • A number: a plain float applies to connect, read, write and pool together.
    • None: today the helper treats None as "omitted". In the SDK itself, None normally means "no timeout", which is different. Keeping None as "use default" is safest for compatibility, and the issue asks for it, but the docstrings must say so. Passing None cannot mean "disable timeouts". Disabling needs a custom async_client.
    • Supported types: keep float | None, as in the raw chat client. Accepting httpx.Timeout for per-phase control would be a separate, larger decision and is out of scope here. I'd suggest not adding it in this change.

    Invalid or non-positive values

    I propose delegating to the SDK and adding no framework check. Reasons:

    • The helper only forwards the value, and the three clients should behave the same.
    • A framework check would duplicate whatever the SDK and httpx do, and could drift from them.
    • The existing raw chat client does no validation.

    Please confirm that the SDK and httpx reject or handle 0 and negative values acceptably for the pinned SDK version. If they do not, add a single check inside load_openai_service_settings, so all three clients share it and no per-class copies exist.

    Proposed plan

    No changes have been made. This is a proposal, and it is not approval of PR #9924. That PR should be reconciled with this plan before anything else starts.

    1. Raw completion and embedding clients. Add timeout: float | None = None to every overload and the runtime signature of RawOpenAIChatCompletionClient and RawOpenAIEmbeddingClient, and pass it to load_openai_service_settings. Use the same docstring as the raw chat client.

    2. High-level clients. Add the same parameter to all overloads and the runtime signature of OpenAIChatClient, OpenAIChatCompletionClient and OpenAIEmbeddingClient. Forward it to the raw class's super().__init__. Include docstring text that says it is ignored when async_client is given, and that None and omitted both mean the SDK default.

    3. Helper. Leave _shared.py logic as it is. Only change it if the SDK check above shows validation is needed.

    4. Tests in python/packages/openai/tests, for each of the three clients:

      • A value reaches the SDK client's timeout on OpenAI routing.
      • A value reaches the client on Azure routing, both the AsyncAzureOpenAI path and the /openai/v1 bridge path.
      • Omitted and None leave the SDK default.
      • An injected async_client keeps its own timeout, and the framework value does not overwrite it.
    5. Docs. Update the package AGENTS.md only if it lists constructor options. Do not touch CHANGELOG.md, because release preparation handles it.

    6. Scope. No retries and no transport abstraction.

    Done when: all three clients accept timeout, the tests above pass, and the docstrings match the runtime behavior.

    Questions for maintainers

    1. Are you comfortable with None meaning "SDK default" rather than "no timeout"?
    2. Do you agree with delegating invalid-value handling to the SDK?

    Generated by Issue Planning for #9923 · copilot · auto · ~28 AIC · ⌖ ~1 AIC · ⊞ 4.8K · ◷

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

agentsUsage: [Issues, PRs], Target: Single agentpythonUsage: [Issues, PRs], Target: Pythonready-for-implementationMaintainers agree the problem and direction are ready for implementation.

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions