Skip to content

Python: Embedding clients have no close()/context manager and leak their internally created SDK clients #9814

Open

Description

Description

#9748 added deterministic lifecycle to chat clients (async with, close(), connector-owned resource release), but the embedding client family was left out:

  • BaseEmbeddingClient has no __aenter__/__aexit__/close(), so async with does not work for any embedding client, and there is no contract for releasing resources.
  • OpenAIEmbeddingClient creates an AsyncOpenAI/AsyncAzureOpenAI SDK client internally (owning an httpx connection pool) when async_client is not supplied, and offers no way to close it — only GC ever releases the pool.
  • BedrockEmbeddingClient creates a boto3 bedrock-runtime client internally with the same gap.
  • OllamaEmbeddingClient creates an AsyncClient (httpx transport) internally with the same gap.
  • GeminiEmbeddingClient already implements close() with _owns_client — the rest of the family is inconsistent with it and with the chat clients.

Repro

from agent_framework_openai import OpenAIEmbeddingClient

client = OpenAIEmbeddingClient(api_key="...")
async with client:  # TypeError: 'async with' received an object that does not implement __aenter__
    ...
# and the internally created AsyncOpenAI is never closed deterministically

Long-lived processes that build embedding clients per unit of work (e.g. per request/tenant) accumulate unclosed httpx pools until GC runs, which is exactly the class of leak #9748 fixed for chat clients.

Expected

Embedding clients follow the same contract #9748 established for chat clients: base-class context trio with a no-op default close(), and resource-owning implementations that close only the SDK client they created (caller-supplied clients stay open), idempotent on repeated close.

Environment

Activity

  1. added
    pythonUsage: [Issues, PRs], Target: Python
    triageUsage: [Issues], Target: All issues that still need to be triaged
    on Oct 29, 2026
  2. changed the title [-]Embedding clients have no close()/context manager and leak their internally created SDK clients[/-] [+]Python: Embedding clients have no close()/context manager and leak their internally created SDK clients[/+] on Oct 29, 2026
  3. eavanvalkenburg commented on Nov 2, 2026

    @eavanvalkenburg
    Member

    Thanks for raising this. The team will consider a common lifecycle contract for embedding clients, including deterministic release of framework-created SDK clients while leaving caller-supplied resources open. This is a foundational cross-provider concern, rather than a vendor-specific integration. This acknowledgement does not promise implementation or timing; the public lifecycle and ownership details need planning, using existing #9815 as input rather than starting competing work.

  4. eavanvalkenburg commented on Nov 2, 2026

    @eavanvalkenburg
    Member

    /plan Plan the accepted core ownership of a uniform embedding-client lifecycle, using existing #9815 and the chat-client/Gemini ownership contracts as input rather than starting another PR. Decide the additive BaseEmbeddingClient close/async-context contract, no-context use, legacy synchronous overrides, idempotency/reuse and close-failure behavior. Audit OpenAI/Azure OpenAI, Bedrock, Ollama and Gemini: release only framework-created SDK clients and leave injected clients, transports, sessions and credentials caller-owned; verify actual supported SDK close APIs and avoid unnecessary private-transport coupling. Cover normal/exceptional context exit, explicit/repeated close, construction/close failures and serialization of internal ownership flags across providers. Distinguish the confirmed lifecycle API gap from the unmeasured accumulating-pool claim. Review the existing implementation/tests for these boundaries; no ready label or new assignment while the public cross-package contract is pending, and retain triage for planning.

  5. eavanvalkenburg commented on Nov 12, 2026

    @eavanvalkenburg
    Member

    /plan Plan the accepted core ownership of a uniform embedding-client lifecycle, using existing #9815 and the chat-client/Gemini ownership contracts as input rather than starting another PR. Decide the additive BaseEmbeddingClient close/async-context contract, no-context use, legacy synchronous overrides, idempotency/reuse and close-failure behavior. Audit OpenAI/Azure OpenAI, Bedrock, Ollama and Gemini: release only framework-created SDK clients and leave injected clients, transports, sessions and credentials caller-owned; verify actual supported SDK close APIs and avoid unnecessary private-transport coupling. Cover normal/exceptional context exit, explicit/repeated close, construction/close failures and serialization of internal ownership flags across providers. Distinguish the confirmed lifecycle API gap from the unmeasured accumulating-pool claim. Review the existing implementation/tests for these boundaries; no ready label or new assignment while the public cross-package contract is pending, and retain triage for planning.

  6. github-actions commented on Nov 12, 2026

    @github-actions
    Contributor

    Summary

    Embedding clients have no shared way to be closed or used with async with. Chat clients got this in #9748. Gemini's embedding client has its own close(). The OpenAI, Bedrock and Ollama embedding clients create SDK clients internally and cannot release them on request.

    Recommendation

    Code changes are warranted. The missing API is a confirmed gap, and the maintainers have said they will consider a shared contract. This plan does not depend on #9815 (open) or its approach.

    The "accumulating pools" claim is unmeasured. Nothing in the issue or repository shows pool growth, so don't use it as justification. The API gap stands on its own: a caller cannot release resources deterministically. Docs and samples alone wouldn't fix that.

    What I read:

    • Core. BaseEmbeddingClient (_clients.py) has only get_embeddings. BaseChatClient has __aenter__, __aexit__ and a no-op close(). __aexit__ also accepts old synchronous close() overrides, and returns nothing so it never suppresses exceptions.
    • Gemini. close() runs only when _owns_client is true. It closes the async transport and then the sync one in finally. It has no _client_closed flag, so it isn't idempotent through that flag.
    • OpenAI embeddings. They store self.client and have no ownership flag or close(). The chat client does this with _owns_client = async_client is None, _client_closed, and await self.client.close().
    • Bedrock embeddings. They build a client from a boto3 session. Only the client the framework creates should be closed. Chat uses asyncio.to_thread(client.close).
    • Ollama embeddings. They create AsyncClient when none is given. Ollama chat closes client._client.aclose() and notes that Ollama has no public close. That is private coupling.
    • Serialization. to_dict skips keys that start with _ (and INJECTABLE), so private ownership flags stay out of serialized output.

    I did not run code, so SDK close behavior is from the existing chat code, not checked against installed SDKs.

    Proposed plan

    This is a proposal. Nothing has been changed.

    1. Core contract. Add __aenter__, __aexit__ and a no-op async close() to BaseEmbeddingClient, matching chat exactly. Context use stays optional, and clients remain usable without it. Exit calls close() on normal and exception exits and does not suppress exceptions. Awaiting is conditional, so legacy sync close() overrides keep working. This is additive.

    2. Idempotency and reuse. Use a private _owns_client flag set at construction, and a _client_closed flag set only after a successful close. A repeated close is then a no-op. If close raises, the flag stays unset so the caller can retry, and the exception propagates. Decide explicitly whether use after close is undefined or raises. Chat doesn't guard it; I'd mirror that and document it.

    3. OpenAI and Azure OpenAI. Own the client only when async_client is None. Close it via the public AsyncOpenAI.close(). Don't touch caller-supplied clients or credentials. For Azure, the credential or token provider stays caller-owned.

    4. Bedrock. Own the client only when client is None. Do not close an injected boto3_session; closing the client the session produced is fine. Call the client's close() through asyncio.to_thread, as chat does.

    5. Ollama. First verify whether the installed ollama version has a public close. If it doesn't, either reuse chat's private _client.aclose() and document the coupling, or leave it as one narrowly-commented exception. Prefer one shared helper over copies.

    6. Gemini. Keep ownership semantics. Align it to the shared flags so repeated close is idempotent, and confirm sync-close failure behavior (finally ordering).

    7. Tests, per provider.

      • Owned vs injected clients, with injected ones never closed.
      • async with on normal exit and on an exception, with the exception propagating.
      • Explicit close, then repeated close.
      • A close that raises, then retry.
      • A failure during construction before the SDK client exists.
      • to_dict / serialization excludes the ownership flags.
      • Legacy sync close() subclasses.
    8. Docs. Add a short note on the embedding lifecycle, mirroring the chat docs.

    Done when all four packages plus core pass these tests and behave like chat clients. Ready label and assignment should wait until the public cross-package contract (steps 1–2) is agreed. The issue stays in triage until then.

    Open points for maintainers

    • Should use after close() be a documented no-guarantee, or raise?
    • For Ollama, accept private-transport coupling, or wait for a public close upstream?

    Generated by Issue Planning for #9814 · copilot · auto · ~17 AIC · ⌖ ~1 AIC · ⊞ 4.8K · ◷

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    pythonUsage: [Issues, PRs], Target: PythontriageUsage: [Issues], Target: All issues that still need to be triaged

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions