Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions python/packages/core/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,28 @@ agent_framework/
for streaming and non-streaming requests. Provides `__aenter__()` returning `self`, `__aexit__()` delegating to
`close()`, and a default no-op async `close()`. Connectors override cleanup for owned connections and credentials;
injected resources remain caller-owned. `SupportsChatGetResponse` does not require lifecycle methods.
- **`FailoverChatClient`** (`_failover.py`, experimental under `CHAT_CLIENT_FAILOVER`) - Tries an ordered list of
chat clients for each model call. It owns the single function-invocation loop and calls each candidate below that
candidate's `FunctionInvocationLayer` (the same entry the candidate's own loop uses per model call), so candidate
chat middleware and telemetry run per attempt while tools run once, in the failover client's loop. A failure after
earlier model calls requested tools therefore never re-executes them. Every model call starts again with the first
client. Any `Exception` fails over unless `should_failover` returns `False`; cancellation is never caught, and
`ResponseInvalidatedException` and `MiddlewareException` always propagate. Streaming fails over only before an
update with contents is released; content-less updates are held until then and discarded with a failed attempt.
History must be client-managed: `store=True` or a service continuation option (the union of candidates'
`SERVICE_CONTINUATION_OPTION_NAMES`, including a nested failover client's computed union) raises
`ChatClientInvalidRequestException` in `get_response` before the function-invocation loop resolves approvals, so
no approved tool runs first; the per-model-call check remains for middleware-modified options. Candidates with
`STORES_BY_DEFAULT` receive `store=False` per attempt when `store` is unset. The caller's session is carried past
the failover client's own chat middleware layer through a request-local client kwarg, so candidate chat middleware
receives it. Reasoning is provider-specific: the client records a hashed client-type/model/service-URL source on
each `text_reasoning` content, in streamed updates and final responses
(`Content.additional_properties["agent_framework.failover"]`; content-level so it survives streamed-history
aggregation and persistence; first writer wins so nested clients keep the innermost producer) and omits that
reasoning, plus the group's compaction `has_reasoning` flag, from the per-attempt copy sent to a candidate that
does not accept that source. Calls and
results are always sent, the caller's transcript is not modified, and unrecorded reasoning is sent unchanged.
Candidates are caller-owned and not closed.

### Types (`_types.py`)

Expand Down
2 changes: 2 additions & 0 deletions python/packages/core/agent_framework/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,7 @@
"tool_called_check",
"tool_calls_present",
),
"._failover": ("FailoverChatClient",),
"._feature_stage": ("ExperimentalFeature", "ReleaseCandidateFeature"),
"._harness._agent": ("DEFAULT_HARNESS_INSTRUCTIONS", "create_harness_agent"),
"._harness._background_agents": (
Expand Down Expand Up @@ -514,6 +515,7 @@
"Executor",
"ExpectedToolCall",
"ExperimentalFeature",
"FailoverChatClient",
"FanInEdgeGroup",
"FanOutEdgeGroup",
"FieldTypes",
Expand Down
2 changes: 2 additions & 0 deletions python/packages/core/agent_framework/__init__.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ from ._evaluation import (
tool_called_check,
tool_calls_present,
)
from ._failover import FailoverChatClient
from ._feature_stage import ExperimentalFeature, ReleaseCandidateFeature
from ._harness._agent import DEFAULT_HARNESS_INSTRUCTIONS, create_harness_agent
from ._harness._background_agents import (
Expand Down Expand Up @@ -469,6 +470,7 @@ __all__ = [
"Executor",
"ExpectedToolCall",
"ExperimentalFeature",
"FailoverChatClient",
"FanInEdgeGroup",
"FanOutEdgeGroup",
"FieldTypes",
Expand Down
578 changes: 578 additions & 0 deletions python/packages/core/agent_framework/_failover.py

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions python/packages/core/agent_framework/_feature_stage.py
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ class ExperimentalFeature(str, Enum):

AGENT_TRIGGERED_COMPACTION = "AGENT_TRIGGERED_COMPACTION"
AGENT_HOOKS = "AGENT_HOOKS"
CHAT_CLIENT_FAILOVER = "CHAT_CLIENT_FAILOVER"
COMPUTER_USE = "COMPUTER_USE"
DECLARATIVE_AGENTS = "DECLARATIVE_AGENTS"
EVALS = "EVALS"
Expand Down
Loading
Loading