.NET: Add RoutingChatClient for routing requests across multiple chat clients

[westey-m]

Motivation & Context

Applications that let a user switch the underlying model/provider mid-conversation (e.g. from OpenAI to Anthropic, or between two models) lose the conversation context on the new provider, forcing manual rehydration of the chat history. This is exactly the scenario described in #6865.

This PR introduces RoutingChatClient, an experimental IChatClient decorator (.NET) that holds multiple named inner chat clients and routes each request to one of them based on a per-session active destination. Because the conversation is carried client-side by the agent's session, switching destinations mid-conversation preserves the full history for whichever client handles the next turn — no manual rehydration needed.

Description & Review Guide

• What are the major changes?
  • New RoutingChatClient (Microsoft.Agents.AI/ChatClient/): an IChatClient decorator over multiple named inner clients, with:
    • Per-session active-destination selection via GetActiveDestinationKey / SetActiveDestinationKey (null key falls back to the first inner client, then to the fallback factory).
    • An optional async (ValueTask) Router callback for custom heuristics (e.g. inference-based selection).
    • An optional fallback factory for keys not backed by a registered inner client; fallback clients are created per request and disposed after use by default, with an opt-out (DisableFallbackChatClientDisposal).
    • Three constructors covering inner-clients-only, fallback-only, and both.
  • Supporting types: RoutingChatClientOptions, RoutingContext, RoutingState, plus JSON serialization registration in AgentJsonUtilities.
  • Full unit-test suite (RoutingChatClientTests).
  • New sample Agent_Step22_MultiModelRouting demonstrating multi-model routing, client-side chat history, disabling encrypted reasoning content across models, and retrieving the routing client via GetService (added to the solution and the Agents README).
• What is the impact of these changes?
  • Net-new, additive, experimental API (MAAI001). No existing behavior is changed.
• What do you want reviewers to focus on?
  • The fallback client lifecycle (per-request create + dispose-after-use, and the opt-out), and the null-key fallback ordering (first inner client → fallback factory → InvalidOperationException).

Related Issue

References #6865. This PR implements the .NET side of the request; the Python side remains tracked by the same issue, so it is intentionally referenced without a closing keyword.

Contribution Checklist

• The code builds clean without any errors or warnings
• All unit tests pass, and I have added new tests where possible
• The PR follows the Contribution Guidelines
• This PR is linked to an issue and there is no other open PR for this issue (see Related Issue above).
• This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

[Copilot]

Pull request overview

This PR adds a new experimental .NET IChatClient decorator (RoutingChatClient) that can route each request to one of multiple named inner chat clients based on per-session routing state, enabling model/provider switching mid-conversation without losing client-side chat history.

Changes:

• Introduces RoutingChatClient plus supporting types (RoutingChatClientOptions, RoutingContext, RoutingState) and registers routing state for JSON serialization.
• Adds a full unit test suite for routing behavior, session isolation, and fallback client lifecycle/disposal.
• Adds a new sample (Agent_Step22_MultiModelRouting) and wires it into the samples README and solution.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
dotnet/src/Microsoft.Agents.AI/ChatClient/RoutingChatClient.cs	Implements the routing decorator, per-session state, fallback factory behavior, and disposal semantics.
dotnet/src/Microsoft.Agents.AI/ChatClient/RoutingChatClientOptions.cs	Adds configuration knobs for routing callback, state key, and fallback disposal behavior.
dotnet/src/Microsoft.Agents.AI/ChatClient/RoutingContext.cs	Defines the callback context passed to router/fallback factory.
dotnet/src/Microsoft.Agents.AI/ChatClient/RoutingState.cs	Defines the session-serializable routing state.
dotnet/src/Microsoft.Agents.AI/AgentJsonUtilities.cs	Registers RoutingState for source-generated JSON serialization.
dotnet/tests/Microsoft.Agents.AI.UnitTests/ChatClient/RoutingChatClientTests.cs	Adds unit tests for routing, session scoping, fallback behavior, and disposal.
dotnet/samples/02-agents/Agents/README.md	Adds the Step 22 sample to the samples list.
dotnet/samples/02-agents/Agents/Agent_Step22_MultiModelRouting/README.md	Documents the new multi-model routing sample and key concepts.
dotnet/samples/02-agents/Agents/Agent_Step22_MultiModelRouting/Program.cs	Implements the sample showing switching active model and using fallback factory.
dotnet/samples/02-agents/Agents/Agent_Step22_MultiModelRouting/Agent_Step22_MultiModelRouting.csproj	Adds the sample project and references.
dotnet/agent-framework-dotnet.slnx	Includes the new sample project in the solution.

[github-actions]

Automated Code Review

Reviewers: 5 | Confidence: 92% | Result: All clear

Reviewed: Correctness, Security Reliability, Test Coverage, Failure Modes, Design Approach

---

Automated review by westey-m's agents

[rogerbarreto]

Suggest proposing this into Microsoft.Extensions.AI package as the source of this abstraction as a MEAI001 experiment seems to be a better fit.

Additionally if this implementation is intended to be rapidly used by the Harness, I would keep it in a separate folder for MEAI-ChatClient's like we had before when proposing different IChatClient impl.

Suggested change

	namespace Microsoft.Agents.AI;
	namespace Microsoft.Extensions.AI;

[rogerbarreto]

At first a ChatClient should not be aware of agents except if used internally, we do that in the OTEL Agent to reuse some logic. I think proposing this as a public API opens a wrong precedent how IChatClient should be implemented.

[joshuajyue]

Hi, this is currently being worked on in dotnet/extensions. See dotnet/extensions Issue #7647.

The desired Agent Framework routing behavior can be achieved with the WIP RoutingChatClient. See this gist for a quick overview and proof-of-concept code.

P.S.
Source code for dotnet/extensions RoutingChatClient can be found here, if you are curious.

[westey-m]

@joshuajyue, thanks for sharing. Nice that this is being picked up in MEAI.

Two suggestions:

1. A nice to have out-of-the-box implementation of the abstract base class would be one that just takes a callback Func as input, that provides the SelectClient functionality, so that users don't need to always implement a class if they just have a simple requirement.
2. The looping, exception handling and maximum attempt counts in RoutingChatClient does seem like they are designed specifically to support failover scenarios. FailoverChatClient uses that functionality and only adds support for picking the next ChatClient from a list. Just in terms of naming it does seem like RoutingChatClient is already a failover chat client of sorts, while FailoverChatClient is a FailoverChatClient that picks from a pre-created list. It would be good to consider naming for if we wanted to add a second FailoverChatClient that uses a different mechanism to select the next chat client after a failed attempt. Or perhaps FailoverChatClient should be a base abstract class itself and the looping and exception handling moves to it from the base RoutingChatClient (although I'm not sure yet if this is right).

CC @jozkee

[joshuajyue]

Hi @westey-m, both are good suggestions.

I think the callback idea is good for simple scenarios -- potentially a RoutingChatClient.Create(Func<...>) factory so users don’t need a derived class.

On failover, the base provides the mechanics without requiring a policy to use them: an implementation can return null when previousAttempt is non-null to avoid fallback behavior and rethrow the previous exception. The concrete client is specifically ordered list-based failover, so renaming it to OrderedFailoverChatClient should address the ambiguity. That said, I agree the current base is failover-oriented, so the separation is worth considering further.

Edit:
I ended up taking both of your suggestions:

RoutingChatClient is now a minimal one-shot abstraction in Microsoft.Extensions.AI.Abstractions, with SelectClientAsync and a callback-based Create(Func<...>) factory.

Failover-specific behavior moved into an abstract FailoverChatClient, which adds SelectNextClientAsync, attempt limits, terminal attempt reporting, and the retry/streaming-safety machinery. OrderedFailoverChatClient is now the concrete list-based implementation.

I think this more clearly separates routing from failover while leaving room for other failover-selection strategies.

[joshuajyue]

Hi @westey-m. The PR is at extensions #7662. We are curious to see if and how Agent Framework can build on top of MEAI to address this issue.