server/discover fallback fails when 400 body isn't JSON-RPC shaped (non-JSON-RPC error envelope)

[philipthomas-tecman]

Description

Related to #1765 and #1790, but a distinct third case neither covers: a server/discover probe response that is HTTP 400 with a JSON body that isn't JSON-RPC shaped at all (no jsonrpc/id fields, a string error code rather than an integer, extra fields like param/type/details) causes the connection to fail hard instead of falling back to initialize, even though the HTTP status (400) is exactly the status #1765's fix already handles for non-JSON-RPC (plain-text) bodies.

Server response observed (Azure AI Foundry's Toolbox MCP proxy, rejecting an unrecognized server/discover method - the server does not yet implement SEP-2575):

HTTP/1.1 400 Bad Request
Content-Type: application/json; charset=utf-8

{
  "error": {
    "code": "invalid_payload",
    "message": "Failed to deserialize request body: Unknown ToolsetMCPMethodName value. (Parameter 'value')\nActual value was server/discover. [Request ID: ...]",
    "param": "$",
    "type": "invalid_request_error",
    "details": [],
    "additionalInfo": { "request_id": "..." }
  }
}

Note there is no jsonrpc field, no top-level id, and error.code is a string, not an integer JSON-RPC error code. This is a generic API-gateway/model-binding-failure envelope (the same shape OpenAI-compatible APIs commonly return for a validation error), not an MCP JSON-RPC error object.

Compare to #1765's repro, where the rejecting server returned a plain-text body ("Invalid session ID") on HTTP 404 - EnsureSuccessStatusCodeWithResponseBodyAsync throws HttpRequestException, and McpClientImpl.ConnectAsync's catch (HttpRequestException exception) when (exception.GetStatusCode() is BadRequest or NotFound) clause (the fix for #1765, present in 2.1.0) catches it and falls back correctly.

In our case, because the body is valid JSON (just not valid JSON-RPC), something upstream of that catch (most likely StreamableHttpClientSessionTransport's attempt to deserialize it as a JSON-RPC error envelope, per #1790's own description: "correctly converts the structured HTTP 400 response to McpProtocolException") appears to fail in a way that doesn't produce either (a) a McpProtocolException that reaches the generic fallback catch, or (b) a preserved HttpRequestException with the original 400 status that the #1765 fix's catch would still match. I have not stepped through this with a debugger against a minimal repro isolated from our specific caller (see below), so I can't point to the exact throw site with certainty - but the net effect is reproducible and consistent: the exception escapes ConnectAsync unhandled, McpClient.CreateAsync throws, and the caller sees a hard connection failure rather than a graceful fallback to initialize.

To Reproduce

I've reproduced the server-side behavior directly (curl), and the client-side behavior through Microsoft.Agents.AI.Foundry.Hosting's AddFoundryToolboxes (which calls McpClient.CreateAsync/ListToolsAsync with McpClientOptions.ProtocolVersion left unset, and doesn't expose a way to set it). I have not written a minimal repro using ModelContextProtocol.Core directly with no Agent Framework layer in between - flagging that as the gap in my own verification, not asserting it as unnecessary.

1. Connect to a Streamable HTTP server that returns the above malformed-for-JSON-RPC 400 body in response to server/discover, with ProtocolVersion unset:

var transport = new HttpClientTransport(new HttpClientTransportOptions
{
    Endpoint = new Uri("<toolbox endpoint>"),
    TransportMode = HttpTransportMode.StreamableHttp,
});
var client = await McpClient.CreateAsync(transport, new McpClientOptions());

2. Observe the connection fail (in our case, surfaced through Agent Framework as the toolbox never producing any tools - the calling agent ends up with zero real tools and no visible error beyond application logs).
3. Confirm the server itself is fine: a direct initialize call (skipping the discover probe) against the identical endpoint succeeds and negotiates 2025-11-25.
4. Confirm version-boundedness: decompiled every published ModelContextProtocol.Core release from 1.2.0 through 2.1.0 (the latest stable) - 1.4.1 and earlier have no server/discover code path at all (goes straight to initialize, unaffected); every 2.0.0-preview.1+ release, including the current stable 2.1.0, has the unconditional discover-first default and reproduces this hard failure.

Expected behavior

A 400 response to server/discover should fall back to initialize when the body cannot be classified as a recognized modern MCP error (-32020/-32021/-32022) - regardless of whether the body parses as JSON-RPC at all. Per #1790's proposed classification rule ("HTTP 400 with any JSON-RPC error: surface it as McpProtocolException; connection logic ... falls back for other errors"), I'd suggest explicitly extending that rule to a third case: HTTP 400 with a body that is not a well-formed JSON-RPC error object should also surface in a way ConnectAsync can catch and fall back on (either as HttpRequestException with the status preserved, matching the existing #1765-fix catch clause, or as a generic McpProtocolException that the existing generic fallback catch already handles) - rather than whatever currently causes it to escape unhandled.

Additional context

• Server is Azure AI Foundry's Toolbox MCP proxy (Microsoft.Agents.AI.Foundry.Hosting's AddFoundryToolboxes connects to it). Worth flagging since Agent Framework's AddFoundryToolboxes also has no way for a caller to set ProtocolVersion explicitly to avoid the probe (FoundryToolboxOptions only exposes ToolboxNames/ApiVersion/StrictMode) - so for that specific caller, a client-side workaround isn't available even in principle without this fix, or without Agent Framework exposing the option itself.
• This is very likely to recur for anyone whose MCP server hasn't yet rolled out SEP-2575 support and returns a non-JSON-RPC error body for an unrecognized method (a generic framework/gateway validation-error response is a common shape for exactly this situation).
• Confirmed workaround: pinning ModelContextProtocol.Core to 1.4.1 (predates the discover probe entirely) restores working tool discovery/calling against the same server.

Related: #1765, #1790.