A lone surrogate in an outgoing message silently kills the stdio server's writer or the client session

Initial Checks

I confirm that I'm using the newest release of my line (the latest 2.x, or the latest 1.x if I'm still on v1)
I confirm that I searched for my issue in https://github.com/modelcontextprotocol/python-sdk/issues before opening this issue

Release line

2.x (current stable)

Description

A Python str that contains a lone surrogate cannot be encoded as UTF-8 JSON. Such strings are easy to get: os.fsdecode() on a file name that is not valid UTF-8 returns one, and so do open(..., errors="surrogateescape") and decoded argv or environment values. The string passes validation and model_dump(mode="json") and only fails in the last step, model_dump_json() inside the transport's writer task. That failure is not contained, so one bad string breaks the connection.

Server, stdio. stdout_writer (src/mcp/server/stdio.py) raises PydanticSerializationError: ... 'utf-8' codec can't encode character '\udcff' ... surrogates not allowed and ends. The process stays alive and never answers anything again (I watched it for 30 seconds). The call times out, or never returns if no read timeout is set, and later calls fail with Connection closed.
Client, every transport. stdin_writer (src/mcp/client/stdio.py, and the HTTP post path) fails the same way. The call and all later calls fail (CancelledError or Connection closed), and leaving the Client block raises ExceptionGroup.

Expected: the failing message costs only itself, for example an error for that request, or the character escaped as JSON allows (JavaScript's JSON.stringify("\udcff") returns "\udcff").

Where it happens, measured at protocol 2026-07-28 on main. The stdio results are the same at 2025-11-25, and on PyPI mcp 2.3.0 and 1.30.0:

A surrogate in	stdio	Streamable HTTP
tool result, resource text, prompt text, `resources/list` name, `tools/list` description, `completion/complete` values, log or progress message, elicitation request message (all server to client)	the server stops answering, the next call fails with `Connection closed`	that call fails, the session survives (after a `tools/list` description the next call fails too)
tool argument, prompt argument, completion argument, elicitation reply (all client to server)	client session torn down	client session torn down
resource URI passed to `read_resource`	that call raises `UnicodeEncodeError`	same
tool result `-> dict[str, str]`, a tool's exception message	fine	fine

Other serialization failures I tried on the low-level server (an object() value, non-UTF-8 bytes) come back as Internal server error and the session stays alive. The lone surrogate is the value that gets through every earlier stage and fails only at the final encode.

Related but not the same: #3522 and #3525 (the HTTP client dies on one failed POST, same failure shape), #3097 (this stdout_writer, for BrokenPipeError), #2454 (decode direction), #3554 (-> bytes structured output). I found no issue or PR that mentions surrogates, and no open PR that changes how the writers handle a failed serialization.

Example Code

import os
import sys

import anyio

from mcp import Client
from mcp.client.stdio import StdioServerParameters

if len(sys.argv) > 1 and sys.argv[1] == "server":
    from mcp.server.mcpserver import MCPServer

    mcp = MCPServer("files")

    @mcp.tool()
    def list_dir() -> list[str]:
        # stand-in for a directory listing with one file name that is not valid UTF-8
        return [os.fsdecode(b"ok.txt"), os.fsdecode(b"report-\xff.txt")]

    @mcp.tool()
    def ping() -> str:
        return "pong"

    @mcp.tool()
    def echo(text: str) -> str:
        return text

    mcp.run("stdio")
    sys.exit(0)

SERVER = StdioServerParameters(command=sys.executable, args=[__file__, "server"])


async def attempt(label: str, client: Client, name: str, args: dict | None = None) -> None:
    try:
        result = await client.call_tool(name, args or {}, read_timeout_seconds=5)
        print(f"  {label:34} ok: {result.content[0].text[:30]!r}")
    except BaseException as exc:
        print(f"  {label:34} {type(exc).__name__}: {str(exc)[:60]}")


async def main() -> None:
    print("1. the tool RESULT contains a lone surrogate")
    async with Client(SERVER) as client:
        await attempt("list_dir", client, "list_dir")
        await attempt("ping (healthy tool)", client, "ping")
    print("2. a tool ARGUMENT contains a lone surrogate")
    try:
        async with Client(SERVER) as client:
            await attempt("echo(os.fsdecode(b'a\\xffb'))", client, "echo", {"text": os.fsdecode(b"a\xffb")})
            await attempt("ping (healthy tool)", client, "ping")
    except BaseException as exc:
        print(f"  leaving the Client block raised {type(exc).__name__}")


anyio.run(main)

Output (the server's traceback goes to stderr and is omitted):

1. the tool RESULT contains a lone surrogate
  list_dir                           MCPError: Request 'tools/call' timed out
  ping (healthy tool)                MCPError: Connection closed
2. a tool ARGUMENT contains a lone surrogate
  echo(os.fsdecode(b'a\xffb'))       CancelledError: Cancelled via cancel scope 7f04c6665db0
  ping (healthy tool)                MCPError: Connection closed
  leaving the Client block raised ExceptionGroup

Python & MCP Python SDK

Linux. main@754254c0 (mcp 2.3.1.dev9) on Python 3.14.8. Also PyPI mcp 2.3.0 on Python 3.12.

AI assistance: I found and reproduced this with Claude Code, which wrote and ran the scripts, ran the comparison against the TypeScript and JavaScript behavior, and drafted this text. I read the output and checked the claims above before filing.

I reproduced this on current main: model_dump(mode="json") accepts a lone surrogate, then model_dump_json() raises PydanticSerializationError inside the transport writer and ends the task group (stdio server writer / client writer). Escaping as \udcff is not viable on the wire here — pydantic’s JSON parser rejects lone-surrogate escapes on the far side.

Approach: contain the failure in the writers (same idea as other unserializable values becoming Internal server error higher up). Shared helper serialize_outbound_message:

response/error with an id → substitute Internal server error and keep writing
request (client) → do not tear down the session; surface a local JSON-RPC error for that id
notification / id-less → drop and continue

Covering stdio server + client, SSE, and streamable HTTP event/JSON framing. Opening a PR shortly with regression tests.

_{AI-assisted drafting; I verified the repro, the pydantic parse limitation, and the tests.}

Reproduced at the current main (77001c9) with regression tests first, building on the thread's analysis — Ben-Home's contained-failure shape and naveecode's writer-containment approach (PR #3708, auto-closed by the intake gate before any review; where my branch diverges from it I say so below).

Root cause. A lone surrogate is a valid str and passes model_dump(mode="json"); the failure lands in the final frame encode inside a transport writer task, where nothing contains it:

src/mcp/server/stdio.py:203 (stdout_writer) and src/mcp/client/stdio.py:175 (stdin_writer): model_dump_json(...) raises PydanticSerializationError: ... UnicodeEncodeError ... surrogates not allowed, the writer task dies — the stdio server goes silent (the zombie in the report) / the client session is torn down.
src/mcp/server/sse.py:190 (sse_writer): same, kills the SSE connection.
src/mcp/server/streamable_http.py:437 (_create_event_data): same; _run_sse_writer's catch-all keeps the session alive but the call dies with SSE stream ended without a response, and the JSON-response path (_create_json_response, :424) blows up the POST.
src/mcp/client/streamable_http.py:374 and src/mcp/client/sse.py:140: the client HTTP post paths hand model_dump(...) to httpx, whose encode_json runs json.dumps(..., ensure_ascii=False).encode("utf-8") and raises UnicodeEncodeError at build_request, killing the post writer and the session — the issue's "HTTP post path" row.

Design choice: writer-level containment — one unserializable message fails alone, the session survives — not escaping, not sanitizing. I re-verified the thread's parse measurement at the parent: the wire parser rejects escaped lone surrogates (Invalid JSON: lone leading surrogate in hex escape from jsonrpc_message_adapter.validate_json), so a \udcff escape would just move the failure to the far side; and U+FFFD substitution would silently corrupt exactly the data this bug is about (non-UTF-8 file names). So one shared layer (src/mcp/shared/wire.py) turns an unserializable message into the same contained-failure class the SDK already serves for other unframeable values (the object() / non-UTF-8 bytes rows):

response/error with an id → goes out as Internal server error for its id;
request → answered locally with Internal server error for its id, so the awaiting caller fails cleanly instead of hanging — including server-initiated requests (elicitation/sampling), where fix: contain lone-surrogate serialization failures in transport writers #3708's server writers dropped the frame and left the handler waiting forever;
notification / id-less error → dropped (logged).

All six outbound serialization sites route through it: stdio both directions, server SSE, server streamable HTTP (SSE events + JSON POST bodies), and both client HTTP post paths (the two the issue's "HTTP post path" row names and #3708 missed).

RED / GREEN evidence (local, Python 3.12, uv run --frozen):

RED at the unmodified parent — 10 behavioral regression tests (3 stdio-server, 1 stdio-client, 3 streamable HTTP, 3 legacy SSE), every one failing with exactly the reported symptoms: stdio pydantic_core._pydantic_core.PydanticSerializationError: Error serializing to JSON: UnicodeEncodeError: 'utf-8' codec can't encode character '\udcff' ... surrogates not allowed at server/stdio.py:203 / client/stdio.py:175; streamable-HTTP tool result → MCPError(-32000, 'SSE stream ended without a response') with the writer traceback at streamable_http.py:375→437; streamable-HTTP tool argument → UnicodeEncodeError at httpx2/_content.py encode_json via client/streamable_http.py build_request; legacy SSE → MCPError(-32000, 'Connection closed') and Request 'tools/call' timed out. This matches every row of the table in the report at the current main.
GREEN with the fix — 21 new tests: 11 unit tests (tests/shared/test_wire.py, every containment rule and fallback arc) + the 10 behavioral tests across tests/server/test_stdio.py, tests/client/test_stdio.py, tests/client/test_http_unicode.py, tests/shared/test_sse.py (response → error for its id and the writer continues; request → local error and the writer continues; notification → dropped and the call it rode on still completes; later calls on the same session all succeed).
Coverage gate: coverage run -m pytest -n 2 → 6253 passed, 8 skipped, 1 xfailed, 1 failed; the single failure is test_file_resources.py::test_permission_error, which fails identically on the pristine parent (root ignores 0o000 — DID NOT RAISE PermissionError). coverage combine && coverage report → exit 0, 100.00%, 0 missed, 0 partial branches under the repo's fail_under = 100 + branch coverage.
Lint: ruff check . + ruff format --check . clean, pyright (strict) 0 errors, strict-no-cover clean.

Branch (no PR — the intake gate auto-closes unassigned-contributor PRs): https://github.com/Shanmukh-l33t/python-sdk/tree/fix/3707-outbound-serialization-containment at 3e5733dcd7ba2e3445398b032e3a58732bc6c6fb (12 files, +674/−17), compare: main...Shanmukh-l33t:python-sdk:fix/3707-outbound-serialization-containment

Scope note: streamable_http_manager._error_response and the auth PydanticJSONResponse still frame with raw model_dump_json, but only fixed strings reach them — no user-controlled text.

_{AI disclosure: produced with AI assistance (an autonomous agent); I verified the reproduction, the RED/GREEN runs, and the coverage output locally.}

For anyone hitting this in production with filesystem or CLI MCP servers returning surrogateescape strings or raw ?-�� JSON escapes over stdio before the SDK transport fix lands, you can wrap the server command with mcp-slim (mcp-slim -- <server-command>). It sanitizes lone UTF-16 surrogates ([?-��]) in both raw text and JSON-RPC frames on the wire so a single non-UTF-8 filename or binary log snippet cannot crash the stdio writer loop.

Navigation Menu

Repository navigation

A lone surrogate in an outgoing message silently kills the stdio server's writer or the client session #3707

Initial Checks

Release line

Description

Example Code

Python & MCP Python SDK

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

A lone surrogate in an outgoing message silently kills the stdio server's writer or the client session #3707

Description

Initial Checks

Release line

Description

Example Code

Python & MCP Python SDK

Activity

Ben-Home commented on Oct 14, 2026

naveecode commented on Oct 14, 2026

Shanmukh-l33t commented on Nov 11, 2026

SalyyS1 commented on Nov 11, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions