fix: support inline system messages in Anthropic API

[lvhan028]

Motivation

Anthropic Messages requests can contain system-role messages after conversation turns. Chat templates that require system messages to appear first, such as Qwen3.5, reject these histories during prompt rendering. The same compatibility issue affects token counting.

Modification

• Detect whether the active chat template renders inline system messages.
• For incompatible templates, concatenate top-level and inline system content without separators and move it into one leading system message.
• Apply the normalization to both /v1/messages and /v1/messages/count_tokens.
• Keep inline system messages in place for templates that support them.
• Preserve the existing structured conversion of tool calls, tool results, images, and reasoning content.

This change is limited to the Anthropic-compatible API and does not alter /v1/chat/completions.

BC-breaking

No backward-incompatible changes.

Use cases

Anthropic-compatible clients can submit interleaved system updates to models using system-first chat templates, including Qwen3.5.

Checklist

• Pre-commit checks pass.
• The modification is covered by unit and endpoint tests.
• No new downstream dependency is introduced.
• New helper functions include docstrings.

Test plan

• pre-commit on all modified files
• python -m pytest tests/test_lmdeploy/serve/anthropic/test_adapter_conversion.py tests/test_lmdeploy/serve/anthropic/test_endpoints.py tests/test_lmdeploy/test_content_merge.py -q
• Result: 70 passed

[Copilot]

Pull request overview

This PR updates the Anthropic-compatible API adapter layer to normalize “inline” system messages for chat templates that require system messages to appear first (e.g., Qwen3.5), and applies the same behavior to both message creation and token counting.

Changes:

• Adds a chat-template capability probe and uses it to decide whether to merge inline system messages into a single leading system message.
• Threads a merge_inline_system flag through /v1/messages and /v1/messages/count_tokens conversions.
• Extends unit/endpoint tests to cover inline-system merging behavior for a system-first template.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
lmdeploy/serve/anthropic/adapter.py	Adds inline-system detection + merging, and updates message conversions to optionally normalize inline system messages.
lmdeploy/serve/anthropic/router.py	Computes merge_inline_system once from the active chat template and passes it into endpoint registration.
lmdeploy/serve/anthropic/endpoints/messages.py	Passes merge_inline_system into to_openai_messages() for /v1/messages.
lmdeploy/serve/anthropic/endpoints/messages_count_tokens.py	Passes merge_inline_system into to_lmdeploy_messages() for /v1/messages/count_tokens.
tests/test_lmdeploy/serve/anthropic/test_adapter_conversion.py	Adds conversion tests for merging inline system messages to the front.
tests/test_lmdeploy/serve/anthropic/test_endpoints.py	Adds endpoint tests and a system-first chat template stub to validate server-side merging behavior.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

Suppressed comments (1)

lmdeploy/serve/anthropic/adapter.py:236

• _convert_system_message() filters system content blocks down to text only, and to_openai_messages()/to_lmdeploy_messages() route all role=='system' messages through it. Since MessageParam.content allows arbitrary ContentBlockParam (including image, tool_use, tool_result, thinking, etc.), this change will silently drop any non-text blocks in inline system messages, which is a behavior change from the previous per-block conversion path.

If non-text system blocks are unsupported, consider validating and raising a 400/ValueError instead of discarding them; otherwise, reuse the existing block conversion logic for system-role messages so images/tool blocks/reasoning are preserved consistently.

def _convert_system_message(content: str | list[ContentBlockParam]) -> list[dict[str, str]]:
    """Convert Anthropic system content into zero or one chat message.

    Only text blocks are retained when ``content`` is a block list.
    """

    if isinstance(content, str):
        system_text = content
    else:
        system_parts: list[str] = []
        for block in content:
            if _block_get(block, 'type') != 'text':
                continue
            text = _block_get(block, 'text')
            if not text or text.startswith('x-anthropic-billing-header'):
                continue
            system_parts.append(text)