serve: Qwen3.6 tool requests corrupt across shared-prefix reuse and concurrency (0.1.7 regression)

[kira-ariaki]

Summary

BaseRT 0.1.7 native /v1/chat/completions does not reliably isolate Qwen3.6 hybrid tool-call requests when the server reuses a common prompt prefix. Sequential requests eventually lose the tool call; concurrent requests return corrupted arguments or exhaust max_tokens.

Restarting the server before every request makes the same requests pass 3/3. Routing the same model through our /v1/completions-based compatibility layer with a global request lock also passes repeatedly. This points to cross-request KV/recurrent-state reuse/isolation in the native chat path, rather than model weights or the new XML parser itself.

Environment

• BaseRT 0.1.7, official macOS arm64 release bundle
• macOS 26.4.1, M1 Max 64 GB
• Qwen/Qwen3.6-35B-A3B, default-q4, converted with BaseRT
• Server (no paged-KV / CB / prefix-cache flags):

basert-serve model.base \
  --host 127.0.0.1 --port 8896 \
  --max-context 262144 --request-timeout 600000

Minimal sequential reproduction

Send this request repeatedly on one fresh server, changing only seq-N:

{
  "model": "Qwen/Qwen3.6-35B-A3B",
  "messages": [{"role":"user","content":"Call lookup_key with key seq-0."}],
  "tools": [{
    "type":"function",
    "function": {
      "name":"lookup_key",
      "description":"Lookup a key",
      "parameters": {
        "type":"object",
        "properties":{"key":{"type":"string"}},
        "required":["key"]
      }
    }
  }],
  "max_tokens": 2048,
  "temperature": 0
}

Observed on one fresh process:

seq-0  tool_calls  lookup_key({"key":"seq-0"})  prompt=274 completion=142
seq-1  tool_calls  lookup_key({"key":"seq-1"})  prompt=9   completion=102
seq-2  tool_calls  lookup_key({"key":"seq-2"})  prompt=9   completion=90
seq-3  tool_calls  lookup_key({"key":"seq-3"})  prompt=9   completion=82
seq-4  tool_calls  lookup_key({"key":"seq-4"})  prompt=9   completion=120
seq-5  length      no tool_calls                    prompt=9   completion=2048

The drop from 274 to 9 reported prompt tokens shows that the server is reusing the common rendered prefix. Restarting basert-serve before each of three requests (fresh-a/b/c) produces the correct single lookup_key call 3/3.

After the failing run, /slots reports:

[{"id":0,"is_processing":false,"cache_tokens":275,"cache_position":403}]

Concurrent reproduction (more severe)

From a fresh process, submit the same shape concurrently with distinct key values.

Two concurrent requests:

requested c2-0 -> lookup_key({"key":"key"})   WRONG
requested c2-1 -> finish_reason=length, 2048 tokens, no tool call

Four concurrent requests:

requested c4-0 -> lookup_key({"key":"4"})     WRONG
requested c4-1 -> lookup_key({"key":"4"})     WRONG
requested c4-2 -> finish_reason=length, no call
requested c4-3 -> lookup_key({"key":"4-3"})   WRONG (lost prefix)

So 0/2 and 0/4 requests preserved their requested argument. Access log timestamps show the POSTs overlap even though this Qwen3.6 hybrid model does not support independent recurrent sequences / continuous batching.

Controls

1. Fresh server per request: correct 3/3.
2. Different tool schemas separated by a plain chat request: alpha_lookup(A1) -> plain response -> beta_fetch(Paris) -> beta_fetch(Rome) all correct in a short run. The failure is therefore sensitive to common-prefix reuse / repeated tool request shape, not simply “the previous schema always leaks.”
3. Same 0.1.7 core via a /v1/completions compatibility layer with a global lock: repeated tool calls correct 3/3.
4. 0xAD/CJK fixes are healthy: 桜の and a 512-token Chinese generation both return HTTP 200 on this build.

Expected

Every HTTP request must have isolated recurrent/KV state. If the hybrid architecture cannot support independent concurrent sequences, basert-serve should serialize/queue requests internally rather than allow overlapping forwards. Common-prefix reuse must restore/branch all hybrid recurrent state correctly, not only attention KV.

Impact / workaround

This prevents us from retiring our compatibility layer after #21: a long-running agent backend eventually misses or corrupts tool calls, and concurrent callers can receive arguments derived from another request.

Current safe workaround is:

• global request lock outside basert-serve; and
• use /v1/completions with externally rendered prompts/tool parsing.

I have a small Python harness that reproduces the sequential and N=2/N=4 cases and can test a diagnostic build immediately.

[prabod]

@kira-ariaki Thanks, I'll take a look