Skip to content

Harden DSML and JSON parsing in the server#104

Open
Chida82 wants to merge 1 commit into
antirez:mainfrom
Chida82:hardeningparserjson
Open

Harden DSML and JSON parsing in the server#104
Chida82 wants to merge 1 commit into
antirez:mainfrom
Chida82:hardeningparserjson

Conversation

@Chida82
Copy link
Copy Markdown

@Chida82 Chida82 commented May 12, 2026

his change hardens the server-side parsers used for tool-call decoding and request parsing.

  • Reject DSML attributes that only partially match required names.
  • Reject non-JSON numeric literals that were previously accepted by the local parser.
  • Reject raw control characters inside JSON strings.
  • Reject malformed Unicode surrogate pairs in JSON strings.
  • Add focused regression tests for the affected parser paths.

Fixed Cases

1. DSML attribute prefix collisions

The DSML parser previously matched attributes too loosely, so malformed attributes such as xname could be interpreted as name.

Incorrect example now rejected:

<|DSML|tool_calls>
<|DSML|invoke xname="bash">
<|DSML|parameter xname="command" string="true">pwd</|DSML|parameter>
</|DSML|invoke>
</|DSML|tool_calls>

2. Non-JSON numeric literals accepted by the request parser

The JSON parser used strtod, which accepted values that are not valid JSON numbers.

Incorrect examples now rejected:

{"messages":[],"temperature":NaN}
{"messages":[],"temperature":Infinity}
{"messages":[],"top_p":+1}
{"messages":[],"top_k":01}

3. Raw control characters inside JSON strings

The JSON parser previously accepted unescaped control characters inside strings instead of rejecting the request.

Incorrect example now rejected:

{
  "messages": [
    {
      "role": "user",
      "content": "line1
line2"
    }
  ]
}

The same applies to raw tabs and other control characters that must be escaped in JSON.

4. Malformed Unicode surrogate pairs

The JSON string parser now rejects invalid surrogate-pair sequences instead of accepting malformed Unicode escapes.

Tests

  • Add a regression test for invalid DSML attribute names.
  • Add a regression test for invalid JSON numeric forms.
  • Add a regression test for raw control characters in JSON strings.
  • Keep the existing nesting-limit JSON test in place.

Validation

make ds4_test && ./ds4_test --server
make ds4-server && make clean

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant