fix(lang-core): use last complete definition for duplicate IDs in streaming parser

[AbhinRustagi]

What

Fix streaming parsing of redefined statement IDs so the last complete definition replaces the previous one, matching the existing non-streaming behavior.

Closes #1127.

root = Stack([a])
a = Title("x")
a = Title("y")

Previously, parse() rendered "y" while the streaming parser rendered "x". Both now render "y", with or without a trailing newline.

Changes and rationale

• Preserve trailing newlines only for streaming. The streaming scanner uses newlines outside strings and brackets as statement boundaries. Trimming the final newline hid that boundary and kept the final statement in the pending path. Preserving it lets the scanner commit the statement to its cache immediately. The boundary newline is not part of the rendered text; newlines inside closed strings were already preserved.
• Allow complete pending replacements. The model may omit the final newline. A pending statement that does not need quotes or brackets supplied by autoClose() can now overwrite an existing ID. This is what lets a complete final replacement take effect without a newline.
• Protect completed values during partial replacements. If autoClose() must supply missing quotes or brackets, the pending statement cannot overwrite a completed ID. For example, streaming a = Title("y keeps the existing a until the replacement is complete.
• Preserve non-streaming behavior. preprocess() defaults to the original full trimming; only the streaming parser enables newline preservation. For example, non-streaming input ending with the unfinished string root = Title("hello followed by a newline still produces text "hello" with incomplete: true.
• Add regression coverage and a patch changeset for @openuidev/lang-core.

The rendered output now agrees for the duplicate-ID examples. The existing metadata difference remains: statementCount counts unique IDs in batch parsing but all definitions in streaming parsing (2 versus 3 in the example above).

Test Plan

• Parser test file: 37 tests passed, including single-push, line-by-line, character-by-character, two-chunk, set(), trailing-newline, and incomplete-replacement cases.
• Direct runtime assertions confirmed that non-streaming unfinished-string newline handling is unchanged and streaming replacements work with and without a final newline.
• ESLint and Prettier checks passed for the modified parser file.

Checklist

• Linked the related issue.
• Considered backwards compatibility and preserved non-streaming preprocessing.
• Added a patch changeset.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
openui-docs	Ready	Preview	Sep 16, 2026 6:15pm UTC