fix: preserve fallback emoji highlights and complete cleanup

[codewarnab]

Summary

When Intl.Segmenter is unavailable, highlights could split joined emoji and flags: matching 💻 inside 👩‍💻 produced 👩‍<mark>💻</mark>. Share fallback boundaries between source mapping and end guards so component matches highlight the whole emoji or flag, while adjacent sequences stay separate. Add fallback regressions, code comments, public API documentation, plan notes, and a patch changeset.

Also complete Phase 1 cleanup: consolidate interactive dataset sizes in scripts/bench-data/dataset-sizes.ts while preserving existing options and defaults; move frozen snapshot artifacts to docs/benchmarks/ and update their runner, checks, and references; delete the two finished plans and stale references.

Fixes #38

Validation

• Regression observed failing before the emoji fix and passing after it.
• bun run test:highlight (fallback disabled-segmenter cases plus normal suite)
• bun run test:normalization and bun run test:normalization:tr (98 checks each)
• bun run check:shaders, bun run typecheck, bun run build, bun run test:mock
• bun run test:benchmarks (82 checks for the cleanup)
• Independent subagent review; moved snapshots remain byte-for-byte identical.

The fallback protects these emoji boundaries; it remains a targeted fallback rather than a complete Unicode grapheme segmentation implementation.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated
webgpu-search	Ignored		Oct 13, 2026 4:44am UTC

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[codewarnab]

Unicode/highlighting check found a fallback bug: when Intl.Segmenter is unavailable, matching 💻 inside 👩‍💻 or the second regional indicator inside a flag could insert highlight tags in the middle of the displayed sequence. Existing normalization and highlighting tests passed because they did not exercise this fallback.

Fixed by sharing fallback boundaries between source mapping and end guards, consuming emoji ZWJ sequences and pairing regional indicators without merging adjacent flags. Added regression coverage with Intl.Segmenter disabled, plus code comments, public API documentation, an improvement-plan note, and a patch changeset. The regression failed on the original implementation and passes with the fix.