Skip to content

feat(tui): native Arabic and RTL (bidi) support for prompts and messages - #48587

Open
muhamedbeshir wants to merge 9 commits into
anomalyco:devfrom
muhamedbeshir:tui-bidi-rtl
Open

feat(tui): native Arabic and RTL (bidi) support for prompts and messages#48587
muhamedbeshir wants to merge 9 commits into
anomalyco:devfrom
muhamedbeshir:tui-bidi-rtl

Conversation

@muhamedbeshir

@muhamedbeshir muhamedbeshir commented Sep 12, 2026

Copy link
Copy Markdown

Issue for this PR

Closes #38524
Closes #40004
Closes #39525
Closes #32984

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

All four linked issues have the same root cause: nothing in the TUI render path applies the Unicode Bidirectional Algorithm. Arabic/Persian/Urdu/Hebrew words come out in logical (left-to-right) order, mixed English identifiers scramble the sentence order, RTL paragraphs stay left-aligned, wrapping reorders lines, and the prompt caret moves on an LTR assumption.

OpenTUI (@opentui/core@0.4.5) does wrapping, layout and cell painting in a native Zig library with no bidi support and no JS fallback, and the published package ships bundled JS, so this can't be fixed by patching OpenTUI. This PR subclasses its public renderables and overrides only the paint and caret entry points:

  • packages/tui/src/util/bidi.ts wraps bidi-js and adds what the renderer needs: paragraph direction from the first strong character (UAX full-auto / permissions / etc #9 P2/P3), LRI/PDI isolates around LTR islands (URLs, paths, dotted/dashed identifiers, numbers, code spans), grapheme- and width-aware wrapping, and logical<->visual boundary maps for the caret.
  • packages/tui/src/component/bidi-textarea.ts is a TextareaRenderable subclass for the prompt. Painting, caret placement and left/right/up/down motion are visual; the edit buffer stays logical, so typing, paste, undo, values and extmarks are untouched and copy returns the original text.
  • packages/tui/src/component/bidi-markdown.ts uses the markdown renderNode hook for assistant paragraphs/headings and captures tree-sitter chunks for styling. Fenced code, tables and diffs keep the stock renderer, so code is always LTR.
  • packages/tui/src/component/bidi-text.ts is a TextRenderable subclass for user message echoes.
  • Wiring: <textarea> -> <bidi_textarea> in packages/tui/src/component/prompt/index.tsx; renderNode + user echo in packages/tui/src/routes/session/index.tsx.

Why it works: the bidi algorithm only runs for paragraphs that contain strong RTL characters; English-only content takes the stock paint path unchanged. Isolates live in a layout-only stream and are never written to any buffer, so selection/copy stays logical. Wrapped logical text is synced into the native buffer so measurement and selection keep working.

How did you verify your code works?

On Windows 11, from a build of this branch:

bun test --cwd packages/tui ./test/util/bidi.test.ts
bun test --cwd packages/tui ./test/component/bidi-render.test.tsx
bun test --cwd packages/tui ./test/bidi-e2e.test.tsx
bun run --cwd packages/tui typecheck
  • unit tests cover direction detection, ordering, isolates, wrapping and cursor round-trips;
  • component tests render through the real OpenTUI test renderer and assert the actual painted cell frame: RTL order, right alignment, fenced code LTR, English-only output identical to the stock renderer, caret geometry, selection copy returning logical text;
  • the e2e test boots the app with a mocked SDK, types Arabic into the real prompt with real key presses, and streams a real assistant message.

I also built the native binary (bun run --cwd packages/opencode script/build.ts --single) and used it interactively. The full tui suite is 226 tests with 1 pre-existing Windows-only failure unrelated to this change (abbreviates paths within home boundaries uses path.join, so it returns backslashes on Windows).

Screenshots / recordings

Text-only change, so here are two of the exact cell-order frames the component tests assert (a real terminal renders these right-to-left correctly because the cells are already in visual order).

Arabic on its own, right-aligned:

                                 ملاعلاب ابحرم

Mixed Arabic + English, with the English run kept intact:

                     ةمزحلا تيبثتل npm install مدختسا

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

Render Arabic, Persian, Urdu and Hebrew text correctly in the native TUI using UAX anomalyco#9, without terminal-side bidi support or destructive text rewriting.

- Add bidi-js based layout engine (direction detection via first strong char, LRI/PDI isolation for URLs/paths/numbers/code spans, grapheme- and width-aware wrapping, logical<->visual caret maps).
- Add BidiTextareaRenderable for the prompt: visual caret placement and visual arrow-key motion while the edit buffer stays logical Unicode.
- Add a markdown renderNode hook for assistant paragraphs/headings; fenced code, tables and diffs keep the stock LTR renderer.
- Add BidiTextRenderable for user message echoes.
- English-only content takes the stock render path unchanged; selection/copy return logical text.
- Tests: engine unit tests, real test-renderer component tests, and an app-level e2e that types Arabic via real key presses and streams an assistant reply.
… scripts

Document the native bidi/RTL TUI support in English (docs/rtl/README.md) and Arabic (docs/rtl/README.ar.md), and add install-windows.ps1 / restore-windows.ps1 to swap the dev build over the npm-installed opencode binaries.
@github-actions github-actions Bot added the needs:compliance This means the issue will auto-close after 2 hours. label Sep 12, 2026
@muhamedbeshir muhamedbeshir changed the title feat(tui): native bidi/RTL rendering for prompts and messages feat(tui): native Arabic and RTL (bidi) support for prompts and messages Sep 12, 2026
@github-actions github-actions Bot removed the needs:compliance This means the issue will auto-close after 2 hours. label Sep 12, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for updating your PR! It now meets our contributing guidelines. 👍

The TUI cannot select fonts; the terminal does. Document recommended Arabic fonts and add a safe, backed-up script that adds Arabic font fallbacks to Windows Terminal without changing the monospaced main face.
The TUI cannot render with its own font, so /fonts is an application-level bridge: it lists installed font families (recommended Arabic fonts first) and writes the choice into Windows Terminal or the VS Code terminal settings, keeping a timestamped backup. It never installs fonts.
- Wrap the /fonts dialog footer in <text> so the dialog no longer crashes
  with an orphan-text error and actually opens.
- Register BidiTextRenderable as the global 	ext component so RTL
  ordering/alignment works in dialogs, lists, tool output and toasts,
  not only in the prompt and message bodies. English-only content still
  takes the stock paint path.
- Detect the hosting terminal (Windows Terminal vs VS Code) before writing
  the Arabic font, and preserve existing fallbacks instead of replacing them.
- Add e2e coverage for the dialog and host detection.
- Paint reasoning prose through the bidi code renderer so Arabic thinking
  blocks reorder RTL while code lines without Arabic stay LTR.
- Extend the e2e tests to assert right alignment for assistant paragraphs
  and dialog list rows, not just the presence of the visual order.
Windows Terminal has no font.fallbacks setting; it resolves a
comma-separated font.face chain instead. The previous writer stored an
array the terminal silently ignored, so picking a font never changed
anything. The writer now builds 'Mono, <pick>, <arabic...>' preserving
the user's faces, and removes any stale fallbacks key.
Windows Terminal warns about every missing family in the face chain.
Limit the tail to families that resolve on this machine (the picked font
is always kept); the dialog already labels the rest as not installed.
Drop the /fonts command, its dialog, the settings writer, docs and
tests. Font selection stays where it belongs: the terminal's own
settings. Bidi rendering itself is unchanged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant