feat(tui): native Arabic and RTL (bidi) support for prompts and messages by muhamedbeshir · Pull Request #48753 · anomalyco/opencode

muhamedbeshir · 2026-09-13T04:04:56Z

Issue for this PR

Closes #38524
Closes #40004
Closes #39525
Closes #32984

Type of change

Bug fix
New feature
Refactor / code improvement
Documentation

What does this PR do?

All four linked issues have the same root cause: nothing in the TUI render path applies the Unicode Bidirectional Algorithm. Arabic/Persian/Urdu/Hebrew words come out in logical (left-to-right) order, mixed English identifiers scramble the sentence order, RTL paragraphs stay left-aligned, wrapping reorders lines, and the prompt caret moves on an LTR assumption.

OpenTUI does wrapping, layout and cell painting in a native library with no bidi support and no JS fallback, so this cannot be fixed by patching OpenTUI. This PR subclasses its public renderables and overrides only the paint and caret entry points:

packages/tui/src/util/bidi.ts wraps bidi-js (UAX full-auto / permissions / etc #9) and adds what the renderer needs: paragraph direction from the first strong character (P2/P3), LRI/PDI isolates around LTR islands (URLs, paths, dotted/dashed identifiers, numbers, code spans), grapheme- and width-aware wrapping, and logical-visual boundary maps for the caret.
packages/tui/src/component/bidi-textarea.ts is a TextareaRenderable subclass for the prompt. Painting, caret placement and arrow motion are visual; the edit buffer stays logical, so typing, paste, undo, values and extmarks are untouched and copy returns the original text.
packages/tui/src/component/bidi-markdown.ts uses the markdown renderNode hook for assistant paragraphs/headings and captures tree-sitter chunks for styling. Fenced code, tables and diffs keep the stock renderer, so code is always LTR.
packages/tui/src/component/bidi-text.ts is a TextRenderable subclass for user message echoes, also registered globally so dialogs, lists, tool output and toasts inherit RTL ordering (English-only content takes the stock paint path unchanged).
Wiring: textarea to bidi_textarea in component/prompt/index.tsx; renderNode plus user echo in routes/session/index.tsx.

Why it works: the bidi algorithm only runs for paragraphs containing strong RTL characters. Isolates live in a layout-only stream and are never written to any buffer, so selection/copy stays logical. Wrapped logical text is synced into the native buffer so measurement and selection keep working.

How did you verify your code works?

On Windows 11, from a build of this branch:

bun test --cwd packages/tui ./test/util/bidi.test.ts
bun test --cwd packages/tui ./test/component/bidi-render.test.tsx
bun test --cwd packages/tui ./test/bidi-e2e.test.tsx
bun run --cwd packages/tui typecheck

unit tests cover direction detection, ordering, isolates, wrapping and cursor round-trips;
component tests render through the real OpenTUI test renderer and assert the actual painted cell frame: RTL order, right alignment, fenced code LTR, English-only output identical to the stock renderer, caret geometry, selection copy returning logical text;
the e2e test boots the app with a mocked SDK, types Arabic into the real prompt with real key presses, and streams a real assistant message.

Screenshots / recordings

Text-only change, so here are two of the exact cell-order frames the component tests assert (a real terminal renders these right-to-left correctly because the cells are already in visual order).

Arabic on its own, right-aligned:

                             ملاعلاب ابحرم

Mixed Arabic + English, with the English run kept intact:

                 ةمزحلا تيبثتل npm install مدختسا

Checklist

I have tested my changes locally
I have not included unrelated changes in this PR

Render Arabic, Persian, Urdu and Hebrew text correctly in the native TUI using UAX anomalyco#9, without terminal-side bidi support or destructive text rewriting. - Add bidi-js based layout engine (direction detection via first strong char, LRI/PDI isolation for URLs/paths/numbers/code spans, grapheme- and width-aware wrapping, logical<->visual caret maps). - Add BidiTextareaRenderable for the prompt: visual caret placement and visual arrow-key motion while the edit buffer stays logical Unicode. - Add a markdown renderNode hook for assistant paragraphs/headings; fenced code, tables and diffs keep the stock LTR renderer. - Add BidiTextRenderable for user message echoes. - English-only content takes the stock render path unchanged; selection/copy return logical text. - Tests: engine unit tests, real test-renderer component tests, and an app-level e2e that types Arabic via real key presses and streams an assistant reply.

… scripts Document the native bidi/RTL TUI support in English (docs/rtl/README.md) and Arabic (docs/rtl/README.ar.md), and add install-windows.ps1 / restore-windows.ps1 to swap the dev build over the npm-installed opencode binaries.

The TUI cannot select fonts; the terminal does. Document recommended Arabic fonts and add a safe, backed-up script that adds Arabic font fallbacks to Windows Terminal without changing the monospaced main face.

The TUI cannot render with its own font, so /fonts is an application-level bridge: it lists installed font families (recommended Arabic fonts first) and writes the choice into Windows Terminal or the VS Code terminal settings, keeping a timestamped backup. It never installs fonts.

- Wrap the /fonts dialog footer in <text> so the dialog no longer crashes with an orphan-text error and actually opens. - Register BidiTextRenderable as the global ext component so RTL ordering/alignment works in dialogs, lists, tool output and toasts, not only in the prompt and message bodies. English-only content still takes the stock paint path. - Detect the hosting terminal (Windows Terminal vs VS Code) before writing the Arabic font, and preserve existing fallbacks instead of replacing them. - Add e2e coverage for the dialog and host detection.

- Paint reasoning prose through the bidi code renderer so Arabic thinking blocks reorder RTL while code lines without Arabic stay LTR. - Extend the e2e tests to assert right alignment for assistant paragraphs and dialog list rows, not just the presence of the visual order.

Windows Terminal has no font.fallbacks setting; it resolves a comma-separated font.face chain instead. The previous writer stored an array the terminal silently ignored, so picking a font never changed anything. The writer now builds 'Mono, <pick>, <arabic...>' preserving the user's faces, and removes any stale fallbacks key.

Windows Terminal warns about every missing family in the face chain. Limit the tail to families that resolve on this machine (the picked font is always kept); the dialog already labels the rest as not installed.

Drop the /fonts command, its dialog, the settings writer, docs and tests. Font selection stays where it belongs: the terminal's own settings. Bidi rendering itself is unchanged.

github-actions · 2026-09-13T04:05:34Z

The following comment was made by an LLM, it may be inaccurate:

Potential Duplicate PRs Found

PR #48590 - feat(tui): native Arabic/RTL (bidi) support for the opencode2 beta
#48590

PR #48587 - feat(tui): native Arabic and RTL (bidi) support for prompts and messages
#48587

These two PRs appear to address the same feature: native RTL/bidi support for the TUI with nearly identical titles and scope (bidi-textarea, bidi-markdown, bidi-text components). PR #48590 mentions "opencode2 beta" while your current PR (48753) and #48587 have identical titles. Recommend reviewing the status and approach of #48590 and #48587 to avoid duplicate work.

- Apply UAX anomalyco#9 L2 flips over the full augmented stream including isolate controls, matching the reference application instead of clamping them away. Clamping rotated words and detached punctuation in mixed runs. - Expand each island across adjacent Latin/digit words so one isolate covers every maximal LTR run; sibling sub-flips plus the whole-line flip otherwise swap neighboring runs. - Broaden island seeds: markdown code spans, spaced Windows paths and key:line references. Code spans containing RTL prose stay natural. - Prefer wrapping outside LTR islands so tokens are not split across visual lines unless unavoidable. - Paint multi-unit graphemes with drawText: native setCell drops Arabic tashkeel. Shared paintBidiCell helper for text, textarea and markdown. - Install markdown bidi paint on CodeRenderable itself so list items, blockquotes and streaming updates inherit it; fenced code and English keep the stock painter. Drop the now-redundant renderNode hook. - Regression tests for mixed Arabic/English/code/path/URL/number runs, lists, links, diffs, wrapping and tashkeel in all three paint paths.

…op markdown TUI (review findings on the anomalyco#48753 port): - visualStep: strictly progress across hard-wrapped shared boundaries (caret froze at long-token wraps) - RTL probe: cover Thaana/NKo/Samaritan/Mandaic (U+0780-085F) and RLO - unlabeled fenced code keeps the stock LTR path (filetype must be markdown prose) - URL islands stop at strong RTL characters instead of swallowing them - memoize the RTL probe per content so English-only repaints pay no regex scan Desktop: block wrappers render display:contents so the existing '> * plaintext' rule never applied; set unicode-bidi on the real text blocks (paragraphs, headings, list items, quotes) and on agent question/answer copy.

muhamedbeshir added 9 commits September 12, 2026 05:20

docs(rtl): add Arabic font guidance and Windows Terminal fallback script

d7f5754

The TUI cannot select fonts; the terminal does. Document recommended Arabic fonts and add a safe, backed-up script that adds Arabic font fallbacks to Windows Terminal without changing the monospaced main face.

fix(tui): chain only resolving fonts so the terminal stops warning

247a261

Windows Terminal warns about every missing family in the face chain. Limit the tail to families that resolve on this machine (the picked font is always kept); the dialog already labels the rest as not installed.

feat(tui)!: remove the terminal font picker

c4edbd2

Drop the /fonts command, its dialog, the settings writer, docs and tests. Font selection stays where it belongs: the terminal's own settings. Bidi rendering itself is unchanged.

Navigation Menu

Uh oh!

feat(tui): native Arabic and RTL (bidi) support for prompts and messages - #48753

feat(tui): native Arabic and RTL (bidi) support for prompts and messages#48753
muhamedbeshir wants to merge 10 commits into
anomalyco:devfrom
muhamedbeshir:arabic-upstream

muhamedbeshir commented Sep 13, 2026

Uh oh!

github-actions Bot commented Sep 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

muhamedbeshir commented Sep 13, 2026

Issue for this PR

Type of change

What does this PR do?

How did you verify your code works?

Screenshots / recordings

Checklist

Uh oh!

github-actions Bot commented Sep 13, 2026

Potential Duplicate PRs Found

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants