Skip to content

Fix RTL rendering for Hebrew and Arabic scripts - #76

Open
Jonathan-Asher wants to merge 40 commits into
f:mainfrom
Jonathan-Asher:fix/rtl-hebrew-teleprompter
Open

Fix RTL rendering for Hebrew and Arabic scripts#76
Jonathan-Asher wants to merge 40 commits into
f:mainfrom
Jonathan-Asher:fix/rtl-hebrew-teleprompter

Conversation

@Jonathan-Asher

@Jonathan-Asher Jonathan-Asher commented Aug 9, 2026

Copy link
Copy Markdown

Reading a Hebrew script in the teleprompter is currently not usable. There is partial RTL support already (TextDirection.swift, the layoutDirection flip in WordFlowLayout), but it breaks down in four independent ways.

1. Word spacing collapses entirely

WordFlowLayout spaces words by appending a trailing space to each Text (Text(item.word + " ")). Text layout trims trailing whitespace when measuring, so in an RTL line every gap measures zero and the whole line renders as one unbroken string:

before:  ברוכיםהבאיםלטקסטרים!זהוהטלפרומפטרהאישישלך
after:   ברוכים הבאים לטקסטרים! זהו הטלפרומפטר האישי שלך

Words are now spaced by the enclosing HStack instead, which is direction-agnostic. LTR output is unchanged.

2. Embedded LTR runs render reversed

Flipping layoutDirection reverses every word box on the line, including consecutive Latin words that should stay in reading order among themselves:

For the input ...והיא עובדת מצוין עם Claude Code על מק, the embedded Latin pair renders as Code Claude before this change and Claude Code after it.

Each line is now reordered with the Unicode Bidirectional Algorithm's L2 reordering rule applied at word granularity (bidiVisualOrder), so only the runs that need reversing are reversed. Each word is additionally wrapped in a directional isolate (RLI/LRI…PDI) so its own trailing punctuation and bracket mirroring resolve against the run it belongs to rather than against the ambient layout direction — otherwise שלום, puts the comma on the wrong side.

Numbers are treated as LTR, matching UBA rule I1 (European numbers always take an even embedding level), so 1.6.3 inside Hebrew text keeps its digits in order while still sitting at the correct point in the RTL flow.

3. The editor loses its direction on every keystroke

HighlightingTextEditor.updateWritingDirection sets the base writing direction, but applyHighlighting then calls textStorage.setAttributes over the full range with only .font and .foregroundColor — wiping the paragraph style that carries the direction. In textDidChange the two run in that exact order, so Hebrew snaps back to left-aligned as soon as you type. The paragraph style is now part of the default attribute set and of typingAttributes/defaultParagraphStyle, so it survives the reset.

4. The browser teleprompter treats every Hebrew word as a stage cue

isAnnotation and letterCount in the served HTML test against a hardcoded range list:

/[a-zA-Z0-9À-ɏЀ-ӿ -鿿-]/

Hebrew, Arabic, Greek, Thai, Devanagari and others are absent, so every word in those scripts is classified as emoji-only: rendered dim and italic in the cue colour, skipped by the next-word scan, and never scrolled to. The browser view is completely non-functional for these languages. Both helpers now use \p{L}/\p{N}, which matches the native side's isLetter || isNumber, and the text container gets a dir attribute so the browser's own bidi engine orders each line.

Base direction heuristic

textBaseDirection uses first-strong detection, which misreads a script opening with a Latin product name or speaker label — a single leading Textream lays an entire Hebrew page out left-to-right. Added dominantBaseDirection, which picks whichever script has more letters; scripts are read whole, so the dominant script is the better signal at page level. textBaseDirection is left in place.

Also added dir="auto" to the director page's script panes.

Verification

Rendered WordFlowLayout directly through ImageRenderer across four fixtures — pure Hebrew, Hebrew with embedded English and numbers, Hebrew punctuation, and an English control. All three Hebrew cases are correct after the change; the English control is pixel-identical before and after. The browser-side classification and direction helpers were checked against Hebrew, Arabic, CJK, Greek, Latin, digit-only, emoji and bracketed-cue inputs.

Built and running locally on macOS 15 (Apple Silicon).

🤖 Generated with Claude Code

f and others added 30 commits February 27, 2026 00:57
… add state-change debounce to prevent DoS (CWE-400)
fix: add session token auth and connection limit to DirectorServer
fix: enforce connection limit, offload broadcast to background queue,…
The page sidebar was only visible when multiple pages existed, but the
"Add Page" button lives inside the sidebar — making it impossible to
add pages from a single-page state.

Also, pressing play always reset to page 0 (the welcome text) instead
of reading whichever page the user was currently editing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds customizable color and brightness controls for teleprompter
annotation cues (e.g. [pause], [smile], [breath]). Previously these
were hardcoded to white at fixed opacities.

- Cue Color: 6 color presets (matches highlight color options)
- Cue Brightness: 4 levels (Dim, Low, Medium, Bright)
- Settings preview includes [pause] sample annotation
- Remote viewer (BrowserServer) updated to use cue color
- DirectorServer state includes cue color for consistency

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
fix: always show page sidebar and read current page on play
feat: add cue color and brightness settings for annotation text
In Classic and Voice-Activated modes, the overlay would immediately
switch to a "Done" screen and auto-dismiss 1 second after the
auto-scroll reached the last word. This made timed mode unusable
because speakers typically finish talking 10-20 seconds after the
scroll ends.

Now in timer-based modes (Classic/Voice-Activated), when the scroll
reaches the end on the last page:
- The prompter text stays visible instead of switching to "Done"
- The overlay does not auto-dismiss
- The speaker can close manually via the X button or Esc key

Word Tracking mode behavior is unchanged (auto-dismiss is appropriate
there since it knows when the speaker actually finishes).

Fixes f#29
Address code review findings:
- ExternalDisplayView: gate doneView and speechRecognizer.stop() on
  wordTracking mode, matching NotchOverlayView/FloatingOverlayView
- BrowserServer: suppress isDone in classic/silencePaused modes on
  last page so browser clients keep showing prompter text
- Revert accidental DEVELOPMENT_TEAM change in project.pbxproj
The timerWordProgress was incrementing unboundedly after the scroll
reached the end, unlike the SwiftUI views which guard with !isDone.
Add a scrollDone check before incrementing to stop wasting CPU on
the 100ms broadcast timer.
SFSpeechRecognizer silently returns no results when receiving
multi-channel audio buffers. USB audio interfaces like the RODECaster
Pro II send 2-channel 48kHz audio, and the previous `format: nil` tap
delivered these buffers unchanged to the recognition request.

Create a mono AVAudioFormat at the hardware sample rate when the device
has more than one channel and pass it to installTap, letting
AVAudioEngine handle the downmix automatically.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Introduce an overlay transparency mode that uses a blurred background and adjustable tint opacity.

- Add NotchBlurView (NSVisualEffectView wrapper) to provide behind-window blur.
- Update NotchOverlayView and preview to render either a solid black island or a blurred, clipped island with a dark tint driven by opacity.
- Add NotchSettings properties overlayTransparency and overlayTransparencyOpacity, persisted via UserDefaults (keys: "overlayTransparency", "overlayTransparencyOpacity") and initialized with sensible defaults.
- Expose a Toggle and an opacity Slider in SettingsView to enable transparency and control amount; update reset defaults to include the new settings.

This enables a see-through notch overlay option where desktop content shows through while keeping text readable via a configurable dark tint.
Add overlay transparency with blur and slider
…cognition

fix: downmix multi-channel audio to mono for SFSpeechRecognizer
…pm-scroll

Fix: Keep text visible after WPM auto-scroll reaches the end
Addresses two user-reported bugs: (1) highlight not tracking at the right
speed, jumping erratically or lagging behind speech, and (2) mic appearing
to stall out and stop picking up audio after ~60 seconds.

Root causes identified and fixed:

**Seamless recognition restart (P0)**
- Split cleanupRecognition() so AVAudioEngine stays alive across
  SFSpeechRecognitionTask restarts, eliminating audio gaps
- Add pre-emptive 55-second restart timer to beat Apple's ~60s timeout
- Update matchStartOffset to recognizedCharCount before each restart so
  new sessions match from the correct position
- Thread-safe request swapping via NSLock for audio I/O thread safety
- Add contextualStrings from remaining source text for better STT accuracy

**Fix fuzzy matching false positives (P1)**
- Remove overly permissive `contains` check from isFuzzyMatch that caused
  "and" to match "demand", "the" to match "other", etc.
- Tighten prefix matching to require minimum 3-char words
- Require exact match for 2-char words (no edit distance tolerance)
- Fix charLevelMatch skip-both fallback: no longer advances
  lastGoodOrigIndex on genuine mismatches (gibberish no longer matches)
- Fix wordLevelMatch +1 space overcount on last matched word
- Fix unicode scalar vs Character count mismatch in charLevelMatch

**Confidence gating (P2)**
- Replace blind max(charResult, wordResult) with agreement-based selection
- Add sliding window requiring 2-of-3 recent results to agree before
  committing large forward jumps (small steps always pass through)

**Retry resilience (P3)**
- Distinguish timeout errors (code 1110/216) from real errors
- No retry limit for expected timeouts; immediate soft restart
- Backoff with retry limit only for genuine errors

**Architecture cleanup (P4)**
- Merge two polling timers in observeDismiss() into one
- Fix retain cycle in dismiss() asyncAfter closure
- Add isDismissing guard to prevent double-dismiss
- Fix cancelled-task error callback race in restartTask()

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ned mode

Two bugs fixed:

1. Clicking a page in the sidebar then pressing Play always started from
   page 1 instead of the selected page. Root cause: the `sidebarSelection`
   setter wrapped the `currentPageIndex` update in `DispatchQueue.main.async`,
   deferring it asynchronously. Since SwiftUI binding setters are already
   called on the main thread, this wrapper was unnecessary and caused `run()`
   to read the stale value (0) before the update applied. Fix: remove the
   async dispatch so `currentPageIndex` is updated synchronously on selection.

2. `showPinned()` was the only display mode not calling `installKeyMonitor()`,
   so the ESC key did not work to dismiss the overlay in pinned mode.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix: start playback from selected page and install key monitor in pin…
fix: voice tracking highlight and mic stall bugs
jumpTo tore down the recognition task and started a fresh one, dropping
the audio spoken during the restart window (~1s to warm) — exactly the
words the user re-speaks right after tapping a word to jump, so the
highlight appeared to stop following.

Jumps now keep the task running and record a transcript anchor:
matching ignores everything transcribed before the jump and continues
from the new offset with zero dropped audio. The anchor resets whenever
a new task starts a fresh transcript (soft restart, new session).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
beginRecognition and restartTask both gave up permanently when
SFSpeechRecognizer reported unavailable — a state that is usually
transient (the recognition service churns briefly after a task
cancellation or an audio device config change). beginRecognition's
guard didn't even set isListening = false, so the app went silently
deaf: engine stopped by the preceding cleanup, no retry, no error
surfaced, until a full app relaunch.

Observed live: a USB mic (Elgato Wave) sample-rate renegotiation ~100ms
after engine start triggered restartTask, the cancelled task's error
callback scheduled a fresh beginRecognition, and that hit the
unavailable guard and returned. Both guards now retry with the same
backoff the invalid-format guard already uses.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Findings from an adversarial self-review of the first two commits:

- restartTask/beginRecognition now clear lastSpokenText along with the
  anchor: a jump taken after a task restart but before its first result
  anchored on the old transcript and froze matching until the next
  ~55s restart.
- The anchor is now the transcript prefix string, trimmed by surviving
  common prefix (with bounded slack) instead of a raw char count, so
  the recognizer revising earlier text no longer swallows post-jump
  speech or leaks the pre-jump transcript.
- Results delivered within 300ms of a jump are ignored — they were
  computed against the pre-jump position.
- jumpTo no longer resets retryCount (tapping words could keep a
  failing availability-retry loop alive forever) and falls back to a
  full restart for far jumps (>500 chars, refreshing contextualStrings
  for the new section) or when the engine died silently.
- A nil SFSpeechRecognizer (unsupported locale) fails fast instead of
  silently retrying a permanent condition; exhausted retries in
  restartTask stop the audio engine instead of leaving the mic hot.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
fix: recognition resilience — keep task alive across jumps, retry transient recognizer unavailability
f and others added 10 commits July 9, 2026 22:17
… mode

- Skip bracketed notes and highlight the following spoken words
- Commit multi-word phrases when both match strategies progress
- Catch up during fast reading by preferring the further match
- Widen resync windows for dropped/inserted words
- Voice-Activated mode runs on microphone energy without speech recognition
…cripts (f#63)

- MarqueeTextView: Detect RTL content via isRTLUnicodeScalar(), flip VStack
  alignment to .trailing for RTL, and set .environment(\.layoutDirection) so
  word flow lines render right-to-left.
- HighlightingTextEditor: Add isRTLUnicodeScalar() and containsRTLText()
  helpers, plus updateWritingDirection() that sets textView.baseWritingDirection
  to .rightToLeft when RTL content is detected, ensuring proper caret and text
  alignment in the editor.

Fixes garbled/truncated display when dictating or editing Arabic, Hebrew,
Persian, Urdu, and other RTL text.

Co-authored-by: Hermes Agent <hermes@nousresearch.com>
Co-authored-by: Fatih Kadir Akın <f@users.noreply.github.com>
The teleprompter had partial RTL support that broke down in four ways.

Word spacing collapsed entirely. `WordFlowLayout` spaced words by appending
a trailing space to each `Text` ("word "). Text layout trims trailing
whitespace when measuring, so in an RTL line every gap measured zero and the
whole line ran together as one unbroken string of characters. Words are now
spaced by the enclosing `HStack` instead, which is direction-agnostic.

Embedded LTR runs were reversed. Flipping `layoutDirection` reverses every
word box on the line, including consecutive Latin words that should stay in
reading order among themselves — "Claude Code" inside a Hebrew sentence
rendered as "Code Claude". Each line is now reordered by the Unicode
Bidirectional Algorithm's L2 rule at word granularity, so only the runs that
need reversing are reversed, and each word is wrapped in a directional
isolate so its own punctuation and bracket mirroring resolve against the run
it belongs to.

The editor lost its direction on every keystroke. `applyHighlighting` resets
all attributes over the full range, wiping the paragraph style that carried
the writing direction, so Hebrew jumped back to left-aligned as soon as you
typed. The paragraph style is now part of the default attribute set and of
the typing attributes.

The browser teleprompter treated every Hebrew word as a stage cue. Its
`isAnnotation` and `letterCount` helpers tested against a hardcoded list of
character ranges that omitted Hebrew, Arabic, Greek, Thai and more, so those
words were classified as emoji-only: rendered dim and italic, skipped by the
highlight scan, and never scrolled to. Both helpers now use `\p{L}`/`\p{N}`,
matching the native side's `isLetter || isNumber`, and the text container
gets a `dir` attribute so the browser's own bidi engine orders the line.

Base direction is now taken from the dominant script rather than the first
strong character, so a Hebrew script that opens with a Latin product name or
speaker label still lays out right-to-left. The director page's script panes
get `dir="auto"`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants