Skip to content

Draw script labels, with their taps, and Geyser's gauges - #64

Open
vadi2 wants to merge 10 commits into
mainfrom
ccr-14671218-tf96z6-scriptui-f2u-labels-ui
Open

vadi2 wants to merge 10 commits into
mainfrom
ccr-14671218-tf96z6-scriptui-f2u-labels-ui

Conversation

@vadi2

@vadi2 vadi2 commented Nov 15, 2026 •

Copy link
Copy Markdown
Member

Brief overview of PR changes/additions

  • Script labels now draw on the game screen where the script put them, scaled with their window: rich text, the style sheet subset, raster and SVG images through getImage, tool tips and click-through.
  • Touches reach the label as pointer events: a tap is a left click, a quick second tap a double click, a long press a right click, a drag holds the left button as it moves.
  • Under TalkBack a clickable label is a button named by its accessible name, text or tool tip, with a 48 dp touch target, a "Right-click" action and a "Follow link" action for each link. Text labels read as text; image-only ones are hidden.
  • Geyser gauges are read as one progress bar: the value comes from the front label's width, and the _text label is the state.
  • Links in a label's text (send:, prompt:, web, Lua) are followed by the app until upstream can follow them (gap 1).

Motivation for adding to Mudlet

Most Geyser UIs are built from labels and gauges. Without them a profile's UI was coloured boxes.

Other info (issues closed, discussion etc)


Generated by Claude Code

The engine now tells the UI what scripts draw, from THeadlessSession's
script-window API: windows (only what changed, with removed, and every
window with full to a UI that attaches), each console's last 300 rows as
windowText, the main window's area as mainArea and a console's own
command line as windowCommandLine. The UI can give the main window's
size, ask for any rows of a console, place or close a user window and
send a line on a console's command line. libmudlet keeps the size the UI
gave for each profile that loads; the stand-in has no script windows,
says so on attach and refuses the requests.

Versions 4 to 6 are left for protocol changes still in review. The
window of a link (getLink/followLink) waits for version 5.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016ujaeGdi2MHnkoicGsDqhg
THeadlessSession sends a turn's main console lines as the turn runs, but
its script windows, their text and command lines from the event loop
after it. script_windows() waited for the chat console's windowText and
then for the "main window" line the same runLua echoed, which had
already come, and been collected, before the windowText: the wait timed
out. Each wait now looks first at what came since its request, and a
timeout lists what did come.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016ujaeGdi2MHnkoicGsDqhg
The profile's mudlet-base-ui package attaches its container to the right
border, so after setBorderLeft(100) the main console's text is narrower
than the window less 100 by that border: read it with getBorderRight().

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016ujaeGdi2MHnkoicGsDqhg
ScriptWindows folds the engine's windows, windowText, mainArea and
windowCommandLine into the state the game screen draws, and estimates
Mudlet's character width from each mainArea grid. WindowLayout places
the windows as docs/phase-f.md says: fitted from 600 dp, with docks a
quarter each; stacked below it, with the borders as bands, the side
windows and docks as pages for a switcher, and Mudlet given the area
turned to landscape. It also gives the size for setMainWindowSize and
the placements for placeUserWindow. Plain JVM code, so its tests run
with :protocol:test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016ujaeGdi2MHnkoicGsDqhg
The game screen draws the mini consoles and user windows a profile's
scripts make, with the main console's text renderer, around and over
the console as WindowLayout places them: fitted at 600 dp and wider,
stacked with a window switcher of tabs below that, a tab opening its
window in a pane over the console. It gives Mudlet's main window the
size that makes Mudlet's grid the console's, tells the engine where
the user windows are, and has their close buttons and the windows'
command lines. TalkBack reads the windows by name after the console,
the input line and the bar.

ScriptWindowsClient holds the state and the requests, so EngineClient
only hands it the window messages and each attach. FakeEngine gets the
window messages; ScriptWindowsTest checks both layouts, the size, the
switcher, the close button and a command line, and the emulator smoke
test runs a Geyser MiniConsole and waits for its line in the UI.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016ujaeGdi2MHnkoicGsDqhg
ScriptWindowsClient lives as long as the process and gives a new engine
the last size it wanted as it attaches, so the fitted test's first
setMainWindowSize was the size the command line test left, taken with
the keyboard up: 169 high, not 372. FakeEngine's receiveLast() reads
until the UI goes quiet and gives the last request of that type; both
size checks use it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016ujaeGdi2MHnkoicGsDqhg
The engine now tells the UI what each label shows, from THeadlessSession's
labelContent(): a label message after the windows message, for each label
whose text, image, SVG, movie, tool tip, accessible name or button,
click-through or stylesheet changed, and for every label to a UI that
attaches or drains. windows gives bgAlpha for a background below opaque,
as a Geyser.Gauge's text label is transparent; its back and front labels'
geometry gives its value. The UI can send what its pointer did on a window
(pointer, run by THeadlessSession::pointer(), so a label's click callback
runs) and ask for an image file, scaled or one frame of it, as a PNG
(getImage, answered by image). The stand-in has no labels and refuses
both.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016ujaeGdi2MHnkoicGsDqhg
The app draws a label's content itself (docs/phase-f.md, "Labels"), so
:protocol reads it, without android.*:

- LabelStyle: the subset of Qt's style sheet Geyser and scripts write on
  a QLabel (colours and linear gradients, border, radius, padding,
  margin, font, qproperty-alignment, border and background images), from
  bare declarations and QLabel or TLabel rules only.
- RichText: Qt's rich text as blocks of styled spans, with whitespace
  collapsed as Qt's rich text does and links kept with their targets.
- LabelLink: what a tapped link does until upstream can follow one (gap
  1): send: as expandAlias(), as Mudlet's send expands aliases, prompt:
  as printCmdLine(), a web link opened, other Lua run.
- Gauge: Geyser's _back, _front and _text labels as one gauge, its value
  the front's share of the back (gap 2).

ScriptWindows keeps each label's content, dropping it with the label,
and PlacedWindow gains the window's uncut rectangle, which a label's
pointer coordinates are taken against.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016ujaeGdi2MHnkoicGsDqhg
A label is drawn at its scaled place as QLabel draws it: its style
sheet's margin, background, border and padding, its raster image or
movie frame at its own size, its SVG fitted under its text with tint,
rotation and shear, and its rich text, unwrapped, with its links. The
images come from the engine with getImage, asked once per instance.

Touches reach the label as pointer events in its own pixels, kept inside
it: a tap is enter, press and release, a second tap a double click, a
long press the right button (and the tool tip), and a drag moves with
the button held. A click-through label lets the touch through.

For TalkBack a clickable label is a button named by its accessible name,
text or tool tip, touched within 48 dp around it, with a right-click
action and an action per link; a label with text is text, one with only
an image and no name is left out, and a gauge is one node with its
progress and its text as its state. Labels come after the game screen's
own controls, as the other script windows do.

ScriptLabelsTest checks these against FakeEngine, which gains label and
image messages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016ujaeGdi2MHnkoicGsDqhg
ScriptLabelsTest's image test read the engine side's socket straight after
sending the label. Compose's test clock only advances while the test waits
on Compose, so the label was never recomposed with its image and asked for
nothing: both emulators timed out on 'getImage'. Wait for the label's node
first, as the other tests do before they read what the UI sent.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016ujaeGdi2MHnkoicGsDqhg
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants