Skip to content

Protocol v8: labels, images and pointer input from libmudlet - #62

Open
vadi2 wants to merge 4 commits into
mainfrom
ccr-14671218-tf96z6-scriptui-f2p-labels
Open

vadi2 wants to merge 4 commits into
mainfrom
ccr-14671218-tf96z6-scriptui-f2p-labels

Conversation

@vadi2

@vadi2 vadi2 commented Nov 15, 2026

Copy link
Copy Markdown
Member

Brief overview of PR changes/additions

  • The engine now sends what each script label shows (text, stylesheet, background image or SVG with tint/rotation/shear, movie state, tool tip, accessible name and button role, click-through) as a label message after the windows message it belongs to. Only changed labels are sent, and a UI that attaches or drains gets them all again.
  • New pointer request: the UI's press, release, double-click, move, enter, leave and wheel on a window reach the profile's click and hover callbacks. The engine replies only when it refuses one.
  • New getImage request: returns a label's image file (or an icon) as a PNG, optionally scaled or for one frame. It is answered by image.
  • Gauges can be drawn from the data alone: the back, front and text labels arrive with their geometry. A new bgAlpha field carries see-through backgrounds, so the text label is transparent.
  • PROTOCOL_VERSION is now 8 in docs/protocol.md, SocketBridge, Protocol.kt and protocol_client.py. The Kotlin side only parses and encodes the new messages; there is no UI for them yet. The stand-in engine refuses the new requests.

Motivation for adding to Mudlet

Geyser labels, buttons and gauges are how most scripted interfaces are built. The Android UI needs their content and a way to click them.

Other info (issues closed, discussion etc)


Generated by Claude Code

The engine now tells the UI what scripts draw, from THeadlessSession's
script-window API: windows (only what changed, with removed, and every
window with full to a UI that attaches), each console's last 300 rows as
windowText, the main window's area as mainArea and a console's own
command line as windowCommandLine. The UI can give the main window's
size, ask for any rows of a console, place or close a user window and
send a line on a console's command line. libmudlet keeps the size the UI
gave for each profile that loads; the stand-in has no script windows,
says so on attach and refuses the requests.

Versions 4 to 6 are left for protocol changes still in review. The
window of a link (getLink/followLink) waits for version 5.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016ujaeGdi2MHnkoicGsDqhg
THeadlessSession sends a turn's main console lines as the turn runs, but
its script windows, their text and command lines from the event loop
after it. script_windows() waited for the chat console's windowText and
then for the "main window" line the same runLua echoed, which had
already come, and been collected, before the windowText: the wait timed
out. Each wait now looks first at what came since its request, and a
timeout lists what did come.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016ujaeGdi2MHnkoicGsDqhg
The profile's mudlet-base-ui package attaches its container to the right
border, so after setBorderLeft(100) the main console's text is narrower
than the window less 100 by that border: read it with getBorderRight().

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016ujaeGdi2MHnkoicGsDqhg
The engine now tells the UI what each label shows, from THeadlessSession's
labelContent(): a label message after the windows message, for each label
whose text, image, SVG, movie, tool tip, accessible name or button,
click-through or stylesheet changed, and for every label to a UI that
attaches or drains. windows gives bgAlpha for a background below opaque,
as a Geyser.Gauge's text label is transparent; its back and front labels'
geometry gives its value. The UI can send what its pointer did on a window
(pointer, run by THeadlessSession::pointer(), so a label's click callback
runs) and ask for an image file, scaled or one frame of it, as a PNG
(getImage, answered by image). The stand-in has no labels and refuses
both.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016ujaeGdi2MHnkoicGsDqhg
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants