Protocol v9: button bars and addon commands from libmudlet

docs/architecture.md

@@ -146,6 +146,16 @@ keeps the profiles' passwords in the KeyStore.
 | `hiddenInputWanted` | `hiddenInput` |
 | `commandHistory()`, read at `profileLoaded` and when a UI attaches | `history` |
 | `commandLineTextRequested` (`printCmdLine()` and the like) | `commandLine` |
+| `scriptWindows()`, read at `profileLoaded`, then at each `scriptWindowsChanged` | `windows`, only the windows that changed and those `removed`; all of them, with `full`, to a UI that attaches. A close sends an empty list |
+| `windowLines()`, the last 300 rows, at `profileLoaded` and at each `windowTextChanged` | `windowText`; any rows on request with `getWindowText` |
+| `setMainWindowSize()`, then `mainConsoleGrid()` and `mainConsoleTextRect()` at `profileLoaded` and at each `mainConsoleAreaChanged` | `setMainWindowSize`, kept and given again to each profile as it loads; `mainArea` |
+| `setUserWindowGeometry()`, `closeUserWindow()` | `placeUserWindow`, `closeUserWindow` |
+| `submitWindowCommandLine()`; `windowCommandLineTextChanged` | `sendWindowCommandLine`; `windowCommandLine` |
+| `labelContent()` of each label, read with `scriptWindows()` | `label` after the `windows`, for each label whose content changed; every label to a UI that attaches |
+| `pointer()` | `pointer`, answered only when refused |
+| none: the engine reads the file itself, with `QImageReader` or as a `QIcon` | `getImage`, answered by `image` as a PNG |
+| `buttonBars()` at `profileLoaded` and at each `buttonsChanged`; `pressButton()` | `buttonBars`, whole, when it changed, and to a UI that attaches; `pressButton`, answered only when refused |
+| `addonCommands()` at `profileLoaded` and at each `addonCommandsChanged`; `activateAddonCommand()` | `commands`, the same way; `activateCommand`, answered only when refused |
 
 The tail is sent once per pass of the event loop, so a flood sends its lines and a single tail.
 
@@ -195,6 +205,22 @@ Android options (`scripts/build-libmudlet-desktop.sh`) to run it. It checks:
 - the input line: a game's `WILL ECHO` gives `hiddenInput`, `sendHidden` reaches the game and no frame or log,
   `dismissHidden`, `history` to a UI that attaches and after a restart, `complete` of a word the game sent and from the
   history, and `printCmdLine()` as `commandLine`;
+- script windows: a Geyser `MiniConsole` as `windows` with its geometry, and its `cecho()` as `windowText` with its
+  colours; a Geyser `UserWindow` placed by `placeUserWindow`, which raises `sysUserWindowResizeEvent`; the size from
+  `setMainWindowSize` before the profile loads, and `setBorderLeft()`, as `mainArea`; `enableCommandLine()` with
+  `printCmdLine()` as `windowCommandLine` and `sendWindowCommandLine` running `setCmdLineAction()`'s function;
+  `getWindowText`; hiding, `closeUserWindow`, a `full` `windows` and each console's text to a UI that attaches, and
+  `removed`;
+- labels: a Geyser `Label`'s `echo()`, `setStyleSheet()`, `setAccessibleName()`, tool tip and click callback as
+  `label`, its click callback run by a `pointer` press, and `pointer` refused off the window; a new `echo()` as
+  `label` alone; `setBackgroundImage()` as the label's `image`, and that file as a PNG from `getImage`, scaled; an
+  SVG one with `setSvgTint()` and `setSvgRotation()`; a Geyser `Gauge`'s three labels, whose front width is its value
+  and whose text label is transparent; every label to a UI that attaches; `deleteLabel()` as `removed`;
+- button bars and commands: a package's bar, with its grid, a push-down button and a folder's menu, and
+  `tempButtonToolbar()` bars at the left and floating, as `buttonBars`; `pressButton` sending a push-down button's
+  command for down, then up, and running its script, and a menu entry pressed; `addCommand()` and
+  `setCommandChecked()` as `commands`, `activateCommand` raising `sysCommandClicked` and ticking it; a disabled or
+  unknown one refused, both lists to a UI that attaches, and `removeCommand()`;
 - TLS (see "TLS" below): a self-signed certificate refused, TLS to a port that does not speak it, then the same
   certificate accepted once the profile accepts self-signed ones.
 

docs/roadmap.md

@@ -27,11 +27,13 @@ repository uses. What Phase E needs, and where it stands:
 | Lines with colours and styles | `lineShown`, `localTextStyled`, `mainConsoleCleared` | `line`, `tail`, `clear` |
 | A line changed after it was shown | None: `lineShown` sends each line once | No |
 | Links (MXP, OSC 8, `echoLink()`) | Ids in runs; `link()`, `followLink()`, `linksChanged` | Ids in runs only |
-| Screen size for NAWS | `setScreenSize()`, `setMainWindowSize()` | No: the game is told 90 by 25 |
+| Screen size for NAWS | `setScreenSize()`, `setMainWindowSize()` | `setMainWindowSize` (version 7), which regrids the main console the game is told of; the app does not send it yet, so the game is told 90 by 25 |
 | Default colours, font, wrap | `consoleColors*`, `consoleFont()`, `consoleWrap()` and their signals | No |
 | Command history, completion | `commandHistory()`, `completeCommandLine()`, `completeCommandLineFromHistory()` | `history`, `complete` (version 3) |
 | Password prompts | `hiddenInputWanted`, `submitHiddenInput()` | `hiddenInput`, `sendHidden`, `dismissHidden` |
 | Scripts setting the input line | `commandLineTextRequested`, `setCommandLineText()` | `commandLine` |
+| Button bars and commands (Phase F) | `buttonBars()`, `pressButton()`, `buttonsChanged`; `addonCommands()`, `activateAddonCommand()`, `addonCommandsChanged` | `buttonBars`, `commands`, `pressButton`, `activateCommand` (version 9); the app draws none of it yet |
+| Script windows (Phase F) | `scriptWindows()`, `windowLines()`, `mainConsoleGrid()`, `setUserWindowGeometry()`, `closeUserWindow()`, `submitWindowCommandLine()` and their signals; `labelContent()`, `pointer()` | `windows`, `windowText`, `mainArea`, `windowCommandLine`, `getWindowText`, `placeUserWindow`, `closeUserWindow`, `sendWindowCommandLine` (version 7); `label`, `image`, `pointer`, `getImage` (version 8); the app draws none of it yet |
 | Screen reader announcements | `announcementRequested` (Lua `announce()`) | No |
 | Backgrounded app | `setApplicationActive()`, `setWindowMinimized()`, `saveAsync()`, `attentionRequested` | No |
 | Notifications | `notificationRequested` | `notify`, posted by the engine |

native/api/IEngine.h

@@ -1,11 +1,44 @@
 #pragma once
 
+#include "IFrontend.h"
 #include "Profile.h"
 
+#include <QByteArray>
 #include <QList>
+#include <QPointF>
+#include <QRect>
+#include <QSize>
 #include <QString>
 
 #include <optional>
+#include <variant>
+
+// The UI's pointer on a script window, as THeadlessSession::WindowPointerEvent describes it
+struct PointerEvent
+{
+    // As upstream names them on the wire
+    enum class Type { press, release, doubleClick, move, enter, leave, wheel };
+    Type type = Type::move;
+    // In the window's pixels, from its top left corner
+    QPointF position;
+    // Qt's values: the button that changed, for a press, release or double-click, and those held after it
+    int button = 0;
+    int buttons = 0;
+    // Qt::KeyboardModifiers
+    int modifiers = 0;
+    // The wheel's turn in eighths of a degree, as QWheelEvent::angleDelta().y()
+    int wheelY = 0;
+};
+
+// An image as PNG, and its size in pixels
+struct PngImage
+{
+    // The largest side IEngine::image() scales to or reads, so that a huge file is not decoded whole for a UI that
+    // could not take it in one frame
+    static constexpr int maxSide = 4096;
+    QByteArray png;
+    QSize size;
+};
 
 // What the host asks of an engine: the stand-in, or libmudlet behind MudletEngine
 class IEngine
@@ -33,4 +66,25 @@ class IEngine
     virtual QString deleteProfile(const QString& name) = 0;
     // Starts that profile, first closing and saving another that runs. The password is for this run only.
     virtual QString openProfile(const QString& name, const QString& password, bool connect) = 0;
+
+    // The script windows'. The main window's size in pixels, kept for the next profile too.
+    virtual QString setMainWindowSize(int width, int height) = 0;
+    // Up to count rows of a console from row first, which counts back from the end below 0; none for a name that is
+    // not a console's, or with no profile open
+    virtual std::optional<WindowText> windowText(const QString& window, int first, int count) const = 0;
+    // Where the UI put a user window, in the main window's pixels, docked or floating
+    virtual QString placeUserWindow(const QString& name, const QRect& geometry, bool floating) = 0;
+    // The dock's close button: hides the user window
+    virtual QString closeUserWindow(const QString& name) = 0;
+    // Enter on a window's command line holding text
+    virtual QString sendWindowCommandLine(const QString& name, const QString& text) = 0;
+    // What the UI's pointer did on a window
+    virtual QString pointer(const QString& window, const PointerEvent& event) = 0;
+    // The image file, or icon, at path as PNG, scaled to size unless it is empty, or that frame of an animated one;
+    // else why there is none. Size is at most PngImage::maxSide a side.
+    virtual std::variant<PngImage, QString> image(const QString& path, const QSize& size, std::optional<int> frame) const = 0;
+    // A click on a button or menu entry of IFrontend::buttonBars(), by its id
+    virtual QString pressButton(int id) = 0;
+    // A click on a command of IFrontend::commands(), by its id
+    virtual QString activateCommand(int id) = 0;
 };

native/api/IFrontend.h

@@ -1,9 +1,13 @@
 #pragma once
 
 #include <QList>
+#include <QRect>
+#include <QSize>
 #include <QString>
 #include <QStringList>
 
+#include <optional>
+
 struct ColorRun
 {
     int start = 0;
@@ -19,6 +23,173 @@ struct ColorRun
     bool operator==(const ColorRun&) const = default;
 };
 
+// A window a profile's scripts made, as THeadlessSession::ScriptWindow describes it, for the UI to draw
+struct ScriptWindowInfo
+{
+    QString name;
+    // Upstream's enum name: miniConsole, userWindow, label, commandLine, textBox, scrollBox, mapper or mxpFrame
+    QString kind;
+    // The window it is in, and that window's kind; both empty for the main window's area, a dock or an external frame
+    QString parent;
+    QString parentKind;
+    // In the parent's pixels; empty for a console's own command line
+    QRect geometry;
+    bool visible = false;
+    bool shown = false;
+    int stackOrder = 0;
+    // As ColorRun's, -1 for none; its alpha from 0 to 255, which a label's may have below 255
+    int background = -1;
+    int backgroundAlpha = 255;
+    // Empty and 0 for none
+    QString fontFamily;
+    double fontSize = 0;
+    bool inDock = false;
+    // A user window's
+    struct Dock
+    {
+        bool autoDock = true;
+        // left, right, top or bottom
+        QString area;
+        bool floating = false;
+        QString title;
+        bool operator==(const Dock&) const = default;
+    };
+    std::optional<Dock> dock;
+    // A mini console's, user window's or MXP frame's
+    struct Console
+    {
+        // The row it is scrolled back to, none while it follows new lines
+        std::optional<int> scrolledBackTo;
+        int columns = 0;
+        int rows = 0;
+        bool scrollBar = false;
+        bool timeStamps = false;
+        bool operator==(const Console&) const = default;
+    };
+    std::optional<Console> console;
+    bool operator==(const ScriptWindowInfo&) const = default;
+};
+
+// Rows of a mini console, user window or MXP frame, after its wrap
+struct WindowText
+{
+    struct Row
+    {
+        QString text;
+        // A fg is always a colour, as the UI does not know the console's own; a bg of -1 is the window's
+        QList<ColorRun> runs;
+        bool prompt = false;
+        // Carries on from the row above
+        bool wrapped = false;
+    };
+    // How many rows the console holds, and the one rows starts at, from 0
+    int lineCount = 0;
+    int first = 0;
+    QList<Row> rows;
+    // The last rows a window's text is sent with unasked
+    static constexpr int recentRows = 300;
+};
+
+// What a label shows, as THeadlessSession::LabelContent describes it: its geometry, background and font are its
+// ScriptWindowInfo's
+struct LabelInfo
+{
+    QString name;
+    // Qt rich text; empty while an image or movie shows
+    QString text;
+    // The file of a raster image while it shows, or of an SVG one, drawn under the text with the tint, rotation and shear
+    QString image;
+    bool svg = false;
+    // As ColorRun's, -1 for none, with its alpha from 0 to 255
+    int svgTint = -1;
+    int svgTintAlpha = 255;
+    double svgRotation = 0;
+    double svgShearX = 0;
+    double svgShearY = 0;
+    // The file of a movie while it plays or is paused; the frame it shows while paused, -1 while it plays
+    QString movie;
+    bool moviePaused = false;
+    int movieFrame = -1;
+    // In percent of the file's own speed
+    int movieSpeed = 100;
+    // Invalid for the file's own
+    QSize movieSize;
+    QString toolTip;
+    bool clickThrough = false;
+    QString accessibleName;
+    bool accessibleButton = false;
+    // The label's own, which Mudlet does not interpret
+    QString styleSheet;
+    bool operator==(const LabelInfo&) const = default;
+};
+
+// A button on a button bar, or an entry of a button's menu, as THeadlessSession::Button describes it
+struct BarButton
+{
+    // Its action's id, which IEngine::pressButton() takes
+    int id = 0;
+    QString name;
+    // The icon's file, for IEngine::image(); empty for none
+    QString icon;
+    // Its own, over its bar's; empty for a menu entry
+    QString styleSheet;
+    // A push-down button stays down, checked, from one press to the next
+    bool pushDown = false;
+    bool checked = false;
+    // A folder, whose press opens a menu of its children
+    bool menu = false;
+    QList<BarButton> children;
+};
+
+// A bar of buttons a profile's scripts or packages made, as THeadlessSession::ButtonBar describes it
+struct ButtonBar
+{
+    int id = 0;
+    QString name;
+    QString styleSheet;
+    // Upstream's enum name: top, left or right for the strip of the main console it is in, floating for a toolbar
+    QString place;
+    bool vertical = false;
+    // A horizontal bar fills each column top to bottom with this many buttons, a vertical one each row left to right
+    int columns = 1;
+    // How many places of that grid are left empty before the first button
+    int fillerOffset = 0;
+    QList<BarButton> buttons;
+};
+
+// A command a package's addCommand() added, as THeadlessSession::AddonCommand describes it; its pulse colours and
+// interval are left out
+struct AddonCommandInfo
+{
+    // What addCommand() returned, which IEngine::activateCommand() takes
+    int id = 0;
+    // Empty for one no package's code added
+    QString package;
+    QString name;
+    QString icon;
+    QString tooltip;
+    // The submenus it sits in, separated by '/', and its shortcut as the script wrote it; both empty off the menu
+    QString menuPath;
+    QString shortcut;
+    bool onMenu = false;
+    bool onToolbar = false;
+    bool enabled = true;
+    bool checkable = false;
+    bool checked = false;
+    bool pinned = false;
+    bool pulse = false;
+};
+
+// The main window's area: the size the UI gave (empty for none), the main console's grid in characters, and where its
+// text goes in pixels, which is the area less the borders scripts set
+struct MainArea
+{
+    QSize size;
+    QSize grid;
+    QRect textRect;
+    bool operator==(const MainArea&) const = default;
+};
+
 // Everything the engine tells its host. Every method must be implementable
 // over a byte stream, so arguments are plain values only.
 class IFrontend
@@ -38,4 +209,18 @@ class IFrontend
     virtual void history(const QStringList& lines) = 0;
     // A script set the whole input line
     virtual void commandLine(const QString& text) = 0;
+    // Every window the profile's scripts made, parents first, and what each of its labels shows: whole each time,
+    // empty with no profile open
+    virtual void windows(const QList<ScriptWindowInfo>& windows, const QList<LabelInfo>& labels) = 0;
+    // A console's last WindowText::recentRows rows, as they may have changed; IEngine::windowText() is read at other times
+    virtual void windowText(const QString& window, const WindowText& text) = 0;
+    // Sent when it changes
+    virtual void mainArea(const MainArea& area) = 0;
+    // A script set a window's command line, or Mudlet emptied it after a command
+    virtual void windowCommandLine(const QString& name, const QString& text) = 0;
+    // The profile's button bars, in the order of its actions: whole each time, empty with no profile open
+    virtual void buttonBars(const QList<ButtonBar>& bars) = 0;
+    // The commands the profile's packages added, in the order they were added: whole each time, empty with no profile
+    // open
+    virtual void commands(const QList<AddonCommandInfo>& commands) = 0;
 };