diff --git a/CLAUDE.md b/CLAUDE.md index dc00632..b7a4364 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,5 +1,15 @@ # CLAUDE.md +## Design specs (`docs/spec/`) + +One file per public type or subsystem. There is **no size limit** on spec +files — a spec should be as long as precision requires. These are the +**authoritative design reference** — before making any change to a public type +or subsystem, read its spec file first. The spec captures the reasoning, +invariants, API surface, and design decisions that the code alone doesn't +document. If a change invalidates any part of a spec, update the spec (not +the other way around). + ## Feature documentation (`docs/superpowers/`) One file per feature, compressed reference documentation: diff --git a/README.md b/README.md index 4571586..003b795 100644 --- a/README.md +++ b/README.md @@ -3,27 +3,54 @@ A typed, asynchronous bridge between your UI and business-object models. You call a model from the UI and get the result back on the UI — without -writing any concurrency, serialisation, or transport code, and without the call site -caring whether the model runs in this process or across a socket. +writing any concurrency, serialisation, or transport code, and without the call +site caring whether the model runs in this process or across a socket. -Models may live **in-process** (local mode) or in a **remote server process** (remote -mode). The GUI code is identical in both cases — only the backend implementation -changes. Results come back as a `Completion` whose callbacks always fire on the -executor you chose (e.g. your UI event loop), so the UI never blocks and never has to -deal with a background worker directly. +Models may live **in-process** (local mode) or in a **remote server process** +(remote mode). The GUI code is identical in both cases — only the backend +implementation changes. Results come back as a `Completion` whose callbacks +always fire on the executor you chose (e.g. your UI event loop), so the UI never +blocks and never has to deal with a background worker directly. - Header-only, C++23, namespace `morph` -- JSON reflection via [Glaze](https://github.com/stephenberry/glaze) — no hand-written - serialisation per action +- JSON reflection via [Glaze](https://github.com/stephenberry/glaze) — no + hand-written serialisation per action - Exact, unit-tagged action values (`morph::math::Rational`, `morph::units::Quantity`) and JSON-Schema generation per action (`morph::forms`) so clients can build their forms at runtime +- Ordered, replayable action log (`morph::journal`) and offline/reconnect + primitives (`morph::offline`) - Optional Qt 6 WebSocket transport (built when `MORPH_BUILD_QT=ON`) +## What problem does it solve? + +Desktop and cross-process apps repeatedly re-solve the same plumbing: move a +request off the UI thread, run it, marshal the result back, and — sooner or +later — do the same thing again over a socket when the logic moves to a server. +morph collapses that into one seam. You write the model as **plain, +single-threaded C++**; the framework owns: + +- **Concurrency** — one *strand* per model instance serialises that model's + calls while different models run in parallel, so model authors never touch a + mutex. +- **Marshalling** — `.then(...)` / `.onError(...)` always run on the executor + you pass, so results land on the UI thread and the UI thread never blocks. +- **Transport** — the same registered action travels a JSON wire protocol to a + remote `RemoteServer` with no change to the model or the call site. + +### Who it's for + +A good fit if you are building a C++ GUI (Qt or otherwise) over a +request/response domain model and want the option to relocate that model to a +server later without rewriting the UI, or you want schema-driven forms and exact +decimal values for financial/lab data. Less of a fit if you need a +general-purpose RPC framework, streaming/bidirectional channels, or a hardened +public-internet server — see [Status & limitations](#status--limitations). + ## The main idea -You write a model as plain, single-threaded C++. The framework handles concurrency -(one strand per model), result marshalling, and transport: +You write a model as plain, single-threaded C++. The framework handles +concurrency (one strand per model), result marshalling, and transport: ```cpp struct MyAction { int x = 0; }; @@ -42,6 +69,14 @@ BRIDGE_REGISTER_MODEL (MyModel, "MyModel") BRIDGE_REGISTER_ACTION(MyModel, MyAction, "MyAction") ``` +> **Registration is per–translation-unit.** Put each `BRIDGE_REGISTER_*` call in +> exactly one `.cpp` (the one that owns the model), never in a header — the +> macros emit an explicit template specialisation plus a file-scope initialiser, +> so a header invocation is an ODR violation. If your models live in a **static +> library**, force-link it (`--whole-archive` / `-force_load` / +> `WHOLE_ARCHIVE`), or the linker will drop the unreferenced registration object +> and the model silently never registers. + Then drive it from the GUI — **the same code works local or remote**: ```cpp @@ -62,36 +97,43 @@ handler.execute(MyAction{21}) ``` To go remote, construct the `Bridge` with a different backend (e.g. -`SimulatedRemoteBackend` or `QtWebSocketBackend`) — the model, the registration, and -the call site above are unchanged. +`SimulatedRemoteBackend` or `QtWebSocketBackend`) — the model, the registration, +and the call site above are unchanged. + +> **Lifetime rule:** a `Bridge` must outlive every `BridgeHandler` registered on +> it. A handler deregisters itself from its bridge on destruction, so destroying +> the bridge first is undefined behaviour. Construct the bridge before, and +> destroy it after, all handlers that use it. ### The three pieces - **`guiExecutor`** — a `morph::exec::IExecutor*` that decides *where* result - callbacks run. Models execute on a worker pool, but `.then(...)` / `.onError(...)` - are always marshalled back onto this executor, so they land wherever you want - (typically the GUI event loop) and never block it. Use `MainThreadExecutor` (pump it - with `runFor(...)` from your loop) for a plain app, or `morph::qt::QtExecutor` for - Qt. The worker pool you pass to `LocalBackend` is a *separate* executor — it runs - the models themselves. -- **`Bridge`** — the single, process-wide hub. It owns the active backend (local or - remote) and tracks every live handler so it can re-point them when the backend is - swapped (e.g. going offline → online). You construct **one** `Bridge` and share it. -- **`BridgeHandler`** — your typed, GUI-facing handle to one model type `M`. It - registers `M` on the bridge on construction and deregisters on destruction (RAII). - This is the object you call `.execute(...)` / `.subscribe(...)` on. Create one - per model type, wherever in the UI you need to talk to that model. + callbacks run. Models execute on a worker pool, but `.then(...)` / + `.onError(...)` are always marshalled back onto this executor, so they land + wherever you want (typically the GUI event loop) and never block it. Use + `MainThreadExecutor` (pump it with `runFor(...)` from your loop) for a plain + app, or `morph::qt::QtExecutor` for Qt. The worker pool you pass to + `LocalBackend` is a *separate* executor — it runs the models themselves. +- **`Bridge`** — the single, process-wide hub. It owns the active backend (local + or remote) and tracks every live handler so it can re-point them when the + backend is swapped (e.g. going offline → online). You construct **one** + `Bridge` and share it. +- **`BridgeHandler`** — your typed, GUI-facing handle to one model type `M`. + It registers `M` on the bridge on construction and deregisters on destruction + (RAII). This is the object you call `.execute(...)` / `.subscribe(...)` on. + Create one per model type, wherever in the UI you need to talk to that model. ## Multiple models across multiple files -A real app has several models and constructs handlers in different translation units -(one per screen/widget). The pattern is: **declare each model and its actions in a -header, register it once in a matching `.cpp`, share a single `Bridge`, and construct -a `BridgeHandler` wherever you need it.** +A real app has several models and constructs handlers in different translation +units (one per screen/widget). The pattern is: **declare each model and its +actions in a header, register it once in a matching `.cpp`, share a single +`Bridge`, and construct a `BridgeHandler` wherever you need it.** The model and its action/result types go in a header. Any code that constructs -`BridgeHandler` or calls `.execute(PlaceOrder{...})` needs the complete -`OrdersModel` definition, so put it here rather than forward-declaring it: +`BridgeHandler` or calls `.execute(PlaceOrder{...})` needs the +complete `OrdersModel` definition, so put it here rather than forward-declaring +it: ```cpp // orders_model.hpp @@ -105,9 +147,10 @@ struct OrdersModel { }; ``` -The matching `.cpp` includes that header and registers the model. The registration -macros run at static-init time, so simply linking this `.cpp` into your binary makes -the model known to the framework — there is no central list to maintain: +The matching `.cpp` includes that header and registers the model. The +registration macros run at static-init time, so simply linking this `.cpp` into +your binary makes the model known to the framework — there is no central list to +maintain: ```cpp // orders_model.cpp @@ -120,31 +163,6 @@ BRIDGE_REGISTER_MODEL (OrdersModel, "OrdersModel") BRIDGE_REGISTER_ACTION(OrdersModel, PlaceOrder, "PlaceOrder") ``` -A second model lives in its own header + `.cpp` pair, following the same shape: - -```cpp -// inventory_model.hpp -#pragma once - -struct CheckStock { int sku = 0; }; -struct StockLevel { int onHand = 0; }; - -struct InventoryModel { - StockLevel execute(const CheckStock& a); -}; -``` - -```cpp -// inventory_model.cpp -#include "inventory_model.hpp" -#include - -StockLevel InventoryModel::execute(const CheckStock& a) { /* ... */ return StockLevel{/* ... */}; } - -BRIDGE_REGISTER_MODEL (InventoryModel, "InventoryModel") -BRIDGE_REGISTER_ACTION(InventoryModel, CheckStock, "CheckStock") -``` - In the GUI layer, share one `Bridge` and one `guiExecutor`; each screen owns the handler for the model it drives: @@ -165,41 +183,49 @@ orders.execute(PlaceOrder{.sku = 42, .qty = 3}) .then([](OrderId id) { showConfirmation(id); }); ``` -```cpp -// inventory_widget.cpp — uses the Inventory model only. -#include "inventory_model.hpp" - -morph::bridge::BridgeHandler stock{bridge, &guiExecutor}; - -stock.execute(CheckStock{.sku = 42}) - .then([](StockLevel s) { updateBadge(s.onHand); }); -``` - -The screen files reach the shared `bridge` and `guiExecutor` however your app exposes -them (an `extern` declaration, a context object, dependency injection). Each -`BridgeHandler` is independent and tied to its own model type; they all share the same -`Bridge` and the same callback executor, and when a handler goes out of scope it -cleanly deregisters itself. - -## Offline support - -`morph::offline` adds connectivity primitives for apps that must keep working while -disconnected: - -- **`NetworkMonitor`** — background probe thread; fires `onOffline` / `onOnline` on - state change. -- **`IOfflineQueue`** — durable store of opaque payloads enqueued while offline. -- **`SyncWorker`** — drains the queue and replays each item on reconnect, with - built-in retry and dead-lettering. -- **`ReconnectCoordinator`** — sequences *reconnect → activate → bind → replay* in a - strict, unit-tested order when connectivity returns. Pure policy: all side effects - are injected, so it owns no thread and touches no socket. +The screen files reach the shared `bridge` and `guiExecutor` however your app +exposes them (an `extern` declaration, a context object, dependency injection). +Each `BridgeHandler` is independent and tied to its own model type; they all +share the same `Bridge` and the same callback executor, and when a handler goes +out of scope it cleanly deregisters itself. + +> A model that is used **remotely** must be registered via +> `BRIDGE_REGISTER_MODEL` and be **default-constructible** — the server +> re-creates instances from the string type id through the registry, and cannot +> use an ad-hoc factory closure the way `LocalBackend` can. A model that works +> locally via a capturing factory will fail at `register` time in remote mode if +> it isn't macro-registered. + +## Subsystems + +morph is layered: the async/bridge core is always present; everything else is an +opt-in header you include only if you need it. + +| Namespace | Header(s) | What it gives you | +|---|---|---| +| `morph::exec` | `executor.hpp`, `strand.hpp` | `IExecutor`, `ThreadPoolExecutor`, `MainThreadExecutor`, per-model `StrandExecutor` | +| `morph::async` | `completion.hpp` | `Completion` — move-only result handle with `.then` / `.onError` | +| `morph::model` | `registry.hpp`, `model.hpp` | Registration traits, validators, `ActionDispatcher`, type-erased holders | +| `morph::backend` | `backend.hpp`, `remote.hpp` | `LocalBackend`, `RemoteServer`, `SimulatedRemoteBackend` | +| `morph::bridge` | `bridge.hpp` | `Bridge`, `BridgeHandler` — the user-facing API | +| `morph::wire` | `wire.hpp` | JSON `Envelope` protocol between client and server | +| `morph::session` | `session.hpp` | `Context` (principal/locale/metadata), `IAuthorizer` | +| `morph::journal` | `journal.hpp`, `action_log.hpp`, `file_action_log.hpp` | Ordered, coalescing, replayable action log; undo via replay | +| `morph::offline` | `network_monitor.hpp`, `offline_queue.hpp`, `sync_worker.hpp`, `reconnect_coordinator.hpp` | Connectivity detection, write queue, replay-on-reconnect | +| `morph::math` | `rational.hpp` | Exact `int64` rational values with a decimal-precision tag | +| `morph::units` | `quantity.hpp` | Unit-tagged, optionally-empty values with a compile-time unit algebra | +| `morph::time` | `datetime.hpp` | UTC `DateTime` / optionally-empty `Timestamp` with strict ISO-8601 codec | +| `morph::forms` | `forms.hpp`, `choice.hpp` | JSON-Schema generation and form-field types for runtime-built GUIs | +| `morph::qt` | `qt/*.hpp` | `QtExecutor`, `QtWebSocketBackend`, `QtWebSocketServer` (with `MORPH_BUILD_QT=ON`) | + +The full design rationale for each is in [`docs/spec/`](docs/spec) (one file per +type/subsystem) and [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md). ## Exact values, units, and auto-generated forms Actions can carry exact, unit-tagged values instead of raw doubles, and every -action type can describe itself as a JSON Schema — enough for a client to -render its form at runtime: +action type can describe itself as a JSON Schema — enough for a client to render +its form at runtime: ```cpp #include // pulls in quantity.hpp + rational.hpp @@ -226,28 +252,36 @@ struct LabModel { - Values are exact `int64` fractions with a decimal-precision tag — no floating-point rounding, canonical `{"num":617,"den":50,"dp":2}` on the wire. - Units never travel; they live in the C++ types and the schemas. + Units never travel; they live in the C++ types and the schemas. (The `int64` + numerator/denominator is a fixed-width rational, not a bignum — see + [Status & limitations](#status--limitations).) - Each field *declares* its precision in the type — the unit's default unless overridden (`Quantity`) — which becomes the form's input step (`x-decimalPlaces`); the value's actual precision stays runtime data that propagates through arithmetic and can be retagged (`withDecimalPlaces`). - `morph::time::Timestamp` fields travel as strict ISO-8601 UTC strings and - render as date-time inputs (`"format": "date-time"`); malformed timestamps - are wire *errors*, not clamped values. + render as date-time inputs (`"format": "date-time"`); malformed timestamps are + wire *errors*, not clamped values. - Unit systems may declare convertible entry units with exact ratios - (`UnitTraits::alternatives`): fields grow a unit selector, values - recalculate exactly on switch, and payloads always stay canonical. -- `morph::forms::Choice` declares a combo box: - the options are the rows returned by executing the named action, referenced - from the schema via `x-optionsAction` — the option list stays living data - served by the model, never a copy baked into the form. -- A `Quantity` can be *empty* ("not measured yet"). A field is required unless - it is `std::optional` or listed in the action's `optionalFields` opt-out — - the same declaration drives the schema's `required` array, the form's - submit gate, and `validate()`. + (`UnitTraits::relations`): fields grow a unit selector, values recalculate + exactly on switch, and payloads always stay canonical. The schema's + `x-unitAlternatives` list is derived from the same `relations` that drive + conversion, so there is nothing to keep in sync. +- `morph::forms::Choice` declares a combo box: the + options are the rows returned by executing the named action, referenced from + the schema via `x-optionsAction` — the option list stays living data served by + the model, never a copy baked into the form. (The action name and row fields + are string references, resolved at runtime — a typo fails on the client, not at + compile time.) +- A `Quantity` can be *empty* ("not measured yet"). A field is required unless it + is `std::optional` or listed in the action's `optionalFields` opt-out — the + same declaration drives the schema's `required` array, the form's submit gate, + and `validate()`. Note the schema's `required` is a **client-side** gate; the + server dispatcher runs whatever payload arrives, so a model must re-check its + own preconditions. - `morph::forms::schemaJson()` returns a JSON Schema with - units (`ExtUnits`), decimal steps (`x-decimalPlaces`), field order - (`x-order`), bounds, and descriptions (via `glz::json_schema`). + units (`ExtUnits`), decimal steps (`x-decimalPlaces`), field order (`x-order`), + bounds, and descriptions (via `glz::json_schema`). [`examples/forms`](examples/forms) shows the whole loop with two renderers driven purely by the generated schemas — a self-contained HTML page and a @@ -261,10 +295,119 @@ ninja -C build ./build/examples/forms/gui_qml/morph_forms_qml # same forms, rendered in QML ``` +## Action log (audit & undo) + +`morph::journal` records executed actions as an ordered, replayable log — +distinct from the offline queue: the queue holds *pending* writes and deletes +them once delivered, while the journal is a permanent trail the framework never +prunes. Install one sink in `main()` and every model records automatically: + +```cpp +morph::journal::setActionLog(std::make_shared("audit.ndjson")); +``` + +Each successful, loggable action becomes a `LogEntry` (model type, entity key, +payload/result JSON, principal, timestamp). `SessionLog` adds in-memory history +with `checkpoint()` coalescing and `undoLast()`, which reconstructs state by +*replaying* the remaining actions. This model is exact for pure, deterministic, +in-memory models; a model that owns an external store (SQL, network) needs care, +because replay re-executes actions rather than re-applying stored results — see +the design notes in [`docs/spec/journal.md`](docs/spec/journal.md). + +## Offline support + +`morph::offline` adds connectivity primitives for apps that must keep working +while disconnected: + +- **`NetworkMonitor`** — background probe thread; fires `onOffline` / `onOnline` + on state change. Callbacks run on the probe thread and must not block — set an + atomic or post to an executor, don't run reconnect logic inline. +- **`IOfflineQueue`** — durable store of opaque payloads enqueued while offline. + Only an in-memory implementation ships; a durable (SQL/file) queue is the + caller's to provide against the interface. +- **`SyncWorker`** — drains the queue and replays each item on reconnect, with + retry and dead-lettering. Replay must be idempotent (items are retried). +- **`ReconnectCoordinator`** — sequences *reconnect → activate → bind → replay* + in a strict, unit-tested order. Pure policy: all side effects are injected, so + it owns no thread and touches no socket. + +Conflict resolution is deliberately **not** a framework concern: on a backend +switch morph fires `onBackendChanged()` on the fresh model instance and steps +back; how the model reconciles is its own business. + +## Sessions & authorization + +Each `execute` can carry a `morph::session::Context` (principal, locale, +metadata) across the wire, and `RemoteServer` runs every request through an +`IAuthorizer` before dispatch. This is a **policy** hook, not authentication: +the `Context` is client-supplied data with no built-in integrity, the default +authorizer allows everything, and the local backend does not authorize at all. +Treat authentication as the transport's job (e.g. TLS + a token your authorizer +validates) and enforce security-critical checks inside the model. See +[`docs/spec/session.md`](docs/spec/session.md). + +## Building & dependencies + +- **Compiler:** C++23 (developed against recent Clang; libstdc++/libc++). The + default logger uses `std::println`, so a C++23 standard library is required. +- **Dependencies:** [Glaze](https://github.com/stephenberry/glaze) (JSON + reflection), fetched via [vcpkg](https://vcpkg.io) (`vcpkg.json` manifest). + Optional: Qt 6 for the WebSocket transport and QML example. +- **Build system:** CMake (presets in `CMakePresets.json`) + Ninja. + +```sh +export VCPKG_ROOT=/path/to/vcpkg +cmake --preset clang-release +cmake --build build/clang-release --target morph_tests +./build/clang-release/tests/morph_tests +``` + +Relevant CMake options: `MORPH_BUILD_TESTS`, `MORPH_BUILD_EXAMPLES`, +`MORPH_BUILD_QT`, `MORPH_BUILD_FORMS_QML`, `MORPH_BUILD_DOCUMENTATION`. +Because morph is header-only, using it in your own project is just adding +`include/` to your include path and linking Glaze. + +## Examples + +- [`examples/forms`](examples/forms) — the schema-driven forms loop (HTML + + QML renderers) over an exact-value lab model. +- [`examples/bank`](examples/bank) — a larger app: SQLite-backed models behind + DTO/entity layers, a Qt desktop GUI, and a self-contained single-threaded Qt + **WASM** build. Demonstrates the action log end-to-end and the local/remote + split at app scale. + +## Status & limitations + +morph is a young, actively developed library with thorough test coverage of its +core. It is honest about the following boundaries — read the per-subsystem specs +in [`docs/spec/`](docs/spec) before relying on any of these in production: + +- **Security is app-supplied.** The wire protocol has no version negotiation, + no message-size or timeout bounds, and no built-in authentication; `Context` + identity is unauthenticated and `RemoteServer` model ids are guessable + sequential integers with control messages unauthorized. `RemoteServer` + assumes a trusted, authenticated transport — it is not a hardened + public-internet server as shipped. +- **`Completion` is a leaf callback primitive**, not a composable future: one + handler per outcome, no `T→U` chaining, no `co_await`, no cancellation. +- **Exact numbers are fixed-width.** `Rational` is an `int64` pair; `+`/`-`/`*` + can overflow (undefined behaviour) rather than returning an error, and high + decimal precision shrinks the representable magnitude. Wire input is *clamped*, + not rejected, on malformed values. +- **Journal replay re-executes actions**, so undo/reconstruction is only exact + for pure, in-memory models; there is no transactional outbox tying the log to a + model's own store yet. +- **Offline durability is bring-your-own.** Only an in-memory queue ships; the + crash-safety story depends on a durable queue you implement. +- **Registration is global and macro-driven** (per-TU, static-init, string type + ids); there is no runtime deregistration and unknown ids fail at runtime, not + compile time. + ## Learn more -See [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) for the namespace map, layer -diagram, wire protocol, deployment topologies, and design rationale. +- [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) — namespace map, layer diagram, + wire protocol, deployment topologies, and design rationale. +- [`docs/spec/`](docs/spec) — the authoritative per-type/subsystem design specs. ## License diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index d3bf444..93881f8 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -6,6 +6,15 @@ The framework is header-only (C++23, namespace `morph`), depends on Glaze for JSON reflection, and optionally integrates with Qt 6 via a separate target. +> **Design specs.** This document is the cross-cutting map. The authoritative, +> per-subsystem reference lives in `docs/spec/` — one file per public type or +> subsystem, capturing invariants, API surface, and the reasoning behind each +> design. Consult the matching spec before changing a public type: e.g. +> `docs/spec/security.md` (authenticated sessions and the trust model), +> `docs/spec/completion.md`, `docs/spec/executor.md`, `docs/spec/bridge.md`, +> `docs/spec/journal.md`, `docs/spec/offline.md`, `docs/spec/session.md`, +> `docs/spec/wire.md`. Where this document and a spec disagree, the spec wins. + ## Namespace map The public surface is split per topic so callers always know whether a name is part of the stable API or an implementation detail. @@ -19,7 +28,8 @@ The public surface is split per topic so callers always know whether a name is p | `morph::model` | Model & action traits | `ModelTraits<>`, `ActionTraits<>`, `ActionValidator<>`, `ActionLogPolicy<>`, `Loggable` | | `morph::backend` | Pluggable backends | `LocalBackend`, `RemoteServer`, `SimulatedRemoteBackend` | | `morph::bridge` | Bridge between handler and backend | `Bridge`, `BridgeHandler` | -| `morph::offline` | Connectivity + replay | `NetworkMonitor`, `NetworkMonitorConfig`, `IOfflineQueue`, `QueueItem`, `InMemoryOfflineQueue`, `SyncWorker`, `SyncResult` | +| `morph::offline` | Connectivity + replay | `NetworkMonitor`, `NetworkMonitorConfig`, `IOfflineQueue`, `QueueItem`, `InMemoryOfflineQueue`, `SyncWorker`, `SyncResult`, `ReconnectCoordinator`, `ReconnectOutcome`, `ReconnectCoordinatorConfig` | +| `morph::session` | Per-call session context + authentication | `Context`, `IAuthorizer`, `AllowAllAuthorizer`, `allowAllAuthorizer`, `current`; authenticated sessions (`session_auth.hpp`): `SessionToken`, `TokenIssuer`, `TokenVerifier`, `SigningAuthorizer`, `MacFunction`, `hmacSha256`, `AuthError` | | `morph::journal` | Ordered, replayable action log (issue #3) | `LogEntry`, `IActionLog`, `InMemoryActionLog`, `FileActionLog`, `SessionLog`, `replay()`, `toJson`/`fromJson`, `setActionLog`, `defaultActionLog`, `ScopedActionLog` | | `morph::math` | Exact numeric values for actions | `Rational`, `DecimalPlaces`, `RationalError`, `kMaxDecimalPlaces`, `abs`/`ceil`/`floor`/`trunc` | | `morph::units` | Unit-tagged, optionally-empty values | `Quantity`, `UnitMeta`, `UnitTraits` (app-specialised), `UnitAlternative`, `HasUnitAlternatives`, `UnitEnum`, `isQuantity` | @@ -73,7 +83,7 @@ Every nested `detail` namespace under those topics holds implementation symbols. | `journal.hpp` | `SessionLog`, `replay()` (`morph::journal::`) — full-fidelity session log with `checkpoint()` coalescing and `undoLast()`, built on `action_log.hpp` + the existing `ActionDispatcher`/`ModelRegistryFactory` | | `file_action_log.hpp` | `FileActionLog` (`morph::journal::`) — append-only NDJSON `IActionLog`, `flush()` fsyncs | | `rational.hpp` | `Rational`, `DecimalPlaces`, `RationalError` (`morph::math::`) — exact int64 rational arithmetic with a decimal-precision tag; Glaze wire codec (`{"num","den","dp"}`, canonicalised on read) and `std::formatter` | -| `quantity.hpp` | `Quantity`, `UnitMeta`, `UnitTraits` (`morph::units::`) — unit-tagged optional value over `Rational`; units are application enum NTTPs, schemas get `ExtUnits` automatically | +| `quantity.hpp` | `Quantity`, `UnitMeta`, `UnitTraits` (`morph::units::`) — unit-tagged optional value over `Rational`; units are application enum NTTPs, schemas get `ExtUnits` automatically. See `docs/spec/quantity_type.md` for the full design. | | `datetime.hpp` | `DateTime`, `Timestamp` (`morph::time::`) — UTC instant (ms precision) with a strict ISO-8601 wire codec (malformed input is a read *error*) and the optionally-empty field wrapper; schemas carry `"format": "date-time"` | | `choice.hpp` | `Choice`, `FixedString`, `isChoice` (`morph::forms::`) — a field whose options are the result of executing the named action; surfaces as `x-optionsAction`/`x-optionValue`/`x-optionLabel` in schemas, renders as a combo box | | `forms.hpp` | `schemaJson()`, `allRequiredEngaged()`, `EmptyCapableField` (`morph::forms::`) — JSON Schema per action with derived `required`, `x-decimalPlaces`, `x-order`, `x-options*` | @@ -83,6 +93,9 @@ Every nested `detail` namespace under those topics holds implementation symbols. | `network_monitor.hpp` | `NetworkMonitorConfig`, `NetworkMonitor` — background probe thread, online/offline state machine | | `offline_queue.hpp` | `IOfflineQueue`, `QueueItem`, `InMemoryOfflineQueue` — durable write queue abstraction | | `sync_worker.hpp` | `SyncWorker`, `SyncResult` — drains offline queue on reconnect via caller-supplied replay | +| `reconnect_coordinator.hpp` | `ReconnectCoordinator`, `ReconnectOutcome`, `ReconnectCoordinatorConfig` — sequences reconnect → activate → bind → replay on connectivity return; all side effects injected via `Deps` | +| `session.hpp` | `Context`, `IAuthorizer`, `AllowAllAuthorizer`, `allowAllAuthorizer`, `current` (`morph::session::`) — per-call session bag carried through the bridge to the model, plus the server-side authorization seam; internals in `morph::session::detail` (`ScopedContext`, thread-local current context) | +| `session_auth.hpp` | `SessionToken`, `TokenIssuer`, `TokenVerifier`, `SigningAuthorizer`, `MacFunction`, `hmacSha256`, `AuthError` (`morph::session::`) — opt-in signed bearer tokens and a verifying `IAuthorizer`; self-contained reference HMAC-SHA256 in `morph::session::detail`. See `docs/spec/security.md`. | ### Qt integration headers (`include/morph/qt/`) @@ -150,10 +163,24 @@ field is the discriminator: All envelopes round-trip through Glaze JSON, so the protocol is self-describing, escaping-safe, and easy to extend (add a field, ignore unknowns). -The `session` field is a `morph::session::Context`. The server runs every -incoming `execute` envelope through its configured `IAuthorizer`; a `false` -return causes the server to reply with `err|unauthorized` (callId echoed). The -default authorizer permits everything. +The `session` field is a `morph::session::Context`, which now carries a +verified bearer `token` alongside the (untrusted) client-asserted `principal`. +The server runs every incoming `execute` envelope through its configured +`IAuthorizer`; a `false` return causes the server to reply with +`err|unauthorized` (callId echoed). The default authorizer permits everything. +When a verifying authorizer (`SigningAuthorizer`) is installed, the server also +calls `IAuthorizer::authenticate` and, when it yields a principal from a valid +token, **overwrites** `Context::principal` with it before dispatch — so the +verified principal is authoritative and `session::current()->principal` read +inside a model is the authenticated identity, not the client's claim. See +"Authenticated sessions" below and `docs/spec/security.md`. + +`RemoteServer::handleInline` is the synchronous variant used for control +messages (`register`/`deregister`) — e.g. when a `BridgeHandler` is constructed +from inside an action handler running on the worker pool. It **rejects** +`execute` up front with an `err` reply, because an `execute` reply is produced +asynchronously on the strand, after the synchronous call has already returned +and destroyed the reply buffer the deferred callback would write into. `contextKey` carries a `register`ing instance's stable identity (e.g. an account id) from `HandlerBinding::contextKey` across the wire, so a @@ -167,10 +194,12 @@ identity; the field is ignored on every other envelope kind. All concurrency runs through `morph::exec::IExecutor::post(fn)`: -- **`ThreadPoolExecutor`** — fixed N worker threads, MPMC queue; exceptions are swallowed and logged. -- **`MainThreadExecutor`** — single-threaded queue with `runFor(timeout)` drain; used in non-Qt tests to pump the "GUI" thread. +- **`ThreadPoolExecutor`** — fixed N worker threads, MPMC queue. A task exception is caught and logged via `morph::log` (`std::exception` with its `what()`, anything else as "unknown exception"); it never propagates out of the worker or aborts sibling tasks. +- **`MainThreadExecutor`** — single-threaded queue with `runFor(timeout)` drain; used in non-Qt tests to pump the "GUI" thread. It catches only `std::exception` from a task, logs it via `morph::log`, and continues with the next task. - **`QtExecutor`** — posts via `QMetaObject::invokeMethod(Qt::QueuedConnection)`; safe from any thread; drops silently if the target object is deleted. +`morph::exec::detail::StrandExecutor` (below) is where `Model::execute()` actually runs; like `ThreadPoolExecutor`, it catches a task exception (`std::exception` or unknown) and logs it via `morph::log` so a throw neither stalls the strand nor vanishes — the next queued task for that model still runs. + ### StrandExecutor `morph::exec::detail::StrandExecutor` guarantees that all tasks for the same `ModelId` are serialised while tasks for different models are parallelised. Internally keeps one `std::queue` and a `running` flag per model; tasks are dispatched to the underlying `IExecutor` one at a time. @@ -208,14 +237,39 @@ Bridge ──holds──► weak_ptr (does NOT kee **RAII lifetime:** When `BridgeHandler` goes out of scope, its destructor calls `Bridge::deregisterHandler`, which tells the backend to destroy the model instance and removes the stale `weak_ptr` from the Bridge's list. No manual cleanup is required from application code. +**Destruction order is now safe either way:** the `BridgeHandler` also holds a `weak_ptr` liveness token published by the `Bridge` (`Bridge::liveness()`; the token is destroyed with the bridge). The destructor deregisters only if that token is still live — if the `Bridge` was destroyed first, the token has expired and the handler skips deregistration instead of dereferencing a dangling `Bridge&`. Destroying a bridge before its handlers is still discouraged, but is defined behaviour rather than a use-after-free. + **`atomic currentId`:** Every call to `executeVia` reads `binding->currentId` to find out which backend-assigned ID to use. The value is an atomic so it can be updated during a backend switch without holding the bridge mutex for the duration of every execute call. A value of `0` is the sentinel for "not bound" — `executeVia` returns an immediate error in that case. -**Backend switching:** `Bridge::switchBackend(newBackend)` acquires the bridge mutex, re-registers every live `HandlerBinding` on the new backend (writing the new `ModelId` atomically into `binding->currentId`), replaces the active backend, then calls `notifyBackendChanged()`. `LocalBackend` forwards this to every live model that implements `IBackendChangedSink` (detected at compile time via the `BackendChangedNotifiable` concept). `SimulatedRemoteBackend` is a no-op — its models live inside `RemoteServer`. +**Backend switching:** `Bridge::switchBackend(newBackend)` acquires the bridge mutex, re-registers every live `HandlerBinding` on the new backend, replaces the active backend, then calls `notifyBackendChanged()`. `LocalBackend` forwards this to every live model that implements `IBackendChangedSink` (detected at compile time via the `BackendChangedNotifiable` concept). `SimulatedRemoteBackend` is a no-op — its models live inside `RemoteServer`. + +The switch is **exception-safe and atomic** (stage-all-then-commit): phase 1 registers every binding on the new backend into a staging list *without* touching any `currentId`; if any registration throws (a plausible remote/transport failure) the already-registered instances are rolled back with `deregisterModel` and the exception is rethrown, leaving the old backend and every `currentId` untouched — the switch either fully succeeds or is a no-op. Only after all registrations succeed does phase 2 publish the new `ModelId` values atomically into each `binding->currentId` and swap the backend in. The outgoing backend's pending completions are drained (`cancelPending` with `BackendChangedError`) *outside* the bridge mutex, so user callbacks never run while `_mtx` is held. ### Logger `morph::log` provides a global, mutex-protected, replaceable sink (`std::function`). `LogLevel` is a `uint8_t`-backed enum (`debug < info < warn < error < off`). All framework internals route through `morph::log::detail::log(level, msg)`; call `morph::log::setLogger` at startup to redirect to spdlog, Qt logging, or a test spy. +### Authenticated sessions + +`morph::session` carries a per-call `Context` (principal, verified `token`, +`requestId`, `locale`, and a free-form metadata bag) from the caller through the +bridge to the model — `session::current()` reads it inside `execute()` without +changing the model signature. The server-side authorization seam is +`IAuthorizer` (`authorize` gates each `execute`; `authenticate` returns the +verified principal to make authoritative); `AllowAllAuthorizer` / +`allowAllAuthorizer()` is the default and permits everything. + +`session_auth.hpp` adds the opt-in authenticated variant: signed, stateless +bearer tokens (`SessionToken` claims, `base64url(claims).base64url(mac)`) minted +by `TokenIssuer`, verified by `TokenVerifier`, and enforced per request by +`SigningAuthorizer`. The MAC primitive is pluggable (`MacFunction`); a +self-contained, test-vector-verified `hmacSha256` is the default reference +implementation. `AuthError` enumerates verification failures. + +**`docs/spec/security.md` is the authoritative spec** for the trust model, the +authoritative-principal flow, and the limits of the reference crypto — this +section is only a map, not a substitute. + ### NetworkMonitor `morph::offline::NetworkMonitor` provides a background probe thread that tracks connectivity and fires callbacks on transitions. The framework supplies no probe implementation — the caller provides a `bool()` callable that returns `true` when the system is reachable (TCP connect, HTTP ping, custom check, etc.). This keeps the monitor fully portable across OS and transport. @@ -256,7 +310,7 @@ morph::offline::NetworkMonitor monitor{ `morph::journal` records executed actions as an ordered, replayable log, distinct in purpose from `IOfflineQueue` above: `IOfflineQueue` holds pending writes awaiting retry and deletes them once delivered; the action log is a permanent audit/replay trail — entries are never removed by the framework. -**`IActionLog`** is the durable-sink interface (`append`, `flush`, `entries`), implemented by `InMemoryActionLog` and `FileActionLog` (append-only NDJSON, `flush()` fsyncs). Each `LogEntry` carries `modelType`, `entityKey`, `actionType`, `payload`/`result` JSON, `principal`, and a sink-assigned `seq`. +**`IActionLog`** is the durable-sink interface (`append`, `flush`, `entries`), implemented by `InMemoryActionLog` and `FileActionLog` (append-only NDJSON, `flush()` fsyncs). Each `LogEntry` carries `modelType`, `entityKey`, `actionType`, `payload`/`result` JSON, `principal`, and a sink-assigned `seq`. `FileActionLog::entries()` **tolerates a torn trailing line**: a crash between `append`'s write and the next flush can leave a truncated final line, so a malformed *last* line is logged and skipped rather than throwing — but a malformed line mid-file is genuine corruption and is re-thrown. **Set the sink once, in `main()` — every model uses it automatically.** `morph::journal::setActionLog(log)` installs a process-wide default. `ModelFactory::create()` — the factory behind every ordinary model registration, local *or* remote — attaches that default to each new instance automatically (empty `entityKey`). No per-model, per-handler, or per-backend wiring is required; `RemoteServer`-owned instances get it exactly the same way, since they're constructed through the same factory. `defaultActionLog()` reads the current sink back; `ScopedActionLog` (RAII, mirrors `morph::log::ScopedLoggerOverride`) installs one temporarily and restores the previous one on scope exit — the tool tests use it to avoid leaking a sink across test cases. @@ -276,6 +330,8 @@ Because these are mutually exclusive per topology, recording is automatically se **`SessionLog`** (`journal.hpp`) is where coalescing actually happens. It keeps full, uncoalesced history in memory (the raw material for `undoLast()`), and `checkpoint(durableSink)` reduces everything appended since the last checkpoint by `(modelType, entityKey, actionType)` — keeping only the latest entry where `coalesce == true`, every entry otherwise — before forwarding the reduced set to the real sink. `undoLast()` needs no inverse operations: it drops the most recent entry and calls `journal::replay()` over what remains, reusing the same `ActionDispatcher`/`ModelRegistryFactory` `RemoteServer` already relies on for dispatch. This is not a workaround — a model's entire state genuinely is "initial state plus its ordered actions replayed," so reconstructing it by replay is the direct statement of that fact, not a special case. +**Replay/undo never records into the live default log.** `journal::replay()` builds the reconstructed holder through `ModelRegistryFactory::create`, which (like every factory-built model) auto-attaches the process-wide default action log. Before replaying, `replay()` immediately **detaches** it (`attachActionLog(nullptr, {})`), so re-running the recorded actions does not re-record each one into the live sink — which would corrupt the very audit trail being read from. The reconstructed holder is a detached, throwaway artifact. + **Remote-mode per-instance identity** (`RemoteServer::setLogProvider`) is the advanced escape hatch for when the global default isn't granular enough: `RemoteServer` owns the actual model instances behind any remote/simulated-remote client, so it is the only place able to attach a *different* log (or a specific `entityKey`) to a *specific* instance. `HandlerBinding::contextKey` (client-side) travels through the `register` wire envelope's `contextKey` field; if a `LogProvider` is installed, `RemoteServer` calls it with `(modelType, contextKey)` and attaches whatever `IActionLog` it returns (or nothing, if it returns `nullptr` or no `contextKey` was sent) before the instance ever executes an action — overriding whatever the global default would have attached. **Not yet built** (see the design note linked from issue #3): the outbox pattern an integration against a model that also owns its own durable store would need (to avoid the log and the store's committed state silently diverging) — see `examples/bank`, which demonstrates `setActionLog` end to end against SQLite-backed models but writes to its own DB and the audit log as two independent steps, not one atomic outbox write. A Kafka-backed sink (dropped for now; the `IActionLog` interface is designed so one can be added later without touching call sites) and any read-model built on top of it are noted as future work. @@ -289,6 +345,22 @@ Because these are mutually exclusive per topology, recording is automatically se - `stop()` signals the current `run()` to abort after the current item. Resets at the start of each `run()`, so it is one-shot. - Concurrent `run()` calls are serialised by an internal mutex. +### ReconnectCoordinator + +`morph::offline::ReconnectCoordinator` sequences the steps that must happen, in +order, when connectivity returns: `tryReconnect()` → `activatePrimary()` → +`bindContext()` → `replay()`. All side effects are injected via `Deps` +callables — the class contains only the retry loop, the strict ordering +guarantee (each step waits for the previous), and the abort checks +(`shouldContinue()` is polled before each attempt and again before replay). It +performs no I/O and owns no thread; `onOnline()` / `onOffline()` run +synchronously on the calling thread and are mutually serialised by an internal +mutex (mirroring `SyncWorker::run()`). `onOnline()` returns a `ReconnectOutcome` +(`Reconnected` / `GaveUp` after `maxAttempts` / `Aborted`); retry tuning lives in +`ReconnectCoordinatorConfig` (`maxAttempts`, `retryDelay`). `tryReconnect` and +`shouldContinue` throwing are treated as a failed attempt / "do not continue" +respectively. + ### Conflict Resolution — a domain concern, not a framework concern Conflict resolution during offline-to-online sync belongs entirely in the model. The framework's role is to fire `onBackendChanged()` on the new model instance and then step back. How the model handles that notification is its own business. @@ -308,7 +380,7 @@ Conflict resolution during offline-to-online sync belongs entirely in the model. | `Bridge` | Handler list protected by mutex; register/deregister safe from any thread. | | Logger | Sink and level accesses protected by mutex. | | `StrandExecutor` | Per-strand mutex + atomic running flag; safe from any thread. | -| `Bridge::switchBackend` | Holds bridge mutex for full duration; re-registration and notification are atomic with respect to new `execute` calls. | +| `Bridge::switchBackend` | Holds bridge mutex while staging + committing; re-registration and notification are atomic with respect to new `execute` calls. Exception-safe: a registration failure rolls back and leaves the old backend and all `currentId`s untouched (no-op). Outgoing-backend cancellation runs after the mutex is released. | ## Error propagation @@ -320,7 +392,9 @@ Model::execute(action) throws └─ fn receives exception_ptr; caller rethrows to inspect ``` -If the `Completion` is abandoned (no `.onError` attached), the destructor logs the exception. Non-`std::exception` types are logged as "unknown exception". +If the `Completion` is abandoned (no `.onError` attached, or no callback executor to deliver it on), the destructor logs the exception through the orphan logger. Non-`std::exception` types are logged as "unknown exception". + +Task exceptions on the executors themselves are handled independently: `ThreadPoolExecutor` and `StrandExecutor` catch and log every task throw via `morph::log`, and `MainThreadExecutor::runFor` catches `std::exception`. A throwing task therefore never kills a worker or stalls a strand — see "Executors" above. ## Adding a new model and actions @@ -501,12 +575,14 @@ Two further field types follow the same one-kind-of-empty pattern: and — unlike the clamping `Rational` codec — a malformed timestamp is a JSON **read error**: there is no meaningful clamp for a mistyped instant. Schemas carry the standard `"format": "date-time"` annotation. -- **Unit switching** (`UnitTraits::alternatives`): a unit system may +- **Unit switching** (`UnitTraits::relations`): a unit system may declare convertible entry units per canonical unit with exact rational ratios (grams -> kilograms as `{g, 1, 1000}`). They surface as `x-unitAlternatives` in the schema; renderers offer a unit selector and recalculate the entered value exactly on switch, and payloads always carry - the canonical unit — the model never sees display units. + the canonical unit — the model never sees display units. The alternatives + list is **derived from the same `relations`** that drive `convert` — there is + no separate `alternatives` declaration to keep in sync. - **`Choice`**: declares in the type that the field is *not free input* — its options are the rows returned by executing the named action (itself just a registered action, typically @@ -543,9 +619,11 @@ Callbacks (`onOffline`, `onOnline`) run directly on the probe thread. A blocking `switchBackend` holds `Bridge::_mtx` for its entire duration and calls `notifyBackendChanged()` while still holding it. `onBackendChanged()` is invoked from inside that call. If an `onBackendChanged()` implementation calls `switchBackend` or `registerHandler` / `deregisterHandler` (which also acquire `_mtx`), the thread will self-deadlock. `executeVia` is safe to call from `onBackendChanged()` because it uses a lock-free snapshot of the backend. -### `CompletionState` requires a non-null executor before callbacks fire +### A null callback executor drops the callback (but not the orphan log) + +The `cbExec` pointer on `CompletionState` should be set before `setValue` / `setException` is called (or before `attachThen` / `attachOnError` if the state is already ready). If `cbExec` is null when a callback would fire, that callback is silently dropped — no error is raised. This is enforced by the `Completion` constructor, which accepts an `IExecutor*`; the hazard only arises when using `CompletionState` directly (an internal type). -The `cbExec` pointer on `CompletionState` must be set before `setValue` / `setException` is called (or before `attachThen` / `attachOnError` if the state is already ready). If `cbExec` is null when a callback would fire, the callback is silently discarded — no error is raised. This is enforced by the `Completion` constructor, which accepts an `IExecutor*`; the hazard only arises when using `CompletionState` directly (an internal type). +An abandoned **error**, however, is no longer silenced by a null executor: `onErrAttached` (the flag that suppresses the destructor's orphan log) is only set when an executor actually exists to deliver the handler. With a null executor the error handler is never posted, so `onErrAttached` stays `false` and `~CompletionState` still logs the exception through the orphan logger rather than losing it. ### `MainThreadExecutor::runFor` does not drain on timeout diff --git a/docs/spec/backend.md b/docs/spec/backend.md new file mode 100644 index 0000000..0a8402b --- /dev/null +++ b/docs/spec/backend.md @@ -0,0 +1,390 @@ +# The `morph::backend` types — design + +`morph::backend` provides the execution backends that own model instances and +dispatch actions against them. The namespace is split into two header files: + +- **`backend.hpp`** — `ActionCall`, `IBackend`, error types, `LocalBackend`. +- **`remote.hpp`** — `RemoteServer`, `SimulatedRemoteBackend`. + +`Bridge` (in `bridge.hpp`) holds one active backend at a time and can swap it +atomically via `Bridge::switchBackend()`. Every backend follows the same +contract: register and deregister models, dispatch actions, cancel pending work, +and react to backend changes. + +## Contents + +- [The dispatch struct — `ActionCall`](#the-dispatch-struct--actioncall) +- [The abstract interface — `IBackend`](#the-abstract-interface--ibackend) +- [Error types](#error-types) +- [`LocalBackend` — in-process execution](#localbackend--in-process-execution) +- [`RemoteServer` — server-side message handler](#remoteserver--server-side-message-handler) +- [`SimulatedRemoteBackend` — adapter for testing](#simulatedremotebackend--adapter-for-testing) +- [Lifetime & ownership](#lifetime--ownership) +- [Failure modes](#failure-modes) +- [Thread context](#thread-context) +- [API reference](#api-reference) +- [Design decisions](#design-decisions) +- [Cross-references](#cross-references) +- [Limitations](#limitations) + +## The dispatch struct — `ActionCall` + +`detail::ActionCall` bundles everything needed to dispatch one action. It has +three callables — one for each execution path — so the same `ActionCall` can be +used locally or serialised for a remote round-trip: + +| Field | Type | Purpose | +|---|---|---| +| `modelTypeId` | `std::string` | String id of the target model type (from `ModelTraits`). | +| `actionTypeId` | `std::string` | String id of the action type (from `ActionTraits`). | +| `serializeAction` | `std::function` | Serialises the action to JSON. Only called on the remote path. | +| `deserializeResult` | `std::function(std::string_view)>` | Deserialises a JSON reply into the opaque result. Only called on the remote path. | +| `localOp` | `std::function(IModelHolder&)>` | Executes the action directly against a model holder. Only called on the local path. | +| `session` | `morph::session::Context` | Session context. Local backends thread it through a thread-local before invoking `localOp`; remote backends serialise it into the wire envelope. | + +## The abstract interface — `IBackend` + +`detail::IBackend` is the abstract interface every backend implements. `Bridge` +holds a `unique_ptr` and delegates all model operations to it. + +| Method | Purpose | +|---|---| +| `registerModel(typeId, factory)` | Registers a new model instance, returns its opaque `ModelId`. | +| `registerModelWithContext(typeId, factory, contextKey)` | Same as `registerModel`, additionally passes a stable identity (e.g. account id). Default implementation drops `contextKey` and forwards to `registerModel` — correct for `LocalBackend` where the factory closure already captures identity. `SimulatedRemoteBackend` overrides to carry `contextKey` across the wire. | +| `deregisterModel(mid)` | Removes the model identified by `mid`. | +| `execute(mid, call, cbExec)` | Dispatches `call` against the model identified by `mid`. Returns a `Completion>`. | +| `notifyBackendChanged()` | Called by `Bridge::switchBackend()` after all handlers are re-registered. | +| `cancelPending(exc)` | Resolves every still-pending completion with `exc`. Called on the outgoing backend during `switchBackend()` and in `Bridge`'s destructor. After this call, any later `setValue`/`setException` on those states is a no-op. | +| `setReconnectHandler(handler)` | Installs a callback invoked when the backend reconnects to its peer. Used by backends with transport (e.g. `QtWebSocketBackend`). Default implementation is a no-op. | + +## Error types + +Three exception types are thrown into in-flight `Completion`s: + +| Type | Trigger | Purpose | +|---|---|---| +| `BackendChangedError` | `Bridge::switchBackend()` runs | GUI can retry on the new backend or surface a "backend changed" message. | +| `BridgeDestroyedError` | `Bridge` is destroyed | In-flight completions are cancelled because the bridge is gone. | +| `DisconnectedError` | Transport drops mid-call (e.g. WebSocket disconnect) | Framework retries the call on reconnect if the backend supports it; otherwise the GUI's `.onError(...)` runs. | + +## `LocalBackend` — in-process execution + +`LocalBackend` is the concrete in-process backend. It owns a +`StrandExecutor` (wrapping the `IExecutor&` worker pool, typically a +`ThreadPoolExecutor`) and a `ModelId → shared_ptr` map. + +**Lifecycle:** +- `registerModel` — atomically increments a counter, locks the registry mutex, + calls the factory, stores the holder, returns the new `ModelId`. +- `deregisterModel` — locks the registry mutex, erases the entry. +- `execute` — looks up the holder under the registry lock; if `mid` is unknown + it immediately resolves the completion with + `std::runtime_error("model not found: id=")`. Otherwise it tracks the + completion in the pending list, posts `localOp` on the model's strand + (serialised per-model), sets up a `ScopedContext` (from `call.session`) before + calling `localOp`, and returns the `Completion`. +- `cancelPending` — snapshots the pending list under the pending mutex, delivers + `exc` to every still-live state. +- `notifyBackendChanged` — iterates all models, calls `onBackendChanged()` on + every holder that implements `IBackendChangedSink`. +- `setReconnectHandler` — no-op (no transport to reconnect). + +Each model instance gets its own strand so actions are serialised per-model +without a global lock on the pool. + +## `RemoteServer` — server-side message handler + +`RemoteServer` receives JSON envelopes (`morph::wire::Envelope`) from any +transport and executes the corresponding model operations via an +`ActionDispatcher`. Authorization is delegated to an `IAuthorizer` that defaults +to allow-all. It derives from `std::enable_shared_from_this`. + +**Must be heap-allocated via `std::make_shared`.** `handle()` captures +`shared_from_this()` to prevent use-after-free when the worker pool outlives the +server. + +**Wire format.** All requests and replies are JSON `morph::wire::Envelope`. The +`kind` field discriminates three message types: + +| `kind` | Request fields | Reply | Notes | +|---|---|---|---| +| `register` | `typeId`, `[contextKey]` | `ok` with `modelId` (body empty) | Creates a model via the `ModelRegistryFactory`. Empty `typeId` → `err "register requires a typeId"`. If `contextKey` is non-empty, consults the `LogProvider` (if set) and, when it returns a non-null log, calls `holder->attachActionLog(log, contextKey)`. | +| `deregister` | `modelId` | `ok` | Erases the model from the registry. | +| `execute` | `modelId`, `modelType`, `actionType`, `body`, `session` | `ok` with `body` or `err` | See the execute flow below. | + +**Execute flow (`dispatchExecute`).** In order: + +1. **Authorize.** `_authorizer->authorize(env.session, env.modelType, env.actionType)`. + Denied → `err "unauthorized"` (with the request's `callId`), no dispatch. +2. **Authenticate / make the principal authoritative.** After `authorize` + succeeds, the server calls `_authorizer->authenticate(env.session)`. If it + returns a value, the server **overwrites** `env.session.principal` with that + verified principal *before* building the `ScopedContext`. So model code that + reads `session::current()->principal` on the remote path sees the identity the + authorizer extracted from a valid token, not the client's asserted claim + (`Context::principal` is untrusted wire input on its own). A non-verifying + authorizer — including the default `AllowAllAuthorizer` — returns `nullopt` + from `authenticate` and the client-supplied principal is left unchanged. This + is the only place the principal is made authoritative; the verifying + implementation lives in `SigningAuthorizer` (`session_auth.hpp`, cross-ref + security.md). The rewrite happens on the calling/pool thread, before the + strand task is posted. +3. **Look up the model.** Under `_regMtx`, find `env.modelId`. Missing → + `err "model not found"` (with `callId`), no dispatch. (Note: the remote + message is the bare string `"model not found"`, without the id — unlike the + `LocalBackend` path, which resolves the completion with + `std::runtime_error("model not found: id=")`.) +4. **Dispatch on the strand.** Posts to the model's strand a task that installs a + `ScopedContext` from the (now possibly rewritten) `env.session`, calls + `ActionDispatcher::dispatch(modelType, actionType, *holder, body)`, and replies + `ok` with the serialised result. Any `std::exception` thrown by the dispatch is + caught on the strand and returned as `err exc.what()` with the `callId`. + +Any envelope that fails to decode produces `err` carrying the decode +exception's message. An unrecognised `kind` produces `err "unknown envelope +kind: "`. Any `std::exception` thrown while handling a decoded envelope is +caught and returned as an `err` reply carrying `exc.what()` and the request's +`callId`. + +**`handle(msg, reply)`** — asynchronous entry point. Posts to the worker pool, +calls `dispatchMessage` which decodes, dispatches by `kind`, and calls `reply` +exactly once. + +**`handleInline(msg)`** — synchronous entry point intended for control messages +(`register`, `deregister`) only. It runs `dispatchMessage` directly on the +calling thread instead of posting the message to the worker pool, so it is safe +to call from a thread that *is* the worker pool. It **rejects `execute`** up +front: an `execute` reply is produced asynchronously on the model's strand, +*after* the synchronous call has returned and destroyed the local reply buffer +the deferred callback would write into, so `handleInline` decodes the envelope +first and, if its `kind` is `execute`, returns an `err` reply +(`"handleInline does not support execute (reply is asynchronous)"`) without +dispatching. A malformed envelope falls through to `dispatchMessage`, which +emits the canonical decode-error reply. + +**`setLogProvider(provider)`** — installs a `LogProvider` callable consulted on +every `register` envelope whose `contextKey` is non-empty. This is how +`RemoteServer` attaches action logs to model instances created on behalf of +remote clients — the factory closure (which lives on the client side) cannot +capture the server-side log. Thread-safe. + +```cpp +using LogProvider = std::function( + std::string_view modelType, std::string_view contextKey)>; +``` + +## `SimulatedRemoteBackend` — adapter for testing + +`SimulatedRemoteBackend` implements `IBackend` by forwarding all calls through +a `RemoteServer`. Control operations (`registerModel`, `registerModelWithContext`, +`deregisterModel`) are forwarded synchronously via `RemoteServer::handleInline`; +`execute` is forwarded asynchronously via `RemoteServer::handle` and returns a +`Completion` that resolves on the server's reply. Intended for testing and +in-process simulation of remote execution. + +**Key design choices:** +- `registerModel` and `registerModelWithContext` use `handleInline` (synchronous + control message) — the `factory` argument is ignored because model construction + is delegated to the server's `ModelRegistryFactory`. +- `deregisterModel` likewise uses `handleInline`. +- `execute` serialises the action via `call.serializeAction()`, builds an + `execute` envelope, calls `handle()` (asynchronous), and returns a `Completion` + that resolves when the server's reply is deserialised via + `call.deserializeResult()`. +- `notifyBackendChanged` is a no-op — models live in the `RemoteServer`, not + locally. +- `cancelPending` snapshots and resolves pending completions, same pattern as + `LocalBackend`. + +## Lifetime & ownership + +The backends hold *references*, not owning pointers, to the resources they run +on. Getting the destruction order wrong is a use-after-free, so the invariants +are: + +- **The worker pool must outlive the backend.** Every backend takes an + `IExecutor& workerPool` by reference (`LocalBackend`, `RemoteServer`) and wraps + it in a `StrandExecutor`. The pool (typically a `ThreadPoolExecutor`) must be + destroyed *after* the backend that references it — and, in practice, after the + `Bridge` that owns the backend. Destroying the pool first leaves the strand + pointing at freed storage. +- **`RemoteServer` must be created via `std::make_shared`.** It derives from + `std::enable_shared_from_this`; `handle()` captures + `shared_from_this()` into the pool task. Constructing it on the stack and + calling `handle()` throws `std::bad_weak_ptr` (see ARCHITECTURE.md + "RemoteServer must be heap-allocated"). +- **The `RemoteServer` shared_ptr must outlive every referencing + `SimulatedRemoteBackend` (and every transport `QtWebSocketServer` etc.).** + `SimulatedRemoteBackend` stores `RemoteServer& _server` — a non-owning + reference — and forwards every `registerModel` / `deregisterModel` / `execute` + through it. If the server's owning shared_ptr is released while a backend still + references it, subsequent calls dereference a dangling reference. The `handle()` + path is self-protecting for tasks already *in flight* (each captures a + shared_ptr copy), but `SimulatedRemoteBackend`'s reference member is not — the + caller must keep the server alive for the backend's whole lifetime. +- **Pending strand tasks capture shared_ptr copies, so model destruction + mid-flight is safe.** Both backends' `execute` strand tasks capture the model + `holder` by `shared_ptr` copy (and `RemoteServer`'s also captures the reply + callback and the moved `Envelope`). A `deregisterModel` that erases the map + entry while a task is queued or running only drops the *map's* reference; the + in-flight task holds its own, so the holder stays alive until the task + completes. `RemoteServer`'s pool tasks additionally keep the server itself + alive via `shared_from_this()`. + +## Failure modes + +| Situation | Local (`LocalBackend`) | Remote (`RemoteServer` / `SimulatedRemoteBackend`) | +|---|---|---| +| `register` with an unregistered `typeId` | N/A — the local factory closure constructs the instance directly; there is no registry lookup and no type-id failure. | `ModelRegistryFactory::create(typeId)` fails → the catch in `dispatchMessage` replies `err "unknown model type: "`. Remote registration therefore requires the model to have been macro-registered with `BRIDGE_REGISTER_MODEL`. `SimulatedRemoteBackend::registerModelWithContext` turns that `err` into a thrown `std::runtime_error("register failed: unknown model type: ")`. | +| `register` with an empty `typeId` | N/A | `err "register requires a typeId"`. | +| `execute` against an unknown model id | Completion resolves with an **untyped** `std::runtime_error("model not found: id=")`. | `err "model not found"` (bare, no id); `SimulatedRemoteBackend` surfaces it as a thrown/`onError` `std::runtime_error("model not found")`. | +| Action handler throws | Caught on the strand; completion resolves with the thrown exception. | Caught on the strand; `err exc.what()` reply, which the client re-throws into the completion. | +| Envelope fails to decode | N/A | `err ` (no `callId` echoed — it couldn't be parsed). | +| Unrecognised `kind` | N/A | `err "unknown envelope kind: "`. | + +There is **no typed "model not found" exception** on either path — callers that +need to distinguish it from any other `std::runtime_error` have only the message +string to go on, and the local and remote messages differ (see the table). The +typed error hierarchy (`BackendChangedError`, `BridgeDestroyedError`, +`DisconnectedError`) covers only lifecycle/transport cancellation, not +per-call dispatch failures. + +## Thread context + +An `ActionCall`'s three callables run on three different threads across a remote +round-trip; model and GUI authors must not assume any two share a thread: + +| Callable | Runs on | +|---|---| +| `serializeAction` | The **calling / GUI thread** — `SimulatedRemoteBackend::execute` invokes it synchronously while building the envelope, before handing off to the pool. | +| `deserializeResult` | The **reply / pool thread** — invoked inside the `handle()` reply callback when the server's `ok` arrives (for `SimulatedRemoteBackend`, that is a `RemoteServer` worker-pool thread). | +| `localOp` | The **model strand** (`LocalBackend` only) — posted on the per-`ModelId` `StrandExecutor`, serialised against other actions for the same model. Never invoked on the remote path. | + +On the server side, `RemoteServer` runs authorize/authenticate and the model +lookup on the pool thread that `dispatchMessage` runs on, then runs +`ActionDispatcher::dispatch` (and the `ScopedContext`) on the model strand. +Completion *callbacks* (`.then`/`.onError`) are delivered via the `cbExec` +executor passed to `execute`, independent of all of the above. + +## API reference + +### `detail::ActionCall` + +| Member | Type | Notes | +|---|---|---| +| `modelTypeId` | `std::string` | Target model type id. | +| `actionTypeId` | `std::string` | Target action type id. | +| `serializeAction` | `std::function` | JSON serialiser; remote path only. | +| `deserializeResult` | `std::function(std::string_view)>` | JSON deserialiser; remote path only. | +| `localOp` | `std::function(IModelHolder&)>` | Direct execution; local path only. | +| `session` | `morph::session::Context` | Session context for the call. | + +### `detail::IBackend` + +| Method | Signature | Notes | +|---|---|---| +| `registerModel` | `virtual ModelId registerModel(const string&, function()>)` | Pure virtual. | +| `registerModelWithContext` | `virtual ModelId registerModelWithContext(const string&, function()>, string_view)` | Default: drops `contextKey`, calls `registerModel`. | +| `deregisterModel` | `virtual void deregisterModel(ModelId)` | Pure virtual. | +| `execute` | `virtual Completion> execute(ModelId, ActionCall, IExecutor*)` | Pure virtual. | +| `notifyBackendChanged` | `virtual void notifyBackendChanged()` | Pure virtual. | +| `cancelPending` | `virtual void cancelPending(const exception_ptr&)` | Pure virtual. | +| `setReconnectHandler` | `virtual void setReconnectHandler(const function&)` | Default: no-op. | + +### Error types + +| Type | Base | Message | +|---|---|---| +| `BackendChangedError` | `std::runtime_error` | `"backend changed before completion resolved"` | +| `BridgeDestroyedError` | `std::runtime_error` | `"bridge destroyed before completion resolved"` | +| `DisconnectedError` | `std::runtime_error` | `"transport disconnected before completion resolved"` | + +### `LocalBackend` + +| Method | Notes | +|---|---| +| `explicit LocalBackend(IExecutor& workerPool)` | Constructs with a strand around `workerPool`. | +| `registerModel(typeId, factory)` | Atomically increments `_nextId`, stores the holder under `_regMtx`. `typeId` is accepted for interface compatibility but not used. | +| `deregisterModel(mid)` | Erases from `_models` under `_regMtx`. | +| `notifyBackendChanged()` | Calls `onBackendChanged()` on `IBackendChangedSink` holders. | +| `execute(mid, call, cbExec)` | Posts `call.localOp` on the model's strand with `ScopedContext`. Returns a `Completion`. | +| `cancelPending(exc)` | Snapshots `_pending`, delivers `exc` to each live state. | + +### `RemoteServer` + +| Method | Notes | +|---|---| +| `RemoteServer(workerPool, dispatcher, registry)` | Allow-all authorizer. | +| `RemoteServer(workerPool, authorizer, dispatcher, registry)` | Custom authorizer; null → allow-all. | +| `handle(msg, reply)` | Async: posts to pool, decodes, dispatches, calls `reply` once. Thread-safe. | +| `handleInline(msg)` | Sync: runs `dispatchMessage` on the calling thread and returns the reply JSON; intended for `register`/`deregister` only. **Rejects `execute`** — returns an `err` reply without dispatching, because an `execute` reply is produced asynchronously after this call returns. | +| `setLogProvider(provider)` | Installs a `LogProvider`; `nullptr` clears. Thread-safe. | + +### `SimulatedRemoteBackend` + +| Method | Notes | +|---|---| +| `explicit SimulatedRemoteBackend(RemoteServer& server)` | References the server. | +| `registerModel(typeId, factory)` | Delegates to `registerModelWithContext(typeId, {}, {})`. | +| `registerModelWithContext(typeId, factory, contextKey)` | Sends `register` envelope via `handleInline`. `factory` ignored. | +| `deregisterModel(mid)` | Sends `deregister` envelope via `handleInline`. | +| `execute(mid, call, cbExec)` | Serialises, sends `execute` via `handle`, returns `Completion` that resolves on reply. | +| `notifyBackendChanged()` | No-op. | +| `cancelPending(exc)` | Snapshots `_pending`, delivers `exc` to each live state. | + +## Design decisions + +| Decision | Choice | Why | +|---|---|---| +| Dual-path `ActionCall` | Three callables: `localOp`, `serializeAction`, `deserializeResult` | The same `ActionCall` struct works for both local and remote execution without an `if (isRemote)` branch at the call site — each backend uses the field(s) it needs. | +| `registerModelWithContext` | Virtual with a default that drops `contextKey` | `LocalBackend`'s factory closure already captures identity, so there is nothing to forward. `SimulatedRemoteBackend` overrides to carry `contextKey` across the wire so the server's `LogProvider` can attach an action log. | +| `RemoteServer` heap requirement | `std::enable_shared_from_this` | `handle()` posts to the worker pool capturing `shared_from_this()` — the server must outlive any in-flight message. | +| `handleInline` | Synchronous; caller-restricted to control messages | Safe to call from a worker-pool thread (e.g. from a `BridgeHandler` constructor). It is meant for `register`/`deregister` only; an `execute` envelope is rejected with an `err` reply, because `dispatchExecute` posts to the strand and would reply after `handleInline` returns (writing into an already-destroyed reply buffer). The rejection is now enforced by the code, matching the documented intent. | +| `SimulatedRemoteBackend` factory ignored | Model construction delegated to `RemoteServer`'s `ModelRegistryFactory` | The factory closure lives on the client side; the server owns the actual instances. | +| `cancelPending` snapshots | Weak-ptr snapshot under lock, then resolves outside | Avoids holding the lock while delivering exceptions to each state, preventing deadlock if a callback re-enters the backend. | +| `setReconnectHandler` | Default no-op | Only backends with a transport layer (e.g. `QtWebSocketBackend`) need to react to reconnects. `LocalBackend` and `SimulatedRemoteBackend` never invoke it. | +| Strand-per-model | `StrandExecutor` serialises actions per `ModelId` | Actions against the same model run sequentially; different models can run in parallel. No global lock on the pool. | +| Overwrite `session.principal` on remote execute | `authenticate()` result replaces the client claim before dispatch | The client-asserted `Context::principal` is untrusted; a verifying authorizer makes the token-derived identity authoritative so `session::current()->principal` inside a model is trustworthy. Non-verifying authorizers return `nullopt` and change nothing. | + +## Cross-references + +| Spec | Relationship | +|---|---| +| bridge.md | `Bridge` owns one `IBackend` and swaps it via `switchBackend()`; `BridgeHandler`/`HandlerBinding` carry the `contextKey` that reaches `registerModelWithContext`. `executeVia` builds the `ActionCall`. | +| session.md | `Context`, `IAuthorizer::authorize`/`authenticate`, `ScopedContext`, `session::current()`. The principal-overwrite contract is specified there and enforced here. | +| security.md | Threat model for `RemoteServer`: authorization coverage, the untrusted client principal, and what `register`/`deregister` do *not* check. | +| wire.md | `Envelope`, `encode`/`decode`, `makeOk`/`makeErr`/`makeRegister`/`makeDeregister`, and the `kind` discriminator the server switches on. | +| registry.md | `ModelRegistryFactory::create` (remote model construction, `BRIDGE_REGISTER_MODEL`), `ActionDispatcher::dispatch` (the remote execute call site), and the `Loggable` policy. | +| completion.md | `Completion>` returned by `execute`, the `CompletionState` the backends track for `cancelPending`, and `cbExec` callback delivery. | + +## Limitations + +- **Local and remote are not fully interchangeable.** The GUI-facing API is + identical, but the two paths construct models differently. `LocalBackend` runs + the caller-supplied **factory closure**, which can capture arbitrary + dependencies and need not be default-constructible. `RemoteServer` ignores the + factory and constructs via the `ModelRegistryFactory`, which requires the model + to be **default-constructible and macro-registered** (`BRIDGE_REGISTER_MODEL`). + A model that works locally can therefore fail at remote `register` with + `err "unknown model type: ..."` (or fail to compile the registration if it is + not default-constructible). Parity between the two paths is a property of the + model, not something the framework guarantees. +- **Remote model ids are guessable and register/deregister are unauthorized.** + `RemoteServer` assigns model ids from a sequential `std::atomic` + counter (`_nextId + 1`), so ids are trivially guessable. Only `execute` runs + through the `IAuthorizer`; `register` and `deregister` are **not** authorized + at all — any client that can reach the transport can create and destroy model + instances. See security.md. +- **`notifyBackendChanged` uses RTTI over every model under the registry lock.** + `LocalBackend::notifyBackendChanged` iterates the entire `_models` map holding + `_regMtx` and `dynamic_cast`s each holder to `IBackendChangedSink`. This + depends on RTTI being enabled and its cost scales with the number of live + models while the registry lock is held. (`SimulatedRemoteBackend`'s override is + a no-op — its models live in the server.) +- **Stale `SimulatedRemoteBackend` header doc.** The class comment in + `remote.hpp` still describes a "synchronous request-reply protocol" where "the + calling thread blocks until the reply arrives via `std::promise`". The code + does not match: `execute` is **asynchronous** (it calls `handle()` and returns + a `Completion`), and there is no `std::promise` anywhere in the class — only + the control operations (`registerModel`/`deregisterModel`) are synchronous, via + `handleInline`. Treat this spec, not that comment, as authoritative. \ No newline at end of file diff --git a/docs/spec/bridge.md b/docs/spec/bridge.md new file mode 100644 index 0000000..fff876d --- /dev/null +++ b/docs/spec/bridge.md @@ -0,0 +1,464 @@ +# Bridge types — `Bridge`, `BridgeHandler`, `ActionExecuteRegistry` + +`morph::bridge` owns the client-side dispatch path: a `Bridge` holds an active +`IBackend` and a set of registered model bindings; a `BridgeHandler` is +an RAII handle that registers one model instance and exposes typed +`execute()`, field-by-field `set()`, and subscription APIs. A +`ActionExecuteRegistry` provides dynamic (string-keyed) dispatch for call sites +that only know action names at runtime. + +## Contents + +- [Architecture overview](#architecture-overview) +- [`HandlerBinding`](#handlerbinding) +- [`Bridge`](#bridge) +- [`BridgeHandler`](#bridgehandlermodel) +- [`ActionExecuteRegistry`](#actionexecuteregistry) +- [`BRIDGE_REGISTER_ACTION` and `registerActionExecutorOnce`](#bridge_register_action-and-registeractionexecutoronce) +- [`MemberPointerTraits`](#memberpointertraits) +- [Subscription semantics](#subscription-semantics) +- [Thread safety](#thread-safety) +- [Lifetime & ownership](#lifetime--ownership) +- [API reference](#api-reference) +- [Design decisions](#design-decisions) +- [Limitations](#limitations) +- [Cross-references](#cross-references) + +## Architecture overview + +``` +GUI / call site + │ + ▼ BridgeHandler::execute(action) + │ + ▼ Bridge::executeVia(binding, action, cbExec) + │ + ▼ IBackend::execute(ModelId, ActionCall, cbExec) + │ + ┌─────┴─────┐ + │ Local │ Remote (SimulatedRemoteBackend, QtWebSocketBackend) + └───────────┘ +``` + +A `Bridge` owns one active backend at a time and tracks all `HandlerBinding` +instances. On `switchBackend()` it re-registers every live binding on the new +backend. `BridgeHandler` wraps one binding with an RAII lifecycle: it +registers on construction, deregisters on destruction. + +`BridgeHandler` exposes two execute paths: the typed `execute(action)` +(compile-time dispatch) and `executeJson(actionType, bodyJson)` (runtime +dispatch via `ActionExecuteRegistry`). + +For GUI-led workflows, `subscribe(cb)` registers a result callback, +`set<&Action::field>(value)` fills one field of an in-progress draft, and +`reset()` discards the draft. When all required fields are filled, the +handler automatically fires the action. + +## `HandlerBinding` + +Declared as `morph::bridge::detail::HandlerBinding` (internal linkage record, +not part of the public namespace). + +```cpp +struct HandlerBinding { + std::string typeId; + std::function()> modelFactory; + std::string contextKey; + std::atomic currentId{0}; +}; +``` + +One record per registered model instance. `typeId` is the string +`ModelTraits::typeId()`. `modelFactory` creates a fresh +`IModelHolder` — used by `switchBackend()` to re-register on the new +backend. `contextKey` is an optional stable identity (e.g. account id) that +travels in the `register` wire envelope for remote backends. `currentId` is +the `ModelId` value the active backend assigned; 0 = unbound. + +## `Bridge` + +Central dispatcher. Non-copyable, non-movable. Thread-safe. + +**Construction** takes ownership of an `IBackend` and installs a reconnect +handler so backends with recoverable transports (e.g. `QtWebSocketBackend`) +re-register all live bindings on reconnection. + +**`registerHandler()`** creates a `HandlerBinding` with the default +`ModelFactory::create()` factory and registers it on the active +backend. Returns the `shared_ptr`. An overload accepts a +pre-built binding (for dependency injection, custom `contextKey`, or custom +factory captures). + +**`executeVia(binding, action, cbExec)`** dispatches one +action. Uses a lock-free snapshot of the backend pointer and the binding's +`currentId` so it does not block `switchBackend()`. If `currentId` is 0, +completes immediately with `"handler not bound"`. Constructs an `ActionCall` +with serialization/deserialization lambdas and a `localOp` that calls +`Model::execute(*action)` and optionally records a journal `LogEntry` for +loggable actions. The typed result is unwrapped from `std::shared_ptr` +into the final `Completion`. + +**`switchBackend(newBackend)`** replaces the active backend atomically: the +switch either fully succeeds or leaves everything exactly as it was. It runs +in two phases under `_mtx`: + +- **Phase 1 — stage, do not mutate.** Every live binding is registered on the + new backend and the resulting `(binding, newId)` pairs are collected into a + staging list. No `currentId` is touched and the old backend is still active, + so concurrent `executeVia()` calls continue to hit the old backend. If any + `registerModelWithContext` throws (a plausible remote/transport failure), + the already-staged registrations are rolled back by calling + `deregisterModel` on the new backend for each staged id (rollback-deregister + failures are logged, not propagated), and the original exception is + rethrown. On this path the old backend and every `currentId` are untouched — + the call is a no-op. +- **Phase 2 — commit.** Only reached when *all* registrations succeeded. The + staged ids are published into each `binding->currentId`, the handler list is + replaced with the surviving-bindings list, the backend pointer is swapped via + `exchangeBackend`, and `notifyBackendChanged()` fires. + +After the swap the reconnect handler is re-installed on the new backend and +cleared on the old one, and the old backend's pending completions are cancelled +with `BackendChangedError` — outside `_mtx`, because `cancelPending` delivers +callbacks through user executors and the bridge never runs user code while +holding its mutex. Lock ordering: `_mtx` is acquired before the backend's +internal mutex. `onBackendChanged()` implementations must not call +`registerHandler()` or `deregisterHandler()` (deadlock). + +**`deregisterHandler(binding)`** calls `deregisterModel` on the active backend +(only if the binding still has a non-zero `currentId`), then resets +`binding->currentId` to the `0 = unbound` sentinel and removes the binding from +tracking. Resetting to `0` means a late or concurrent `executeVia()` on a +deregistered binding fails fast on the "handler not bound" guard rather than +sending a now-destroyed `ModelId` to the backend. + +**`setDefaultSession(session)`** / **`defaultSession()`** installs a default +`morph::session::Context` that is attached to every `executeVia()` call. +Thread-safe, separate mutex from `_mtx`. + +**Destructor** cancels every pending completion on the active backend with +`BridgeDestroyedError`. In-flight replies that arrive after destruction are +no-ops (`CompletionState::setValue`/`setException` is idempotent). + +**`liveness()`** (private, exposed only to `BridgeHandler` via friendship) +returns a `std::weak_ptr` observing the bridge's `_liveness` +member — a `shared_ptr` that is created with the bridge and +destroyed with it. Each `BridgeHandler` captures this weak token at +construction and consults it in its destructor so that destroying the `Bridge` +before its handlers is a safe no-op rather than a use-after-free. See +[Lifetime & ownership](#lifetime--ownership). + +## `BridgeHandler` + +RAII handle. Registers a `HandlerBinding` on construction, deregisters on +destruction. Non-copyable. + +**Construction** takes a `Bridge&` and a GUI executor. Optionally accepts a +pre-built `HandlerBinding` (for dependency injection). It captures the bridge's +`liveness()` weak token into `_bridgeAlive`, holds a strong `Bridge&`, and +stores a `SubscriberState` shared pointer that supports the field-by-field and +subscription APIs. + +**Destruction** deregisters the binding via `Bridge::deregisterHandler` — but +only if `_bridgeAlive.lock()` still succeeds. If the `Bridge` was already +destroyed the token has expired, so the destructor skips deregistration +instead of dereferencing a dangling `Bridge&`. Destroying the bridge before +its handlers is still discouraged (see [Lifetime & ownership](#lifetime--ownership)), +but it is now defined behaviour, not a use-after-free. + +**`execute(action)`** delegates to `Bridge::executeVia` with the +handler's binding and GUI executor. Default session is attached +automatically by the bridge. + +**`executeJson(actionType, bodyJson)`** type-erased variant. Looks up the +executor in `ActionExecuteRegistry::instance()` and dispatches. The +registry's executor deserializes the JSON body via +`ActionTraits::fromJson`, calls `execute`, and serializes +the result back to JSON. Throws `std::runtime_error` if the action was never +registered. + +**`subscribe(cb)`** / **`subscribe(cb, errCb)`** stores a +result (and optional error) callback keyed by +`ActionTraits::typeId()`. Callbacks execute on the GUI executor. + +**`unsubscribe()`** clears both result and error callbacks for the +action type. + +**`set<&Action::field>(value)`** updates one field of the in-progress draft. +Uses `MemberPointerTraits` to recover the action and field types from the +pointer-to-member. After setting the value, checks +`ActionValidator::ready(snapshot)`. If all required fields are +present, fires the action via `Bridge::executeVia` and delivers the result +to the registered `sink` callback. If a flight is already in progress, marks +`pending = true` and refires when the current flight completes +(debounce-like coalescence). On failure, the registered `errSink` is invoked; +if none is registered, the error is logged via `morph::log::logError` +(tagged `[subscription:]`) rather than silently dropped. + +**`reset()`** discards the in-progress draft. + +**`guiExecutor()`** returns the executor passed at construction. + +### SubscriberState + +```cpp +struct SubscriberState { + std::mutex mtx; + Bridge* bridge; + std::shared_ptr binding; + IExecutor* guiExec; + std::unordered_map entries; +}; +``` + +Keys are `std::string_view` pointing to `ActionTraits::typeId()` string +literals (static storage duration — keys never dangle). Each `SubscriberEntry` +holds a `draft` (`std::any` of the action struct), `sink`, `errSink`, and +`running`/`pending` flags for flight tracking. + +## `ActionExecuteRegistry` + +Process-level singleton (`instance()`). Maps `(modelTypeId, actionTypeId)` +pairs to `Executor` values (`std::function(void*, +string_view)>`). Populated by +`registerActionExecutorOnce()`, which +`BRIDGE_REGISTER_ACTION` calls during static initialization. + +**`execute(modelId, actionId, handler, bodyJson)`** looks up the executor and +invokes it. The executor `static_cast`s the `void*` back to +`BridgeHandler*`, deserializes the JSON body, calls +`handler->execute()`, and serializes the result back to JSON. +Throws `std::runtime_error` if no executor is registered for the pair. + +**Requirement**: every translation unit calling `BRIDGE_REGISTER_ACTION` +must include `bridge.hpp` (directly or transitively), because +`registerActionExecutorOnce` is defined in this header and the static +initializer will fail to link otherwise. + +## `BRIDGE_REGISTER_ACTION` and `registerActionExecutorOnce` + +```cpp +// namespace morph::model::detail +template +inline bool registerActionExecutorOnce(std::string_view modelId, + std::string_view actionId) noexcept; +``` + +Lives in `morph::model::detail`. It is *forward-declared* (non-`inline`) in +`registry.hpp` and *defined* `inline` here in `bridge.hpp` (after +`ActionExecuteRegistry`), breaking the `registry.hpp` → `bridge.hpp` include +cycle. The body calls +`ActionExecuteRegistry::instance().registerAction()` and +returns `true`. + +The `BRIDGE_REGISTER_ACTION` macro is defined in `registry.hpp`. Its +expansion (a) specialises `morph::model::ActionTraits` with JSON +codecs, `typeId()`, and a `loggable` flag, and (b) emits two anonymous- +namespace `const bool` initializers that call +`morph::model::detail::registerActionOnce` (server-side dispatcher) and +`morph::model::detail::registerActionExecutorOnce` (this registry) at +static-init time. The `inline` keyword lets the definition in this header be +included and instantiated across many translation units without an ODR/link +violation; the registration itself runs from the macro's static initializer. + +## `MemberPointerTraits` + +Declared as `morph::bridge::detail::MemberPointerTraits`. + +```cpp +template struct MemberPointerTraits; + +template +struct MemberPointerTraits { + using ClassType = A; + using ValueType = V; +}; +``` + +Compile-time decomposition of a pointer-to-data-member type. Used by +`BridgeHandler::set` to recover both the action type (`A`) +and the field type (`V`) from a single non-type template parameter, so +callers write `handler.set<&MyAction::c>(7.0)` with no redundant type +arguments. + +## Subscription semantics + +The fielded/reactive surface (`subscribe`, `set`, `unsubscribe`, `reset`) is +built on the per-handler `SubscriberState`. Exactly one `SubscriberEntry` +exists per action type, keyed by `ActionTraits::typeId()`. The rules: + +- **One subscriber per `(handler, action type)`.** `subscribe(cb)` (or the + two-argument `subscribe(cb, errCb)`) *replaces* any callback previously + registered for `A` on this handler — there is no fan-out. The two-argument + overload additionally stores the error sink; the one-argument overload leaves + the existing `errSink` untouched. +- **Fire without a subscriber.** A `set<>`-triggered fire runs whether or not a + subscriber is installed. If no `sink` is registered when the result arrives, + the result is simply dropped (the entry exists because `set<>` created the + draft, but its `sink` is empty). Errors are different: with no `errSink` the + error is logged via `morph::log::logError` tagged `[subscription:]`, + never silently dropped. +- **Default validator fires on the first `set<>`.** `ActionValidator::ready` + returns `true` for any action without a `BRIDGE_REGISTER_VALIDATOR` + specialisation, so the very first `set<>` puts the (single-field-populated) + draft into a ready state and dispatches immediately. Actions that need + several fields before firing must specialise the validator. +- **Every ready `set<>` re-fires.** Each `set<>` landing a `ready()==true` + snapshot dispatches the action again — live recomputation. Rapid patches + coalesce: while a flight is running, further `set<>` calls set + `pending=true`, and exactly one re-fire with the latest snapshot is issued + when the in-flight completion resolves (`consumeFlight`). +- **Draft lifetime.** A draft is created lazily on the first `set<>` for its + action type and persists across fires, across `unsubscribe()`, and across + `Bridge::switchBackend` (the draft lives in the handler's `SubscriberState`, + not the backend). It is destroyed only when the handler is destroyed or when + `reset()` is called. `unsubscribe()` clears both callbacks but leaves + the draft intact. + +## Thread safety + +`Bridge` is fully thread-safe (see the `Bridge` section: separate +`_backendMtx`, `_mtx`, and `_sessionMtx`, with lock-free backend snapshots in +`executeVia`). + +`BridgeHandler`'s individual mutating operations — `set`, `subscribe`, +`unsubscribe`, `reset` — are each internally safe: they take the +`SubscriberState::mtx` while touching the entry map. Result and error callbacks +never run under that mutex; they are marshalled to the `guiExec` executor +passed at construction, and `tryFireImpl` captures a `weak_ptr` +so a callback that fires after the handler is destroyed is a no-op. The +intended usage is nonetheless **single-GUI-thread affinity**: a handler and its +subscriptions belong to one GUI thread, and interleaving concurrent `set<>` +storms from multiple threads onto the same handler, while memory-safe, has no +defined ordering guarantee beyond the per-operation locking. + +## Lifetime & ownership + +**Binding ownership (shared/weak split).** `BridgeHandler` owns the +`shared_ptr`; the `Bridge` holds only a +`weak_ptr` in its `_handlers` list. The binding therefore lives +exactly as long as its handler. The bridge can enumerate live bindings (for +`switchBackend` and reconnect re-registration) and skip dead ones via +`weak.lock()`, but it never keeps a handler alive. + +**Bridge-vs-handler teardown order.** The bridge holds a +`shared_ptr _liveness`; every handler captures a matching +`weak_ptr` (`_bridgeAlive`) at construction. This makes *teardown* +order-independent: + +- Handler destroyed first (the normal case): its destructor deregisters the + binding from the still-live bridge. +- Bridge destroyed first: the handler's `_bridgeAlive` token has expired, so + its destructor skips deregistration — a safe no-op instead of a use-after-free. + +**The lifetime rule.** The liveness token makes only *destruction* safe in +either order. It does **not** make a `Bridge` optional for live handlers: any +`execute()`, `executeJson()`, or `set<>`-triggered fire dereferences the +`Bridge&` and must run while the bridge is alive. In other words, the bridge +must still outlive all normal use of its handlers; only mis-ordered +*destruction* is now defined behaviour. (This corrects an earlier claim that +"weak references let `BridgeHandler` outlive `Bridge`" — they do not; they only +make teardown order-independent.) + +## API reference + +### `ActionExecuteRegistry` + +| Member | Signature | Notes | +|---|---|---| +| `instance` | `static ActionExecuteRegistry& instance()` | Process-level singleton. | +| `registerAction` | `template void registerAction(string_view modelId, string_view actionId)` | Registers an executor that deserializes JSON → `ActionTraits::fromJson`, calls `BridgeHandler::execute<>`, serializes result back. Defined out-of-line after `BridgeHandler`. | +| `execute` | `Completion execute(string_view modelId, string_view actionId, void* handler, string_view bodyJson) const` | Lookup + invoke. Throws `runtime_error` on unknown pair. | + +### `Bridge` + +| Member | Signature | Notes | +|---|---|---| +| ctor | `explicit Bridge(unique_ptr)` | Installs reconnect handler on the backend. | +| dtor | `~Bridge()` | Cancels all pending completions with `BridgeDestroyedError`. | +| `registerHandler` | `shared_ptr registerHandler()` | Default factory. | +| `registerHandler(binding)` | `void registerHandler(const shared_ptr&)` | Pre-built binding. | +| `switchBackend` | `void switchBackend(unique_ptr)` | Atomic: stages all re-registrations on the new backend, commits (publishes new ids + swaps) only if all succeed, else rolls back and rethrows leaving old backend + `currentId`s intact. Cancels old backend's pending ops with `BackendChangedError`. | +| `deregisterHandler` | `void deregisterHandler(const shared_ptr&)` | Deregisters from active backend (if bound), resets `currentId` to 0, removes from tracking. | +| `executeVia` | `Completion executeVia(const shared_ptr&, Action, IExecutor*)` | Lock-free dispatch. Attaches default session. Records journal for loggable actions. | +| `setDefaultSession` | `void setDefaultSession(session::Context)` | Installs default session context. | +| `defaultSession` | `session::Context defaultSession() const` | Returns snapshot of default session. | + +### `BridgeHandler` + +| Member | Signature | Notes | +|---|---|---| +| ctor (default) | `BridgeHandler(Bridge&, IExecutor*)` | Registers via `Bridge::registerHandler()`. | +| ctor (custom binding) | `BridgeHandler(Bridge&, IExecutor*, shared_ptr)` | Registers pre-built binding. | +| dtor | `~BridgeHandler()` | Deregisters via `Bridge::deregisterHandler`, but only if the bridge's liveness token is still alive; a no-op if the `Bridge` was already destroyed. | +| `execute` | `Completion execute(Action)` | Typed dispatch through the bridge. | +| `executeJson` | `Completion executeJson(string_view actionType, string_view bodyJson)` | Type-erased dispatch through `ActionExecuteRegistry`. | +| `subscribe(cb)` | `void subscribe(function)` | Result callback. | +| `subscribe(cb, errCb)` | `void subscribe(function, function)` | Result + error callbacks. | +| `unsubscribe` | `void unsubscribe()` | Clears both callbacks. | +| `set` | `void set(MemberPointerTraits::ValueType value)` | Field-by-field update; auto-fires when ready. | +| `reset` | `void reset()` | Discards in-progress draft. | +| `guiExecutor` | `IExecutor* guiExecutor() const noexcept` | Returns the callback executor. | +| `binding` | `const shared_ptr& binding() const` | Returns the underlying binding. | + +### `HandlerBinding` + +| Field | Type | Notes | +|---|---|---| +| `typeId` | `string` | `ModelTraits::typeId()`. | +| `modelFactory` | `function()>` | Factory for re-registration on backend switch. | +| `contextKey` | `string` | Stable identity for remote backends (optional, empty by default). | +| `currentId` | `atomic` | Backend-assigned model id; 0 = unbound. | + +## Design decisions + +| Decision | Choice | Why | +|---|---|---| +| Binding storage | **`vector>`** | `Bridge` does not own the bindings — `BridgeHandler` holds the `shared_ptr`. Weak references let `switchBackend` and the reconnect handler skip dead bindings without keeping handlers alive. (Handler *teardown* after the bridge is made safe separately, by the `_liveness` token — not by this weak storage.) | +| Teardown order | **`shared_ptr _liveness` + per-handler `weak_ptr`** | Makes bridge-vs-handler destruction order-independent: a handler outliving its bridge skips deregistration instead of dereferencing a dangling `Bridge&`. Normal `execute`/`set` still require the bridge to outlive its handlers. | +| Backend pointer | **Lock-free snapshot under `_backendMtx`** | `executeVia()` reads the backend through a `loadBackend()` helper that copies the `shared_ptr` under a dedicated mutex, so it never blocks `switchBackend()`. | +| Session storage | **Separate `_sessionMtx` from `_mtx`** | Session access is a hot path (every `executeVia` reads it). A separate mutex avoids contention with handler registration/switchBackend. | +| Reconnect handler | **Weak‑backend guard + stale check** | The lambda captures a `weak_ptr`. On invocation it checks `pinned == loadBackend()` — if a switch occurred since the handler was installed, the reconnect is ignored. | +| Fielded actions | **`SubscriberState` shared across `BridgeHandler` copy-unsafe design** | The handler is non-copyable; the subscriber state is `shared_ptr` so `tryFireImpl` can capture a `weak_ptr` and survive handler destruction. Flight tracking (`running`/`pending`) coalesces rapid `set` calls. | +| Action readiness | **`ActionValidator::ready(snapshot)`** | Framework-agnostic validation — each action struct defines its own required-field semantics. The bridge never interprets action fields. | +| Subscription keys | **`string_view` into static storage** | `ActionTraits::typeId()` returns `constexpr` string literals with static duration. The `unordered_map` holds non-owning keys; no allocation, no lifetime issues. | +| `executeJson` | **Separate registry, not a vtable** | The action type is unknown at the call site. A flat `unordered_map<(modelId, actionId), Executor>` lets any translation unit register its actions without central registration or RTTI. | +| `registerActionExecutorOnce` | **`inline` definition in header** | The function is forward-declared in `registry.hpp` (`morph::model::detail`) but defined `inline` in `bridge.hpp`, after `ActionExecuteRegistry`. `inline` lets that definition be instantiated in every TU that transitively includes `bridge.hpp` without an ODR/link violation. The registration runs from the anonymous-namespace initializer the macro emits. Because the definition lives only in `bridge.hpp`, any TU expanding `BRIDGE_REGISTER_ACTION` must include it (directly or transitively) or the link fails with an unresolved symbol. | + +## Limitations + +- **Backend switch interrupts in-flight fielded edits.** When `switchBackend` + commits, it cancels the old backend's pending completions with + `BackendChangedError`. A fielded subscriber whose `set<>`-triggered flight is + still in the air will therefore receive `BackendChangedError` on its + `errSink` mid-edit (or see it logged if no `errSink` is registered). The + client-side draft survives the switch, so the next `set<>` re-fires cleanly + against the new backend, but the interrupted flight itself surfaces as an + error rather than being transparently retried. +- **`ActionExecuteRegistry::execute` trusts its `void* handler`.** The + type-erased entry point takes a `void*` that each registered executor + `static_cast`s back to `BridgeHandler*` for the model type it was + registered under. Passing a handler whose model type does not match the + `modelId` is undefined behaviour — there is no runtime type check. In + practice `BridgeHandler::executeJson` always passes `this` with a matching + `ModelTraits::typeId()`, so the invariant holds by construction; the + hazard only exists for callers that invoke the registry directly. + +## Cross-references + +- [`backend.md`](backend.md) — `IBackend`, `LocalBackend`, + `SimulatedRemoteBackend`, `registerModelWithContext`, `cancelPending`, + `BackendChangedError`/`BridgeDestroyedError`, reconnect handlers. +- [`session.md`](session.md) — `session::Context` attached to every + `executeVia` call via the default session. +- [`security.md`](security.md) — how the session principal drives + authorization on the execute path. +- [`wire.md`](wire.md) — the `register` envelope carrying `contextKey` and the + action call/result serialization used by remote backends. +- [`completion.md`](completion.md) — `Completion`/`CompletionState` + semantics, executor marshalling, and idempotent value/exception setting. +- [`registry.md`](registry.md) — `ModelTraits`, `ActionTraits`, + `ActionValidator`, `Loggable`, `BRIDGE_REGISTER_ACTION`, and the server-side + `ActionDispatcher` counterpart. +- [`concurrency_and_lifetimes.md`](concurrency_and_lifetimes.md) — the broader + mutex-ordering and object-lifetime rules this type participates in. \ No newline at end of file diff --git a/docs/spec/choice.md b/docs/spec/choice.md new file mode 100644 index 0000000..e6d80e8 --- /dev/null +++ b/docs/spec/choice.md @@ -0,0 +1,295 @@ +# The `Choice` type — design + +`morph::forms::Choice` is a form field whose options come from +executing another action at runtime. In the UI it renders as a combo box: the +client calls the named action, reads the result rows, and offers `labelField` as +display text while submitting `valueField` as the payload. + +The key design property: **the options metadata lives in the C++ type, never on +the wire**. The wire carries only the nullable underlying value (`T`), exactly +like `Quantity` and `Timestamp`. The schema bridges the gap — the declaration +surfaces `x-optionsAction` / `x-optionValue` / `x-optionLabel` so a client knows +which action to call and which result fields to use without hardcoding anything. + +## Contents + +- [FixedString — NTTP string](#fixedstring--an-nttp-compile-time-string) +- [FixedString notes](#fixedstring-notes) +- [Choice — structure](#choice--structure) +- [Empty state](#empty-state) +- [Wire and schema](#wire-and-schema) +- [Schema representation](#schema-representation) +- [API reference](#api-reference) +- [Design decisions](#design-decisions) +- [Usage example](#usage-example) +- [Author's obligations](#authors-obligations) +- [Failure modes](#failure-modes) +- [Limitations](#limitations) +- [Cross-references](#cross-references) + +## FixedString — an NTTP compile-time string + +`FixedString` is a structural type that can appear as a non-type template +parameter. It stores `N` characters (including the terminating null) in a +`std::array` and is constructed `consteval` from a string literal. + +| Member | Signature | Notes | +|---|---|---| +| `data` | `std::array data{}` | Null-terminated storage. | +| ctor | `consteval FixedString(const char (&literal)[N]) noexcept` | From a string literal of the same length. | +| `view()` | `[[nodiscard]] constexpr std::string_view view() const noexcept` | Excludes the null terminator — length is `N - 1`. | + +The `consteval` constructor guarantees that `FixedString` is only ever +initialised from a compile-time literal, so every template instantiation is +visible to the compiler at the point of use. + +## FixedString notes + +A few subtleties matter because `FixedString` is the vehicle that carries the +options metadata into the type system: + +- **`N` counts the terminating null.** The literal `"id"` is `const char[3]`, + so `FixedString<3>`; `view()` returns `{data.data(), N - 1}` — a two-char + view `"id"` that stops before the null. The null is stored but never part of + the string. +- **An empty literal yields a zero-length name.** `""` is `const char[1]`, so + `FixedString<1>` and `view()` is a length-0 `string_view`. Nothing rejects + this — an empty action name or field name compiles and simply produces an + empty `x-option*` annotation that no client can act on. +- **Type identity depends on structural NTTP equality.** `FixedString` is a + *structural type* (only public, non-mutable data members: the `std::array`), + so it is usable as a non-type template parameter and two `FixedString` + values compare member-wise. Two separate `"id"` literals therefore produce + the *same* `FixedString<3>` value, so `Choice` written + in two translation units is one and the same type. This is what makes + `Choice` type identity stable across the codebase and what lets `glz::meta` + and `isChoice` match on the instantiation. +- **`FixedString` is public only as an NTTP vehicle.** It exists so a string + literal can travel as a template parameter; it is not a general-purpose + string type and is not intended for direct use in application code. The + only supported way to produce one is a string literal in a `Choice` + template-argument position. + +## Choice — structure + +```cpp +template +struct Choice { + std::optional value; + // ... +}; +``` + +| Template parameter | Purpose | +|---|---| +| `T` | Underlying value type submitted on the wire (e.g. `std::int64_t` for ids, `std::string` for codes). | +| `OptionsAction` | Type id of the registered action whose result provides the options (executed with an empty body). | +| `ValueField` | Field of each result row submitted as the value. | +| `LabelField` | Field of each result row shown to the user. | + +The four template parameters — the type parameter `T` plus the three +`FixedString` NTTPs — make every `Choice` a distinct type +whose options source is known at compile time. The same action name can appear +in multiple `Choice` fields across different form types. + +## Empty state + +The payload is `std::optional`. A default-constructed `Choice` is empty — no +value has been selected. `hasValue()` queries the state; `operator*` provides +unchecked access (UB when empty, exactly like `std::optional`). + +Equality is total: `operator==` returns `true` when both are empty or both hold +equal values. + +## Wire and schema + +On the morph JSON wire a `Choice` is its nullable underlying value — the +options metadata never travels with payloads. The glaze `meta` specialisation +maps the type name to `"Choice"` and serialises through the `value` member +directly: + +```cpp +template +struct glz::meta> { + static constexpr auto value = &morph::forms::Choice::value; + static constexpr std::string_view name = "Choice"; +}; +``` + +In the generated JSON Schema (`morph::forms::schemaJson`) the property receives +`x-optionsAction`, `x-optionValue`, and `x-optionLabel` annotations so a client +knows which action to call and which result fields to use. A non-optional +`Choice` member is required by the `morph::forms` rules. + +## Schema representation + +The `glz::meta` specialisation sets `name = "Choice"` for *every* +instantiation, regardless of `T`, `OptionsAction`, `ValueField`, or +`LabelField`. glaze uses that name as the type's key in the schema's `$defs` +block and as the `$ref` target for each property. Two consequences follow, both +intentional: + +- **`$defs` collapses all `Choice<...>` to one entry.** `Choice` and `Choice` + both resolve to `$defs/Choice` and share a single `$ref`. glaze does not + suffix the name to keep them apart, so whichever it emits describes only the + common shape. +- **That shared shape is the bare nullable value.** Because `meta::value` + points at the `value` member, the `$defs/Choice` entry describes only + `std::optional` — a nullable scalar/string, carrying none of the options + metadata. Nothing that distinguishes one `Choice` field from another lives + in `$defs`. + +The collision is therefore benign: the parts that *do* differ between fields — +which action to call, which result fields to read — are emitted by +`mergeSchemaExtras` as **property-level** `x-optionsAction` / `x-optionValue` / +`x-optionLabel` annotations, one set per property, alongside `x-order` and the +derived `required` array. A renderer reads those from the property, not from +`$defs`, so it never depends on the shared `$defs/Choice` node to tell two +`Choice` fields apart. The one caveat, when `T` varies across `Choice` fields +in the same action, is that the single `$defs/Choice` payload type cannot be +correct for all of them; renderers that submit the raw nullable value observe +no problem, since the wire value is validated by the action, not by the schema. + +## API reference + +### `Choice` + +| Member | Signature | Notes | +|---|---|---| +| `value` | `std::optional value` | Public data member; the payload. | +| default ctor | `constexpr Choice() noexcept` | Empty state — `std::nullopt`. | +| value ctor | `constexpr Choice(T selected) noexcept(std::is_nothrow_move_constructible_v)` | Implicit; engages, moving from `selected`. | +| optional ctor | `constexpr Choice(std::optional payload) noexcept(std::is_nothrow_move_constructible_v)` | Implicit; adopts an optional payload as-is. | +| `optionsAction()` | `static constexpr std::string_view optionsAction() noexcept` | The action name from the type. | +| `valueField()` | `static constexpr std::string_view valueField() noexcept` | The result-row field submitted as the value. | +| `labelField()` | `static constexpr std::string_view labelField() noexcept` | The result-row field shown to the user. | +| `hasValue()` | `constexpr bool hasValue() const noexcept` | Engaged? No implicit `bool` conversion. | +| `operator*` | `constexpr const T& operator*() const noexcept` | Unchecked access (UB when empty). | +| `operator==` | `constexpr bool operator==(const Choice&) const noexcept` | Total; empty==empty is `true`. | + +### Trait + +| Symbol | Kind | Notes | +|---|---|---| +| `isChoice` | `constexpr bool` | `true` when `T` (cvref-stripped) is a `morph::forms::Choice`. | + +## Design decisions + +| Decision | Choice | Why | +|---|---|---| +| Options source | **NTTP `FixedString` naming an action** | The options action is known at compile time and baked into the type; the schema then tells the client which action to call without the client hardcoding any names. | +| Value / label fields | **NTTP defaults `"id"` / `"name"`** | Most list actions return rows with these conventional field names; override when the result schema differs. | +| Wire representation | **Nullable `T` only** | Options metadata never travels with payloads — it lives in the C++ type and the generated schema. Keeps the wire compact and stable. | +| Options metadata delivery | **Schema annotations (`x-optionsAction` etc.)** | Generated by `schemaJson` from the NTTP parameters so a client can discover the options source without hardcoding. | +| Blank state | **`std::nullopt` in the payload** | Same pattern as `Quantity` and `Timestamp`; a non-optional `Choice` member is required by the forms rules. | +| Glaze serialisation | **Through the `value` member directly** | The glaze `meta` specialisation maps `value` so the type serialises as its nullable payload, with the type name `"Choice"`. | +| Equality | **Total, through `std::optional`'s semantics** | Empty equals empty; engaged values compare by `T`'s equality. | + +## Usage example + +```cpp +#include +#include + +struct SampleInfo { std::int64_t id = 0; std::string name; }; +struct SampleList { std::vector samples; }; +struct ListSamples {}; // registered like any other action (a pure query) + +struct RecordMeasurement { + morph::forms::Choice sampleId; + // ... +}; +``` + +The generated schema surfaces `sampleId` as a `Choice` with `x-optionsAction = +"ListSamples"`, `x-optionValue = "id"`, `x-optionLabel = "name"`. The client +calls `ListSamples`, reads the `SampleList` result, and offers each +`SampleInfo::name` as a display option while submitting `SampleInfo::id` as the +value. On the wire the field is just a nullable integer — `null` or `42`. + +## Author's obligations + +Declaring a `Choice` places three obligations on the author that the type +system cannot check, because the strings are opaque NTTPs: + +- **The `OptionsAction` must be a registered action.** The name (`"ListSamples"` + above) has to resolve to an action the executor knows, and that action must + succeed **when invoked with an empty body** — the renderer calls it with no + arguments to populate the combo box. An action that requires input fields + cannot serve as an options source. +- **The result rows must expose the value and label fields.** Each row the + options action returns must have fields literally named by `ValueField` and + `LabelField` — `id` and `name` by default. The renderer reads those two + fields off every row: `valueField()` becomes the submitted payload, + `labelField()` the display text. +- **Those names are WIRE (JSON) field names, not C++ member names.** They must + match what the result row serialises as on the wire, not what the C++ member + is called. A `glz::meta` rename (or any glaze naming customisation) on the + result type changes the wire name, and `ValueField`/`LabelField` must track + that renamed wire name — not the original member identifier. Defaulting to + `"id"`/`"name"` works only when the result rows serialise with exactly those + keys. + +## Failure modes + +### Validation & staleness + +`Choice` participates in `morph::forms` required-field validation only through +`hasValue()`, which reports **engagement** — whether *some* value is selected — +and nothing more. That leaves gaps neither the client nor the server closes: + +- **A required `Choice` with an empty options list is permanently + unsubmittable.** If the `OptionsAction` returns zero rows, there is nothing to + select, so `hasValue()` can never become `true`, so `allRequiredEngaged` + never passes. The form cannot be submitted until the options action yields at + least one row — there is no "no valid options" escape hatch for a required + field. +- **`allRequiredEngaged` checks engagement, never membership.** It verifies the + payload is engaged; it never re-checks that the selected value still exists + among the *current* options. A value chosen when the list contained id `42` + stays "valid" to `allRequiredEngaged` even after `42` disappears from the + options action's result. A **stale id submits silently** — the draft looks + complete and goes out with a value that no current option backs. +- **Neither side validates option membership.** The schema's `x-option*` + annotations tell a client how to *fetch* options; they do not constrain the + submitted value to that set. The client renderer does not re-validate the + payload against a freshly fetched list, and the wire schema (a bare nullable + `T`) has no enumeration to check against on the server. Membership is an + assumption, not an enforced invariant, at both ends. + +## Limitations + +- **The action and field names are unchecked strings.** `OptionsAction`, + `ValueField`, and `LabelField` are resolved *at runtime* by the client and + executor. A typo in the action name, an action that is not registered, or a + value/label field that does not match the result row's wire keys all + **compile cleanly** and fail only later — when some client executes the + options action and finds nothing, or reads a field that is not there. There + is no compile-time link between a `Choice` and the action or result type it + names. Renaming a result field (e.g. via `glz::meta`) without updating the + `Choice` is the same silent failure. +- **The accessor surface is unchecked-only.** The type offers `hasValue()` and + an *unchecked* `operator*` (UB when empty, by design). There is no + `value_or`, no checked accessor, and no `operator bool`. Callers must guard + every dereference with `hasValue()` themselves; the type will not do it for + them, and "read the value if present, else a default" has to be written by + hand. + +## Cross-references + +- **[forms.md](forms.md)** — how a `Choice` member becomes *required* (the + `EmptyCapableField` concept plus the not-`std::optional`/not-`optionalFields` + rule), and where `mergeSchemaExtras` emits the `x-optionsAction` / + `x-optionValue` / `x-optionLabel` property annotations. +- **[quantity_type.md](quantity_type.md)** and **[datetime.md](datetime.md)** — + `Quantity` and `Timestamp` share the *one kind of empty* pattern: the blank + state lives inside the value as `std::optional`, `hasValue()` reports + engagement, and a non-optional member is required. `Choice` is the third + member of that family. +- **[security.md](security.md)** — the options metadata and any + membership expectation are enforced (if at all) only on the client; the + server sees a bare nullable value. This is the client-only-validation trust + boundary — never trust a submitted `Choice` value to be a current, valid + option without server-side re-checking. \ No newline at end of file diff --git a/docs/spec/completion.md b/docs/spec/completion.md new file mode 100644 index 0000000..e3d072c --- /dev/null +++ b/docs/spec/completion.md @@ -0,0 +1,274 @@ +# `Completion` — design + +`morph::async::Completion` is a move-only handle representing the eventual +result of an asynchronous operation. It delivers a single success value or an +error to callbacks registered via `then()` / `onError()`, posting the +callback to the executor supplied at construction (if any) so it runs on the +intended thread (e.g. the GUI thread). When the executor is `nullptr`, +callbacks are never posted — the handle is a write-only endpoint for the +producer. An undelivered *value* is dropped silently, but an undelivered +*error* is preserved: it surfaces through the destructor's orphan logger rather +than vanishing (see [Failure modes](#failure-modes)). + +## Contents + +- [Shared state — `CompletionState`](#shared-state--completionstatet) +- [Orphan detection](#orphan-detection) +- [Move-only handle — `Completion`](#move-only-handle--completiont) +- [Thread safety](#thread-safety) +- [Failure modes](#failure-modes) +- [Empty state](#empty-state) +- [API reference](#api-reference) +- [Design decisions](#design-decisions) +- [Limitations](#limitations) +- [Cross-references](#cross-references) + +## Shared state — `CompletionState` + +`detail::CompletionState` is the heap-allocated backing that both the +producer and the consumer reference through `std::shared_ptr`. All mutation is +guarded by `std::mutex mtx`. + +| Member | Type | Purpose | +|---|---|---| +| `mtx` | `std::mutex` | Guards all state and callback registration | +| `value` | `std::optional` | The success value, set once | +| `error` | `std::exception_ptr` | The error, set once via `setException` | +| `ready` | `bool` | `true` once either `value` or `error` is set | +| `onOk` | `std::function` | Stored success callback, moved out on dispatch | +| `onErr` | `std::function` | Stored error callback, moved out on dispatch | +| `onErrAttached` | `bool` | Suppresses orphan logging when `true`; set to `(cbExec != nullptr)` — never set on a null-executor state | +| `cbExec` | `::morph::exec::IExecutor*` | Executor for callback dispatch; may be `nullptr` | + +**Setting a value or exception.** `setValue(T)` and `setException(exception_ptr)` +are called by the producer. If the state is already `ready`, the call is a no-op +(only the first result wins). When a callback is already registered (via +`attachThen` / `attachOnError`), a fire-once closure is built under the lock and +posted to the executor outside the lock, so the callback never runs under the +mutex. The closure is posted only when `cbExec != nullptr`; with a null executor +it is built but never delivered. + +`setException` additionally sets `onErrAttached = (cbExec != nullptr)` — but +only along the branch where an `onErr` handler was already registered. It marks +the error handled (suppressing the orphan logger) **only when an executor exists +to actually deliver it**. With a null executor the handler is present but the +closure is never posted, so `onErrAttached` stays `false` and the abandoned +error still reaches the destructor's orphan logger rather than vanishing +silently. + +**Attaching callbacks.** `attachThen(handler)` and `attachOnError(handler)` are +called by `Completion::then()` / `onError()`. If the state is already ready with +the corresponding kind of result (value for `then`, exception for `onError`), a +fire-now closure is built and posted to the executor. Otherwise, if the state is +not yet ready, the handler is stored in `onOk` / `onErr` for later dispatch. If +the state is already ready with the *opposite* kind of result (e.g. `attachThen` +on an error state, or `attachOnError` on a value state), neither branch runs: no +closure is built and no handler is stored — the attach is a silent no-op. + +`attachOnError` sets `onErrAttached = (cbExec != nullptr)` unconditionally on +entry, before inspecting the state. So attaching an error handler on a +null-executor state does **not** suppress orphan logging: the handler will never +be posted, so the error is preserved for the destructor's orphan logger instead +of being both dropped and silenced. + +## Orphan detection + +If a `CompletionState` is destroyed while `ready == true`, `error` is set, and +`onErrAttached` is `false`, the destructor logs the unhandled exception via +`::morph::log::logError` with the prefix `[orphan]`. The exception is +re-thrown solely to extract a message: + +- If it derives from `std::exception`, the log reads + `[orphan] unhandled exception: `. +- Otherwise (a `catch (...)` branch), the log reads + `[orphan] unhandled unknown exception`. + +The `logError` call itself is wrapped in a `try { ... } catch (...) {}` (an +empty catch that swallows any exception `logError` might throw), so the +`noexcept` destructor never lets an exception escape. This prevents silent +loss of error information when a `Completion` goes out of scope without an +`onError` handler. + +`onErrAttached` is set by both `attachOnError` (unconditionally, on entry) and +`setException` (on the branch where an `onErr` handler was already registered), +but in both cases the value written is `(cbExec != nullptr)`, **not** an +unconditional `true`. The consequences: + +- **With an executor:** once an error handler has been attached (or an error has + been dispatched to an already-registered handler), `onErrAttached` is `true` + and the destructor treats the error as handled — no orphan is logged. +- **With a null executor:** the handler can never be posted, so `onErrAttached` + stays `false` and the destructor still logs the orphan. This closes a hole + where a null-executor error handler used to both drop the error (no executor + to post on) *and* silence the orphan logger, losing the error entirely. + +In short, orphan logging is suppressed precisely when the error has a real +delivery path; if the error can never be delivered, it is never silenced. + +## Move-only handle — `Completion` + +`Completion` wraps a `shared_ptr>`. Move-only — no copy +construction or copy assignment. The default constructor produces an empty +(no-op) completion with a null state pointer. + +The two-argument constructor takes a shared state and an executor pointer, +storing the executor in `state->cbExec`. All subsequent `then()` / `onError()` +calls forward to the state's `attachThen` / `attachOnError`, which use `cbExec` +for posting. + +`then()` and `onError()` return `*this` for chaining: + +```cpp +completion + .then([](int val) { /* ... */ }) + .onError([](std::exception_ptr e) { /* ... */ }); +``` + +## Thread safety + +- `then()` and `onError()` may be called from any thread — the mutex guards + `value`, `error`, `ready`, and the callback slots (`onOk` / `onErr`), so + registration and result-setting race safely. +- Callbacks are never invoked directly from the producing thread. They are + posted to `cbExec` via `IExecutor::post()` and run on the executor's thread. +- If `cbExec` is `nullptr`, no callback is posted (the fire-now/fire-once + closure is built but never delivered, and stored callbacks are never + invoked). See [Failure modes](#failure-modes) for what happens to an + abandoned error in this case. + +**`cbExec` is not mutex-guarded.** The `mtx` protects `value` / `error` / +`ready` / `onOk` / `onErr`, but `cbExec` is read outside the lock (after the +scoped lock is released) in `setValue`, `setException`, `attachThen`, and +`attachOnError`. This is safe only because of a happens-before requirement, not +a lock: the `Completion` handle writes `cbExec` in its constructor, and the +state must be fully constructed (with its executor assigned) **before** it is +published to any producer or consumer thread. Once published, `cbExec` is never +reassigned. There is no atomic and no lock around it — the ordering guarantee is +structural (construct-then-share), not enforced at runtime. + +## Failure modes + +These are the sharp edges of the single-shot, single-consumer design. None of +them raise or throw — they are silent by construction. + +- **Last-writer-wins callback slots.** Each state has exactly one `onOk` slot + and one `onErr` slot. A second `then()` overwrites the first success handler + (`onOk = std::move(handler)`); a second `onError()` overwrites the first error + handler. Only one handler per outcome survives, and it is the most recent one + registered *while the state was not yet ready*. There is no fan-out to + multiple consumers. + +- **Mismatched attach on a ready state is a silent no-op.** `then()` on a state + that is already `ready` with an *error* does nothing — no closure, no stored + handler, no error surfaced to the `then` handler. Symmetrically, `onError()` + on a state that is already `ready` with a *value* does nothing. Only an + attach that matches the settled outcome (or precedes readiness) has any + effect. + +- **Null-executor error drop, but no silencing.** With `cbExec == nullptr`, an + attached or pending error handler is never delivered — there is no executor to + post it on. Crucially, `onErrAttached` is left `false` in that case (it is set + to `(cbExec != nullptr)`), so the abandoned error still reaches the + destructor's orphan logger. The error is *undelivered* but never *lost*: it + surfaces as an `[orphan]` log line instead. (A null-executor **value** is + simply dropped with no diagnostic — only errors have orphan logging.) + +- **Overwriting a delivered handler has no effect.** Once a handler has fired + (its slot was moved out on dispatch), re-registering is governed by the rules + above against the now-`ready` state — i.e. a matching-outcome re-attach fires + again with the settled result, a mismatched one is a no-op. + +## Empty state + +A default-constructed `Completion` has a null `_state` pointer. `then()` and +`onError()` are no-ops (they check for `nullptr` and return `*this`). The +`state()` accessor returns `nullptr`. This is used for placeholder completions +that will never signal. + +## API reference + +### `Completion` (namespace `morph::async`) + +| Member | Signature | Notes | +|---|---|---| +| default ctor | `Completion() = default` | Empty, no-op completion (null state). | +| value ctor | `Completion(shared_ptr>, IExecutor*)` | Backed by user-supplied state; executor may be `nullptr`. | +| move ctor | `Completion(Completion&&) noexcept = default` | Transfers state ownership. | +| move assign | `Completion& operator=(Completion&&) noexcept = default` | Transfers state ownership. | +| copy ctor | `Completion(Completion const&) = delete` | Move-only handle. | +| copy assign | `Completion& operator=(Completion const&) = delete` | Move-only handle. | +| `then(handler)` | `Completion& then(std::function)` | Registers success callback; returns `*this` for chaining. | +| `onError(handler)` | `Completion& onError(std::function)` | Registers error callback; returns `*this` for chaining. | +| `state()` | `shared_ptr> state() const` | Returns the underlying shared state (advanced / internal use). | + +### `CompletionState` (namespace `morph::async::detail`) + +| Member | Signature | Notes | +|---|---|---| +| `setValue(T)` | `void setValue(T)` | Producer-side; no-op if already ready. Posts callback if one was registered. | +| `setException(exception_ptr)` | `void setException(std::exception_ptr const&)` | Producer-side; no-op if already ready. Posts callback if one was registered. | +| `attachThen(function)` | `void attachThen(std::function)` | Consumer-side; fires immediately if ready with value, stores otherwise. | +| `attachOnError(function)` | `void attachOnError(std::function)` | Consumer-side; fires immediately if ready with error, stores if not yet ready, no-op if ready with a value. Sets `onErrAttached = (cbExec != nullptr)`, so orphan logging is suppressed only when an executor exists to deliver on. | +| destructor | `~CompletionState()` | Orphan-detection: logs unhandled exceptions when destroyed with an error and no `onErr` attached. | + +## Design decisions + +| Decision | Choice | Why | +|---|---|---| +| Callback dispatch | **Posted to `IExecutor`, never direct** | Ensures callbacks run on the intended thread (e.g. GUI/main thread) regardless of which thread completes the operation. | +| Mutex scope | **Lock held only during state access, not during callback invocation** | Callback closures are built under the lock but invoked outside it, preventing callback re-entrancy into the mutex and avoiding deadlock. | +| Orphan detection | **Destructor logs through `logError`** | Prevents silent loss of error information when a `Completion` is destroyed without an `onError` handler. The exception is re-thrown just to extract a message (`what()` for a `std::exception`, a generic string otherwise), which is logged; the `logError` call is itself wrapped in an empty `catch (...)` so the `noexcept` destructor never lets an exception escape. | +| No executor callback | **`cbExec == nullptr` disables posting** | A `Completion` without an executor is a write-only endpoint — the producer can set a value or error, but stored callbacks are never invoked. This is by design for internal patterns where the consumer never attaches. An abandoned *error* is not silenced, though: `onErrAttached` tracks `(cbExec != nullptr)`, so a null-executor error still reaches the destructor's orphan logger. | +| First-result-wins | **`setValue`/`setException` are no-ops after `ready`** | An asynchronous operation should complete exactly once; subsequent calls are silently ignored. | +| Move-only handle | **`Completion` is move-only, `CompletionState` is shared via `shared_ptr`** | The handle is owned by one consumer at a time; the shared state is owned jointly by the producer and any consumer that has moved the handle. | +| Empty completion | **Null state pointer makes `then`/`onError` no-ops** | Default-constructed `Completion` is a safe placeholder that never signals. | + +## Limitations + +`Completion` is deliberately a **leaf callback primitive**, not a general +future/promise or a monadic async type. Its scope is narrow by design: + +- **No transformation, no chaining.** `then()` returns `*this` (the same + `Completion&`), purely so a `then().onError()` pair reads fluently. It does + **not** return a new `Completion` for a transformed result — there is no + `T → U` mapping and no way to chain one asynchronous step onto another. To + sequence work, the consumer must start a fresh operation from inside the + handler. +- **No `co_await`.** `Completion` is not an awaitable; it has no coroutine + promise/awaiter machinery. Consumption is callback-only. +- **No cancellation.** There is no handle to cancel an outstanding operation; + once started, it runs to completion (or is abandoned). +- **Single consumer, one handler per outcome.** The handle is move-only and each + state has exactly one `onOk` and one `onErr` slot. There is no multicast / + fan-out; a later registration overwrites an earlier one (see + [Failure modes](#failure-modes)). +- **No synchronous blocking.** There is no `wait()` or `get()`. + +**Orphan logging fires from `~CompletionState`, not from handle destruction.** +The orphan check lives in `CompletionState::~CompletionState`, which runs when +the *last* `shared_ptr` to the state drops — jointly held by the producer and +any consumer that moved the handle. Destroying a `Completion` handle does not +by itself trigger orphan logging if the producer still holds a reference to the +state; the log is emitted only when the state itself is finally destroyed with a +`ready` error and `onErrAttached == false`. + +## Out of scope + +- Cancellation — there is no mechanism to cancel an outstanding operation. +- Multiple values — `Completion` is a single-result primitive. +- Synchronous blocking — there is no `wait()` or `get()`; the API is + callback-only. +- Transformation / composition — see [Limitations](#limitations). + +## Cross-references + +- [`executor.md`](executor.md) — `IExecutor` and its implementations; `cbExec` + is the executor on which every callback is posted. +- [`logger.md`](logger.md) — `morph::log::logError`, the error-handling sink + used by orphan detection when an error is abandoned. (There is no dedicated + `error_handling.md` spec; the orphan-logging contract lives in this file and + the logging sink in `logger.md`.) +- [`bridge.md`](bridge.md) — `BridgeHandler` produces `Completion` from + `execute()` and posts callbacks on the GUI executor. +- [`backend.md`](backend.md) — backends resolve the pending `Completion` when a + response arrives. \ No newline at end of file diff --git a/docs/spec/concurrency_and_lifetimes.md b/docs/spec/concurrency_and_lifetimes.md new file mode 100644 index 0000000..3e816ca --- /dev/null +++ b/docs/spec/concurrency_and_lifetimes.md @@ -0,0 +1,357 @@ +# Concurrency & lifetimes + +Cross-cutting spec for morph's threading model and its object-ownership / +destruction-ordering rules. The per-type specs each describe their own local +guarantees; this document collects the framework-wide invariants — *which code +runs on which thread* and *who must outlive whom* — in one place, because the +subtle footguns live in the seams **between** subsystems, not inside any one of +them. + +Read this before wiring up a `Bridge`, a `RemoteServer`, or a +`ThreadPoolExecutor`/`StrandExecutor` pair, and before changing any teardown +sequence. + +## Contents + +- [The one rule: everything goes through `IExecutor::post`](#the-one-rule-everything-goes-through-iexecutorpost) +- [Thread roles — what runs where](#thread-roles--what-runs-where) +- [The strand model — one strand per `ModelId`](#the-strand-model--one-strand-per-modelid) +- [Completion callback marshalling](#completion-callback-marshalling) +- [Destruction ordering — who must outlive whom](#destruction-ordering--who-must-outlive-whom) +- [Synchronisation specifics per subsystem](#synchronisation-specifics-per-subsystem) +- [`Completion` / `CompletionState` thread-safety](#completion--completionstate-thread-safety) +- [Quick cheat-sheet](#quick-cheat-sheet) +- [Cross-references](#cross-references) + +## The one rule: everything goes through `IExecutor::post` + +morph has no ad-hoc threads scattered through the dispatch path. All asynchronous +work is scheduled by calling `IExecutor::post(std::function)` +(`executor.hpp`). The interface says nothing about *where* the task runs — that +is the concrete executor's job: + +| Executor | Thread(s) | Role | +|---|---|---| +| `ThreadPoolExecutor` | N fixed worker threads, FIFO MPMC queue | Runs model work (`Model::execute`) and remote message processing. | +| `MainThreadExecutor` | The thread that calls `runFor()` | Stand-in "GUI" thread in non-Qt tests; pumped manually. | +| `QtExecutor` | The Qt GUI thread | Real GUI executor; posts via `QMetaObject::invokeMethod(Qt::QueuedConnection)`. | +| `StrandExecutor` | *Borrows* a base `IExecutor` (usually the pool) | Serialises tasks per `ModelId` on top of the base executor. It owns no thread. | + +Because everything funnels through `post`, the concurrency model is fully +determined by *which executor a task is posted to*. Model code never blocks the +GUI, and the GUI thread never runs model work — the executors enforce the split. + +## Thread roles — what runs where + +| Work | Runs on | Scheduled by | +|---|---|---| +| `Model::execute(action)` (local mode) | Worker pool, inside a per-`ModelId` strand | `LocalBackend::execute` → `StrandExecutor::post` | +| `ActionDispatcher::dispatch` → `Model::execute` (remote mode) | `RemoteServer`'s worker pool, inside a per-`ModelId` strand | `RemoteServer::dispatchExecute` → `StrandExecutor::post` | +| Remote message decode / envelope handling | `RemoteServer`'s worker pool | `RemoteServer::handle` → `_pool.post` | +| `Completion::then` / `onError` callbacks | The `cbExec` executor supplied at dispatch (the GUI executor for `BridgeHandler`) | `CompletionState::setValue`/`setException` → `cbExec->post` | +| Subscription result / error sinks (`BridgeHandler::subscribe`) | The handler's `guiExec` | Same as `Completion` callbacks — they *are* completion callbacks | +| Connectivity probe + `onOffline`/`onOnline` callbacks | `NetworkMonitor`'s dedicated **probe thread** | `NetworkMonitor::run` | +| `ReconnectCoordinator::onOnline`/`onOffline` | The caller's thread (host posts it to a worker; **not** the probe thread) | Host wiring | +| `SyncWorker::run` (offline-queue replay) | The caller's thread; concurrent calls serialised | Host wiring / `ReconnectCoordinator::replay` | +| Backend reconnect handler (re-register bindings) | The backend's transport thread | `IBackend::setReconnectHandler` callback | +| Log sink invocation | Whatever thread called `log*()` | `morph::log::detail::log` | + +Key consequences: + +- **A model author writes single-threaded code.** For a given `ModelId`, the + strand guarantees `execute()` is never re-entered concurrently, so per-model + state needs no locking. Different models run in parallel across pool threads. +- **The GUI thread is never blocked by dispatch.** `executeVia` returns a + `Completion` immediately; the actual work runs on the pool and the result is + marshalled back to the GUI executor. +- **Probe callbacks must not block** (see below) — they run on the single probe + thread, and a blocking callback stalls all future probes. + +## The strand model — one strand per `ModelId` + +`StrandExecutor` (`strand.hpp`) sits on top of an arbitrary base `IExecutor` and +turns it into a set of per-key serial queues: + +- `post(ModelId key, task)` appends `task` to the strand for `key`. Tasks with + the same key run in FIFO order with **no overlap**; tasks with different keys + may run concurrently on different pool threads. This is what removes the need + for per-model mutexes. +- Each strand is a `shared_ptr` in a map guarded by `_mapMtx`. When a + strand's queue drains, the map entry is erased — but the "keep-running vs. + drain-and-erase" decision is made atomically under **both** `_mapMtx` and the + strand's own `mtx`. Doing it in two steps once opened a window where a + concurrent `post()` re-armed a strand between the unlock and the erase, + orphaning a live strand and letting two strands run the same model's tasks + concurrently (a data race). The combined lock closes that window. +- `_inFlight` counts strand lambdas currently dispatched to the base executor. + The destructor waits on `_cv` until `_inFlight == 0` before destroying the + map, so no pool thread can touch `_strands` after the executor is gone. + +`LocalBackend` owns one `StrandExecutor` over the worker pool; `RemoteServer` +owns another over its worker pool. Both post model work keyed by `ModelId`. + +## Completion callback marshalling + +`Completion` (`completion.hpp`) is the seam between the producing thread (a +pool/strand thread) and the consuming thread (the GUI executor). The invariant: +**`.then` / `.onError` callbacks are always posted to the `cbExec` executor +supplied at construction, never invoked directly on the producing thread.** So a +callback attached from the GUI runs back on the GUI thread even though the value +was produced on a pool thread. + +If `cbExec` is `nullptr`, callbacks are never delivered (silently dropped); a +failed-but-unhandled state logs an orphan error from `~CompletionState`. + +See [`Completion` / `CompletionState` thread-safety](#completion--completionstate-thread-safety) +for the internal locking and the one non-mutex-guarded field. + +## Destruction ordering — who must outlive whom + +This is the section to read before writing any teardown code. Several of these +rules encode recent fixes to real deadlocks and use-after-frees. + +| This… | must outlive / be destroyed after… | Consequence if violated | +|---|---|---| +| base `IExecutor` (e.g. `ThreadPoolExecutor`) | the `StrandExecutor` built on it | **Deadlock** in `~StrandExecutor` (see below) | +| `Bridge` | its `BridgeHandler`s (for normal `execute`/`set` calls) | Fine at teardown (order-independent, see below); a *call* on a handler whose bridge is gone is still UB | +| `RemoteServer` (heap, `make_shared`) | every `SimulatedRemoteBackend`/transport holding `RemoteServer&` | Dangling `RemoteServer&` → use-after-free | +| worker pool | the backend that posts to it (`LocalBackend`, `RemoteServer`) | Same deadlock/UAF family as the strand rule | +| `session::Context` passed to `ScopedContext` | the scope in which the model runs | Dangling thread-local `Context*` | + +### base `IExecutor` must outlive its `StrandExecutor` — and keep running + +This is the sharpest edge in the framework. `~StrandExecutor` **blocks** until +`_inFlight == 0`, i.e. until every lambda it dispatched to the base executor has +actually run. `~ThreadPoolExecutor` **drops** queued-but-unstarted tasks and +joins its workers. + +Therefore, if you destroy the pool **first**, the strand lambdas that were still +queued are dropped, `_inFlight` never reaches 0, and `~StrandExecutor` waits +forever → **deadlock**. The pool must be destroyed *after* every `StrandExecutor` +(and hence after `LocalBackend` / `RemoteServer`, which own the strands). + +Corollary: **no `post()` may race or follow `~StrandExecutor`.** Once the strand +executor's destructor has started, posting to it is undefined. Stop feeding a +backend before you tear it down. + +Correct teardown order (innermost-first): + +``` +handlers → Bridge → backend (LocalBackend / SimulatedRemoteBackend) + → RemoteServer (if remote) → worker pool (ThreadPoolExecutor) +``` + +Declared as members, list the pool **first** so it is destroyed **last**. + +### `Bridge` vs. `BridgeHandler` — teardown is now order-independent + +Normal operation still requires the `Bridge` to outlive its handlers: every +`execute` / `set` call dereferences `_bridge`. But **teardown order no longer +matters** (recent fix). Each `BridgeHandler` captures a `weak_ptr` +liveness token from the `Bridge` (`Bridge::liveness()`). In +`~BridgeHandler`, it locks the token first: + +- token still valid → the `Bridge` is alive → deregister normally. +- token expired → the `Bridge` is already gone → **no-op**, skipping the + deregistration that would otherwise dereference a dangling `Bridge&`. + +Previously, destroying the `Bridge` before its handlers was a use-after-free. +It is now defined behaviour (a silent no-op deregistration). Destroying the +bridge first is still discouraged, but it is no longer unsafe. + +### `RemoteServer` must be `make_shared` and outlive its transports + +`RemoteServer` derives from `enable_shared_from_this` and **must** be created via +`std::make_shared`. `handle()` captures `shared_from_this()` into the pool task, +so the server object survives until every dispatched message completes, even if +the owning `shared_ptr` is dropped while work is in flight — this is why the +worker pool can safely outlive the server *reference*. + +`SimulatedRemoteBackend` (and any real transport) stores a bare `RemoteServer&`. +That reference must remain valid for the backend's whole life: the +`RemoteServer` (its owning `shared_ptr`) must outlive every backend/transport +that points at it. The `make_shared` requirement guarantees in-flight *tasks* +are safe; it does **not** rescue a dangling `RemoteServer&` held by a backend. + +Model-destruction-mid-flight is safe on both backends: the strand task captures +a `shared_ptr` copy of the `IModelHolder` (`holder = std::move(holder)` in the +`post` lambda), so a concurrent `deregisterModel` that erases the map entry +cannot free the model out from under a running action. + +## Synchronisation specifics per subsystem + +### `Bridge::switchBackend` — atomic, exception-safe, self-deadlock-prone + +`switchBackend` holds `Bridge::_mtx` for its **entire** duration. Within that +lock it uses a **stage-all-then-commit** protocol (recent fix): + +1. **Phase 1** — register every live binding on the new backend, staging + `(binding, newId)` pairs *without mutating any `currentId`*. If any + registration throws (a plausible remote/transport failure), it rolls back the + registrations already made and rethrows, leaving the old backend and every + `currentId` untouched. The switch is therefore **atomic**: it either fully + succeeds or is a complete no-op. +2. **Phase 2** — commit: publish the new ids, swap the backend pointer, and call + `notifyBackendChanged()` (still under `_mtx`). + +Cancellation of the outgoing backend's pending completions +(`cancelPending(BackendChangedError)`) happens **outside** `_mtx`, because it +delivers callbacks through the caller's GUI executor and the bridge must never +hold `_mtx` while user code runs. + +Two hard rules: + +- **Do not call `switchBackend`, `registerHandler`, or `deregisterHandler` from + inside `onBackendChanged()`.** `notifyBackendChanged()` runs while `_mtx` is + held, and those methods re-acquire `_mtx` → **self-deadlock**. Lock ordering is + `Bridge::_mtx` before `LocalBackend::_regMtx`. +- **`executeVia` IS safe from `onBackendChanged()`.** It never takes `_mtx`; it + reads a **lock-free snapshot** of the backend `shared_ptr` (via `_backendMtx`, + a short separate lock) and the binding's `std::atomic currentId`. A concurrent + switch cannot free the old backend out from under the call because its + `shared_ptr` refcount is still > 0; the call either succeeds or fails with + "model not found", both safe. + +### `NetworkMonitor` — probe-thread callbacks must not block + +`onOffline` / `onOnline` run **directly on the probe thread** (`NetworkMonitor::run`). +Constraints: + +- **Callbacks must not block.** A blocking callback stalls the probe loop and + delays or prevents all subsequent probes. The intended body is short — set an + atomic flag or `post()` to an executor and return. +- **Callbacks must not throw** through the monitor's expectations (the probe + itself is wrapped in `safeProbe`, which swallows exceptions). +- **`stop()` from within a callback self-detaches.** `stop()` normally joins the + probe thread, but joining from the probe thread itself would deadlock. It + detects `this_thread == probe thread` and **detaches** instead; the destructor + then spin-waits on `_runExited` until the thread exits. `stop()` is idempotent. +- `isOnline()` reads an `std::atomic` — safe from any thread at any time. + +### `ReconnectCoordinator` — mutex held across the whole retry loop + +`onOnline()` holds `_mtx` for the **entire** reconnect → activate → bind → replay +loop, including all retry sleeps; `onOffline()` takes the same mutex. So the two +are mutually exclusive and a second concurrent caller blocks until the first +finishes. The coordinator owns no thread and does no I/O — it runs synchronously +on the caller's thread, and the host is expected to post it onto a worker +executor, **not** call it on the probe thread. The strict step order +(reconnect → activatePrimary → bindContext → replay) is an invariant: replay +never runs before context is bound. + +### Registries — populated at static-init, then read-only + +`ModelRegistryFactory`, `ActionDispatcher` (`registry.hpp`), and +`ActionExecuteRegistry` (`bridge.hpp`) are process-level singletons populated by +the `BRIDGE_REGISTER_MODEL` / `BRIDGE_REGISTER_ACTION` macros at **static-init +time**, single-threaded, before `main` runs. After that they are **read-only**. +This is precisely what makes concurrent dispatch safe without locking them: many +pool threads look up runners/factories concurrently, and concurrent reads of a +never-again-mutated `unordered_map` need no synchronisation. Registering a new +action *after* threads are running is unsupported and would be a data race. + +### Logger — lock-free reject, mutex-guarded sink, non-recursive + +`morph::log` (`logger.hpp`) has two tiers: + +- **Fast path:** the minimum level is an `std::atomic`. A message below + the threshold is rejected **lock-free**, before touching the mutex or + formatting the string (`logFormat` checks the level before `std::format`). +- **Sink path:** the sink is invoked while a global `std::mutex` is held, so sink + calls are serialised and `setLogger`/`ScopedLoggerOverride` swap safely. + +Because that mutex is **non-recursive**, a sink **must not** call back into +`morph::log` — no `log*()`, no `setLogger`, no `ScopedLoggerOverride` — from +inside the sink, or it **self-deadlocks**. Sinks should also not block for long, +since they serialise all logging. + +### Thread-local session `Context` — valid only on the dispatch thread + +`session::current()` (`session.hpp`) returns a thread-local `Context*` installed +by a `ScopedContext` around the model call — `LocalBackend::execute` on the local +path, `RemoteServer::dispatchExecute` on the remote path. It is valid **only on +the dispatch (strand/pool) thread, only for the duration of that `execute()`**. +It is **not** visible from the GUI thread or from a `Completion` callback, and +its address dangles once the scope exits. Models read it synchronously inside +`execute()`; they must not stash the pointer for later use. + +Recent addition: the `Context` also carries the **authoritative principal**. A +verifying authorizer's `authenticate()` (e.g. `SigningAuthorizer`) overwrites +`Context::principal` with the identity it extracted from a valid token +**before** dispatch, so `session::current()->principal` read inside a model is +the authenticated identity, not the client's unverified claim. See +[security.md](security.md). + +## `Completion` / `CompletionState` thread-safety + +`CompletionState` (`completion.hpp`) is shared between the producing thread +and the attaching thread, so its mutable state is mutex-protected: + +| Field | Protection | +|---|---| +| `value`, `error`, `ready`, `onOk`, `onErr`, `onErrAttached` | `mtx` (all reads/writes) | +| `cbExec` | **Not** mutex-guarded — happens-before, see below | + +`setValue` / `setException` are idempotent: once `ready` is set, later calls +return immediately. This is what lets a backend `cancelPending(...)` a completion +and then have a late server reply arrive as a harmless no-op. + +**`cbExec` is a happens-before requirement, not a lock.** It is written once, in +the `Completion` constructor, before the state is published to any other thread, +and only read afterward. The contract is: *construct the `Completion` handle +(which sets `cbExec`) before the producing thread can call +`setValue`/`setException`.* The backends honour this — they build the +`Completion` object before posting the strand/transport task that resolves the +state. Guarding `cbExec` with `mtx` would be redundant given that ordering, so it +is deliberately left unguarded. + +## Quick cheat-sheet + +Destroy in this order (or declare members so the reverse holds): + +``` +BridgeHandler(s) ← first to go (or any order vs. Bridge, thanks to the liveness token) + Bridge + backend ← LocalBackend / SimulatedRemoteBackend + RemoteServer ← only in remote mode; keep its shared_ptr alive this long + ThreadPoolExecutor ← LAST: it must outlive every StrandExecutor it backs +``` + +One-liners to remember: + +- Never destroy the pool before the strand/backend → `~StrandExecutor` deadlocks. +- Never `post()` to a `StrandExecutor` whose destructor has begun. +- Never call `switchBackend`/`registerHandler`/`deregisterHandler` from + `onBackendChanged()`; `executeVia` is fine there. +- Never block or re-enter from a `NetworkMonitor` callback (probe thread). +- Never log from inside a log sink (non-recursive mutex). +- Never read `session::current()` off the dispatch thread or after `execute()` + returns. +- Register all models/actions at static-init; treat the registries as read-only + afterward. + +## Cross-references + +- [`executor.md`](executor.md) — `IExecutor`, `ThreadPoolExecutor`, + `StrandExecutor`, `ModelId`; the "destroy strand before base pool" rule in + detail. +- [`completion.md`](completion.md) — `Completion` / `CompletionState` + internals and orphan-error logging. +- [`bridge.md`](bridge.md) — `Bridge`, `BridgeHandler`, `switchBackend`, + `executeVia`, the liveness token. +- [`backend.md`](backend.md) — `LocalBackend`, `RemoteServer`, + `SimulatedRemoteBackend`, `cancelPending`, the `make_shared` requirement. +- [`offline.md`](offline.md) — `NetworkMonitor`, `ReconnectCoordinator`, + `SyncWorker` wiring and the reconnect ordering guarantee. +- [`registry.md`](registry.md) — `ActionDispatcher` / `ModelRegistryFactory` and + the static-init registration model. +- [`logger.md`](logger.md) — the logging fast path and sink contract. +- [`session.md`](session.md) — `Context`, the thread-local, and `IAuthorizer`. +- [`security.md`](security.md) — where the authoritative principal comes from and + the `RemoteServer` enforcement points. +- `error_handling.md` — the framework-wide error-propagation story that the + per-subsystem exception handling here plugs into (also summarised under + *Error propagation* in `../ARCHITECTURE.md`). +- *Thread safety* table in `../ARCHITECTURE.md` — the high-level summary this + spec expands on. +``` diff --git a/docs/spec/datetime.md b/docs/spec/datetime.md new file mode 100644 index 0000000..90e1a36 --- /dev/null +++ b/docs/spec/datetime.md @@ -0,0 +1,285 @@ +# The `DateTime` / `Timestamp` types — design + +Two types in namespace `morph::time`, mirroring the `Rational` / `Quantity` +split: + +- **`morph::time::DateTime`** — the *value*: a UTC instant with millisecond + precision over `std::chrono::sys_time` (adapted + from LASTRADA `Toolbox/Chrono.hpp`). Cheap to copy, ordered, duration + arithmetic. +- **`morph::time::Timestamp`** — the *field*: an optionally-empty `DateTime` + with the same one-kind-of-empty semantics as `Quantity` (a form draft starts + blank; required-ness is derived by `morph::forms`). + +## Contents + +- [Wire format](#wire-format) +- [Schema](#schema) +- [`DateTime`](#datetime) + - [Construction](#datetime-construction) + - [Access and inspection](#datetime-access-and-inspection) + - [Arithmetic and ordering](#datetime-arithmetic-and-ordering) +- [`Timestamp`](#timestamp) + - [Free functions (namespace scope)](#free-functions-namespace-scope) + - [Empty-state semantics](#empty-state-semantics) +- [Glaze codec](#glaze-codec) +- [`std::format` support](#stdformat-support) +- [Design decisions](#design-decisions) +- [Range & precision](#range--precision) +- [Failure modes — what is rejected](#failure-modes--what-is-rejected) +- [Limitations](#limitations) +- [Cross-references](#cross-references) +- [Out of scope](#out-of-scope) + +## Wire format + +A `DateTime` travels as the ISO-8601 string `YYYY-MM-DDTHH:MM:SS[.mmm]Z` +(UTC only). On output the fractional part and trailing `Z` are **always** +written; on input both are optional. The parser is a strict, hand-rolled +routine — no locale, no `std::chrono::parse` dependency — so behaviour is +identical across libstdc++, libc++ (WASM), and MSVC. + +Unlike the `Rational` codec, which clamps hostile input into a valid value, a +malformed timestamp has no meaningful clamp: the codec rejects it as a JSON +**read error**. + +## Schema + +A `Timestamp` field renders as `{"type": ["string","null"], +"format": "date-time"}` — the standard JSON-Schema vocabulary form renderers +key on (the demo clients render a date-time input for it). + +## `DateTime` + +The underlying instant is `std::chrono::sys_time` — +Unix time with leap seconds ignored. + +### `DateTime` — construction + +| Member | Signature | Notes | +|---|---|---|---| +| `value` | `std::chrono::sys_time` | The underlying UTC instant (public data member). | +| default ctor | `constexpr DateTime() noexcept` | The Unix epoch (1970-01-01T00:00:00.000Z). | +| sys_time ctor | `constexpr DateTime(sys_time) noexcept` | Wraps an existing UTC instant. | +| calendar ctor | `constexpr DateTime(std::chrono::year, std::chrono::month, std::chrono::day, std::chrono::hours, std::chrono::minutes, std::chrono::seconds, std::chrono::milliseconds = milliseconds{0}) noexcept` | Composes from calendar/clock components. Performs **no** validation (unlike `fromIso8601`): a valid `year_month_day` is a caller precondition, and out-of-range components yield an unspecified instant. | +| `now()` | `static DateTime now() noexcept` | The current UTC instant, truncated to milliseconds. | +| `fromIso8601(text)` | `static std::optional fromIso8601(string_view) noexcept` | Strict parser (see [Wire format](#wire-format)); returns `nullopt` when malformed. | + +### `DateTime` — access and inspection + +| Member | Signature | Returns | +|---|---|---| +| `toIso8601()` | `std::string toIso8601() const` | `YYYY-MM-DDTHH:MM:SS.mmmZ` via `std::format`. | + +### `DateTime` — arithmetic and ordering + +| Member | Signature | Notes | +|---|---|---| +| `operator<=>` | `constexpr std::strong_ordering operator<=>(DateTime const&) const noexcept = default` | Delegates to the underlying `sys_time`. | +| `operator+=` | `template constexpr DateTime& operator+=(duration) noexcept` | Shifts by any chrono duration (`duration_cast` to ms). | +| `operator-=` | `template constexpr DateTime& operator-=(duration) noexcept` | Shifts backwards. | +| `operator+(lhs, delta)` | `template friend constexpr DateTime operator+(DateTime, duration) noexcept` | Returns a new shifted instant. | +| `operator-(lhs, delta)` | `template friend constexpr DateTime operator-(DateTime, duration) noexcept` | Returns a new shifted instant. | +| `operator-(lhs, rhs)` | `friend constexpr auto operator-(DateTime const&, DateTime const&) noexcept` | Returns `lhs.value - rhs.value` as a chrono duration. | + +## `Timestamp` + +The blank state ("not entered / not recorded") lives inside the struct, exactly +like `Quantity`: action structs never wrap a `Timestamp` in `std::optional`, +and a non-optional `Timestamp` member is *required* by `morph::forms` rules. + +| Member | Signature | Notes | +|---|---|---|---| +| `value` | `std::optional` | The payload; `std::nullopt` means empty (public data member). | +| default ctor | `constexpr Timestamp() noexcept` | The **empty** state. | +| DateTime ctor | `constexpr Timestamp(DateTime) noexcept` | Engages with the given instant. | +| optional ctor | `constexpr Timestamp(std::optional) noexcept` | Adopts an optional payload as-is. | +| `now()` | `static Timestamp now() noexcept` | `Timestamp{DateTime::now()}`. | +| `hasValue()` | `constexpr bool hasValue() const noexcept` | Engaged? No implicit `bool` conversion. | +| `operator*` | `constexpr DateTime const& operator*() const noexcept` | Unchecked access to the engaged value (UB when empty, like `std::optional`). | +| `operator<=>` | `constexpr auto operator<=>(Timestamp const&) const noexcept = default` | Default ordering on the optional payload; empty sorts before engaged. | + +### Free functions (namespace scope) + +| Symbol | Signature | Notes | +|---|---|---| +| `operator-(lhs, rhs)` | `constexpr std::optional operator-(Timestamp const&, Timestamp const&) noexcept` | The signed duration between two engaged timestamps; `nullopt` when either is empty. | + +### Empty-state semantics + +- **Default construction.** `Timestamp{}` is empty. +- **Querying.** `hasValue()` returns `false` when empty; no implicit `bool`. +- **Comparison.** `operator<=>` is total — empty compares less than any engaged + timestamp; two empties compare equal (`std::optional` default ordering). +- **Wire.** An empty `Timestamp` serializes as JSON `null` (or, as a struct + member, is omitted — the glaze `meta` delegates to `std::optional`). +- **Difference.** `lhs - rhs` returns `nullopt` if either operand is empty. + +## Glaze codec + +Specialisations in `glz` (and `glz::detail` for schema): + +- `from` — reads a JSON string and calls + `DateTime::fromIso8601`; malformed input sets `error_code::syntax_error`. +- `to` — writes `DateTime::toIso8601()` as a JSON string. +- `meta` — declares the wire layout as the underlying + `std::optional` (nullable on the wire, null = empty). +- `to_json_schema` — produces a JSON-Schema string with + `"format": "date-time"`. + +## `std::format` support + +`std::formatter` renders the ISO-8601 UTC string. +An empty format spec `{}` or `{:}` is accepted; any non-empty spec throws +`std::format_error`. No `operator<<` is provided. + +## Design decisions + +| Decision | Choice | Why | +|---|---|---| +| Precision | **Milliseconds** | Sufficient for domain timestamps without the cost/scope of microsecond/nanosecond. | +| Default state | **Unix epoch, not empty** | `DateTime` is a pure value wrapper; the empty/optional distinction lives in `Timestamp`. | +| Parser | **Hand-rolled, strict** | Identical behaviour across all standard libraries (no `std::chrono::parse` dependency, no locale). `from_chars` for each numeric field; `year_month_day.ok()` for calendar validity. | +| Malformed input | **Read error, not clamped** | Unlike `Rational`, there is no meaningful fallback for "yesterday" — reject at the wire boundary. | +| Wire output | **Always `.mmmZ`** | The canonical form is the most precise, unambiguous representation; optional precision on input is a leniency, not the output contract. | +| Timestamp empty state | **Inside the struct**, not `std::optional` | Same one-kind-of-empty design as `Quantity`; required-ness is a `morph::forms` property, not a type property. | +| Formatting | **`std::formatter` only** | Single formatting path; no `operator<<`. | +| Time zone support | **UTC only** | All application timestamps are UTC; no time zone offset parsing, no local time storage. A non-UTC input (`+02:00`) is rejected as malformed. | + +## Range & precision + +The *storage* — `std::chrono::sys_time` — spans far +more than a millennium in each direction (the `sys_days` count alone reaches +well beyond `year{-32767}`/`year{32767}`). The *wire codec*, however, is +narrower than the value it carries: + +- **Output** (`toIso8601`) formats the year with `{:04}` — a minimum-width-4, + zero-padded field, *not* a fixed-width one. Years 1–9999 emit exactly four + digits; year `10000` emits five (`"10000-…"`) and a negative year emits a + leading `-` (`"-001-…"`). +- **Input** (`fromIso8601`) reads the year as *exactly four digits* + (`number(0, 4)`) and then requires a `-` at offset 4. A five-digit year shifts + the `-` past offset 4, and a leading `-` puts a digit where the separator is + expected; **both fail the separator check**. + +The consequence: **only years 0001–9999 round-trip.** Anything outside that band +is representable as a `DateTime` value (and can be constructed, compared, and +arithmetic-shifted in memory) but cannot survive a serialize→parse cycle — the +serialized text either won't re-parse or would re-parse as a *different* +instant. This is treated as a **producer precondition**: application timestamps +live inside the four-digit era, and feeding an out-of-band instant to the wire +codec is a lossy operation the codec does not guard against. Millisecond +precision is the other axis of the same contract — a 4th fractional digit is not +silently dropped (see [Failure modes](#failure-modes--what-is-rejected)). + +## Failure modes — what is rejected + +`fromIso8601` is deliberately strict. RFC 3339 permits several forms this parser +rejects, and the rejections are load-bearing (they keep the wire form canonical +and cross-stdlib-identical), so they are enumerated here. + +| Input | Result | Why | +|---|---|---| +| `2026-07-05T14:30:15.123456Z` | **read error** | A 4th (or later) fractional digit is *not* truncated — the loop consumes at most three digits, then the leftover `456Z` is trailing input and `cursor != text.size()` fails. Precision loss is a parse failure, never a silent round-down. | +| `2026-07-05T14:60:00Z` | **read error** | Leap second (`:60`). RFC 3339 *allows* `:60`; this parser bounds seconds at `> 59`. | +| `2026-07-05t14:30:15z` | **read error** | Lowercase `t`/`z`. RFC 3339 *allows* both cases; this parser matches only the uppercase literals `'T'` and `'Z'`. | +| `2026-07-05T14:30:15+02:00` | **read error** | Non-UTC zone offset — trailing input after the seconds field. UTC only. | +| `2026-07-05T14:30:15.` | **read error** | A `.` with no following digit (`digits == 0`). | +| `2026-02-30T10:00:00Z` | **read error** | Calendar date that does not exist (`year_month_day::ok()` is false). | +| **`2026-07-05T14:30:15`** (no `Z`) | **accepted, interpreted as UTC** | The trailing `Z` is *optional* on input. | + +**The silent-UTC asymmetry.** A zone offset (`+02:00`) is rejected, yet a +zone-*less* string is silently accepted and assumed UTC. So the parser refuses +to *convert* a stated non-UTC time but happily *assumes* UTC for a string that +stated no zone at all — a string that names the wrong zone is safer (it fails +loudly) than one that names none (it is taken at face value). Producers that +cannot guarantee UTC should append the explicit `Z` so the intent is on the +wire, even though the parser does not require it. + +**Diagnostics are coarse.** `fromIso8601` returns a bare `std::optional` — every +failure above collapses to `std::nullopt`, discarding *which* field or check +failed. The glaze `from` adapter then maps that single +`nullopt` to a single `error_code::syntax_error`. There is no "bad month" vs. +"bad fraction" vs. "trailing junk" distinction at the wire boundary; the caller +learns only that the string was not a valid canonical UTC timestamp. + +## Limitations + +- **No date-only, time-of-day, or duration types.** There is one temporal field, + `Timestamp`, and it always carries a full instant. A birthday, a due-date, or + an opening time all drag a `HH:MM:SS.mmm` component whether the domain wants + one or not, and the schema always emits `format: date-time` — a renderer has + **no signal** to offer a bare date picker (there is no `format: date` / + `format: time` / duration emission). Callers encode "midnight UTC" or similar + conventions by hand. +- **The cross-stdlib-identical claim is test-backed, not proven.** The parser is + hand-rolled precisely so that libstdc++, libc++ (WASM), and MSVC agree; that + agreement rests on the round-trip and boundary tests + (`DateTime::Iso8601::RoundTrip` and `DateTime::Iso8601::RejectsMalformedInput` + in `tests/test_datetime.cpp`) exercising the same inputs everywhere, not on a + formal argument. A stdlib-specific `year_month_day::ok()` or `from_chars` + discrepancy would only surface as a test failure on that platform. +- **An empty `Timestamp` sorts *before* every real instant.** `Timestamp`'s + defaulted `operator<=>` delegates to `std::optional`, whose ordering + puts a disengaged optional below every engaged one. So a blank/not-yet-entered + timestamp is the *smallest* value — sorting a list of records by a + `Timestamp` column floats the un-entered ones to the top, which is frequently + the opposite of the domain intent ("undated = unknown = should sort last"). + This is a `std::optional` artifact, not a deliberate temporal choice; callers + that need "empty sorts last" must special-case it (mirroring the + `hasValue()`-before-ordering guidance in `quantity_type.md`). +- **Several converting constructors are implicit.** `DateTime(sys_time)`, + `Timestamp(DateTime)`, and `Timestamp(std::optional)` are all + non-`explicit`. This is convenient (`Timestamp t = DateTime::now();`, brace-init + of action members) but means a stray `sys_time`, `DateTime`, or + `optional` converts silently at call sites and in overload + resolution. + +## Cross-references + +- **`forms.md`** — `Timestamp` satisfies the `EmptyCapableField` concept via + `hasValue()`, and `allRequiredEngaged()` treats a non-optional `Timestamp` + member as a **required-field gate**: the action is not "ready" until that + timestamp is engaged. This is the whole reason the empty state lives *inside* + `Timestamp` rather than in a `std::optional` wrapper. +- **`quantity_type.md`** and **`choice.md`** — the same *one-kind-of-empty* + pattern: exactly one representation of "not entered" (`value == nullopt` / + the disengaged variant), a `hasValue()` query, no implicit `bool`, and an + unchecked `operator*`. `Timestamp` is the third member of this family. +- **`rational.md`** — the instructive **contrast**. `Rational`'s codec *clamps* + hostile input into a valid value (there is a meaningful fallback); `DateTime`'s + codec *rejects* it as a read error (there is no meaningful "nearest valid + timestamp"). Same framework, opposite boundary policy, chosen per type. + +### Worked example — a required `Timestamp` field + +An action struct declares the timestamp as a plain, non-optional member. That +non-optionality is what makes it *required*: + +```cpp +struct RecordMeterReading { + morph::time::Timestamp observedAt; // required: not std::optional, not defaulted-away + std::string meterId; +}; + +RecordMeterReading draft{}; // observedAt is empty (blank form) +morph::forms::allRequiredEngaged(draft); // false — observedAt not engaged +draft.observedAt = morph::time::DateTime::now(); // implicit DateTime -> Timestamp +morph::forms::allRequiredEngaged(draft); // observedAt gate now satisfied +``` + +The schema fragment generated for the field (the nullable string + `date-time` +format the demo renderers key on): + +```json +"observedAt": { "type": ["string", "null"], "format": "date-time" } +``` + +An empty `observedAt` serializes as `null` (or is omitted as a struct member); +a `null` on read means "not entered", not "epoch". + +## Out of scope + +- Sub-millisecond precision — the storage is `sys_time`. +- Time zone conversion or zone-aware arithmetic — UTC exclusively. +- `std::chrono::parse`-based parsing — the hand-rolled parser is deliberate. diff --git a/docs/spec/error_handling.md b/docs/spec/error_handling.md new file mode 100644 index 0000000..7774299 --- /dev/null +++ b/docs/spec/error_handling.md @@ -0,0 +1,295 @@ +# Error handling — design (cross-cutting) + +This is the single authoritative reference for how errors, exceptions, and +failures propagate across **every** layer of morph — from a throwing +`Model::execute` on a worker thread all the way back to a GUI-thread +`.onError` callback, and every typed failure the value codecs, wire protocol, +registry, journal, and auth layers can raise along the way. Per-type specs +link here rather than re-deriving the whole story. + +The design has one organising principle: **an error must never vanish +silently, and it must never surface on the wrong thread.** Everything below is +a consequence of holding those two invariants together. + +## Contents + +- [The async happy-vs-error path](#the-async-happy-vs-error-path) +- [Orphan logging and single-shot callbacks](#orphan-logging-and-single-shot-callbacks) +- [Executor exception handling](#executor-exception-handling) +- [Backend error types](#backend-error-types) +- [Remote wire errors](#remote-wire-errors) +- [Value-codec philosophy: clamp vs. reject](#value-codec-philosophy-clamp-vs-reject) +- [Other typed failures](#other-typed-failures) +- [How to observe failures](#how-to-observe-failures) +- [Cross-references](#cross-references) + +## The async happy-vs-error path + +Every action dispatched through a backend flows through the same shape. On the +local path (`morph::backend::LocalBackend`), `Model::execute` runs inside a +strand task on the worker pool: + +``` +Model::execute(action) throws + └─ LocalBackend's strand task catches via try/catch (backend.hpp) + └─ CompletionState::setException(std::current_exception()) + └─ .onError(fn) handler posted to the GUI (callback) executor + └─ fn receives exception_ptr; caller rethrows to inspect +``` + +The exact code (`LocalBackend::execute`) is: + +```cpp +_strand.post(mid, [localOp, holder, compState, session]() mutable { + try { + ::morph::session::detail::ScopedContext const scoped{session}; + compState->setValue(localOp(*holder)); // happy path + } catch (...) { + compState->setException(std::current_exception()); // error path + } +}); +``` + +Both `setValue` and `setException` resolve the shared +`morph::async::detail::CompletionState>` exactly once +(first writer wins; a second call while `ready == true` is a no-op). When a +callback is already registered, they build it under the state mutex and then +`post()` it to `cbExec` — the executor supplied at `Completion` construction, +which for GUI code is the GUI executor. **The producing (worker) thread never +invokes the callback directly**, so `.then` / `.onError` bodies always run on +the intended thread. + +To inspect the failure, the GUI's `.onError` handler rethrows: + +```cpp +handler.execute(action) + .then([](Result r) { /* GUI thread */ }) + .onError([](std::exception_ptr e) { + try { std::rethrow_exception(e); } + catch (const morph::backend::DisconnectedError&) { /* retry */ } + catch (const std::exception& ex) { showError(ex.what()); } + }); +``` + +The remote path (`SimulatedRemoteBackend::execute`, and real WebSocket +backends) is identical from the caller's side: the reply-handling lambda calls +`state->setValue(deser(reply.body))` on an `"ok"` reply, or throws a +`std::runtime_error(reply.message)` (caught into `setException`) on an `"err"` +reply. See [Remote wire errors](#remote-wire-errors) for how the server side +produces those `err` messages. + +## Orphan logging and single-shot callbacks + +`CompletionState` (see `completion.hpp`) protects a value slot, an +`std::exception_ptr error` slot, two callback slots (`onOk` / `onErr`), an +`onErrAttached` flag, and a `cbExec` pointer, all under one mutex. + +**Single-shot, last-writer-wins.** `then()` and `onError()` each register at +most one handler: + +| Situation | Behavior | +|---|---| +| `then` registered before ready | Stored in `onOk`, fired when `setValue` lands | +| `then` on an already-value state | Fired immediately (posted to `cbExec`) | +| `then` on an already-**error** state | Silent no-op — no success value exists | +| `onError` registered before ready | Stored in `onErr`, fired when `setException` lands | +| `onError` on an already-**error** state | Fired immediately (posted to `cbExec`) | +| `onError` on an already-value state | Silent no-op — no error exists | +| Second `then`/`onError` after ready | Overwrites the stored slot; but if already fired, the slot is empty so nothing re-fires | + +**Orphan logging.** If a `Completion` is destroyed while its state holds an +unhandled error, `~CompletionState` logs it through `morph::log::logError`: + +- `std::exception` subclass → `"[orphan] unhandled exception: " + what()` +- any other type → `"[orphan] unhandled unknown exception"` + +The destructor logs only when `ready && error && !onErrAttached`. So attaching +an `.onError` handler suppresses the orphan log — **but only when there is an +executor to actually deliver it on.** + +**Recent fix — null callback executor no longer silences the error.** Both +`setException` and `attachOnError` set `onErrAttached = (cbExec != nullptr)`. +Previously, attaching `.onError` unconditionally marked the error handled; with +a null `cbExec` the callback is never posted (`post` runs only when +`cbExec != nullptr`), so the error would be both undelivered *and* +unsuppressed-from — it vanished. Now, with a null executor `onErrAttached` +stays `false`, so even though the handler can't be delivered, the destructor's +orphan logger still fires and the error reaches the log rather than +disappearing silently. An undelivered *value* is still dropped silently (a +value that no one is waiting for denotes nothing); an undelivered *error* +always surfaces. + +## Executor exception handling + +Executors (`executor.hpp`, `strand.hpp`) are the thread boundaries. A task that +throws must not kill its worker, must not abort sibling tasks, and must not +vanish. **Recently changed:** the pool and strand executors now catch-and-log +rather than swallow. + +| Executor | Catch behavior | Log prefix | +|---|---|---| +| `ThreadPoolExecutor` | Catches `std::exception` and `...` per task; worker loops on | `[thread-pool] task threw: ` / `[thread-pool] task threw unknown exception` | +| `StrandExecutor` | Catches `std::exception` and `...` per task; next queued task for the same key still runs | `[strand] task threw: ` / `[strand] task threw unknown exception` | +| `MainThreadExecutor::runFor` | Catches **only** `std::exception`; continues with the next task | `[main-thread] callback threw: ` | + +Notes that matter: + +- The `StrandExecutor` is where `Model::execute` actually runs (for both + `LocalBackend` and `RemoteServer`). In normal operation the backend's own + `try/catch` converts a throwing `execute` into `setException` **before** the + strand's catch could see it, so `[strand] task threw:` fires only for + exceptions escaping the backend's own lambda body (e.g. a throw from + `setValue`/`setException` machinery itself), not for ordinary action + failures. +- `MainThreadExecutor::runFor` deliberately catches only `std::exception`. + Any non-`std::exception` type (e.g. a bare `throw 42;` or a foreign SEH-style + type) **propagates out of `runFor`** to the caller's main loop. The pool and + strand executors, by contrast, catch `...` too and never propagate. +- The base `IExecutor::post` contract documents that exceptions are "silently + swallowed unless the implementation documents otherwise" — the two concrete + executors above document otherwise (they log). + +## Backend error types + +`backend.hpp` defines three typed errors, all `std::runtime_error` subclasses +with canned messages, thrown *into* in-flight completions (delivered via +`.onError`) rather than out of any call: + +| Type | Message | Thrown when | +|---|---|---| +| `BackendChangedError` | `"backend changed before completion resolved"` | `Bridge::switchBackend()` calls `cancelPending()` on the outgoing backend; every still-pending completion resolves with this | +| `BridgeDestroyedError` | `"bridge destroyed before completion resolved"` | `Bridge`'s destructor cancels pending completions | +| `DisconnectedError` | `"transport disconnected before completion resolved"` | A transport (e.g. Qt WebSocket) drops mid-call; framework retries on reconnect if supported, else `.onError` runs | + +`cancelPending(exc)` walks the backend's tracked-pending list and calls +`setException(exc)` on each live state. Because `setException` is a no-op once +`ready`, an in-flight server reply arriving *after* the cancel cannot resurrect +a cancelled completion — the cancel wins. + +**One failure is deliberately untyped.** When `LocalBackend::execute` is given +a model id it doesn't hold, it resolves the completion with a plain +`std::runtime_error("model not found: id=" + id)` — there is no dedicated +error class for it. The remote path has an analogous plain +`"model not found"` `err` reply (see below). + +## Remote wire errors + +The wire protocol (`wire.hpp`) uses a single `Envelope` struct with a `kind` +discriminator. Failures come back as `kind == "err"` envelopes carrying a +free-text `message` and an echoed `callId`. `RemoteServer` (`remote.hpp`) +produces them: + +| Error `message` | Raised in | Cause | +|---|---|---| +| `"unauthorized"` | `dispatchExecute` | `IAuthorizer::authorize` returned `false` (bad/expired/absent token, or policy denial) | +| `"register requires a typeId"` | `dispatchMessage` (`register`) | `register` envelope with empty `typeId` | +| `"unknown model type: "` | `ModelRegistryFactory::create` | `register` for a type-id with no registered factory | +| `"model not found"` | `dispatchExecute` | `execute` for a `modelId` the server doesn't hold | +| `"unknown envelope kind: "` | `dispatchMessage` | `kind` is not `register`/`deregister`/`execute` | +| `"envelope decode failed: "` | `wire::decode` in `dispatchMessage` | Malformed request JSON | +| any handler exception's `exc.what()` | outer `try/catch` in `dispatchMessage`, and the strand lambda in `dispatchExecute` | A throw from `ActionDispatcher::dispatch` or `Model::execute` — its `what()` becomes the `err` message | +| `"handleInline does not support execute (reply is asynchronous)"` | `handleInline` | An `execute` envelope handled via the synchronous inline path (reply would dangle) | + +**callId echoing.** The `callId` is copied from the request into the `err` +reply so the client can correlate it. The one exception: when `wire::decode` +itself fails, no envelope could be parsed, so `env.callId` is still its default +`0` and the decode-error reply carries `callId == 0`. + +**Encoding/decoding failures.** `wire::encode` and `wire::decode` both throw +`std::runtime_error` on Glaze failure (`"envelope encode failed: ..."` / +`"envelope decode failed: ..."`). `decode` failures on the server become the +canonical decode-error reply above; on the client side (in +`SimulatedRemoteBackend`'s reply lambda) any throw — decode failure or an +`"err"` reply turned into `throw std::runtime_error(reply.message)` — is caught +and routed to `setException`, then to `.onError`. + +## Value-codec philosophy: clamp vs. reject + +The two exact value types take deliberately **opposite** stances on hostile +wire input, because the two kinds of value denote differently: + +| Type | Hostile input policy | Rationale | +|---|---|---| +| `math::Rational` (`rational.hpp`) | **Clamps** | A number always denotes *some* quantity; the nearest valid value is a meaningful answer | +| `time::DateTime` (`datetime.hpp`) | **Rejects** (JSON read error) | A malformed instant denotes *nothing*; there is no meaningful "nearest" timestamp | + +**`Rational` clamps.** Its Glaze read side (`from_json`/`read`) rebuilds +through the canonicalising constructor: `dp` is run through +`detail::clampWireDecimalPlaces` into `[1, kMaxDecimalPlaces]` (18), a zero +denominator is clamped to `1`, and the fraction is gcd-reduced — silently, with +no error. `Rational` itself **never throws**: fallible arithmetic +(`operator/`, `reciprocal`, `dividedBy`, `from`, `fromFloat`) returns +`std::expected` where `RationalError` is +`{ DivisionByZero, NotFinite, Overflow }`, and mixed expressions +short-circuit the first error left-to-right. There is a two-tier clamp: +`clampWireDecimalPlaces` is silent (untrusted wire), `clampDecimalPlaces` +asserts in debug first (a precision stated in code out of range is a bug). + +**`DateTime` rejects.** `fromIso8601` strictly parses `YYYY-MM-DDTHH:MM:SS` +with a hand-rolled routine (no locale, no `std::chrono::parse`); a non-existent +calendar date (`2026-02-30`) or out-of-range clock field returns +`std::nullopt`. Its Glaze read op turns that `nullopt` into +`ctx.error = error_code::syntax_error` — a JSON **read error** that fails the +whole decode, exactly as if the field were the wrong type. There is no clamp +because a mistyped instant has no valid nearby value. + +## Other typed failures + +| Type / message | Where | Trigger | +|---|---|---| +| `journal::SerializationError` (`std::runtime_error`) | `journal::toJson` / `journal::fromJson` (`action_log.hpp`) | `LogEntry` (de)serialisation fails. `fromJson`'s path is real (malformed JSON); `toJson`'s branch is shared but structurally unreachable for `LogEntry` | +| `model::detail::ParseError` (`std::runtime_error`) | `ActionTraits::toJson`/`fromJson`/`resultToJson`/`resultFromJson` (the `BRIDGE_REGISTER_ACTION` codec) | Glaze read/write of an action payload or result fails | +| `std::runtime_error("unknown action: /")` | `ActionDispatcher::dispatch` (`registry.hpp`) | `(modelType, actionType)` pair was never registered | +| `std::runtime_error("unknown model type: ")` | `ModelRegistryFactory::create` (`registry.hpp`) | Model type-id was never registered — also surfaces as the `register` wire `err` | +| `std::runtime_error` from `journal::replay` | `journal.hpp` | Replay hits an unregistered model type-id or an entry with an unregistered action type (it delegates to `dispatch`/`create`) | +| `session::AuthError` (`enum { Malformed, BadSignature, Expired }`) | `TokenVerifier::verify` returns `std::expected` (`session_auth.hpp`) | `Malformed`: not `payload.sig`, bad base64url, or unparseable claims. `BadSignature`: MAC mismatch (forged/tampered). `Expired`: `expiresAtMs` in the past | + +Note the auth asymmetry with the wire layer: `AuthError` is a **typed value** +returned to the server's authorizer, never a thrown exception and never sent +to the client verbatim. `SigningAuthorizer::authorize` collapses any `AuthError` +into a `false`, which `RemoteServer` turns into the single opaque +`"unauthorized"` `err` reply — the client is never told *why* authorization +failed. The MAC is checked before the payload is parsed, so untrusted JSON is +never handed to the parser until authenticity is established. + +## How to observe failures + +Errors reach three different places depending on where they occur; observe them +accordingly. + +1. **Attach `.onError`.** This is the primary channel for any failure that + resolves a `Completion` — a throwing action, a wire `err` reply, or a + backend cancel error. The handler runs on the GUI executor; rethrow the + `exception_ptr` to inspect the concrete type. + +2. **Set `morph::log::setLogger` early — before any backend or executor runs.** + Several failure classes surface **only** through the log: + - executor task exceptions (`[thread-pool]` / `[strand]` / `[main-thread]` + prefixes) — caught, logged, and otherwise swallowed; + - orphan errors (`[orphan] ...`) — a failed `Completion` destroyed with no + `.onError`, or with an `.onError` but a null callback executor. + + The default logger is a no-op sink; without `setLogger` these are invisible. + Point it at spdlog, Qt logging, or a test spy at startup. + +3. **The log is the only signal for exceptions on worker threads.** A throw on + a pool or strand thread never propagates to the main thread (the sole + exception being a non-`std::exception` type escaping + `MainThreadExecutor::runFor`, which does propagate to that thread's caller). + If a worker-side action misbehaves and no `Completion` carried the error + out, the log entry is the only trace you get. + +## Cross-references + +- [`completion.md`](completion.md) — `Completion` / `CompletionState`, orphan logging, single-shot callbacks. +- [`executor.md`](executor.md) — `IExecutor`, `ThreadPoolExecutor`, `MainThreadExecutor`, and the strand. +- [`backend.md`](backend.md) — `IBackend`, `LocalBackend`, `cancelPending`, the three backend error types. +- [`wire.md`](wire.md) — the `Envelope` protocol and `encode`/`decode`. +- [`registry.md`](registry.md) — `ActionDispatcher`, `ModelRegistryFactory`, `ParseError`, the codec macros. +- [`journal.md`](journal.md) — `LogEntry`, `SerializationError`, `replay`, `SessionLog`. +- [`rational.md`](rational.md) — `Rational`, `RationalError`, the clamping wire codec. +- [`datetime.md`](datetime.md) — `DateTime` and its strict, rejecting ISO-8601 codec. +- [`session.md`](session.md) / [`security.md`](security.md) — `Context`, `IAuthorizer`, `AuthError`, `SigningAuthorizer`. +- [`logger.md`](logger.md) — `morph::log`, `setLogger`, log levels. +- [`ARCHITECTURE.md`](../ARCHITECTURE.md#error-propagation) — the "Error propagation" overview this spec expands. +``` diff --git a/docs/spec/executor.md b/docs/spec/executor.md new file mode 100644 index 0000000..9b60bd1 --- /dev/null +++ b/docs/spec/executor.md @@ -0,0 +1,292 @@ +# Executor framework — design + +`morph::exec` provides a small set of executor abstractions that control +*where* and *when* posted tasks run. The design is intentionally minimal: +every executor accepts `std::function` tasks and guarantees they +execute at some point after `post()` returns, but the concurrency model, +threading, and serialisation semantics differ per implementation. + +## Contents + +- [Type overview](#type-overview) +- [`IExecutor` — the abstract interface](#iexecutor--the-abstract-interface) +- [`ThreadPoolExecutor`](#threadpoolexecutor) +- [`MainThreadExecutor`](#mainthreadexecutor) +- [`StrandExecutor` and `ModelId`](#strandexecutor-and-modelid) +- [Lifetime & ownership](#lifetime--ownership) +- [Thread safety](#thread-safety) +- [Failure modes](#failure-modes) +- [API reference](#api-reference) +- [Design decisions](#design-decisions) +- [Limitations](#limitations) +- [Cross-references](#cross-references) + +## Type overview + +There are six types, split across `morph::exec` (in `executor.hpp`) and +`morph::exec::detail` (in `strand.hpp`): + +| Type | Namespace | Purpose | +|---|---|---| +| `IExecutor` | `morph::exec` | Abstract base: a single pure-virtual `post(task)`. | +| `ThreadPoolExecutor` | `morph::exec` | Fixed-size thread pool, FIFO queue, task exceptions logged (never propagate). | +| `MainThreadExecutor` | `morph::exec` | Collects tasks from any thread, drains on `runFor()` from the owning thread. | +| `ModelId` | `morph::exec::detail` | Opaque 64-bit identifier for a model instance, used as a strand key. | +| `ModelIdHash` | `morph::exec::detail` | Hash functor so `ModelId` can be an `unordered_map` key. | +| `StrandExecutor` | `morph::exec::detail` | Per-key serialising wrapper — tasks with the same `ModelId` never overlap. | + +`IExecutor` and the two concrete executors live in the public `morph::exec` +namespace. `StrandExecutor`, `ModelId`, and `ModelIdHash` live in +`morph::exec::detail` because they are implementation details of the morph model +framework, not general-purpose utilities. + +## `IExecutor` — the abstract interface + +A pure-virtual `post(std::function)` that schedules a callable for +asynchronous execution. Thread-safe. How a task's exception is handled is left +to the implementation; the header's default wording says an exception "is +silently swallowed unless the implementation documents otherwise", and both +concrete executors here *do* document otherwise — see +[Failure modes](#failure-modes). No implementation lets a task exception escape +`post()` (the task has not run yet when `post()` returns). + +## `ThreadPoolExecutor` + +A fixed-size thread pool. The constructor spawns `n` worker threads; each worker +loops, waiting for tasks on a shared condition variable. Tasks are dispatched in +FIFO order from a single mutex-protected queue. + +The destructor signals stop, notifies all workers, and joins every thread. +Remaining queued tasks that have not started are dropped — there is no drain +phase. Exceptions from tasks are caught in the worker `loop` and logged via +`morph::log::logError` (see [Failure modes](#failure-modes)); they never +propagate out of a worker, so one failing task neither kills its thread nor +aborts sibling tasks. + +## `MainThreadExecutor` + +A single-thread executor that does **not** spawn its own thread. Tasks posted +from any thread are enqueued and only executed when the owning thread calls +`runFor(timeout)`. Useful in tests or event loops that lack a native dispatcher. + +`runFor()` runs queued tasks one by one for up to the given wall-clock +duration. It uses a condition-variable `wait_until` on the deadline, so it does +**not** return early when the queue drains — while time remains it keeps waiting +for newly posted tasks and only returns once the deadline is reached. If a task +throws a `std::exception`, the exception is logged (via `morph::log::logError`, +prefixed `"[main-thread] callback threw: "`) and execution continues with the +next task. Note that only `std::exception`-derived exceptions are caught; any +other thrown type propagates out of `runFor()`. + +## `StrandExecutor` and `ModelId` + +A per-key serialising executor built on top of any `IExecutor`. Tasks posted +with the same `ModelId` key execute in FIFO order with no overlap, even when the +underlying executor is a thread pool. Tasks with different keys may run +concurrently. + +`ModelId` is an opaque 64-bit identifier. Zero is reserved and means "not +bound". Non-zero values are assigned by the backend and are stable for the +lifetime of the model. It supports three-way comparison and can be used as an +`unordered_map` key via `ModelIdHash`. + +Internally `StrandExecutor` maintains a map of `ModelId → shared_ptr` +(shared state per key). A `Strand` holds a pointer to the base `IExecutor`, a +mutex, a pending queue, and a `running` flag. The executor also tracks an +`_inFlight` counter (guarded by the map mutex) that the destructor waits on. The +drain-and-erase logic in `scheduleNext` acquires both the map lock and the +strand lock together in a single scoped lock to close a race window where a +concurrent `post()` could re-arm a strand being erased. + +Each strand task is the point where the model's own code actually runs, so the +task wrapper catches exceptions and logs them via `morph::log::logError` (see +[Failure modes](#failure-modes)) before deciding whether to keep the strand +running. A throw therefore neither stalls the strand nor skips the drain-and-erase +bookkeeping: the next queued task for that key still runs. + +The destructor waits for all in-flight tasks to complete (`_inFlight == 0`) +before destroying the strand map. + +## Lifetime & ownership + +`StrandExecutor` stores a raw pointer to the base `IExecutor` (`_base`, copied +into each `Strand::base`). It does **not** own the base and never extends its +lifetime. Two invariants make the arrangement safe, and violating either is a +latent bug: + +1. **The base `IExecutor` must outlive the `StrandExecutor`.** Every strand + dispatch calls `strand->base->post(...)`, and `~StrandExecutor` blocks until + the last of those dispatched lambdas has run. So the base must still be alive + for the whole life of the strand, *including* the destructor's wait. + +2. **The base must actually run — not drop — every task the strand posts.** + `~StrandExecutor` only returns once `_inFlight` reaches 0, and `_inFlight` is + decremented *inside* the dispatched lambda, after the task runs. If the base + silently discards a queued lambda instead of running it, that decrement never + happens and the strand destructor waits forever. + +These two combine into the framework's most important ordering rule for these +types. `~ThreadPoolExecutor` **drops** queued-but-unstarted tasks (no drain), +whereas `~StrandExecutor` **blocks** until `_inFlight == 0`. Therefore: + +> **Always destroy the `StrandExecutor` before the base pool it wraps.** + +Destroying the base `ThreadPoolExecutor` first drops any strand lambdas still +sitting in the pool queue; their `--_inFlight` never runs, and the subsequent +`~StrandExecutor` deadlocks on its condition variable forever. With member +declaration order this means the pool must be declared *before* the strand +(members destroy in reverse order), or the two must be torn down explicitly in +that order. + +A second rule follows from the same wait: **no `post()` may race with or follow +`~StrandExecutor`.** The destructor takes `_mapMtx` and waits for +`_inFlight == 0`, but it does not block new `post()` calls. A `post()` that +arrives concurrently with (or after) destruction can enqueue work and re-arm a +strand after the destructor believed it had quiesced, reintroducing exactly the +data race the `_inFlight` wait exists to prevent. Callers must ensure all task +sources are shut down before the `StrandExecutor` is destroyed. + +The strand map is self-cleaning: when a strand drains (its `pending` queue is +empty), `scheduleNext` clears `running` and erases the map entry under the +combined `{_mapMtx, strand->mtx}` lock. Live memory therefore tracks the set of +*currently active* models rather than every model ever seen — there is no +per-model registration to leak. The cost is allocation churn: a model that is +posted to in bursts allocates a fresh `Strand` each time its queue empties and +refills, rather than keeping one long-lived strand per key. + +## Thread safety + +All three executors' `post()` methods are safe to call from any thread +concurrently. + +- `ThreadPoolExecutor` guards its queue and `_stop` flag with a single mutex + `_m` and coordinates workers on `_cv`. Multiple workers pop under the lock, so + the FIFO order is a total order across producers; task *execution* is + concurrent across the `n` workers. +- `MainThreadExecutor` guards its queue with `_m`. `post()` may be called from + any thread, but `runFor()` must be called only from the single owning + ("main") thread; concurrent `runFor()` calls are not supported. +- `StrandExecutor` uses two lock levels: `_mapMtx` protects the `_strands` map + and the `_inFlight` counter, and each `Strand::mtx` protects that strand's + `pending` queue and `running` flag. The drain-and-erase step in `scheduleNext` + acquires both `{_mapMtx, strand->mtx}` in one `scoped_lock` (deadlock-free via + `std::lock`'s ordering) to close the race where a concurrent `post()` re-arms a + strand between clearing `running` and erasing the map entry — which would + otherwise orphan a live strand and let two strands for one key run + concurrently. The net guarantee: tasks with the same `ModelId` never overlap; + tasks with different keys may run in parallel on the base pool. + +## Failure modes + +| Executor | What happens when a task throws | +|---|---| +| `ThreadPoolExecutor` | The worker `loop` catches it. `std::exception` is logged as `"[thread-pool] task threw: " + what()`; any other type is logged as `"[thread-pool] task threw unknown exception"`. The worker keeps looping. | +| `StrandExecutor` | The strand task wrapper catches it. `std::exception` is logged as `"[strand] task threw: " + what()`; any other type is logged as `"[strand] task threw unknown exception"`. The strand's drain/erase bookkeeping and `_inFlight` decrement still run, so the next task for the key proceeds. | +| `MainThreadExecutor` | `runFor` catches **only** `std::exception`, logged as `"[main-thread] callback threw: " + what()`, then continues with the next task. **Any non-`std::exception` type propagates out of `runFor()`** and is the caller's problem. | + +All logging goes through `morph::log::logError`. The design principle: a task +failure must never kill a worker/strand or abort sibling tasks, but it must also +never be *invisible*. Previously these exceptions were swallowed silently; they +are now logged. `ThreadPoolExecutor` and `StrandExecutor` catch `(...)` and so +contain every exception type; `MainThreadExecutor` deliberately narrows its +`catch` to `std::exception` (a non-standard throw surfaces on the drain thread +rather than being hidden). + +## API reference + +### `IExecutor` + +| Member | Signature | Notes | +|---|---|---| +| `post` | `virtual void post(std::function task) = 0` | Thread-safe. Task runs after the call returns. Per-implementation exception handling (both concrete executors log; see Failure modes). | +| dtor | `virtual ~IExecutor() = default` | | + +### `ThreadPoolExecutor : IExecutor` + +| Member | Signature | Notes | +|---|---|---| +| ctor | `explicit ThreadPoolExecutor(std::size_t n)` | Spawns `n` worker threads. `n` must be > 0. | +| dtor | `~ThreadPoolExecutor() override` | Signals stop, joins all workers. Remaining queued tasks are dropped. | +| `post` | `void post(std::function task) override` | Enqueues to FIFO; notifies one worker. Thread-safe. Task exceptions caught and logged in the worker loop. | + +### `MainThreadExecutor : IExecutor` + +| Member | Signature | Notes | +|---|---|---| +| `post` | `void post(std::function task) override` | Enqueues; notifies waiters. Thread-safe. Not executed until `runFor()`. | +| `runFor` | `void runFor(std::chrono::milliseconds timeout)` | Runs tasks until the `timeout` deadline (blocks for new tasks while time remains; does not return early on an empty queue). Must be called from the owning thread. `std::exception`s logged and skipped; other exception types propagate. | + +### `ModelId` (`morph::exec::detail`) + +| Member | Signature | Notes | +|---|---|---| +| `v` | `uint64_t v{0}` | Raw id. 0 = unbound. | +| `operator<=>` | `auto operator<=>(const ModelId&) const = default` | Three-way comparison. | + +### `ModelIdHash` + +| Member | Signature | Notes | +|---|---|---| +| `operator()` | `std::size_t operator()(ModelId mid) const noexcept` | Hashes `mid.v`. | + +### `StrandExecutor` (`morph::exec::detail`) + +| Member | Signature | Notes | +|---|---|---| +| ctor | `explicit StrandExecutor(IExecutor& base)` | Wraps `base`. | +| dtor | `~StrandExecutor()` | Blocks until `_inFlight == 0`. Requires the base to outlive it and to run every posted task — otherwise deadlocks (see Lifetime & ownership). | +| `post` | `void post(ModelId key, std::function task)` | Enqueues for strand `key`. FIFO per key, concurrent across keys. Thread-safe. Task exceptions caught and logged. Must not race/follow the destructor. | + +## Design decisions + +| Decision | Choice | Why | +|---|---|---| +| Task signature | `std::function` | Simple, universal. Every executor accepts the same callable type. No return value, no cancellation. | +| Exception handling | **Caught and logged, never propagated out of a worker/strand** | A task failure must not crash unrelated tasks *or* vanish. `ThreadPoolExecutor` and `StrandExecutor` catch `(...)` and log via `morph::log::logError`; `MainThreadExecutor` narrows its catch to `std::exception` so a non-standard throw surfaces on the synchronous drain thread. See [Failure modes](#failure-modes). | +| ThreadPoolExecutor drain-on-dtor | **Drop remaining, don't drain** | No way to know which tasks are safe to run during shutdown. The caller must synchronise externally if in-flight tasks matter. | +| MainThreadExecutor's `runFor` | **Wall-clock deadline, not a `runOnce()`** | Lets the caller batch-process tasks without spinning. The condition-variable wait avoids busy-waiting. | +| ModelId zero | **Reserved — "not bound"** | A natural sentinel for optional/uninitialised model handles. | +| StrandExecutor in `detail` | **Not a general-purpose utility** | Exists only for the morph model framework's per-model serialisation. The `ModelId` key is specific to model instances. | +| StrandExecutor destructor | **Waits for in-flight tasks** | Without this, a pool thread running `scheduleNext` can access `_strands` after it has been destroyed (TSan: data race on destructor vs erase). | +| Strand drain-and-erase | **Locks `_mapMtx` and `strand->mtx` atomically** | Prevents a race: a concurrent `post()` re-arms a strand after `running` is cleared but before the map entry is erased, then the erase orphans a live strand, causing two strands for the same key to run concurrently. | +| No `std::future` / return value | **Fire-and-forget only** | Executors schedule side-effect tasks. Callers that need results use shared state or futures externally. | +| No `std::executor` conformance | **Custom interface, not `std::executor`** | C++26 `std::executor` is not yet widely available. This is a minimal in-house abstraction. | + +## Limitations + +These are honest, known gaps — accepted trade-offs, not bugs: + +- **Unbounded queues / no backpressure.** `ThreadPoolExecutor`, `MainThreadExecutor`, + and each `Strand::pending` are all unbounded `std::queue`s. A producer that + outruns consumption grows memory without limit; `post()` never blocks or + rejects. There is no bounded-queue option, no high-water mark, and no way for a + caller to learn the queue is backing up. +- **No cancellation.** Once posted, a task cannot be cancelled or removed. The + signature is fire-and-forget `std::function` with no token, handle, or + future. The only mitigation anywhere is `QtExecutor`, which drops a callable if + its target `QObject` has been deleted — that covers only the GUI leg, not the + thread pool or strands. +- **No graceful drain / `waitIdle` on `ThreadPoolExecutor`.** The destructor drops + queued-but-unstarted tasks, and there is no method to wait until the queue is + empty or to flush pending work before shutdown. Callers who need in-flight work + to finish must synchronise externally. (`StrandExecutor` waits for `_inFlight`, + but that is a lifetime-safety wait, not a general drain API, and it relies on the + base pool still running the strand's dispatched lambdas.) +- **Strand allocation churn.** The self-cleaning map (see + [Lifetime & ownership](#lifetime--ownership)) is good for memory — live entries + track active models — but a bursty model re-allocates a `Strand` every time its + queue empties and refills, instead of reusing one long-lived strand per key. + +## Cross-references + +- [`completion.md`](completion.md) — `Completion` marshals its `.then` / + `.onError` callbacks through an `IExecutor`; the executor is *how* async + results land on the right thread. +- `error_handling.md` — the framework-wide error-propagation story that the + per-task logging here plugs into (currently also summarised under + *Error propagation* in `../ARCHITECTURE.md`). +- `concurrency_and_lifetimes.md` — the broader threading and teardown-ordering + model; the "destroy strand before base pool" rule above is a concrete instance + of it (see also *Thread safety* in `../ARCHITECTURE.md`). +- [`bridge.md`](bridge.md) — the bridge wires backends to a GUI executor and a + strand-backed dispatcher; it is the primary consumer of these types. diff --git a/docs/spec/forms.md b/docs/spec/forms.md new file mode 100644 index 0000000..dc0f071 --- /dev/null +++ b/docs/spec/forms.md @@ -0,0 +1,358 @@ +# `morph::forms` — schema generation & readiness for action types + +Given a plain-aggregate action type `A` (registered with `BRIDGE_REGISTER_ACTION`), +`morph::forms` produces a standard JSON Schema a client can render a form from, +and provides a compile-time `validate()` body that gates submission until every +required empty-capable field is filled in. It builds on glaze's +`write_json_schema` (which already contributes types, `$defs`, per-field +metadata from `glz::json_schema`, and `ExtUnits` from +`morph::units::Quantity`) and closes the gaps glaze leaves open. + +## Contents + +- [Empty state — `EmptyCapableField` concept](#empty-state--emptycapablefield-concept) +- [`Choice` — server-sourced picklist](#choice--server-sourced-picklist) +- [`FixedString` — NTTP compile-time string](#fixedstring--nttp-compile-time-string) +- [`schemaJson()` — schema generation](#schemajsona--schema-generation) +- [Renderer contract: the schema key vocabulary](#renderer-contract-the-schema-key-vocabulary) +- [`allRequiredEngaged()` — readiness check](#allrequiredengageda--readiness-check) +- [Support traits and helpers](#support-traits-and-helpers) +- [API reference](#api-reference) +- [Design decisions](#design-decisions) +- [Failure modes](#failure-modes) +- [Limitations](#limitations) +- [Cross-references](#cross-references) +- [Out of scope](#out-of-scope) + +## Empty state — `EmptyCapableField` concept + +A field type that has an internal blank state (nothing entered / nothing +selected) exposes `hasValue() -> bool`. This is the **only** thing the forms +module needs to know about a field to decide whether it counts as "engaged". + +```cpp +template +concept EmptyCapableField = requires(const T& field) { + { field.hasValue() } -> std::convertible_to; +}; +``` + +Satisfied by: +- `morph::units::Quantity` — `hasValue()` returns `true` when the + `Rational` payload is present. +- `morph::forms::Choice` — `hasValue()` returns `true` when its + `std::optional` is engaged. +- `morph::time::Timestamp` — `hasValue()` returns `true` when its `DateTime` + payload is present. +- Any user type that exposes `bool hasValue() const`. + +A **non**-empty-capable field (plain `int64_t`, `std::string`, …) is always +considered engaged — forms cannot know whether it has been "filled in" without +application-specific logic, so `allRequiredEngaged` simply skips it. + +## `Choice` — server-sourced picklist + +A field whose value is chosen from options served by another registered action. +Options are not hardcoded on the client — they come from executing the named +action over the same wire, and the result rows are mapped to a combo box. + +```cpp +template +struct Choice { + std::optional value; + // ... +}; +``` + +- `T` — the value type submitted on the wire (`int64_t` for ids, `string` for codes). +- `OptionsAction` — the registered action type id whose result provides options + (executed with an empty body, returns `{valueField, labelField, ...}` rows). +- `ValueField` / `LabelField` — which result-row fields carry the submitted value + and the display label; both default to `"id"` / `"name"`. + +On the wire a `Choice` is just its nullable `T` — the options metadata lives in +the C++ type and the generated schema only, never in payloads. The +`glz::meta>` specialisation reflects `value` directly, so glaze +serialises it as `T | null`. + +## `FixedString` — NTTP compile-time string + +A structural type that lets string literals be used as non-type template +parameters (C++20 NTTP): + +```cpp +template +struct FixedString { + std::array data{}; + consteval FixedString(const char (&literal)[N]) noexcept; + constexpr std::string_view view() const noexcept; +}; +``` + +Used by `Choice` to embed the options-action name, value field, and label field +in the type itself. + +## `schemaJson()` — schema generation + +Produces a complete JSON Schema string for action type `A`, post-processing the +output of `glz::write_json_schema()` to add five annotation groups: + +| Annotation | Scope | Contents | +|---|---|---| +| `required` | Top-level | Array of field names that are **not** `std::optional<...>` and not listed in `A::optionalFields`. | +| `x-order` | Every property | The member's declaration index (0‑based), so a renderer lays fields out in declaration order regardless of JSON key ordering. | +| `x-decimalPlaces` | `Quantity` properties | The field's declared precision (`Quantity::declaredDecimals`). | +| `x-unitAlternatives` | `Quantity` properties | Convertible display/entry units derived from `UnitTraits::relations`, each with `{id, display, decimals, num, den}` — `id`/`display`/`decimals` come from the alternative unit's `UnitMeta`, and `num`/`den` are the exact alternative-to-canonical ratio. Omitted entirely when the field's unit declares no convertible units. | +| `x-optionsAction` / `x-optionValue` / `x-optionLabel` | `Choice` properties | The action that serves the options and which result fields to use. | + +The result is **computed once per type and cached** in a `static const std::string` +inside `schemaJson()`. On internal failure (malformed intermediate JSON, +etc.) the unmerged glaze schema is returned — or an empty string when even +glaze's own `write_json_schema()` failed, since `schemaJson` feeds +`mergeSchemaExtras` with `write_json_schema().value_or(std::string{})`. +Schema generation never throws. + +### Required-ness rule + +Required is the default. A member is *optional* (and therefore not added to +`required`) when either: +1. Its type is `std::optional<...>`, or +2. Its name appears in `A::optionalFields` — a `static constexpr` iterable of + `std::string_view` that the action declares: + +```cpp +struct RecordMeasurement { + std::int64_t sampleId = 0; + Density density{}; + Moisture moisture{}; // optional + + static constexpr std::array optionalFields{std::string_view{"moisture"}}; + [[nodiscard]] bool validate() const { return morph::forms::allRequiredEngaged(*this); } +}; +``` + +### `mergeSchemaExtras` — DOM post-processing + +The actual workhorse behind `schemaJson`. It parses the glaze schema into a +`glz::generic_u64` DOM (preserving `int64`/`uint64` bounds in `$defs`), +iterates reflected members via `forEachNamedMember`, and patches the DOM in +place. If the input schema is not valid JSON the raw string passes through +unchanged. + +## Renderer contract: the schema key vocabulary + +This is the **normative** list of every key a renderer must understand to build +a form from a morph action schema. Standard JSON-Schema keywords (`type`, +`properties`, `$defs`, `$ref`, numeric bounds, `description`, …) are emitted by +glaze and behave per the JSON-Schema 2020-12 spec; the table below covers the +keys morph either **synthesises** (`required`, the `x-*` extensions) or **relies +on glaze to stamp** (`format`, `ExtUnits`). A renderer that ignores an `x-*` key +still produces a usable form — it just loses the affordance that key carries +(unit selector, field order, combo box, decimal step). + +| Key | Where | JSON type | Meaning / renderer obligation | +|---|---|---|---| +| `required` | top-level (object) | array of strings | Names of members that must be engaged before submit. A member is listed unless it is a `std::optional<...>` or appears in `A::optionalFields`. Always emitted (an explicit `[]` when nothing is required). The renderer blocks submission until every listed field has a value. | +| `x-order` | every property | non-negative integer | The member's 0-based **declaration index**. Renderers lay fields out in ascending `x-order`, not in JSON key order (object key order is not preserved across DOMs). | +| `x-decimalPlaces` | `Quantity` property | non-negative integer | The field's *declared* precision (`Quantity::declaredDecimals`, unit default unless the type overrides it). The numeric input step / rounding granularity for entry in the canonical unit. | +| `x-unitAlternatives` | `Quantity` property | array of objects | Convertible display/entry units for the field, derived from `UnitTraits::relations`. **Omitted entirely** when the unit declares no convertible peers. Each element has the five subfields below. The renderer offers these as a unit selector and recomputes the entered value *exactly* on switch; the submitted payload is always in the canonical unit (the one named by `ExtUnits`). | +| ↳ `id` | alternative entry | string | Stable ascii id of the alternative unit (`UnitMeta::id`). | +| ↳ `display` | alternative entry | string | Human display text of the alternative unit (`UnitMeta::display`). | +| ↳ `decimals` | alternative entry | non-negative integer | The alternative unit's own default decimals (`UnitMeta::defaultDecimals`) — the input step to use while that unit is selected. | +| ↳ `num` | alternative entry | signed integer | Numerator of the exact **alternative→canonical** ratio. | +| ↳ `den` | alternative entry | signed integer | Denominator of that ratio. `value_in_canonical = value_in_alternative · num / den`; `num`/`den` are the `Rational` numerator/denominator of the composed relation, so the recompute is exact (no floating-point drift). | +| `x-optionsAction` | `Choice` property | string | Type id of the registered action whose result rows populate this field's combo box (executed with an empty body). | +| `x-optionValue` | `Choice` property | string | Which result-row field carries the value submitted on the wire (default `"id"`). | +| `x-optionLabel` | `Choice` property | string | Which result-row field carries the display label (default `"name"`). | +| `format` | `Timestamp` property | string, value `"date-time"` | Standard JSON-Schema vocabulary (stamped by glaze, not by morph). The renderer shows a date-time input; the wire value is the ISO-8601 string `Timestamp` serialises to. No `x-*` extension is used for timestamps. | +| `ExtUnits` | `Quantity` property | object | Glaze-stamped block describing the field's **canonical** unit. Two fields: `unitAscii` (the stable ascii id, e.g. `"kg_per_m3"` — sourced from `UnitMeta::id`) and `unitUnicode` (the human display text, e.g. `"kg/m³"` — from `UnitMeta::display`). This is the unit a payload value is always denominated in, and the reference point the `num`/`den` of every `x-unitAlternatives` entry converts *to*. A renderer needs `ExtUnits.unitAscii`/`unitUnicode` to label the field and to anchor the unit selector. | + +### Versioning stance + +The emitted schema is **unversioned**. There is no `$id`, `$schema` version +marker, or morph-specific version field anywhere in the output — a renderer +cannot detect at runtime which revision of this vocabulary a schema was produced +against. The vocabulary is therefore treated as a stable framework contract: +**changing the semantics of any key above (renaming it, changing its type, or +altering how a value is interpreted) is a breaking change and ships only in a +breaking framework release.** Adding a new, optional `x-*` key that older +renderers can safely ignore is not breaking. + +## `allRequiredEngaged()` — readiness check + +```cpp +template +[[nodiscard]] constexpr bool allRequiredEngaged(const A& action) noexcept; +``` + +Returns `true` when every **required** empty-capable member of `action` has +`hasValue() == true`. Required means: not `std::optional<...>` and not listed +in `A::optionalFields`. Non-empty-capable members (plain ints, strings, etc.) +are skipped — they cannot express "not filled in". Intended as the body of the +action's `validate()` (the `ActionValidator` machinery picks it up +automatically). + +## Support traits and helpers + +| Symbol | Kind | Purpose | +|---|---|---| +| `detail::IsStdOptional` | trait | `true` when `T` is a `std::optional<...>`. | +| `detail::isStdOptional` | variable template | cvref-stripped alias of the trait. | +| `detail::HasOptionalFields` | concept | `true` when `A` has a `static constexpr` iterable `optionalFields`. | +| `detail::declaredOptional(name)` | constexpr function | `true` when `name` appears in `A::optionalFields`. | +| `detail::forEachNamedMember(action, visitor)` | function template | Calls `visitor.operator()(name, member)` for every reflected member of `action` (uses glaze pure reflection). | +| `detail::mergeSchemaExtras(raw)` | function | Post-processes a glaze-generated schema to inject `required`, `x-decimalPlaces`, `x-order`, `x-unitAlternatives`, `x-optionsAction` etc. Called by `schemaJson()`. | + +## API reference + +### `schemaJson()` + +| Signature | Returns | +|---|---| +| `template std::string schemaJson()` | The merged schema JSON. Cached per type. Never throws. On internal failure returns the raw glaze schema, or an empty string if glaze's own schema generation failed. | + +### `allRequiredEngaged()` + +| Signature | Returns | +|---|---| +| `template bool allRequiredEngaged(A const&)` | `true` when every required empty-capable field is engaged. | + +### `EmptyCapableField` concept + +| Signature | Checks | +|---|---| +| `template concept EmptyCapableField` | `const T&` has `.hasValue()` returning convertible-to-`bool`. | + +### `Choice` and `FixedString` + +Both types are **owned by `choice.hpp` and specified in full in +[choice.md](choice.md)** — this spec does not restate their member-by-member API, +to avoid two copies drifting apart. In brief: `Choice` is an optionally-empty value (`std::optional` payload, `hasValue()`, +unchecked `operator*`, defaulted `operator==`) whose options come from executing +a named registered action; `optionsAction()`/`valueField()`/`labelField()` +expose the compile-time metadata that `mergeSchemaExtras` reads to emit +`x-optionsAction`/`x-optionValue`/`x-optionLabel`. `FixedString` is the +`consteval` NTTP string that carries those names inside the `Choice` type. The +`isChoice` trait (`true` for any cvref-stripped `Choice`) is what +`mergeSchemaExtras` and `allRequiredEngaged` branch on. See [choice.md](choice.md) +for the exhaustive tables and design rationale. + +## Design decisions + +| Decision | Choice | Why | +|---|---|---| +| Required default | **All members required unless explicitly opted out** | The safer default for domain forms — forgetting to mark a field optional would leak data, not lose it. Opt out via `std::optional` or `optionalFields` list. | +| Optional mechanism | **Two orthogonal opt-outs** | `std::optional` handles library types (glaze already knows how to serialise them); `optionalFields` handles custom types like `Quantity` whose emptiness is not expressed through `optional`. | +| Schema caching | **`static const std::string` inside the template** | Same schema for the same type in every translation unit. No synchronisation needed — schema generation does not mutate anything. | +| Failure mode | **Returns raw glaze schema (or empty) rather than throwing** | Schema generation is a description facility; crashing a server over a malformed schema would be wrong. | +| `Choice` metadata | **In the type, not the payload** | The set of options for a field is a compile-time property of the action, not a runtime property of each submission. The generated schema communicates it to the client; payloads carry only the selected value. | +| Wire serialisation | **Glaze `meta` reflects `value` directly** | `Choice` serialises as `T \| null` — the options metadata never travels. | +| Options action | **A registered action type id** | The same action dispatch mechanism handles queries for picklist data, so no separate protocol or endpoint is needed. | +| `x-order` | **Always emitted, on every property** | JSON object key order is not reliable across DOM implementations; the explicit index gives renderers a deterministic layout. | +| `x-unitAlternatives` | **Derived from `UnitTraits::relations`** | The same `UnitRelation` entries that drive `convert` also drive the display-unit selector — no separate declaration to keep in sync. | +| `Timestamp` | **Uses standard `"format": "date-time"`** | No extension annotation needed; standard JSON-Schema vocabulary is sufficient. | + +## Failure modes + +### Scope: flat actions only + +Annotation and `required`-derivation operate **exclusively on the action's +top-level members**. `mergeSchemaExtras` reflects `A`'s members with +`forEachNamedMember(probe, …)` and patches `dom["properties"][name]` for each — +it never descends into member types. A member that is itself an aggregate is +emitted by glaze into `$defs` and referenced by `$ref`; the generator does not +recurse into that definition, so its sub-members receive **none** of the `x-*` +annotations and are **not** part of any synthesised `required` array (the nested +`$def` gets no `required` at all). Actions meant to drive a generated form must +therefore be **flat**: every field the renderer should understand has to be a +direct member of the action type. Nesting is not a documented form-generation +path. + +The action type must also be **default-constructible**: `mergeSchemaExtras` +builds a probe instance (`A probe{}`) purely to enumerate member names and types +via reflection. A type with no accessible default constructor will not compile +`schemaJson()`. + +### Total schema failure yields an empty string + +`schemaJson()` calls `mergeSchemaExtras(glz::write_json_schema().value_or(std::string{}))`. +When glaze's own schema writer fails, `value_or` hands `mergeSchemaExtras` an +**empty string**; `read_json` then fails on it and the function returns that +same empty string. So a total failure surfaces as `""` — **not** valid JSON, not +an empty JSON object `{}`. Renderers must treat an empty string as "schema +unavailable" and refuse to build a form from it. Because the fallback path +carries no diagnostic, an empty result is **indistinguishable** from any other +failure mode (there is no error code, message, or partial schema to inspect). +Schema generation never throws, so the empty string is the only failure signal a +caller receives. + +## Limitations + +### Security / trust boundary + +`required`-ness and `allRequiredEngaged` gate the **client only**. The morph +dispatcher runs **no server-side validators** before invoking a handler — it +does not consult the schema's `required` array and does not call `validate()`. +Consequently a hand-crafted wire payload that omits or blanks a "required" field +is accepted by the wire/dispatch layer and reaches the handler unmodified: the +`required` array and `allRequiredEngaged` are **UX affordances, not a security or +integrity boundary**. Model authors must therefore **re-check required +quantities inside the handler**. The `examples/forms` model does exactly this — +its `execute(RecordMeasurement)` calls `action.validate()` (the same +`allRequiredEngaged` predicate the client gates on) and throws +`std::invalid_argument` if it fails, so schema, form, and server agree by +construction rather than by trust. See [security.md](security.md) for the +dispatcher's validation stance. + +### One cached schema per type — no localisation + +Each type's schema is memoised in a function-local `static const std::string` +(`schemaJson()`), computed by the first caller and shared process-wide +thereafter (first-caller-wins; no synchronisation, since generation mutates +nothing). This baking-in **precludes localised / i18n schemas**: the human +display strings that land in the schema (unit `display`/`unitUnicode`, and any +`description` text) are fixed at first call. There is no per-request or +per-locale schema variant — a translated form would need a different mechanism +entirely. + +### Load-bearing assumptions about glaze's schema shape + +The generator is coupled to concrete details of glaze's schema **output shape**, +not just its public API: + +- **A top-level `properties` object.** `mergeSchemaExtras` indexes + `dom["properties"][name]` directly. If glaze stopped emitting a `properties` + object at the root (or nested it), the merge would create the wrong structure. +- **`$defs` numeric-bound preservation via `generic_u64`.** The DOM is parsed in + u64 number mode specifically because `$defs` carries `int64`/`uint64` bounds + that the default double-only DOM would silently round. This depends on glaze + emitting those bounds as integers in a place the round-trip preserves. +- **Reflection key order feeding `x-order`.** `x-order` is the index `I` from + `glz::reflect::keys`, and it is trusted to equal source **declaration + order**. If glaze's reflection reordered keys, `x-order` would misdescribe the + layout while still looking well-formed. + +A change to any of these in glaze could make the generator silently produce a +wrong or un-merged schema rather than fail loudly. + +## Cross-references + +| Spec | Why | +|---|---| +| [choice.md](choice.md) | Full `Choice` / `FixedString` API and design (this spec cross-refs rather than duplicates them). | +| [quantity_type.md](quantity_type.md) | `Quantity`, its unit tags, `UnitTraits::relations`, and `convert` — the source of `x-decimalPlaces`, `x-unitAlternatives`, and `ExtUnits`. | +| [datetime.md](datetime.md) | `DateTime` / `Timestamp`, the ISO-8601 wire format, and the `"format": "date-time"` schema annotation. | +| [rational.md](rational.md) | Exact `Rational` values; the `num`/`den` in each `x-unitAlternatives` entry are a `Rational` numerator/denominator, which is why unit switches recompute exactly. | +| [security.md](security.md) | The dispatcher's trust boundary — why `required` gates only the client and handlers must re-validate. | + +## Out of scope + +- Generating schemas for non‑action types (the module assumes glaze reflection is + available on `A`). +- Validating payloads against the schema — the schema is for the *client*. +- Executing the options action — `choices` metadata tells the client *which* + action to call, but the forms module does not invoke it. +- `morph::time::Timestamp` definition — it is only consumed here via + `EmptyCapableField`. \ No newline at end of file diff --git a/docs/spec/journal.md b/docs/spec/journal.md new file mode 100644 index 0000000..bf81dba --- /dev/null +++ b/docs/spec/journal.md @@ -0,0 +1,438 @@ +# The `journal` subsystem — design + +`morph::journal` is a durable, append-only record of every action executed +against a model instance. It is the audit trail and the raw material for +state reconstruction — the journal, not the live model, is the source of truth +for "what happened." + +Three concerns live here, spread across two headers (`action_log.hpp` and `journal.hpp`): + +1. **The log entry format** — `LogEntry`, a flat struct of what one action + execution produced, plus `toJson`/`fromJson` for wire/file encoding. +2. **The storage interface** — `IActionLog`, plus three implementations: + `InMemoryActionLog` (`action_log.hpp`), `FileActionLog` (`file_action_log.hpp`), + and `SessionLog` (`journal.hpp`). +3. **The process-wide default** — `setActionLog`, `defaultActionLog`, + `ScopedActionLog` (all in `action_log.hpp`), and how model instances + auto-attach to it. `replay()` and `SessionLog::undoLast()` live in + `journal.hpp` and depend on `ModelRegistryFactory`/`ActionDispatcher`. + +## Contents + +- [LogEntry — one recorded action execution](#logentry--one-recorded-action-execution) +- [Serialization](#serialization) +- [IActionLog — the storage interface](#iactionlog--the-storage-interface) +- [InMemoryActionLog](#inmemoryactionlog) +- [FileActionLog](#fileactionlog) +- [SessionLog](#sessionlog) +- [replay()](#replay) +- [Process-wide default log](#process-wide-default-log) +- [ScopedActionLog](#scopedactionlog) +- [API reference](#api-reference) +- [Design decisions](#design-decisions) +- [Invariants](#invariants) +- [Failure modes / durability](#failure-modes--durability) +- [Limitations](#limitations) +- [Cross-references](#cross-references) + +## LogEntry — one recorded action execution + +`LogEntry` is produced automatically by +`morph::model::detail::IModelHolder::recordIfAttached` after every successful +loggable action. Application and model code never construct or append these +directly. + +| Field | Type | Meaning | +|---|---|---| +| `seq` | `uint64_t` | Monotonic order assigned by the sink on `append()`. Callers pass `0`. | +| `modelType` | `std::string` | String type-id of the model (`ModelTraits::typeId()`). | +| `entityKey` | `std::string` | Stable identity of the model instance (e.g. account id), stamped from `attachActionLog()`. Empty if none was set. | +| `actionType` | `std::string` | String type-id of the action (`ActionTraits::typeId()`). | +| `payload` | `std::string` | JSON-encoded request (`ActionTraits::toJson`). | +| `result` | `std::string` | JSON-encoded result (`ActionTraits::resultToJson`), captured after successful execution. | +| `principal` | `std::string` | Auth principal from `morph::session::current()`, if any. Empty if unset. | +| `timestampMs` | `int64_t` | Wall-clock time, milliseconds since the Unix epoch. | + +`LogEntry` is a plain aggregate — Glaze reflects it without a `glz::meta` +specialisation, the same automatic reflection `BRIDGE_REGISTER_ACTION` relies on. + +## Serialization + +### `toJson(LogEntry const&) -> std::string` + +Encodes a `LogEntry` as JSON via Glaze. Throws `SerializationError` on failure +(not realistically reachable for a flat struct of strings/integers — see +`detail::throwOnGlazeError`). + +### `fromJson(std::string_view) -> LogEntry` + +Decodes JSON into a `LogEntry`. Throws `SerializationError` if the input is not +valid. + +### `SerializationError` + +`struct SerializationError : std::runtime_error` — thrown by `toJson`/`fromJson` +when (de)serialisation fails. Inherits `std::runtime_error`'s constructors. + +### `detail::throwOnGlazeError` + +Shared non-template helper used by both `toJson` and `fromJson` so their error +paths compile through the same branch. `fromJson`'s failure path is easy to +exercise (malformed JSON is everyday input); `toJson`'s is not — Glaze's +buffer-writer has no reachable failure mode for a flat struct like `LogEntry`. + +## IActionLog — the storage interface + +A pure-virtual interface for durable, append-only storage of action entries. +Entries are never removed by the framework — this is a permanent record, unlike +`morph::offline::IOfflineQueue` (whose `markDone()` deletes items once retried). + +| Method | Signature | Purpose | +|---|---|---| +| `append` | `virtual void append(LogEntry)` | Appends an entry. Implementations assign `entry.seq`. | +| `flush` | `virtual void flush()` | Pushes buffered entries to the durable backend. No-op for sinks with nothing to buffer. | +| `entries` | `[[nodiscard]] virtual std::vector entries(std::string_view entityKey = {}) const` | Returns recorded entries in append order, optionally filtered by `entityKey`. | + +## InMemoryActionLog + +Thread-safe in-memory implementation of `IActionLog`. Suitable for testing and +applications that do not need cross-process durability. Mirrors +`morph::offline::InMemoryOfflineQueue`'s shape. + +- `append`: assigns a monotonically increasing `seq`, pushes to an internal + vector under a mutex. +- `flush`: no-op. +- `entries`: returns a snapshot under a mutex; filters by `entityKey` if + non-empty. + +## FileActionLog + +Append-only, newline-delimited-JSON `IActionLog` backed by a local file. Each +entry is one `toJson`-encoded line. `flush()` flushes the C stdio buffer and +then issues a real `fsync` (POSIX `fsync` / Windows `_commit`), so a crash +immediately after `flush()` returns cannot lose data. + +Open (creating if necessary) via `FileActionLog(std::filesystem::path)`. +Throws `std::runtime_error` if the file cannot be opened. Closes the file in +the destructor. Copy and move are deleted. + +**Process-local `seq`.** `seq` is assigned fresh per process instance — it does +not resume from the highest `seq` already on disk. Entries remain correctly +ordered on disk (append-only), but `seq` alone is not a cross-restart unique +key; use `entries()`' natural file order for that. + +**Thread safety.** All public methods are thread-safe (guarded by an internal +mutex). Safe from multiple threads within one process; not safe for multiple +processes to append to the same path concurrently. + +`entries()` re-reads the file from disk and decodes every line. Reads whatever +is currently on disk, including anything written but not yet `flush()`ed if the +platform's stdio buffering has already handed it to the OS. `flush()` first for +a guaranteed-durable view. Blank lines are skipped before decoding. + +**Torn-write tolerance.** A crash between `append()`'s `fwrite` and the next +`flush()` can leave a truncated final line (bytes written, never completed). +`entries()` tolerates exactly that case: if decoding fails on the **last** +non-empty line, it skips that line, logs a warning via `morph::log::logWarn` +(naming the path and the parse error), and returns everything before it. A +decode failure on any line **mid-file** is treated as genuine corruption and +the `fromJson` exception is re-thrown — the log is not silently truncated at an +interior tear. So a single trailing torn record is recoverable; interior +damage is fatal and surfaced to the caller. + +## SessionLog + +Full-fidelity, in-memory log of one model instance's executed actions. Attach +directly via `IModelHolder::attachActionLog` — every successfully executed +loggable action is appended here in order, regardless of any +`ActionLogPolicy::coalesce` setting. This full history is the raw material for +`undoLast()`. + +`checkpoint()` is where coalescing happens: entries accumulated since the last +checkpoint are reduced by `(modelType, entityKey, actionType)` — keeping only +the latest occurrence where the action's policy says `coalesce == true`, keeping +every occurrence otherwise — and only that reduced set reaches the durable sink. + +All public methods are thread-safe (guarded by an internal mutex). + +| Method | Signature | Purpose | +|---|---|---| +| `append` | `void append(LogEntry)` | Always full fidelity — nothing is coalesced or dropped here. | +| `flush` | `void flush()` | No-op — `checkpoint()` is the real commit point. | +| `entries` | `std::vector entries(std::string_view entityKey = {})` | Full history (or one entity's slice) in append order. | +| `undoLast` | `std::unique_ptr undoLast(modelTypeId, registry, dispatcher)` | Drops the most recent entry and replays the remainder against a **fresh, detached** model instance, reconstructing pre-undo state. Returns that new holder — the caller must install/use it (it does not mutate any live instance). No-op (returns a freshly created, un-replayed holder) if the log is empty. | +| `checkpoint` | `void checkpoint(IActionLog& durableSink, ActionDispatcher& dispatcher = defaultDispatcher())` | Coalesces entries since the last checkpoint and forwards the reduced set to `durableSink`; then flushes it. Advances the internal checkpoint watermark to the current tail *before* forwarding anything, so it stays advanced regardless of whether the subsequent `durableSink.append()`/`durableSink.flush()` throws — this makes the batch **at-most-once / forward-only** (a throwing sink drops it permanently), NOT the at-least-once its `IOfflineQueue` shape might suggest. See [Failure modes / durability](#failure-modes--durability). No-op if nothing has been appended since the last checkpoint. | + +### `undoLast()` + +No inverse/undo operations are needed on the action types themselves — this +reuses `replay()` over a shorter prefix of the same log. Takes optional +`modelTypeId`, `registry`, and `dispatcher` — `registry` and `dispatcher` +default to the process-level singletons. Dropping the last entry also clamps +the checkpoint watermark down to the new tail, so a previously-committed final +entry does not leave the watermark past the end of the (now shorter) history. +Because the durable sink is append-only, `undoLast()` only rewinds `SessionLog`'s +own in-memory history — it cannot remove an entry already forwarded to a +durable sink by a prior `checkpoint()`. + +`undoLast()` returns a **fresh, detached** holder (created by `replay()`, which +immediately detaches the auto-attached default log — see [replay()](#replay)), +not a mutation of any live model instance. The caller is responsible for +installing or otherwise using the returned holder; the pre-undo instance, if +any, is left untouched. Like `replay()`, `undoLast()` assumes the entries it +walks all belong to one model instance — a `SessionLog` is intended to be +attached per instance, so this holds by construction; if a single `SessionLog` +is ever shared across instances, filter by `entityKey`/model type before +reconstructing (see [Invariants](#invariants)). + +### `checkpoint()` + +Coalescing is `(modelType, entityKey, actionType)`-keyed: the latest occurrence +overwrites earlier ones (in place, preserving first-seen position) wherever the +dispatcher says the action coalesces (unknown/unregistered `(modelType, actionType)` +pairs default to *not* coalescing, so every such entry is kept); every other +entry is kept as-is. The checkpoint watermark is advanced to the current tail +*before* any entry is forwarded, so it stays advanced regardless of whether +`durableSink.append()` or `durableSink.flush()` then throws. + +**This is at-most-once, forward-only — not the at-least-once its `IOfflineQueue` +shape suggests.** Because the watermark advances *before* forwarding, a +`durableSink` that throws on `append()`/`flush()` causes that batch to be +**dropped permanently**: the next `checkpoint()` starts from the already-advanced +watermark and never re-sends it. A checkpoint is a forward-only commit point, +not a transaction that will be retried. If a durable sink can fail transiently, +the caller must treat a throwing `checkpoint()` as data loss for that batch, not +as a retryable no-op. See [Failure modes / durability](#failure-modes--durability). + +> Note: `journal.hpp`'s doxygen still describes this as "at-least-once"; that +> phrasing is imprecise for the actual watermark-advances-first behavior. This +> spec is the authoritative description. + +## replay() + +Reconstructs model state by replaying entries, in order, against a freshly +created model instance. Builds on the same `ModelRegistryFactory` / +`ActionDispatcher` machinery `RemoteServer` uses — no separate replay engine. + +```cpp +std::unique_ptr replay( + std::string_view modelTypeId, + const std::vector& entries, + ModelRegistryFactory& registry = defaultRegistry(), + ActionDispatcher& dispatcher = defaultDispatcher()); +``` + +Throws `std::runtime_error` if `modelTypeId` or any entry's action type is +unregistered. + +**Reconstruction does not pollute the live audit trail.** `registry.create(...)` +(via `ModelFactory::create`) auto-attaches the process-wide default action log +to the new holder, exactly as for any ordinary model instance. `replay()` +therefore calls `holder->attachActionLog(nullptr, {})` *immediately* after +creating the holder and *before* dispatching any entry, detaching that default. +Without this detach, each replayed dispatch would re-record into the live sink — +corrupting the very audit trail being read from (and, since replay re-runs the +recorded actions, doubling every entry on every reconstruction). As a result, +both `replay()` and `SessionLog::undoLast()` reconstruct state in isolation: +the replayed actions are **not** written back into `defaultActionLog()`. Undo +and reconstruction are read-only with respect to the audit trail. + +**Single-instance precondition.** `replay()` creates *one* holder of +`modelTypeId` and dispatches every entry against it in order. The caller is +responsible for passing entries that all belong to that one model instance — +typically obtained by filtering a log with `entries(entityKey)` and by matching +`modelType`. Mixing entries from several instances (or several model types) into +one `replay()` call replays them all onto a single object and produces a +meaningless state. This is a precondition, not something `replay()` validates. + +## Process-wide default log + +Every model instance created via `ModelFactory::create()` — every model +registered the ordinary way, whether the active backend is local or remote — +automatically gets the default log attached (with an empty `entityKey`) from +that point on. + +### `setActionLog(std::shared_ptr)` + +Installs a log as the process-wide default. Pass `nullptr` to stop +auto-attaching (existing instances keep whatever they already have). Thread-safe. + +### `defaultActionLog() -> std::shared_ptr` + +Returns the currently installed default, or `nullptr` if none has been set. +Thread-safe. + +The state is a function-local static (`detail::defaultActionLogState()`), not a +namespace-scope global, so it is safe regardless of translation-unit init order. + +## ScopedActionLog + +RAII helper that installs a default action log for its lifetime and restores the +previous one on destruction. Mirrors `morph::log::ScopedLoggerOverride`. Intended +for tests and for temporarily redirecting auto-attached logging. + +```cpp +{ + morph::journal::ScopedActionLog guard{std::make_shared()}; + // models created here auto-attach guard's log +} // previous default restored +``` + +Copy and move are deleted. + +## API reference + +All symbols live in `namespace morph::journal`. + +### Log entry and serialization + +| Symbol | Kind | Signature / Notes | +|---|---|---| +| `LogEntry` | struct | Flat aggregate: `seq`, `modelType`, `entityKey`, `actionType`, `payload`, `result`, `principal`, `timestampMs`. Glaze-reflected (no `glz::meta`). | +| `toJson` | free function | `std::string toJson(const LogEntry&)` — encodes as JSON. Throws `SerializationError`. | +| `fromJson` | free function | `LogEntry fromJson(std::string_view)` — decodes from JSON. Throws `SerializationError`. | +| `SerializationError` | struct | `: std::runtime_error`. Thrown by `toJson`/`fromJson`. | +| `detail::throwOnGlazeError` | inline function | `void throwOnGlazeError(const glz::error_ctx&, std::string_view)` — shared error path for `toJson`/`fromJson`. | + +### IActionLog and implementations + +| Symbol | Kind | Notes | +|---|---|---| +| `IActionLog` | abstract struct | `virtual ~IActionLog() = default`; `append(LogEntry)`, `flush()`, `entries(entityKey)`. | +| `InMemoryActionLog` | class | `: IActionLog`. Thread-safe `std::vector`-backed. `flush()` no-op. | +| `FileActionLog` | class | `: IActionLog`. Newline-delimited JSON, fsync on `flush()`. `explicit FileActionLog(std::filesystem::path)`. Copy/move deleted. | +| `SessionLog` | class | `: IActionLog`. Full-fidelity in-memory log + `undoLast()` + `checkpoint()`. | + +### Process-wide default + +| Symbol | Kind | Notes | +|---|---|---| +| `setActionLog` | free function | `void setActionLog(std::shared_ptr)`. Thread-safe. | +| `defaultActionLog` | free function | `[[nodiscard]] std::shared_ptr defaultActionLog()`. Thread-safe. | +| `ScopedActionLog` | class | RAII: saves previous default, restores on destruction. `explicit ScopedActionLog(std::shared_ptr)`. Copy/move deleted. | + +### State reconstruction + +| Symbol | Kind | Notes | +|---|---|---| +| `replay` | free function | `std::unique_ptr replay(modelTypeId, entries, registry, dispatcher)`. | + +## Design decisions + +| Decision | Choice | Why | +|---|---|---| +| `LogEntry` is a plain aggregate | **No `glz::meta`** | Same automatic reflection `BRIDGE_REGISTER_ACTION` uses; no manual schema maintenance. | +| Error path sharing | **`detail::throwOnGlazeError` for both `toJson`/`fromJson`** | `fromJson`'s failure is easy to test (malformed input); `toJson`'s is structurally unreachable for `LogEntry`. Routing both through one non-template function means the same compiled branch covers both, so `toJson`'s error path is exercised by `fromJson`'s tests. | +| Entries are never removed | **Append-only, no deletion API** | Permanent audit trail — unlike `IOfflineQueue` whose `markDone()` deletes retried items. | +| Default log is a function-local static | **`detail::defaultActionLogState()` returns a `pair`** | Safe regardless of translation-unit init order, unlike a namespace-scope global. | +| `SessionLog::checkpoint` advances the watermark *before* forwarding | **At-most-once / forward-only** | A checkpoint is a forward-only commit point, not a transaction to retry: the watermark advances first, so a throwing durable sink drops that batch permanently. (`IOfflineQueue`'s retry semantics do *not* carry over — the shared shape is superficial.) | +| `SessionLog::undoLast` uses `replay()` | **No inverse operations on actions** | Replays the shorter prefix against a fresh model — no per-action undo logic needed. | +| `FileActionLog::seq` is process-local | **Fresh per process, not resumed from disk** | `seq` is a monotonic order key within one process instance, not a cross-restart durable identifier. On-disk order is append order; `entries()` returns in that order regardless of `seq` gaps. | +| `FileActionLog` uses C stdio + `fsync` | **`fopen`/`fwrite`/`fflush`/`fsync`** | `fwrite` is buffered; `flush()` calls `fflush` then `fsync` (or `_commit` on Windows) for real durability. POSIX `write`/`fsync` would bypass stdio buffering entirely; C stdio gives buffering by default with explicit flush control. | +| `FileActionLog::entries` tolerates a torn trailing line | **Skip + warn on the last line only; re-throw mid-file** | A crash between `append`'s `fwrite` and the next flush can truncate the final line. Skipping it keeps the log readable after a crash; re-throwing on interior damage refuses to silently hide real corruption. | + +## Invariants + +These hold for every sink and are relied on by `replay()`/`undoLast()`: + +- **Only successful, loggable actions are recorded.** A `LogEntry` is produced + by `IModelHolder::recordIfAttached` *after* an action executes successfully. + Actions that throw (business-rule failures), drafts rejected by a validator + (`ActionValidator::validate`), and any action registered `Loggable::No` + (typically pure queries like `GetAccount`/`ListAccounts`) never appear in the + log. The log is a record of committed facts, not of attempts. +- **`result` reflects post-execution state.** `payload` is the request JSON; + `result` is captured only after success, so replaying `payload` re-derives an + equivalent `result` for a deterministic model. +- **`seq` is sink-local and re-stamped on every forward.** Each sink's + `append()` overwrites `entry.seq` with its own `++_nextSeq`, ignoring any + incoming value. When `SessionLog::checkpoint()` forwards entries to a durable + sink, that sink re-stamps them again. `seq` is therefore an ordering key + *within one sink instance in one process run* — it is **not** a stable, + cross-sink or cross-restart identifier. Use `entries()`' natural append order + for identity/ordering across sinks; do not persist or compare raw `seq` + values as keys. (`FileActionLog::seq` is likewise fresh per process — it does + not resume from the highest `seq` on disk.) +- **Reconstruction is single-instance.** `replay()` and `undoLast()` expect + entries already filtered to a single model instance — filter by `entityKey` + (via `entries(entityKey)`) and by `modelType` first. Feeding mixed instances + into one `replay()` call is undefined at the domain level (all entries dispatch + onto one holder). +- **`undoLast()` yields a detached holder the caller must install.** It does not + mutate a live instance; the reconstructed pre-undo state lives only in the + returned holder (whose default log is detached, so using it records nothing + into the live trail). + +## Failure modes / durability + +The durability contract is narrower than the "durable audit trail" framing +alone implies. Understand these before relying on the log for recovery: + +- **`checkpoint()` is at-most-once / forward-only.** The watermark advances to + the current tail *before* any entry is forwarded to the durable sink. If the + sink's `append()` or `flush()` throws, the batch is **lost permanently** — the + next `checkpoint()` resumes past it and never retries. Despite sharing the + `IOfflineQueue` shape, this is the opposite of `IOfflineQueue`'s at-least-once + retry-until-`markDone` behavior. Callers that need at-least-once must layer + their own retry around a sink whose `append`/`flush` are idempotent, or treat + a throwing `checkpoint()` as an explicit data-loss event. +- **Entries are durable only after `checkpoint()` *and* `flush()`.** A + `SessionLog`'s history is pure in-memory until `checkpoint()` forwards it to a + durable sink and that sink's `flush()` returns. For `FileActionLog`, `flush()` + is what fsyncs; before it, entries may sit in stdio/OS buffers. A crash before + `checkpoint()` loses the entire uncheckpointed session's history. +- **No transactional link to the model's own store.** The log and a model's own + durable store (e.g. the SQLite in `examples/bank`) commit as two independent + steps. A crash can leave the store committed but the log missing the + corresponding entries (uncheckpointed), or — with a separately-flushed + file — the log ahead of the store. There is no outbox tying the two into one + atomic write; recovery must reconcile them out of band. +- **`FileActionLog` torn-write recovery is trailing-only.** A single truncated + *final* line (from a crash mid-`append`) is skipped with a warning; any + malformed *interior* line makes `entries()` throw. So the file self-heals from + the one crash shape it is designed for, and refuses to silently drop data for + any other. +- **Single-writer file assumption.** `FileActionLog` is thread-safe within one + process but not safe for concurrent appenders across processes on the same + path; interleaved writes from two processes can corrupt lines. + +## Limitations + +Honest boundaries of the current design: + +- **Replay re-executes actions.** `replay()`/`undoLast()` reconstruct state by + *re-running* each recorded action through the dispatcher — not by replaying a + captured state diff. For a model whose actions have external side effects + (SQL writes, network calls, RNG, clock reads), replaying re-triggers those + side effects. Undo and reconstruction are therefore **exact only for pure, + deterministic, in-memory models**. A model backed by an external store will, + on replay, attempt to re-apply its writes. +- **No transactional outbox.** As above, nothing ties a log entry to the + commit of the model's own store. Divergence between the two is possible and + is the application's problem to detect and reconcile. +- **Unbounded in-memory growth; O(n²) repeated undo.** `SessionLog` retains full + uncoalesced history for the lifetime of the instance — memory grows without + bound. Each `undoLast()` replays the entire remaining prefix from scratch, so + undoing the last *k* actions one at a time is O(n·k) ≈ O(n²) in the history + length. There is no incremental/snapshot fast-path. +- **No schema version or migration path for persisted NDJSON.** `FileActionLog` + writes `LogEntry` as Glaze-reflected JSON with no embedded schema version. A + change to `LogEntry`'s shape has no defined migration story for files written + by an earlier build; on-disk compatibility rests entirely on the struct + staying additive-and-compatible under Glaze. + +## Cross-references + +- **`registry.md`** — `ModelRegistryFactory`/`ActionDispatcher` and + `ModelFactory::create`, which auto-attach the default log and which `replay()` + reuses for dispatch. Also the `ActionDispatcher::coalesce` lookup driving + `checkpoint()`. +- **`bridge.md`** — the two (mutually exclusive) recording call sites + (`Bridge::executeVia`'s `localOp` for local mode; the `RemoteServer` dispatch + path for remote/Qt), and `HandlerBinding::contextKey`/`RemoteServer::setLogProvider` + for per-instance identity. +- **`backend.md`** — how local vs. remote topology decides which recording site + is live, and why recording is automatically server-side wherever a client/server + split exists. +- **`error_handling.md`** — `SerializationError` and the failure/validator-rejection + paths that explain *why* unsuccessful actions never reach the log. \ No newline at end of file diff --git a/docs/spec/logger.md b/docs/spec/logger.md new file mode 100644 index 0000000..6c8fb8f --- /dev/null +++ b/docs/spec/logger.md @@ -0,0 +1,293 @@ +# The `morph::log` logging system — design + +`morph::log` is a lightweight, thread-safe logging facility built on C++23 +`` and ``. It provides a global log sink, runtime level +filtering, and a RAII guard for test isolation. + +There is no logger object to construct — the API is a set of namespace-scope +free functions operating on a hidden `LogState` singleton. + +## Contents + +- [Log levels](#log-levels) +- [Global state](#global-state) +- [Level helpers](#level-helpers) +- [Scoped override](#scoped-override) +- [Thread safety](#thread-safety) +- [Failure modes](#failure-modes) +- [Lifetime](#lifetime) +- [API reference](#api-reference) +- [Design decisions](#design-decisions) +- [Limitations](#limitations) +- [Cross-references](#cross-references) + +## Log levels + +Five severity levels, ordered from most to least verbose: + +| Enumerator | Value | Meaning | +|---|---|---| +| `debug` | 0 | Fine-grained diagnostic output | +| `info` | 1 | General informational messages | +| `warn` | 2 | Recoverable conditions worth noting | +| `error` | 3 | Errors that should be investigated | +| `off` | 4 | Suppresses all output when set as minimum level | + +The ordering is used for runtime filtering: a message is emitted only when +the sink is non-null **and** `level >= logState().minLevel` (the `uint8_t` +value comparison). Setting `minLevel` to `off` silences everything, and +installing a null sink (`setLogger(nullptr)`) suppresses all output without +crashing. + +The level comparison is the hot path and is deliberately made cheap: `minLevel` +is a `std::atomic`, so the reject decision is a single relaxed atomic +load with **no mutex acquisition** (see [Thread safety](#thread-safety)). + +## Global state + +A single `detail::LogState` singleton holds: + +- **`sink`** — a `std::function` called for + every message that passes the level filter. Default: writes `[LEVEL] msg` to + `stderr`. Guarded by `mtx`. +- **`minLevel`** — the minimum `LogLevel` to emit, as a + `std::atomic`. Default: `debug` (everything passes). Read/written + lock-free with relaxed ordering; **not** guarded by `mtx`. +- **`mtx`** — a `std::mutex` protecting `sink` and serialising sink invocation. + +The two fields have different concurrency disciplines: + +- **`minLevel` is lock-free.** `setLogLevel` is a relaxed atomic `store` and + `getLogLevel` is a relaxed atomic `load` — neither touches `mtx`. This makes + the level a cheap, contention-free fast path that both `detail::log` and + `detail::logFormat` consult before doing any expensive work. +- **`sink` is mutex-guarded.** `setLogger` acquires `mtx` to swap the sink, and + `detail::log` holds `mtx` for the duration of the sink call, so sink + invocations are serialised with respect to each other and to `setLogger` / + `ScopedLoggerOverride`. + +The relaxed memory order is intentional: level filtering is advisory (a message +racing a concurrent `setLogLevel` may be emitted or dropped depending on +timing), so no cross-thread happens-before relationship on the level is needed, +and relaxed is the cheapest correct choice. + +## Level helpers + +Eight public free functions — four taking a plain `std::string_view`, and four +overloaded on `std::format_string` + variadic args — each hardcoding one level: + +```cpp +logDebug("message"); // plain string +logInfo("count: {}", n); // format string +logWarn("timeout after {}ms", ms); +logError("{}: {}", err.what(), code); +``` + +The format variants delegate to `detail::logFormat`, which checks the level +against `minLevel` **before** calling `std::format` and returns early if the +message is suppressed — so a call like `logDebug("… {} …", expensiveArgs)` +under a higher `minLevel` pays no `std::format` and no string-allocation cost; +only the (already-evaluated) argument expressions and the atomic load are paid. +When the level passes, `logFormat` formats and forwards to `detail::log`, which +re-checks the level (again lock-free) before taking the mutex. + +The plain (`std::string_view`) variants call `detail::log` directly, which +performs the same lock-free level reject before locking. + +## Scoped override + +`ScopedLoggerOverride` is a RAII guard that swaps the global sink and level +for the lifetime of the object, restoring them in the destructor. Designed for +tests that want to capture log output without leaking the custom sink into +other tests. + +Two constructors: + +- **Default** — snapshots the current sink and level without changing them. + Use when test code will install its own sink mid-test via `setLogger()` / + `setLogLevel()` and just wants automatic restoration. +- **Explicit** — takes a new sink and optional level (defaults to `debug`). + Installs them immediately; restores the previous values on destruction. + +Copy and move are deleted — the guard is not copyable. + +## Thread safety + +The design splits work between a lock-free fast path and a mutex-guarded slow +path: + +- **Level check — lock-free.** Both `detail::log` and `detail::logFormat` + compare `level` against `minLevel.load(relaxed)` and bail out before touching + the mutex or doing any formatting. `setLogLevel` / `getLogLevel` are a relaxed + atomic `store` / `load`. Reading or changing the level never contends with + logging on another thread. +- **Sink invocation — serialised under `mtx`.** When a message passes the + level check, `detail::log` takes `state.mtx` and calls the sink **while + holding the lock**. This means at most one sink call runs at a time, and it + cannot race a concurrent `setLogger` or a `ScopedLoggerOverride` + construction/destruction (all of which also take `mtx`). The default + `stderr` sink is independently thread-safe, but serialising also keeps + interleaved messages from arbitrary user sinks intact. + +Because the sink runs under the global, **non-recursive** `std::mutex`, a sink +**MUST NOT** call back into `morph::log` from within its own invocation: + +- calling any `logDebug`/`logInfo`/`logWarn`/`logError` (or `detail::log` + directly) — the nested `detail::log` would try to re-lock `mtx`; +- calling `setLogger` / `setLogLevel`-via-sink paths that lock `mtx`; +- constructing or destroying a `ScopedLoggerOverride`. + +Any of these re-enters the held mutex and **self-deadlocks** (`std::mutex` is +not recursive — re-locking from the same thread is undefined behaviour, in +practice a hang). Note `setLogLevel` / `getLogLevel` alone are safe to call +from a sink because they do not take `mtx` — but there is no legitimate reason +to, and doing so is still poor form. + +A sink should also **not block for long**: it stalls every other thread trying +to log (they queue on `mtx`) for the full duration of the call. Do expensive or +blocking work (network, disk with fsync, cross-thread handoff) off the logging +thread. + +## Failure modes + +- **A single string argument is never a format string.** `logInfo(s)` for a + `std::string`/`std::string_view`/`const char*` `s` binds the plain + `std::string_view` overload, not the `std::format_string` template. Any `{}` + in `s` is therefore emitted **literally** and cannot throw + `std::format_error` — the text is passed straight through to the sink. Use + the variadic overload (`logInfo("{}", s)`) only when you actually want `s` + treated as a format string with arguments. This is why untrusted or + brace-containing strings are safe to pass as the sole argument. +- **A throwing sink propagates.** Exceptions from the sink are **not** caught. + They unwind out of `detail::log` (releasing `mtx` via the `scoped_lock` + destructor as the stack unwinds) and out of the originating + `logDebug`/`logInfo`/… call into the caller. The logging layer adds no + `try/catch`; a sink that must not disrupt its caller has to swallow its own + exceptions internally. +- **Format-time errors (variadic overload).** Argument formatting is checked at + compile time by `std::format_string`, so mismatched placeholders are a build + error, not a runtime one. A runtime `std::format_error` is only possible from + a dynamically-constructed format string, which this API does not accept. + +## Lifetime + +`detail::logState()` returns a reference to a function-local +`static LogState` — a Meyers singleton. First-use initialisation is +thread-safe (the C++ runtime guards the static's construction), and the object +lives until static-destruction at program exit. + +**Do not log from static-destruction paths.** Once `LogState` itself is +destroyed, any later `morph::log` call (e.g. from another static object's +destructor, or an `atexit` handler ordered after `LogState`) touches a +destroyed `std::mutex`/`std::function`/`std::atomic` — a use-after-free with +undefined behaviour. There is no destruction-order guarantee between `LogState` +and other translation-unit statics, so avoid logging from any code that can run +during static teardown. + +## API reference + +### `LogLevel` + +| Member | Signature | Notes | +|---|---|---| +| `debug` | `enum class LogLevel : std::uint8_t` | Value 0 | +| `info` | | Value 1 | +| `warn` | | Value 2 | +| `error` | | Value 3 | +| `off` | | Value 4 | + +### Configuration functions + +| Symbol | Signature | Notes | +|---|---|---| +| `setLogger` | `void setLogger(std::function)` | Replaces the global sink. Thread-safe: **takes `mtx`**. Pass a no-op lambda (or `nullptr`) to silence all output — a null sink is skipped by the emit path. | +| `setLogLevel` | `void setLogLevel(LogLevel)` | Sets the minimum level. Messages below this level are silently dropped. Thread-safe and **lock-free** — a relaxed atomic `store`, does not take `mtx`. | +| `getLogLevel` | `LogLevel getLogLevel()` | Returns the current minimum level. Thread-safe and **lock-free** — a relaxed atomic `load`, does not take `mtx`. | + +### Level helpers + +| Symbol | Overloads | Notes | +|---|---|---| +| `logDebug` | `(std::string_view)` / `(std::format_string, Args&&...)` | Emits at `LogLevel::debug` | +| `logInfo` | `(std::string_view)` / `(std::format_string, Args&&...)` | Emits at `LogLevel::info` | +| `logWarn` | `(std::string_view)` / `(std::format_string, Args&&...)` | Emits at `LogLevel::warn` | +| `logError` | `(std::string_view)` / `(std::format_string, Args&&...)` | Emits at `LogLevel::error` | + +### `ScopedLoggerOverride` + +| Member | Signature | Notes | +|---|---|---| +| default ctor | `ScopedLoggerOverride()` | Snapshots current sink + level; does not change them. Thread-safe: the snapshot reads are taken under the global mutex. | +| explicit ctor | `ScopedLoggerOverride(std::function, LogLevel = debug)` | Installs the given sink and level; saves previous values. Thread-safe: snapshot-then-install is done under the global mutex. | +| dtor | `~ScopedLoggerOverride()` | Restores saved sink and level. Thread-safe. | +| copy/move | deleted | Not copyable or movable. | + +### Internal detail (not for direct use) + +| Symbol | Signature | Notes | +|---|---|---| +| `detail::levelName` | `constexpr std::string_view levelName(LogLevel) noexcept` | Returns `"DEBUG"`, `"INFO "`, `"WARN "`, `"ERROR"`, `"OFF "` (padded to 5 chars). Falls back to `"? "` for any out-of-range enum value. | +| `detail::Logger` | `using Logger = std::function` | Sink function type. | +| `detail::log` | `void log(LogLevel, std::string_view)` | Internal emit entry point. First does a **lock-free** `level >= minLevel` check (relaxed atomic load) and returns early if suppressed; only then acquires `mtx` and calls the sink if it is non-null. The sink runs **while `mtx` is held** (see [Thread safety](#thread-safety)). | +| `detail::logFormat` | `template void logFormat(LogLevel, std::format_string, Args&&...)` | Checks `level >= minLevel` **before** calling `std::format` (lock-free) and returns early if suppressed, so a filtered formatted call pays no formatting/allocation cost. When the level passes, formats via `std::format` and forwards to `detail::log`. | + +## Design decisions + +| Decision | Choice | Why | +|---|---|---| +| API surface | **Namespace-scope free functions, no logger object** | The simplest possible integration — no dependency injection, no global instance to manage. A single hidden `LogState` singleton is sufficient for application-level logging. | +| Level filtering | **Runtime `uint8_t` comparison, not template-based** | Levels are known at runtime from configuration; a virtual dispatch or template per level adds complexity with no benefit. The `>=` comparison on the enum's underlying value is branch-predictable and cheap. | +| Sink type | **`std::function`** | Erases any callable (lambda, function pointer, callable object) without requiring the user to subclass an interface. The cost of a `std::function` invocation is negligible relative to I/O. | +| Thread safety | **Split: lock-free `std::atomic` for the level, `std::mutex` for the sink** | The level is the per-call hot path, read on every log call; making it an atomic keeps filtering contention-free and lets a suppressed formatted call reject before `std::format`. The sink is a `std::function` that cannot be swapped atomically, so it keeps the mutex — which also serialises sink calls. The default `stderr` sink is independently thread-safe, but serialising keeps arbitrary user sinks' output from interleaving. Consequence: the sink runs under a non-recursive mutex and must not re-enter `morph::log` (see [Thread safety](#thread-safety)). | +| Default sink | **`std::println(stderr, "[{}] {}", levelName(lvl), msg)`** | C++23 `std::println` is the modern replacement for `fprintf` — type-safe, no format string mismatch, and it appends a newline automatically. The `[LEVEL]` prefix is concise and grep-friendly. | +| Format support | **Two overloads per level: `std::string_view` and `std::format_string` + variadic args** | The plain overload avoids forcing `std::format` on callers with a raw string; the variadic overload provides type-safe formatting. SFINAE on `std::format_string` ensures the format args are checked at compile time. | +| `ScopedLoggerOverride` | **RAII guard with two constructors (snapshot-only and install), deleted copy/move** | The default constructor enables a pattern where test fixtures set up their own logger mid-test and still get automatic restoration. The explicit constructor is the common case. Deleted copy/move prevents double-restore. | +| No `operator<<` | **`std::format` / `std::print` only** | Consistent with C++23 direction — no iostreams dependency, no ADL pitfalls, no locale contamination. | +| No compile-time level filtering | **All levels compiled in; filtering is runtime only** | The system is small enough that the dead code from unused levels is negligible. A compile-time toggle would require `if constexpr` at every call site or a macro-based approach, neither of which is justified here. | + +## Limitations + +These are deliberate scope choices, not defects — but they bound where this +logger is the right tool: + +- **One global sink, one serialised call.** There is a single process-wide + sink invoked under `mtx`, so all logging from all threads funnels through one + serialised call. A slow or blocking sink stalls every other thread's logging + for the duration of each call. There is no fan-out to multiple sinks and no + async/queued handoff — if you need those, wrap them inside your own sink + (and keep the part that runs under `mtx` short). +- **Sink signature is only `(LogLevel, std::string_view)`.** The sink receives + a level and a fully-formatted message and nothing else — no source location, + no category/logger name, no timestamp, no thread id. Structured routing + (per-category filtering, per-field indexing, correlation ids) is not possible + without smuggling that data into the message string. Callers that need it + must encode it into the text and parse it back out in the sink. +- **No compile-time level elision.** Every call site is compiled in and reaches + at least the atomic level check at runtime; the argument expressions of a + formatted call are always evaluated even when the message is suppressed + (only the `std::format` step is skipped). There is no macro or `if constexpr` + that removes suppressed call sites from the binary. +- **`ScopedLoggerOverride` swaps *global* state.** The override replaces the one + global sink/level, so it isolates only tests that run **serially**. Two tests + running concurrently (or code on a background thread) share the same override + window and will see each other's sink. It is a serial-test-isolation helper, + not per-thread or per-scope logging. + +## Cross-references + +- **`completion.md`** and ARCHITECTURE.md's *Error propagation* section — the + error-handling path is a primary user of this logger: an abandoned + `Completion` with no `.onError` handler logs the pending exception through the + configured sink from its destructor (non-`std::exception` types are logged as + `"unknown exception"`). This is exactly the kind of destructor-time logging + the [Lifetime](#lifetime) note warns about — safe during normal execution, + but not from static-destruction ordering after `LogState` is gone. +- **`ScopedLoggerOverride` usage** — the standard pattern for capturing log + output in tests without leaking a sink across cases; it mirrors + `morph::journal::ScopedActionLog` (see `journal.md`), which applies the same + RAII save/restore idiom to the action-log sink. Because both swap global + state, tests using them must run serially (see [Limitations](#limitations)). +- ARCHITECTURE.md *Logger* section — the one-paragraph framework-level summary: + all framework internals route through `morph::log::detail::log`, and + applications call `setLogger` once at startup to redirect to spdlog, Qt + logging, or a test spy. \ No newline at end of file diff --git a/docs/spec/offline.md b/docs/spec/offline.md new file mode 100644 index 0000000..f30085c --- /dev/null +++ b/docs/spec/offline.md @@ -0,0 +1,505 @@ +# `morph::offline` — offline support + +`morph::offline` provides the building blocks for a network-aware application +that degrades gracefully when the backend is unreachable. It covers four +concerns: + +1. **Detecting** the connectivity state (`NetworkMonitor`). +2. **Queuing** actions that could not be delivered (`IOfflineQueue`, + `InMemoryOfflineQueue`). +3. **Replaying** queued actions on reconnect, with retry and dead-letter + semantics (`SyncWorker`). +4. **Orchestrating** the reconnect → activate → bind → replay sequence + (`ReconnectCoordinator`, `ReconnectOutcome`, `ReconnectCoordinatorConfig`). + +All types live in `morph::offline`. + +## Type overview + +| Type | Header | Role | +|---|---|---| +| `NetworkMonitor` / `NetworkMonitorConfig` | `network_monitor.hpp` | Background probe thread + online/offline state machine. | +| `QueueItem`, `IOfflineQueue`, `InMemoryOfflineQueue` | `offline_queue.hpp` | Passive store of undelivered actions (opaque payloads). | +| `SyncWorker` / `SyncResult` | `sync_worker.hpp` | Drains + replays a queue with retry/dead-letter. | +| `ReconnectCoordinator`, `ReconnectOutcome`, `ReconnectCoordinatorConfig`, `ReconnectCoordinator::Deps` | `reconnect_coordinator.hpp` | Orders reconnect → activate → bind → replay, with abort checks. | + +## Contents + +- [NetworkMonitor](#networkmonitor) +- [NetworkMonitor callback constraint](#networkmonitor-callback-constraint) +- [Offline queue](#offline-queue) +- [Ownership: who enqueues](#ownership-who-enqueues) +- [SyncWorker](#syncworker) +- [ReconnectCoordinator](#reconnectcoordinator) +- [End-to-end integration](#end-to-end-integration) +- [Failure modes](#failure-modes) +- [Limitations](#limitations) +- [Design decisions](#design-decisions) +- [Cross-references](#cross-references) + +## NetworkMonitor + +A background thread calls a user-supplied probe function at regular intervals. +The monitor starts *online* and transitions to *offline* only after +`failureThreshold` consecutive failures. It returns to *online* after +`onlineThreshold` consecutive successes. Callbacks fire on the probe thread. + +The monitor is non-copyable and non-movable. Destroy it to stop monitoring. + +### `NetworkMonitorConfig` + +| Field | Type | Default | Purpose | +|---|---|---|---| +| `probeInterval` | `std::chrono::milliseconds` | `5s` | Time between probe calls. | +| `failureThreshold` | `int` | `3` | Consecutive failures before going offline. | +| `onlineThreshold` | `int` | `1` | Consecutive successes before going online. | + +Declared outside `NetworkMonitor` so its default member initialisers are fully +parsed before any constructor default argument evaluates — a nested incomplete +type breaks constructor-default-argument lookup on clang/GCC. + +### `NetworkMonitor` API + +| Member | Signature | Notes | +|---|---|---| +| `ProbeFunction` | `std::function` | Returns `true` when the network is reachable. | +| `Callback` | `std::function` | Called on state change. | +| `Config` | `NetworkMonitorConfig` | Alias for the config struct. | +| ctor | `NetworkMonitor(ProbeFunction, Callback onOffline, Callback onOnline, Config = {})` | Launches the probe thread immediately. | +| dtor | `~NetworkMonitor()` | Calls `stop()` then spin-waits on `_runExited` to handle the case where `stop()` was called from within a probe callback (avoiding deadlock on `join()`). | +| `isOnline()` | `bool isOnline() const noexcept` | Reads an atomic flag; safe from any thread. | +| `stop()` | `void stop()` | Signals the thread to stop. Idempotent. If called from the probe thread itself, detaches instead of joining. | + +**Probe exceptions are swallowed** — a throwing probe is treated as a failed +probe (`safeProbe` catches everything and returns `false`). + +## NetworkMonitor callback constraint + +**`onOffline` and `onOnline` run on the probe thread, inline inside the probe +loop.** Look at `run()`: it waits on the condition variable for `probeInterval`, +calls `safeProbe`, then calls `handleProbeResult`, which invokes the callback +*before* the loop can circle back to wait for the next interval. There is no +executor, no second thread, and no queue between the probe result and the +callback — whatever the callback does, the probe thread does. + +Consequences: + +- **A blocking callback stalls all probes.** While the callback runs, the next + `wait_for` has not started, so no further connectivity checks happen. A + callback that blocks for 30s means 30s of connectivity blindness. +- **Running the coordinator or `SyncWorker` inline is a mistake.** A + `ReconnectCoordinator::onOnline()` can spin for up to + `maxAttempts * retryDelay` (≈20s at defaults) of retry-and-sleep, and a + `SyncWorker::run()` executes arbitrarily long replay work. Doing either + directly inside a callback runs *seconds of retry loop on the probe thread*, + which is exactly the thread that is supposed to be watching the network. +- **The safe shape is: set an atomic, or post to an executor, and return.** + The callback should do O(1) work — flip a flag, `post()` a lambda onto a + worker executor — and let the heavy sequencing run elsewhere. This is why + `ReconnectCoordinator::onOnline()`/`onOffline()` are documented as + "posted onto a worker executor by the host, not called on the probe thread." + +Calling `stop()` from within a callback is supported (it detaches rather than +joins to avoid a self-deadlock — see the dtor/`stop()` notes above), but it is +still a callback running on the probe thread and must not block first. + +See `concurrency_and_lifetimes.md` for the framework-wide rule that +notification callbacks marshal work off the thread that raised them. + +## Offline queue + +### `QueueItem` + +| Field | Type | Purpose | +|---|---|---| +| `id` | `uint64_t` | Stable identifier assigned at enqueue time. | +| `payload` | `std::string` | Opaque serialised representation of the queued action. | + +The payload format is the caller's choice — JSON, binary-hex, plain text, etc. + +### `IOfflineQueue` + +Minimal interface for durable storage of undelivered actions. Accepts items +while offline; `SyncWorker` drains and replays them on reconnect. + +| Member | Signature | Notes | +|---|---|---| +| `enqueue` | `uint64_t enqueue(std::string payload)` | Appends payload. Returns a stable id. | +| `drain` | `std::vector drain()` | Returns all pending items in enqueue order, without removing them. Safe to call multiple times — items survive between `drain()` and the corresponding `markDone()`. | +| `markDone` | `void markDone(uint64_t itemId)` | Removes the item identified by `itemId`. No-op if not found. | + +### `InMemoryOfflineQueue` + +Thread-safe in-memory implementation of `IOfflineQueue`. Items live in a +`std::deque` protected by a `std::mutex`. Ids are monotonically +increasing. Suitable for testing and applications that do not require +persistence across restarts. + +## Ownership: who enqueues + +**The queue is passive.** `IOfflineQueue` exposes `enqueue` / `drain` / +`markDone` and nothing else — it has no notion of a backend, a transport, or a +"failed request." It never fills itself. The framework supplies no transport +layer that would notice a write failed and drop it into the queue, so +**detecting an offline/failed `execute()` and calling `enqueue()` is the +application's job.** + +The seam is on the *write path*, not inside `morph::offline`. A host that wants +offline durability wraps its own dispatch: + +```cpp +// Application code — the framework does not write this for you. +void submit(const MyAction& action) { + if (!monitor.isOnline()) { // known offline: don't even try + queue.enqueue(serialise(action)); + return; + } + try { + bridge.execute(action); // attempt delivery + } catch (const std::exception&) { // delivery failed at the edge + queue.enqueue(serialise(action)); // trap it into the queue + } +} +``` + +`SyncWorker` closes the loop on the *read path*: on reconnect it `drain()`s the +same queue and replays each payload. The two halves share one `IOfflineQueue` +instance (see [End-to-end integration](#end-to-end-integration)) — the +application owns the "enqueue on failure" half, the framework owns the "drain +and replay" half. Neither `NetworkMonitor` nor `ReconnectCoordinator` enqueues +anything; they only *observe* and *sequence*. + +Because the framework never calls `enqueue`, the serialisation format is +entirely the caller's (`QueueItem::payload` is an opaque `std::string`), and it +is the caller's responsibility that the same format round-trips through the +`SyncWorker::ReplayFunction`. + +## SyncWorker + +Replays queued actions from an `IOfflineQueue` on reconnect. Drains the queue +and calls a caller-supplied `ReplayFunction` for each item. + +### `SyncResult` + +| Field | Type | Default | Purpose | +|---|---|---|---| +| `successful` | `int` | `0` | Items replayed and removed from the queue. | +| `failed` | `int` | `0` | Items that failed and remain in the queue for retry. | +| `deadLettered` | `int` | `0` | Items that exhausted their retry budget and were dropped (logged at `morph::log::LogLevel::error`). | + +### `SyncWorker` API + +| Member | Signature | Notes | +|---|---|---| +| `ReplayFunction` | `std::function` | Return `true` → success, `false` → failure. Throwing is treated as failure. | +| ctor | `SyncWorker(IOfflineQueue&, ReplayFunction)` | References the queue and the replay callable. | +| `run()` | `SyncResult run()` | Drains the queue and replays each item. Concurrent calls are serialised by an internal mutex. Returns immediately if `stop()` was called before acquiring the lock. | +| `stop()` | `void stop()` | Signals an in-progress `run()` to stop after the current item. One-shot — the flag resets at the start of the next `run()`. | + +**Retry & dead-letter (hard-coded defaults):** + +- Each item is retried up to **5 attempts** across successive `run()` calls. +- Items that fail their 5th attempt are dropped and logged at + `morph::log::LogLevel::error` (the payload appears in the log line). +- Items that succeed implicitly reset their attempt counter (they are removed). +- There are intentionally no public knobs — the framework guarantees obvious, + safe defaults. + +The per-item attempt counter lives in a `std::unordered_map` +keyed by `QueueItem::id`. + +## ReconnectCoordinator + +Sequences the reconnect → activate → bind → replay steps when the network comes +back. All side effects are injected via `Deps`; the coordinator contains only +the retry loop, the ordering guarantees, and the abort checks. It performs no +I/O and owns no thread — `onOnline()` / `onOffline()` run synchronously on the +calling thread. + +### `ReconnectOutcome` + +| Enumerator | Meaning | +|---|---| +| `Reconnected` | Backend reopened, made active, context bound, queue replay invoked. | +| `GaveUp` | Exhausted `maxAttempts` without a successful reconnect; stayed offline. | +| `Aborted` | `shouldContinue()` returned false before any reconnect attempt. | + +### `ReconnectCoordinatorConfig` + +| Field | Type | Default | Purpose | +|---|---|---|---| +| `maxAttempts` | `int` | `10` | Max reconnect attempts per `onOnline()` call. | +| `retryDelay` | `std::chrono::milliseconds` | `2s` | Delay between failed attempts. | + +### `ReconnectCoordinator` API + +| Member | Signature | Notes | +|---|---|---| +| `Config` | `ReconnectCoordinatorConfig` | Alias. | +| `Deps` | struct | Injected side-effect callbacks (see below). | +| ctor | `explicit ReconnectCoordinator(Deps, Config = {})` | Non-copyable, non-movable. Null `Deps` members are logged in debug builds. | + +#### `Deps` struct + +| Field | `std::function` signature | Purpose | +|---|---|---| +| `tryReconnect` | `bool()` | Attempt to (re)open the primary backend. Throwing → failed attempt. | +| `activatePrimary` | `void()` | Make the freshly-reconnected primary the active backend. Called once per successful `onOnline()`, after `tryReconnect()` succeeds. | +| `activateLocal` | `void()` | Switch the active backend to the local/offline one. Called by `onOffline()`. | +| `bindContext` | `void()` | Rebind per-connection/per-session context to the active backend. Called after every `activate*` step. Must not throw. | +| `replay` | `void()` | Replay the offline queue against the now-active primary. Typically wraps `SyncWorker::run()`. Called last in `onOnline()`. | +| `shouldContinue` | `bool()` | Return `false` to abort the current `onOnline()` early (e.g. monitor reports offline mid-retry). Polled before each attempt and once more before replay. | +| `sleep` | `void(std::chrono::milliseconds)` | Sleep between failed attempts. Tests substitute a no-op/counter; hosts wire to `std::this_thread::sleep_for`. | + +#### Ordering guarantees (the reason this class exists) + +Within a successful `onOnline()`, the steps run in strict order: + +1. `tryReconnect()` returns `true`. +2. `activatePrimary()` — make primary the active backend. +3. `bindContext()` — rebind per-connection/per-session state. +4. `replay()` — drain + replay the offline queue. + +Step 4 MUST NOT run before step 3, and step 3 MUST NOT run before step 2. + +#### `onOnline()` + +```cpp +ReconnectOutcome onOnline(); +``` + +Synchronous. Runs the retry loop. For each attempt: + +1. Check `shouldContinue()` — abort if false. +2. Call `tryReconnect()` — skip to sleep if false. +3. On success: `activatePrimary()`, `bindContext()`, re-check + `shouldContinue()` before `replay()`, return `Reconnected`. +4. Sleep `retryDelay` (except after the final attempt). +5. After `maxAttempts` failures, log a warning and return `GaveUp`. + +#### `onOffline()` + +```cpp +void onOffline(); +``` + +Calls `activateLocal()` then `bindContext()`. Idempotent — safe to call when +already local. + +#### Thread safety + +`onOnline()` and `onOffline()` are mutually serialised by an internal mutex. +They are intended to be posted onto a worker executor by the host, not called +directly on the probe thread. + +## End-to-end integration + +The four types compose into one pipeline. The rule that ties them together: +**the probe callback does no work of its own — it posts, and the coordinator +does the sequencing on a worker executor, and the coordinator's `replay` +dependency wraps `SyncWorker::run()` over the same queue the application +enqueues into.** + +```cpp +morph::offline::InMemoryOfflineQueue queue; // shared by both halves +morph::exec::SomeExecutor worker; // host's worker executor + +morph::offline::SyncWorker sync{ + queue, + [&](const std::string& payload) { return deliver(payload); } // ReplayFunction +}; + +morph::offline::ReconnectCoordinator coordinator{{ + .tryReconnect = [&] { return backend.reopen(); }, + .activatePrimary = [&] { bridge.switchBackend(makePrimary()); }, + .activateLocal = [&] { bridge.switchBackend(makeLocal()); }, + .bindContext = [&] { session.rebind(); }, + .replay = [&] { sync.run(); }, // <-- SyncWorker over the shared queue + .shouldContinue = [&] { return monitor.isOnline(); }, + .sleep = [](std::chrono::milliseconds d) { std::this_thread::sleep_for(d); }, +}}; + +// Callbacks run on the probe thread, so they ONLY post — never run the +// coordinator inline (see "NetworkMonitor callback constraint"). +morph::offline::NetworkMonitor monitor{ + [] { return tcpProbe(); }, // ProbeFunction: bool() + [&] { worker.post([&] { coordinator.onOffline(); }); }, // onOffline + [&] { worker.post([&] { coordinator.onOnline(); }); }, // onOnline +}; +``` + +Flow: the `bool()` probe drives `NetworkMonitor`'s state machine → on a +transition the callback *only* posts a lambda to `worker` (it must not run +reconnect logic inline on the probe thread) → the worker runs +`ReconnectCoordinator::onOffline()` / `onOnline()` → a successful `onOnline()` +calls `activatePrimary` → `bindContext` → `replay`, and `replay` runs +`SyncWorker::run()`, which drains and replays the `queue` the application filled +on the write path ([Ownership: who enqueues](#ownership-who-enqueues)). + +### Reconciling with ARCHITECTURE.md's direct wiring + +`ARCHITECTURE.md` shows a simpler wiring where the callbacks call +`bridge.switchBackend(...)` directly: + +```cpp +morph::offline::NetworkMonitor monitor{ + myTcpProbe, + [&] { bridge.switchBackend(std::make_unique(localPool)); }, + [&] { bridge.switchBackend(std::make_unique(server)); } +}; +``` + +Both are legitimate; they are different points on a spectrum: + +- **Direct `switchBackend` — the minimal path.** No retry, no ordered + replay, no abort-on-flap. `switchBackend` is a bounded mutex operation (it is + not a seconds-long retry loop), so calling it inline on the probe thread is + acceptable *as a minimal demo*. It does not replay a queue and has no + `shouldContinue` guard. +- **`ReconnectCoordinator` — the ordered, tested path.** Use it when reconnect + can *fail and need retries*, when replay must run strictly *after* activate + + bind, and when a mid-retry flap-back-offline must abort cleanly. This is the + path with the ordering invariant and the guarantees this file documents. Its + own callbacks must be posted off the probe thread precisely because the retry + loop can run for seconds. + +Rule of thumb: a demo or a backend switch with no pending writes can use direct +`switchBackend`; anything that must not lose queued writes on a flaky link uses +the coordinator, with `replay` wrapping `SyncWorker::run()`. + +## Failure modes + +The pipeline has several sharp edges that callers must design around. None are +bugs — they are consequences of the deliberately minimal contracts. + +### No head-of-line blocking in replay + +`SyncWorker::run()` does **not** stop at the first failing item. When +`_replay` returns `false` (or throws) it increments that item's attempt counter +and *continues to the next item*, replaying and `markDone`-ing later items that +succeed. Therefore **"enqueue order is preserved" holds only when every item +succeeds.** If item #2 fails and item #3 succeeds, #3 is delivered and removed +while #2 stays queued for a later `run()` — the backend sees #3 before a +subsequent retry of #2. Callers that need strict ordering across failures must +enforce it themselves (e.g. a replay function that refuses to process #3 until +#2 lands). + +### Retry counter is in-memory and resets on restart + +The per-item attempt count lives in `SyncWorker::_attempts` +(`std::unordered_map`), a plain member — **not** in the queue. +It is lost when the process exits. A queue that survives restarts (a SQL-backed +`IOfflineQueue`) will therefore re-present a poison item with its counter back +at zero after every restart, so it can never actually dead-letter across +restarts. **Durable dead-lettering requires storing the attempt count in the +queue**, which the current `QueueItem` (id + payload only) does not carry. + +### `Reconnected` can be returned without replaying + +`onOnline()` returns `ReconnectOutcome::Reconnected` after a successful +`tryReconnect` + `activatePrimary` + `bindContext`, but it re-checks +`shouldContinue()` *once more before `replay()`*. If that final check is false +(the backend went away again during activate/bind), **`replay()` is skipped and +the outcome is still `Reconnected`.** `Reconnected` means "we reconnected and +bound," not "the queue was replayed." A caller that keys off the outcome to +decide whether the queue is drained will be wrong in this window. + +### First offline report is delayed, and `onOnline` never fires at startup + +`NetworkMonitor::run()` `wait_for`s `probeInterval` *before* the first probe, so +the first probe is at `t = probeInterval`, and `failureThreshold` consecutive +failures are needed to flip offline. **The earliest an `onOffline` can fire is +`probeInterval * failureThreshold`** — ≈15s at defaults (5s × 3). An app that is +offline from the very start still reports online for that whole window. +Separately, the monitor **starts in the online state**, and callbacks fire only +on *transitions*, so **`onOnline` never fires at startup** — there is no +online→online edge. Startup activation is the host's job (call `onOnline()` / +`activatePrimary` explicitly at boot if the backend is expected up). + +### Null `Deps` construct successfully then crash + +`ReconnectCoordinator`'s constructor only *logs* null `Deps` members (in debug +builds, via `assertDepsNonNull`); it does not throw. A coordinator built with a +null `tryReconnect`/`replay`/etc. constructs fine and later crashes when +`onOnline()`/`onOffline()` invokes the null `std::function`. Treat the debug log +line as the only warning you get. + +### `onOnline()` holds the mutex for the entire retry loop + +`onOnline()` takes `_mtx` at entry and holds it across the whole loop — +including every `sleep(retryDelay)` — for up to `maxAttempts * retryDelay` +(≈20s at defaults). Because `onOffline()` shares that mutex, **a +flap-back-offline cannot preempt an in-progress `onOnline()` by acquiring the +lock**; it can only take effect through `shouldContinue()` returning `false` at +the next poll. Wire `shouldContinue` to the live monitor state +(`monitor.isOnline()`) so a flap is actually observed, rather than to a stale +snapshot. + +## Limitations + +Honest boundaries of what ships today: + +- **Opaque `std::string` payload discards the typed-codec machinery.** The rest + of morph moves typed actions through the wire codec (`wire.md`); the offline + queue stores an opaque blob and hands an opaque blob to the replay function. + Serialisation, versioning, and type-safety across the enqueue→replay boundary + are entirely on the caller — the compiler will not catch a format mismatch. +- **No idempotency contract, but replay must be idempotent anyway.** `drain()` + is non-destructive and `markDone` runs only *after* a successful replay, so a + crash (or a `false` return) *after* the side effect has committed re-invokes + `replay` on the same payload on the next `run()`. Retries and post-commit + failures both re-run the payload. The framework provides no dedup or + "exactly-once" guarantee — **the replay function MUST be idempotent** and the + spec cannot enforce it. +- **Only an in-memory queue ships.** `InMemoryOfflineQueue` loses everything on + exit. A durable/SQL-backed `IOfflineQueue` is the caller's to write (the + interface is designed for it, but no implementation is provided here). +- **Dead-lettering is log-only.** A poison item that exhausts its 5 attempts is + `markDone`-d (dropped) and written to `morph::log` at error level. There is + **no recovery hook** — no dead-letter queue, no callback, no way to inspect or + requeue it programmatically. If the log sink drops it, it is gone. +- **Null `Deps` are not rejected at construction** (see Failure modes) — a + misconfigured coordinator is a latent crash, not a constructor error. +- **`onOnline()` serialises the whole retry loop under one mutex**, so + responsiveness to a mid-retry state change is bounded only by the + `shouldContinue()` poll cadence, not by lock hand-off. + +## Design decisions + +| Decision | Choice | Why | +|---|---|---| +| Monitor probe interval | **Caller-chosen, default 5s** | Tunable per application; 5s is polite for most backends. | +| State transitions use thresholds | **`failureThreshold` / `onlineThreshold`** | A single failed probe does not flip state — hysteresis avoids flapping on transient blips. | +| Probe exceptions | **Swallowed, treated as `false`** | A crashing probe should not tear down the monitor; the host fixes the probe. | +| Queue interface | **Minimal virtual interface (`IOfflineQueue`)** | Lets callers swap in SQLite, file-backed, or test queues without framework changes. | +| `drain` is non-destructive | **Items survive between `drain()` and `markDone()`** | Crash safety: a crash after `drain()` but before `markDone()` does not lose items. | +| SyncWorker retry count | **Hard-coded at 5, no public knob** | The framework guarantees obvious, safe defaults; apps that need different math wrap or replace `SyncWorker`. | +| SyncWorker thread safety | **Internal mutex serialises `run()`** | Second caller blocks — safe to fire from multiple executors. | +| Reconnect retry loop | **Synchronous, no background thread** | The host owns the executor; the coordinator is pure orchestration with no hidden threads. | +| Reconnect step ordering | **Explicit in the `onOnline()` body** | The strict order (reconnect → activate → bind → replay) is the class's reason to exist — callers should never have to get it right themselves. | +| `onOnline()` / `onOffline()` serialised | **Same internal mutex** | Prevents a race where a concurrent `onOffline()` replays into a local backend during an in-progress `onOnline()`. | +| `shouldContinue` re-checked before replay | **Second poll after bind** | The backend may have gone away during `activatePrimary()` / `bindContext()` — never replay into a backend that just became unreachable. | +| No sleep after final attempt | **`retryDelay` skipped on last iteration** | Wasting 2s after we already know we're giving up serves no purpose. | + +## Cross-references + +- **`bridge.md`** — `Bridge::switchBackend` is the mechanism the coordinator's + `activatePrimary`/`activateLocal` dependencies drive (re-registers live + handlers on the new backend, fires `onBackendChanged`). ARCHITECTURE.md's + minimal wiring calls `switchBackend` straight from the monitor callback; see + [Reconciling with ARCHITECTURE.md's direct wiring](#reconciling-with-architecturemds-direct-wiring). +- **`journal.md`** — the action log is a permanent, append-only audit/replay + trail; `IOfflineQueue` is transient (holds pending writes, deletes them on + delivery). The two are distinct: the journal's ordering is authoritative and + never dropped by the framework, whereas offline replay ordering only holds + when every item succeeds (see [Failure modes](#failure-modes)). Do not conflate + the offline queue's replay with journal replay. +- **`concurrency_and_lifetimes.md`** — the framework-wide rule that notification + callbacks marshal work off the raising thread (the reason + [NetworkMonitor callbacks must only post](#networkmonitor-callback-constraint)), + plus the monitor's teardown/`stop()`-from-callback contract. +- **`error_handling.md`** — how a failed `execute()` surfaces to the application + (the signal that drives [enqueue-on-failure](#ownership-who-enqueues)), and + the framework's swallow-and-treat-as-failure policy that this file mirrors in + `safeProbe`, `SyncWorker`'s replay `try/catch`, and the coordinator's + `callTryReconnect`/`callShouldContinue`. diff --git a/docs/quantity_type.md b/docs/spec/quantity_type.md similarity index 71% rename from docs/quantity_type.md rename to docs/spec/quantity_type.md index e8da065..e3d4035 100644 --- a/docs/quantity_type.md +++ b/docs/spec/quantity_type.md @@ -12,26 +12,42 @@ itself: 3. **How it was computed** — its *derivation*. A computed quantity can render itself as an equation with named variables, shown symbolically and evaluated. -The third point is the reason the type exists rather than a plain `double`. In -a domain application (financial, engineering, metering) the answer alone is not -enough — you have to be able to show the working: *why* is the density -`0.83 kg/m³`, *which* inputs fed the surplus, *what* was 3 % of what. A +The first two points — an exact value, in a known unit, that may be empty — are +the **core contract** and the reason `Quantity` exists rather than a plain +`double`: domain math must be exact, units must not mix by accident, and "not +entered" must be a first-class state rather than a sentinel. That core is always +present, whatever the build. + +The third point — **provenance** — is an *opt-in feature layered on top of the +core*. It is on by default but compiled out entirely by +`MORPH_QUANTITY_PROVENANCE=0` (see *Provenance*), and it is what makes a domain +application reach for `Quantity` over a bare exact number: the answer alone is +often not enough — you have to show the working: *why* is the density +`0.83 kg/m³`, *which* inputs fed the surplus, *what* was 3 % of what. A traced `Quantity` carries that explanation with the value instead of reconstructing it after the fact. +> **Reading this spec.** Much of this document describes provenance — the larger +> surface, but the optional layer. If you only need the core, read *Units are +> types*, *Exact value and precision*, *How a value prints*, and *Wire and +> schema*; everything under *Provenance*, *Named symbols*, and the `equation()` +> rules is skippable when tracing is off. + ## Contents - [Units are types](#units-are-types) - [Exact value and precision](#exact-value-and-precision) ([Empty state](#empty-state--stdnullopt-behavior)) +- [How a value prints](#how-a-value-prints) - [Provenance — a build-time toggle](#provenance--a-build-time-toggle-on-by-default) - [Named symbols](#named-symbols) -- [Formatting](#formatting--stdformatter-only) - [Wire and schema](#wire-and-schema) - [Unit conversion — `UnitRelation` and `convert`](#unit-conversion--unitrelation-and-convert) - [API reference](#api-reference) - [Design decisions](#design-decisions) - [Usage example](#usage-example) +- [Cross-references](#cross-references) +- [Limitations](#limitations) - [Out of scope](#out-of-scope) ## Units are types @@ -43,7 +59,7 @@ mixed accidentally. The application supplies: places, and a flat list of **peer-to-peer `UnitRelation` entries** that declare exact conversion ratios between same-dimension units with a `morph::math::Rational`. Each entry `{Unit::g, Unit::kg, Rational{1, 1000}}` - generate a `convert` in both directions; a `UnitTraits::convert` static + generates a `convert` in both directions; a `UnitTraits::convert` static takes precedence when the conversion is not a simple ratio (e.g. Celsius ↔ Fahrenheit). - a `consteval` unit algebra (`operator*` / `operator/` on the enum) so @@ -162,9 +178,13 @@ side first if you mean to compare across units. Comparison is on the exact value alone and **ignores the runtime `DecimalPlaces` tag** — two engaged quantities with equal value but different precision compare equal. -**Formatting.** `std::format("{}", emptyQ)` renders the unit's `N/A` text -(defined in `UnitTraits`) alone — `"N/A"`, `"N/A kW"`, etc. The formatting -path never attempts to access the absent `Rational`. +**Formatting.** `std::format("{}", emptyQ)` renders `N/A` suffixed with the +unit's display text — `"N/A"` (scalar, empty display), `"N/AkW"`, `"N/A%"`. +The `"N/A"` marker is a **fixed literal in the formatter**, *not* a field of +`UnitTraits` (`UnitMeta` carries only `id`, `display`, `defaultDecimals`); the +unit contributes only the trailing `display` string, appended with no +separating space. The formatting path never attempts to access the absent +`Rational`. **`equation()`.** An empty quantity has no derivation. `equation()` returns a single-element vector containing the same `N/A` text that `std::format` would @@ -184,6 +204,51 @@ There is no `Rational` to retag. JSON `null` — the payload's `std::nullopt` maps directly to a null JSON token, and the unit metadata travels only in the schema, never in the instance. +## How a value prints + +Every printed form of a `Quantity` — `std::format`, and every number inside +`equation()` — goes through **one** helper, `detail::formatRationalDecimal`, so +a value reads identically everywhere. There is a single formatting path and +**no `operator<<`** (streaming is done by formatting to a `std::string` first). + +**The decimal form.** `formatRationalDecimal` renders the exact `Rational` as a +fixed decimal at its **runtime `DecimalPlaces`** (`{:.Nf}`) and then trims +trailing zeros (and a bare trailing point). So a whole value prints with no point +(`2`), and `0.30` / `1.360` print as `0.3` / `1.36`. This is deliberately **not** +`Rational`'s own `std::format("{}", r)`, which prints an integer numerator or a +`num/den` *fraction* — `Quantity` always prints a trimmed decimal. The runtime +`DecimalPlaces` tag is the single source of truth for how many decimals appear; +no field's *declared* precision is consulted at print time. + +**With the unit.** The `Quantity` formatter (`std::formatter>`) +appends the unit's `display` text to the number with no separating space: +`5.2kW`, `6kWh`, `0.3` (empty display for a scalar). `NamedQuantity`'s formatter +forwards to it. + +**Empty prints `N/A`.** An empty quantity renders as the fixed literal `"N/A"` +suffixed with the unit's `display` — `"N/A"`, `"N/AkW"`, `"N/A%"`. The `"N/A"` +marker lives in the formatter, **not** in `UnitMeta` (which carries only `id`, +`display`, `defaultDecimals`); the format path never touches the absent +`Rational`. + +**The `DecimalPlaces` flow, end to end.** The runtime tag that fixes the printed +form flows through a calculation deterministically: + +1. **Origin.** `fromDouble(d)` stamps the new leaf's `Rational` with the field's + *declared* precision — the one moment declared precision touches a value. A + `Rational` built directly carries whatever tag it was given. +2. **Arithmetic.** `+`, `-`, `*`, `/` set the result's tag to the **maximum** of + the engaged operands' tags (max-propagation — see *Exact value and + precision*). +3. **Conversion.** A `convert` step multiplies by the exact ratio and carries the + operand's tag through unchanged — scaling a value does not change how many + decimals it is specified to. +4. **Formatting.** `formatRationalDecimal` renders at that tag, as above; nothing + else in `Quantity` re-implements number formatting. + +`atDeclaredPrecision()` / `withDecimalPlaces()` retag a value between steps if a +caller wants a specific width; they change only the tag, never the exact value. + ## Provenance — a build-time toggle, on by default Whether a quantity carries its derivation is a **build-wide decision, selected @@ -222,8 +287,10 @@ types: - **`ASTNode`** — a node in the DAG: its own `ASTUnit current` step, an optional symbol `name` (set by `named()` / `NamedQuantity`, which makes the node opaque and stops `equation()` expanding it), and `shared_ptr left` / `right` - handles onto the nodes that fed this one. An `ASTNode` is **move-only**: sharing - is done through the `shared_ptr`, never by copying the node. + handles onto the nodes that fed this one. The struct is a plain aggregate, but + nodes are **never copied** in practice: sharing is done through the `shared_ptr`, + never by duplicating a node — every operation allocates a fresh `ASTNode` and + links the (shared) prior ones. - **`Context`** — the per-`Quantity` handle: a single `shared_ptr node` pointing at the root of that value's derivation. Copying a `Quantity` copies its `Context`, which just bumps the node's refcount — the tree itself is never @@ -269,14 +336,10 @@ so a caller emits them verbatim), in this fixed order: 3. **Result** — element `[2]` — the overall result value — `1.36` — in the same indented `= ` continuation form. 4. **`where` legend** — elements `[3]…`, one per placeholder, in order of first - appearance; the first begins `where `, any further lines align under it. A - placeholder exists only for a reused value, and its legend form depends on - what that value is: - - a **reused unnamed leaf** (a raw input used more than once): `c1 = ` — - e.g. `where c1 = 3`; - - a **reused unnamed computed value**: its work written once, as - `c1 = = = `. - A value used exactly once never reaches the legend — it is inlined at its one + appearance; the first line begins `where `, later lines align under it. A + placeholder exists only for a **reused** value; its exact legend form + (leaf/conversion vs computed) is given in *How each node renders* below. A + value used exactly once never reaches the legend — it is inlined at its one point of use instead. **Degenerate roots return a single element.** When the value has no expandable @@ -286,6 +349,12 @@ substitution/result/legend lines): - an **empty** quantity, or any quantity with tracing compiled out → the formatted value alone (the `N/A` text when empty); - a **bare unnamed leaf** (a raw input never operated on) → its formatted value; +- an **engaged value with no recorded derivation node** — one materialised + directly (aggregate init or the wire codec writing `payload`), bypassing the + value constructors that call `recordLeaf()` → its formatted value; +- a value whose **root is a bare unit conversion** (a `static_cast` / implicit + conversion never further operated on, and left unnamed) → its formatted + (converted) value — a lone `convert` node is treated as an atom; - a value whose **root is named** (e.g. a `NamedQuantity`, or the result of `.named(...)`) → just `"the name"`, since a name is opaque and its derivation is deliberately not expanded. Introspect the *unnamed* computed value if you @@ -295,54 +364,35 @@ So the symbols (the shape you'd write on paper) and the numbers (the audit of what was actually computed) live in one coherent artifact rather than in two methods that echo each other. -**How the numbers are rendered.** `equation()` carries no precision of its own -and never consults any field's *declared* precision. Every number it prints — -each substituted value, the result, and each `where` value — is produced by -running that node's stored `Rational` through the ordinary `Rational` formatter. -The decimals therefore come from the value itself: the `Rational`'s own runtime -`DecimalPlaces` tag, the single source of truth for how a value prints, exactly -as `std::format("{}", someRational)` would render it elsewhere. The unit is not -shown (the tree is unit-erased); pair the lines with `std::format("{}", q)` for -the united answer. - -**The `DecimalPlaces` flow, end to end.** How a number prints is fixed entirely -by the runtime `DecimalPlaces` tag it carries, and that tag flows through the -calculation deterministically: - -1. **Origin.** `fromDouble(d)` stamps the new leaf's `Rational` with the field's - *declared* precision — the one moment declared precision touches a value. A - `Rational` built directly carries whatever tag it was given. -2. **Arithmetic.** `+`, `-`, `*`, `/` set the result's tag to the **maximum** of - the operands' tags (max-propagation, above). -3. **Conversion.** A `convert` step multiplies by the (exact) ratio and carries - the operand's tag through unchanged — scaling a value does not change how many - decimals it is specified to. -4. **Formatting.** `std::format` and `equation()` both defer to the `Rational` - formatter, which is the single authority on the printed form: the default - (no format spec) prints an exact integer when the denominator is `1` - (so `2/1` → `2`) and otherwise renders at the value's `DecimalPlaces`. Nothing - in `Quantity` re-implements number formatting. - -`atDeclaredPrecision()` / `withDecimalPlaces()` retag a value between steps if a -caller wants a specific width; they change only the tag, never the exact value. - -Two things decide how any sub-value reads: whether it is **named**, and whether -it is **reused**. These compose into one uniform rule that applies to leaves and -computed values alike: - -- **Named** → appears as its name and is not expanded; the name summarises its - own sub-derivation (a conversion behind `"heater"`, say, folds into the name). -- **Unnamed, used once** → *inlined* at its point of use: a computed value as its - arithmetic (`"a" - "b"`), a leaf as its own number (`3`). Leave a value unnamed - precisely when you *do* want its arithmetic shown. -- **Unnamed, reused** (appears more than once) → earns a single placeholder - (`c1`, `c2`, …) so the shared work is written once, in the `where` legend, - rather than duplicated at each use — whether the reused value is a computed - expression or a raw leaf. - -The distinction between a leaf and a computed value is therefore *not* what -triggers a placeholder — reuse is. A once-used leaf is just its number in the -formula; a placeholder appears only to avoid repeating a shared value. +**Numbers print exactly as `std::format` prints them.** `equation()` carries no +precision of its own and never consults any field's *declared* precision: each +substituted value, the result, and each `where` value runs through the same +`detail::formatRationalDecimal` helper described in *How a value prints*, at the +node's own runtime `DecimalPlaces`. So a value reads identically in `equation()` +and in `std::format("{}", q)`. The unit is **not** shown (the tree is +unit-erased); pair the lines with `std::format("{}", q)` for the united answer. + +**How each node renders.** Two properties of a sub-value decide its appearance: +whether it is **named**, and whether it is **reused** (appears more than once +along the displayed paths). These compose into one rule for leaves, conversions, +and computed values alike — **reuse, not leaf-vs-computed, is what mints a +placeholder**: + +| Node | Formula (`[0]`) | Substitution (`[1]`) | `where` legend (`[3…]`) | +|---|---|---|---| +| **Named** (any kind) | its name `"x"`, not expanded | its value | never — a name is opaque and is not counted for reuse | +| **Unnamed leaf**, used once | its number (`3`) | its number | — (inlined) | +| **Unnamed leaf**, reused | placeholder `cN` | its value | `cN = ` | +| **Unnamed conversion**, used once | its converted value (an atom) | its value | — (inlined) | +| **Unnamed conversion**, reused | placeholder `cN` | its value | `cN = ` | +| **Unnamed computed**, used once | its inlined expression (`"a" - "b"`) | the substituted expression | — (inlined) | +| **Unnamed computed**, reused | placeholder `cN` | its value | `cN = = = ` | + +A value used exactly once is inlined at its one point of use and never reaches +the legend; only a *reused* value earns a `cN` placeholder, so shared work is +written once instead of duplicated at each use. Naming stops both expansion and +reuse-counting at that node — a conversion behind `"heater"` folds into the name. +Leave a value unnamed precisely when you *do* want its arithmetic written out. ## Named symbols @@ -360,13 +410,6 @@ value on construction and slices losslessly back to a plain `Quantity` wherever one is expected — the name lives in the shared history, not as extra data on the value. -## Formatting — `std::formatter` only - -A `std::formatter>` renders value + unit (`5.2kW`, `N/A%`), -driving off the `UnitTraits` display text and the `Rational` formatter. -`NamedQuantity` forwards to it. **No `operator<<` is provided** — formatting -goes through `std::format` exclusively. - ## Wire and schema On the morph JSON wire a quantity is just its nullable `Rational` payload — @@ -378,11 +421,19 @@ cannot send a mismatched unit, and provenance stays a local, in-process concern. **Display units come from `relations`, not a second list.** `UnitRelation` is the *single* source of within-dimension conversion. A renderer's display/entry unit selector — surfaced in the schema as `x-unitAlternatives` — is **derived -from the same `relations`**: the alternatives for a field's unit are the units -it can convert to (directly or by chaining), each with the exact ratio the -conversion uses. There is no separate `alternatives` declaration to keep in -sync; declaring a `UnitRelation` both enables `convert` and offers the unit in -the selector. +from the same `relations`**: the alternatives for a field's unit are its +**direct relation neighbours** — every `UnitRelation` edge that touches the +unit, in either direction — each with the exact ratio that edge declares. There +is no separate `alternatives` declaration to keep in sync; declaring a +`UnitRelation` both enables `convert` and offers the unit in the selector. + +Note the scope difference: the alternatives list is the **direct edges only**, +*not* the full transitive closure. `convert` itself chains through intermediate +units (see *Unit conversion*), so a value can convert to more units than the +selector lists; the selector deliberately offers only the one-hop neighbours the +`relations` array names directly. (In the lab system `kg` lists both `g` and `t` +as alternatives, but `g` lists only `kg` — even though `g → t` converts by +chaining through `kg`.) ## Unit conversion — `UnitRelation` and `convert` @@ -500,8 +551,12 @@ The search is a **compile-time breadth-first search** over `relations`, so it finds a path with the **fewest hops**; ties are broken by the entries' declaration order in the array. The conversion factor is the **product of the edge ratios** along the chosen path (`g → t` via `kg` composes `g→kg` and -`kg→t`). Provenance records **each hop as its own step**, so the derivation shows -the units it passed through, not just the endpoints. +`kg→t`), composed inside `convert` before a single scaling multiply. Provenance +records the conversion as **one endpoint-to-endpoint step** — a single +`convert -> ` node whose operation names only the source and +target ids (e.g. `convert g -> t`). The intermediate units the search passed +through are **not** recorded as separate hops; the composed ratio is applied in +one multiplication and one node. Only ratio-based `UnitRelation` edges take part in composition. A user-provided `convert` (a non-ratio conversion such as °C ↔ °F) is a **direct edge only** — it @@ -514,6 +569,32 @@ This coexists with the `consteval` unit algebra: the algebra combines *different dimensions* (`kg * m3`, `kg / m3`, same-unit `→ scalar`), while `convert` *scales within one dimension*. +### 5. Conversion edge cases (failure modes) + +- **Chained composition overflows at *compile* time, not run time.** + `conversionRatio` is `consteval` and multiplies the edge ratios *during + constant evaluation*, so composing a path whose product exceeds the `int64` + numerator/denominator range is a **hard compile error** at the call site that + requested the conversion — never a silent runtime overflow. (A *direct* + single-edge ratio is likewise composed at compile time.) The runtime scaling + multiply that `convert` then applies to the value can still overflow per the + usual `Rational` envelope — that part is the documented magnitude limitation, + distinct from this compile-time guard. +- **The search always terminates.** The BFS runs over a fixed-capacity queue + (`capacity = 2 * relationCount + 1`) and refuses to revisit a unit already + enqueued (the `seen` scan over visited nodes), so it visits each reachable + unit at most once and cannot loop, whatever the shape of the `relations` + graph. +- **Ties resolve by declaration order, and paths are *not* cross-checked.** + When two equal-length paths reach the target, the BFS takes whichever edge + appears **first in the `relations` array**. The framework composes that one + path and does **not** verify that a different path would yield the same + factor. If an application's declared ratios are mutually inconsistent (a + redundant edge that disagrees with the rest of the graph), **reordering + `relations` can silently change a chained conversion factor**. Keeping the + declared ratios mutually consistent is the application's responsibility (the + framework composes whatever shortest path it finds). + ## API reference The public surface, grouped by role. `U` / `From` / `To` are enumerators of an @@ -528,10 +609,11 @@ readability). | Symbol | Kind | Purpose | |---|---|---| | `UnitMeta { id, display, defaultDecimals }` | struct | Static per-unit description returned by `UnitTraits::meta`. `id` is wire/schema vocabulary, `display` is human text, `defaultDecimals` seeds declared precision. | -| `UnitTraits` | class template | **Customisation point.** The application specialises it for its unit enum `E`: `static constexpr UnitMeta meta(E)` and `static constexpr std::array, N> relations`. | +| `UnitTraits` | class template | **Customisation point.** The application specialises it for its unit enum `E`: a required `static constexpr UnitMeta meta(E)` (satisfies `UnitEnum`) and an *optional* `static constexpr std::array, N> relations` (detected by `HasUnitRelations`; a unit system without it simply offers no `convert`/alternatives). | | `UnitRelation { from, to, fromTo }` | struct | One exact peer-to-peer ratio (`1 from == fromTo·to`, a positive `Rational`). Entries drive the auto-generated `convert`, conversion chaining, **and** the display-unit selector. | -| `UnitAlternative { unit, num, den }` | struct | A *derived* per-unit view (computed from `relations`, not declared): one convertible display/entry unit and its exact ratio. Returned by `Quantity::unitAlternatives()`; feeds `x-unitAlternatives`. | +| `UnitAlternative { unit, num, den }` | struct | A *derived* per-unit view (computed from `relations`, not declared): one direct-neighbour display/entry unit and its exact alternative-to-canonical ratio (`num`/`den`, `std::int64_t`). Returned by `Quantity::unitAlternatives()`; feeds `x-unitAlternatives`. | | `UnitEnum` | concept | Satisfied by an enum with a `UnitTraits::meta`. Constrains `Quantity`. | +| `isQuantity` | `inline constexpr bool` variable template | Compile-time test: `true` when `T` is a `Quantity<...>`. | | `operator*`, `operator/` (on `E`) | `consteval` | Application-supplied unit algebra deducing cross-dimension result units; an unsupported combination fails to compile. | ### `Quantity` — compile-time members @@ -542,18 +624,21 @@ readability). | `declaredDecimals` | `static constexpr std::uint32_t` | The field's declared decimal count. | | `declaredPrecision()` | `static constexpr DecimalPlaces` | `declaredDecimals` as the strong type. | | `unitMeta()` | `static constexpr UnitMeta` | The unit's `UnitMeta`. | -| `unitAlternatives()` | `static constexpr std::span>` | The convertible display/entry units for this field's unit, **derived from `UnitTraits::relations`** (empty when none). Drives the schema's `x-unitAlternatives`. | +| `unitAlternatives()` | `static constexpr std::span>` | The **direct-neighbour** display/entry units for this field's unit — every `UnitRelation` edge touching the unit, **derived from `UnitTraits::relations`** (empty when none; not the transitive closure). Drives the schema's `x-unitAlternatives`. | ### `Quantity` — construction | Member | Signature | Notes | |---|---|---| | default ctor | `constexpr Quantity() noexcept` | The **empty** state. | -| value ctor | `constexpr Quantity(Rational engaged) noexcept` | Engages, keeping the value's own runtime precision. | -| optional ctor | `constexpr Quantity(std::optional) noexcept` | Adopts a payload as-is. | -| cross-precision ctor | `constexpr Quantity(Quantity) noexcept` | Same unit, different declared precision; value carries over unchanged. | -| `fromDouble` | `static Quantity fromDouble(double raw) noexcept` | Tags the leaf at `declaredPrecision()`; **never empty from a finite value** (empty only when `raw` is non-finite / doesn't fit). | -| `fromOptional` | `static Quantity fromOptional(std::optional) noexcept` | Empty in → empty out, preserving the declared-precision arg. | +| value ctor | `Quantity(Rational engaged)` | Engages, keeping the value's own runtime precision. | +| optional ctor | `Quantity(std::optional)` | Adopts a payload as-is. | +| cross-precision ctor | `Quantity(Quantity)` | Same unit, different declared precision; value carries over unchanged. | +| `fromDouble` | `static Quantity fromDouble(double raw)` | Tags the leaf at `declaredPrecision()`; **never empty from a finite value** (empty only when `raw` is non-finite / doesn't fit). | +| `fromOptional` | `static Quantity fromOptional(std::optional)` | Empty in → empty out, preserving the declared-precision arg. | + +> The declared-decimals template argument `Dec` is constrained to `[1, kMaxDecimalPlaces]` (18). +> Values outside that range cause a `static_assert` failure at compile time. ### `Quantity` — access, precision, provenance @@ -563,8 +648,8 @@ readability). | `value()` | `const std::optional& value() const noexcept` | The payload; pattern-match or `->` it. | | `value_or(fallback)` | `Rational value_or(Rational const&) const` | Payload if engaged, else the fallback. | | `operator*` | `const Rational& operator*() const` | Unchecked access to the engaged value (UB when empty, like `std::optional`). | -| `withDecimalPlaces(p)` | `constexpr Quantity withDecimalPlaces(DecimalPlaces) const noexcept` | Retags actual precision; no-op on empty. Value unchanged. | -| `atDeclaredPrecision()` | `constexpr Quantity atDeclaredPrecision() const noexcept` | Retags actual precision to the declared one; no-op on empty. | +| `withDecimalPlaces(p)` | `Quantity withDecimalPlaces(DecimalPlaces) const` | Retags actual precision (silently clamped to `[1, kMaxDecimalPlaces]`); no-op on empty. Value unchanged. | +| `atDeclaredPrecision()` | `Quantity atDeclaredPrecision() const` | Retags actual precision to the declared one; no-op on empty. | | `named(label)` | `Quantity named(std::string label) const` | Returns a same-unit quantity marked as the symbol `label`; builds a fresh history node (no-op returning empty on empty, or with tracing off). | | `equation()` | `std::vector equation() const` | The worked formula as print-ready lines (see *Provenance*). Single-element (the formatted value) when empty or tracing off. | | `operator Quantity()` | `operator Quantity() const` | Implicit same-dimension conversion; delegates to `convert`, propagates empty, records a provenance step. | @@ -592,9 +677,14 @@ yields empty. | `std::formatter>` | specialisation | Renders value + unit (`5.2kW`, `N/A%`). No `operator<<`. | | `std::formatter>` | specialisation | Forwards to the `Quantity` formatter. | -On the wire (`glz::meta` / `to_json_schema`) a quantity is its nullable -`Rational` payload; the unit surfaces only in the generated JSON Schema -(`ExtUnits`) and the declared precision as `x-decimalPlaces`. +On the wire, `glz::meta` reduces the instance to its nullable +`Rational` `payload`. `to_json_schema` stamps the unit onto the +schema as `ExtUnits{ unitAscii = meta.id, unitUnicode = meta.display }` — that +is the *only* thing the quantity's own schema hook adds. The precision and +alternatives keys (`x-decimalPlaces`, `x-unitAlternatives`) are **not** emitted +by `to_json_schema`; the forms schema-merge layer (`morph::forms`' +`mergeSchemaExtras`) adds them per `Quantity` member from `declaredDecimals` +and `unitAlternatives()`. ## Design decisions @@ -805,6 +895,62 @@ parentheses appear only because `/` over a sum requires them. Had `diff` been used once, it would have inlined as `"a" - "b"` in place, with no `c1` at all — the placeholder exists precisely to avoid writing shared work twice. +## Cross-references + +- **[`rational.md`](rational.md)** — the payload type, and the one to read + first for anything about values, precision, or overflow. `Quantity` inherits + *all* of its numeric behaviour from `Rational`: the `int64` overflow envelope + (silent UB in the no-error-channel operators), the `DecimalPlaces` precision + tag and its `std::max` propagation, the half-away-from-zero rounding of + `fromFloat` (via `llround`) that `Quantity::fromDouble` relies on, and the + shared decimal formatter (`formatRationalDecimal` builds on `Rational`'s + `{:.Nf}` output). This spec does not restate those rules. +- **[`forms.md`](forms.md)** — how a `Quantity` field reaches a generated form: + `ExtUnits` (unit id/display, emitted by `to_json_schema`) plus the + schema-merge keys `x-decimalPlaces` (from `declaredDecimals`) and + `x-unitAlternatives` (from `unitAlternatives()`), added per member by + `morph::forms`' `mergeSchemaExtras`. +- **[`choice.md`](choice.md)** and **[`datetime.md`](datetime.md)** — + one-kind-of-empty siblings. `Choice` and `Timestamp` share `Quantity`'s + `std::optional`-backed single empty state and total `==` (empty == empty). + `Timestamp` differs on ordering — its `operator<=>` is total (empty sorts + before any engaged value) where `Quantity`'s *throws* on an empty operand. + +## Limitations + +Honest constraints of the current design, distinct from *Out of scope* (things +deliberately not attempted): + +- **The unit algebra does not scale.** `operator*` / `operator/` on the unit + enum are hand-written, O(pairs) `consteval` tables the application maintains + by listing every supported combination. There is **no dimensional-analysis + engine**: past roughly a dozen units the table grows unwieldy, and a + *missing* rule surfaces as a compile error at a **distant call site** (the + arithmetic expression that attempted the combination), not at the trait + declaration. The type system catches the mistake, but the diagnostic points + away from the fix. +- **Provenance is on by default and allocates eagerly.** With + `MORPH_QUANTITY_PROVENANCE=1` (the default), **every** construction, + arithmetic op, conversion, and `named()` heap-allocates a fresh `ASTNode` + (`recordLeaf` and the `MORPH_Q_BUILD` macro) — even though the derivation + never crosses the wire and has a **single consumer**, `equation()`. For hot + paths that never call `equation()`, build with `MORPH_QUANTITY_PROVENANCE=0`: + the API stays callable and no nodes are allocated. +- **`int64` ratio overflow for wide-range unit systems.** Conversion ratios are + exact `Rational`s of 64-bit integers. A unit system spanning many orders of + magnitude (pico- to tera-, say) risks overflowing a composed chained ratio — + which, per *Conversion edge cases*, is a compile error — or the runtime + scaling multiply. Keep within-dimension ratios inside the `int64` envelope. +- **`named()` / `equation()` output differs materially between build flags.** + With tracing off, `equation()` collapses to the bare value and `named()` is a + no-op that discards the name. Code that depends on the *content* of those + outputs (tests, generated reports) behaves differently across the two builds — + the toggle changes observable behaviour, not just performance. + +> **Doc note (not a code change).** The header comments in `quantity.hpp` and +> `detail/quantity_equation.hpp` still cite the pre-move path +> `docs/quantity_type.md`; this spec now lives at `docs/spec/quantity_type.md`. + ## Out of scope - Serializing history on the wire — only the `Rational` payload travels. diff --git a/docs/spec/rational.md b/docs/spec/rational.md new file mode 100644 index 0000000..13a2368 --- /dev/null +++ b/docs/spec/rational.md @@ -0,0 +1,370 @@ +# `Rational`, `DecimalPlaces`, `RationalError` — design + +`morph::math::Rational` is a small, value-semantic, trivially-copyable struct +representing the exact rational number `numerator/denominator` with +`std::int64_t` components. It carries a runtime decimal-precision tag as a +strong type (`DecimalPlaces`). Arithmetic is exact — sums, differences, +products, and quotients are reduced to canonical form with no floating-point +rounding error. The precision tag affects only decimal scaling +(`Rational::fromFloat`) and rounding (`Rational::toDouble`, formatting); it +never changes a stored value. + +Adapted from LASTRADA `JPMath/Rational.hpp`, with the `boxed` strong-type +dependency replaced by a self-contained `DecimalPlaces` and a Glaze wire codec +so the type round-trips through the morph JSON wire with its invariants restored +on read. + +## Contents + +- [Invariants](#invariants) +- [Support types](#support-types) +- [Construction](#construction) +- [Arithmetic](#arithmetic) +- [Overflow & value-range envelope](#overflow--value-range-envelope) +- [Mixed-type expressions (expected propagation)](#mixed-type-expressions-expected-propagation) +- [Conversion helpers](#conversion-helpers) +- [Rounding helpers (free functions)](#rounding-helpers-free-functions) +- [Comparison](#comparison) +- [Formatting](#formatting) +- [Wire and schema](#wire-and-schema) +- [API reference](#api-reference) +- [Design decisions](#design-decisions) +- [Cross-references](#cross-references) +- [Limitations](#limitations) + +## Invariants + +Every public operation that produces a `Rational` restores: + +- `denominator > 0` (strictly positive — never zero, never negative) +- `gcd(|numerator|, denominator) == 1` +- canonical zero is `0/1` +- `1 <= decimalPlaces.value <= kMaxDecimalPlaces` + +All sign lives in the numerator. + +**No default precision.** Every call site states the precision it intends, e.g. +`Rational{1, 3, DecimalPlaces{9}}`. Precision is capped at `kMaxDecimalPlaces` +(18, the largest `k` for which `10^k` fits in `int64_t`); out-of-range values +assert in debug and clamp into `[1, kMaxDecimalPlaces]` in release. + +The struct never throws. Operations that may fail (zero divisor, non-finite +floating-point input, overflow during decimal scaling) return +`std::expected`. + +## Support types + +| Type | Role | +|---|---| +| `DecimalPlaces` | Strong type for a decimal-precision count. Prevents the precision from being confused with a numerator or denominator at a call site. Explicit construction; default-constructed `value` is `0`. | +| `Numerator` | Strong type for a Rational numerator. Prevents numerator/denominator argument swapping at construction sites. Explicit construction. | +| `Denominator` | Strong type for a Rational denominator. Explicit construction; must never be zero after canonicalisation. | +| `RationalError` | `enum class : std::uint8_t` with three values: `DivisionByZero`, `NotFinite` (non-finite float input), `Overflow` (scaled magnitude exceeds `int64_t`). | +| `kMaxDecimalPlaces` | `constexpr std::uint32_t = 18` — largest decimal precision supported. | + +## Construction + +| Path | Signature / example | Notes | +|---|---|---| +| Default | `Rational() noexcept` | Canonical zero (`0/1`) at precision 1. | +| Whole integer | `Rational(int64_t, DecimalPlaces) noexcept` | Stores `whole/1` at the given precision; clamped. | +| Full | `Rational(Numerator, Denominator, DecimalPlaces) noexcept` | Canonicalises: flips sign, reduces by gcd, clamps zero denominator to 1. | +| `from` | `static expected from(Numerator, Denominator, DecimalPlaces) noexcept` | Validating factory — rejects `denominator == 0` with `DivisionByZero` instead of clamping. | +| `fromFloat` | `static expected fromFloat(double/float/long double, DecimalPlaces) noexcept` | Lifts a floating-point value to a rational scaled to the requested precision. Returns `NotFinite` for NaN/Inf, `Overflow` when scaled magnitude exceeds `int64_t`. Not `constexpr`. Uses `llround` internally. | +| `zero(p)` | `static constexpr Rational zero(DecimalPlaces) noexcept` | `0/1` at the given precision. | +| `one(p)` | `static constexpr Rational one(DecimalPlaces) noexcept` | `1/1` at the given precision. | + +**Wire path.** The Glaze deserialisation path (`setWire`) rebuilds through the +canonicalising constructor, silently clamping hostile input (`den == 0`, +out-of-range `dp`, `INT64_MIN` components whose negation would overflow) instead +of asserting. + +## Arithmetic + +**Binary arithmetic propagates `std::max` of the two operands' `decimalPlaces`** +— the wider precision wins. Comparison (`<=>`, `==`) is purely value-based on +the canonical `(numerator, denominator)` pair and ignores `decimalPlaces`. + +| Operation | Returns | Notes | +|---|---|---| +| `operator+`, `operator-`, `operator*` (plain `Rational` × `Rational`) | `Rational` | `noexcept`, return a bare `Rational` — no error channel. This means *representable* results never fail; it does **not** mean the operation cannot go wrong. Reduced int64 cross-terms exceeding ~2^63 are **undefined behaviour**, not a reported error (see [Overflow & value-range envelope](#overflow--value-range-envelope)). Reduce-before-multiply (Knuth 4.5.1) to extend safe int64 range. Cross-cancellation before multiplication. | +| `operator/`, `dividedBy` (plain `Rational` ÷ `Rational`) | `expected` | `DivisionByZero` when divisor is zero. Implemented via reciprocal and multiplication. | +| `operator-` (unary) | `Rational` | Negates numerator. **Negating `INT64_MIN` overflows.** | +| `reciprocal` | `expected` | Multiplicative inverse. `DivisionByZero` when value is zero. | +| `operator+=`, `-=`, `*=` (in-place) | `Rational&` | Mutate `*this`, widen precision to `max`, canonicalise. | + +## Overflow & value-range envelope + +`Rational` is **fixed-width `int64` arithmetic, not a bignum.** Both the +numerator and denominator are `std::int64_t`, and the additive/multiplicative +operators do their intermediate math in that same 64-bit type. This gives the +type a hard value-range envelope that the `noexcept` signatures do not advertise. + +**`operator+` / `operator-` / `operator*` are `noexcept` and return a bare +`Rational` — but they can still be wrong.** The reduce-before-multiply and +cross-cancellation steps (Knuth 4.5.1) push the point at which the intermediate +products overflow, but they do not eliminate it. When a reduced cross-term +exceeds ~2^63 the signed multiplication/addition is **undefined behaviour**, not +a trapped or reported error: + +- `operator+=` / `operator-=` compute `numerator * rightDenominatorScaled ± + rhs.numerator * leftDenominatorScaled` and `denominator * + rightDenominatorScaled`. Adding two fractions over large *coprime* + denominators (nothing to cancel) grows the common denominator toward + `d1 * d2`; both the scaled numerator and the product denominator can pass 2^63. +- `operator*=` cross-cancels first, but a genuinely large coprime product + (`reducedLeftNumerator * reducedRightNumerator`, likewise the denominators) + still overflows. + +Because these operators have no error channel, an overflow here is **silent** — +the result is a garbage `Rational` (or a sanitizer trap under UBSan), never a +`RationalError`. Contrast the *only* fallible plain operator, `operator/` +(division), whose sole failure mode is a trivial divisor-is-zero check yet which +returns `std::expected`. The fallibility is inverted: the operation that almost +cannot fail is the one that reports, and the ones that carry real UB do not (see +[Limitations](#limitations)). + +**`dp` → approximate maximum representable magnitude.** A value scaled to +precision `dp` (as `fromFloat` builds it) has denominator `10^dp`, so the +largest magnitude whose scaled numerator still fits `int64` is roughly +`INT64_MAX / 10^dp ≈ 9.22e18 / 10^dp`: + +| `dp` | denominator `10^dp` | approx. max magnitude | +|---|---|---| +| 1 | 10 | ~9.2e17 | +| 2 | 100 | ~9.2e16 | +| 4 | 10^4 | ~9.2e14 | +| 6 | 10^6 | ~9.2e12 | +| 9 | 10^9 | ~9.2e9 (≈ 9.2 billion) | +| 12 | 10^12 | ~9.2e6 (≈ 9.2 million) | +| 15 | 10^15 | ~9223 | +| 18 | 10^18 | **≈ 9.2** | + +At the maximum precision `dp = 18` the representable magnitude is only about +**±9.2** — a value tagged with 18 decimal places has essentially spent its whole +`int64` budget on the fractional part. `fromFloat` guards this edge explicitly +(it returns `RationalError::Overflow` when the scaled magnitude reaches 2^63), +but the arithmetic operators downstream do **not** re-check it, so intermediate +results that leave the envelope are UB regardless of the entry guard. + +**`INT64_MIN` negation hazards.** `INT64_MIN` (`-2^63`) has no positive +counterpart in `int64`, so every place that negates a component is a latent UB +site when that exact value reaches it: + +- **unary `operator-`** — `Rational{Numerator{-numerator}, ...}`: negating an + `INT64_MIN` numerator overflows. +- **`from`** — guards **only** `denominator == 0`; it does not screen + `INT64_MIN` components, so a hostile-but-nonzero `(INT64_MIN, …)` pair flows + straight into the canonicalising constructor. +- **`reciprocal`** — negates the numerator in the `numerator < 0` branch; + `INT64_MIN` there overflows. +- **`canonicalise`** — flips sign for a negative denominator (`numerator = + -numerator`) and takes `absoluteNumerator = numerator < 0 ? -numerator : + numerator`; both negate `INT64_MIN`. This is the shared sink for every + constructor and operator, so any path that lets `INT64_MIN` reach + canonicalisation is unsafe. + +Only the wire codec (`setWire`) defends against this: it maps an `INT64_MIN` +`num`/`den` to `-INT64_MAX` *before* constructing, so untrusted input never +negates the trap value. In-code call sites get no such guard — keep operands +well inside the envelope above. + +## Mixed-type expressions (expected propagation) + +Whenever an arithmetic expression contains an +`std::expected` sub-expression or a floating-point +operand, the whole expression evaluates to +`std::expected`. The float operand is lifted via +`Rational::fromFloat` — its precision is taken from the Rational operand's +`getDecimalPlaces()`. Errors short-circuit left to right. + +``` +auto const a = Rational{7, 2, DecimalPlaces{9}}; +auto const b = Rational{2, DecimalPlaces{9}}; +auto const c = Rational{1, 2, DecimalPlaces{9}}; +auto result = a / b + c * 3.5; +// decltype(result) == std::expected +``` + +Implemented through constrained templates with `RationalLike`, `LiftableOperand`, +and `NeedsLifting` concepts. Valid operand combinations: Rational × Rational, +Rational × ExpectedRational, ExpectedRational × (anything liftable), +float × Rational/ExpectedRational. Two floats are not accepted (at least one +Rational-family operand must be present). + +## Conversion helpers + +| Member | Signature | Notes | +|---|---|---| +| `toDouble()` | `double toDouble() const noexcept` | Converts to `double`, rounded to this value's `decimalPlaces`. | +| `toDouble(n)` | `double toDouble(uint32_t) const noexcept` | Converts to `double`, rounded to `n` decimal places. Falls back to unrounded conversion when `n > 18`. | + +**`toDouble` is a display/interop reading, never an exact path.** The +implementation computes `double(numerator) / double(denominator)` *first*, then +rounds the quotient to `n` decimal places (`std::round(raw * 10^n) / 10^n`). +The division happens in IEEE-754 `double`, whose mantissa holds only 53 bits: +any numerator or denominator beyond 2^53 (~9.0e15) is already rounded to the +nearest representable `double` **before** the decimal rounding runs, so the +result can differ from the exact rational even at magnitudes the `int64` +components represent perfectly. Treat `toDouble` as the way to *show* or hand a +`Rational` to a floating-point consumer — never as a lossless round-trip. The +exact value lives only in the `(numerator, denominator)` pair; the empty-spec +`"{}"` format (`"n/d"`) and the wire codec are the exactness-preserving +readings. + +## Rounding helpers (free functions) + +Free functions in `morph::math`, found by ADL: + +| Function | Signature | Notes | +|---|---|---| +| `abs` | `constexpr Rational abs(Rational) noexcept` | Absolute value. Precision preserved. | +| `ceil` | `constexpr int64_t ceil(Rational) noexcept` | Rounds toward positive infinity. | +| `floor` | `constexpr int64_t floor(Rational) noexcept` | Rounds toward negative infinity. | +| `trunc` | `constexpr int64_t trunc(Rational) noexcept` | Truncates toward zero. | + +## Comparison + +| Operation | Notes | +|---|---| +| `operator<=>` | Three-way comparison. **Value-only: ignores `decimalPlaces`.** Exact for the full int64 range: cross-products computed in 128 bits (via `detail::mulU64`, a 64×64→128-bit product using `unsigned __int128` on GCC/Clang, portable 32-bit limbs on MSVC). Sign-checked first; zero short-circuits. | +| `operator==` | **Value-only: ignores `decimalPlaces`.** Returns `true` when canonical pairs are identical. | + +## Formatting + +`std::format` support is split by the supplied spec: + +| Spec | Output | Example | +|---|---|---| +| empty `"{}"` | Exact rational form — `"n/d"`, or `"n"` when integer | `"7/2"`, `"3"` | +| non-empty `"{:.3f}"` | Delegated to `std::formatter` on `toDouble()` | `"3.500"` | + +Implemented as a `std::formatter` specialisation with a +`delegateToDouble` flag set during `parse`. + +## Wire and schema + +Over the morph JSON wire a `Rational` travels as the object +`{"num":617,"den":50,"dp":2}`. Reading goes through the canonicalising +constructor, so a non-canonical payload (`1234/100`) or a hostile one +(`den == 0`, out-of-range `dp`) always lands as a valid, reduced value. + +The Glaze codec (`glz::meta`) uses `glz::custom` to +route serialisation through the `Wire` struct. A `to_json_schema` +specialisation preserves schema shape by delegating to `Wire`'s schema. + +## API reference + +### `Rational` — data members + +| Member | Type | Invariant | +|---|---|---| +| `numerator` | `int64_t` | Carries the sign of the rational value. | +| `denominator` | `int64_t` | Strictly positive. Never zero, never negative. | +| `decimalPlaces` | `DecimalPlaces` | `1 <= value <= kMaxDecimalPlaces`. | + +### `Rational` — accessors + +| Member | Returns | +|---|---| +| `getDecimalPlaces()` | `DecimalPlaces` — the current precision tag. | +| `isZero()` | `bool` — `numerator == 0`. | +| `isInteger()` | `bool` — `denominator == 1`. | +| `isNegative()` | `bool` — `numerator < 0`. | + +### `Rational` — wire helpers + +| Member | Signature | +|---|---| +| `setWire(Wire)` | `void noexcept` — rebuilds through canonicalising constructor, clamping hostile input. | +| `getWire()` | `Wire noexcept` — canonical members ready for JSON encoding. | +| `struct Wire` | `{ int64_t num; int64_t den; uint32_t dp; }` — flat JSON representation. | + +### `Rational` — `constexpr` non-member operators + +``` +Rational operator+(Rational, Rational) noexcept; +Rational operator-(Rational, Rational) noexcept; +Rational operator*(Rational, Rational) noexcept; +expected operator/(Rational, Rational) noexcept; +``` + +### `Rational` — mixed-type operator templates (namespace `morph::math`) + +``` +// At least one operand must be Rational-family; at least one must be +// ExpectedRational or float; both operands must be liftable. Two plain +// Rationals fail the NeedsLifting clause and take the non-template path. +template + requires (RationalLike || RationalLike) + && (NeedsLifting || NeedsLifting) + && (LiftableOperand && LiftableOperand) +expected operator+(Left const&, Right const&) noexcept; +// Same for -, *, / +``` + +## Design decisions + +| Decision | Choice | Why | +|---|---|---| +| Precision | **Runtime tag (`DecimalPlaces`), not compile-time** | Precision is a property of the *value*, not the *type* — it is set by the data source and propagates through arithmetic dynamically. A compile-time tag would make every precision a distinct type, breaking cross-precision arithmetic without explicit conversion. | +| No default precision | **Caller must supply `DecimalPlaces` at every construction** | Prevents accidental use of a wrong or unknown precision. Every call site states what it intends. | +| Strong types for numerator/denominator | **`Numerator` / `Denominator` / `DecimalPlaces`** | Prevents argument-order mistakes (`Rational{3, 5, dp}` vs `Rational{5, 3, dp}`). `Numerator` / `Denominator` are explicit so the type must be spelled out. | +| Error handling | **`std::expected`** | The struct never throws. Fallible operations return an expected type; the caller decides how to handle errors. Short-circuit via mixed-type expression templates propagates failures. | +| Canonicalisation on `setWire` | **Silently clamps hostile input** | Untrusted wire data (den==0, out-of-range dp, INT64_MIN) always produces a valid reduced value rather than asserting or propagating UB. | +| Comparison ignores precision | **Value-only `<=>` and `==`** | Two values equal in magnitude should compare equal regardless of how many decimals they claim. Precision is a display/rounding concern, not a value property. | +| Max-precision propagation | **Result precision = max of operands** | A computation is never less precise than its most precise input. There is no in-place retag helper; a caller needing a different tag constructs a fresh `Rational` with the desired `DecimalPlaces`. | +| 128-bit cross-product comparison | **`detail::mulU64`** | Exact ordering over the full int64 range without overflow. Uses `unsigned __int128` when available (GCC/Clang), portable 32-bit limb decomposition on MSVC. | +| Negation limitation | **`INT64_MIN` overflows** | Documented limitation. The wire codec clamps `INT64_MIN` components away for untrusted input. | +| `fromFloat` not `constexpr` | **Uses `std::llround` / `std::isfinite`** | These standard library functions are not `constexpr`. The `fromFloat` overloads are `inline` out-of-class, `noexcept` but not `constexpr`. | + +## Cross-references + +| Spec | Relationship | +|---|---| +| [`quantity_type.md`](quantity_type.md) | `Rational` is the **runtime substrate** for `Quantity`. A `Quantity`'s declared precision and the forms layer's `x-decimalPlaces` schema annotation both resolve, at runtime, to a `Rational`'s `DecimalPlaces` tag — the `dp` value carried on the wire and propagated through arithmetic here is exactly the precision a `Quantity` declares. The overflow envelope and `INT64_MIN` hazards documented above therefore bound `Quantity` too. | +| [`forms.md`](forms.md) | The form generator reads `x-decimalPlaces` (and the `Rational` wire shape `{"num","den","dp"}`) to build precision-aware numeric inputs; a form value is a `Rational` under the hood, so its display uses `toDouble`/formatting and its exact value uses the wire codec. | +| [`security.md`](security.md) | `setWire` performs the untrusted-wire **clamping** (`den == 0 → 1`, out-of-range `dp` → `[1, 18]`, `INT64_MIN` → `-INT64_MAX`). This is the boundary defence that keeps a hostile payload from reaching the UB-prone negation/overflow sites; see the clamping semantics discussion there. | +| [`datetime.md`](datetime.md) | Contrast case for wire-decode policy: the `DateTime` codec is **strict** (rejects malformed input) whereas `Rational::setWire` is **lenient/clamping** (silently repairs it). See [Limitations](#limitations) for why the difference matters. | + +## Limitations + +- **Fixed-width `int64`, not a bignum — with silent overflow UB in the + "safe-looking" operators.** `+`, `-`, and `*` are `noexcept` and hand back a + bare `Rational`, which reads as "infallible" but means the opposite for + out-of-envelope inputs: a reduced cross-term past ~2^63 is undefined + behaviour, produced *silently*. The fallibility is inverted — `operator/`, + whose only failure is a trivial divisor-is-zero check, returns + `std::expected`, while the genuinely dangerous `+`/`-`/`*` do not. See + [Overflow & value-range envelope](#overflow--value-range-envelope) for the + `dp` → magnitude table. +- **`setWire` clamps hostile input rather than rejecting it.** `den == 0` + becomes `1`, an `INT64_MIN` component becomes `-INT64_MAX`, an out-of-range + `dp` is pulled into `[1, 18]`. The invariants are always restored, but a + *corrupt amount silently becomes a specific wrong number* — e.g. a payload + meant to carry `x/0` lands as `x/1`, a completely different value, with no + error surfaced. This is deliberate (a `Rational` never propagates UB from the + wire) but it trades detectability for robustness. It is the opposite policy + from the strict `DateTime` codec, which rejects malformed input outright + (cross-ref [`datetime.md`](datetime.md)); a caller that needs "reject, don't + guess" semantics for amounts must validate before decode. +- **`DecimalPlaces` has a floor of 1.** The invariant is `1 <= value <= 18`, so + precision **0 is unrepresentable**. This excludes zero-decimal currencies + (JPY, KRW) and plain integer counts from being tagged with their true + precision — they must borrow `dp = 1` and carry a spurious fractional digit. +- **`==` and `<=>` ignore precision, so equality is not substitutability.** + Comparison is purely value-based on the canonical `(numerator, denominator)` + pair. Two `Rational`s can therefore satisfy `a == b` while + `a.toDouble() != b.toDouble()`, because `toDouble()` rounds to each value's + *own* `decimalPlaces`: e.g. `7/8` tagged `dp = 1` reads `0.9` while the same + `7/8` tagged `dp = 2` reads `0.88`, yet the two compare equal. Equal values + are not freely interchangeable in a floating-point context — precision is a + display property the comparison does not see. +- **Float operands in mixed expressions are lifted at the Rational operand's + `dp`.** In a mixed expression (`someRational * 3.5`) the float is converted + via `fromFloat` using the precision of the Rational-family operand + (`liftPrecision`), which **snaps the literal onto that decimal grid before the + operation**. A coarse `dp` silently quantises the float: with a `dp = 1` + Rational, `3.57` is lifted to `36/10` (i.e. `3.6`) before multiplying, so the + literal you wrote is not the value used. The grid is chosen by the Rational, + not the float. \ No newline at end of file diff --git a/docs/spec/registry.md b/docs/spec/registry.md new file mode 100644 index 0000000..88af1ae --- /dev/null +++ b/docs/spec/registry.md @@ -0,0 +1,593 @@ +# The model registration system — design + +`morph::model` uses a trait-plus-singleton-registry pattern to map C++ model +and action types to string identifiers at static-init time, so that remote +frontends and schema-driven GUIs can discover, instantiate, and execute models +without knowing their concrete types. + +## Contents + +- [Overview](#overview) +- [Registration rules and invariants](#registration-rules-and-invariants) +- [Customisation traits](#customisation-traits) + - [ModelTraits](#modeltraits) + - [ActionTraits](#actiontraits) +- [Validation and logging policy](#validation-and-logging-policy) + - [ActionValidator](#actionvalidator) + - [Loggable](#loggable) + - [ActionLogPolicy](#actionlogpolicy) +- [Type-erased holders and factory](#type-erased-holders-and-factory) + - [IModelHolder](#imodelholder) + - [ModelHolder](#modelholder) + - [ModelFactory](#modelfactory) +- [Singleton registries](#singleton-registries) + - [ActionDispatcher](#actiondispatcher) + - [ModelRegistryFactory](#modelregistryfactory) + - [ActionExecuteRegistry](#actionexecuteregistry) +- [Registration macros](#registration-macros) + - [BRIDGE_REGISTER_MODEL](#bridge_register_model) + - [BRIDGE_REGISTER_ACTION](#bridge_register_action) + - [BRIDGE_REGISTER_VALIDATOR](#bridge_register_validator) +- [API reference](#api-reference) +- [Design decisions](#design-decisions) +- [Thread safety](#thread-safety) +- [Failure modes](#failure-modes) +- [Limitations](#limitations) +- [Cross-references](#cross-references) + +## Overview + +Every model and action type that participates in morph's remote or schema-driven +infrastructure must be registered with a string id. The registration system +provides: + +- **Traits** — `ModelTraits` and `ActionTraits` that map types to string + ids and JSON codecs. Users specialise them directly or use macros. +- **Validators** — `ActionValidator` that decides whether a partially-built + action draft is ready to execute. +- **Logging policy** — `ActionLogPolicy` and `Loggable` that control whether + an action's executions are recorded and how duplicates are coalesced. +- **Type-erased holders** — `IModelHolder` / `ModelHolder` that own a model + instance and carry an optional action log attachment. +- **Singleton registries** — `ActionDispatcher` (server-side dispatch), + `ModelRegistryFactory` (model instantiation by string id), and + `ActionExecuteRegistry` (client/schema-driven generic execute). +- **Macros** — `BRIDGE_REGISTER_MODEL`, `BRIDGE_REGISTER_ACTION`, + `BRIDGE_REGISTER_VALIDATOR` that specialise traits and register into + singletons at static-init time. + +## Registration rules and invariants + +Registration is not a runtime call the application makes; it is a side effect of +static initialisation of file-scope objects the macros emit. That machinery only +works if a handful of invariants hold. **Read this section before adding a model +or action to any target other than a single executable** — the most common +failure (a model that silently never registers) is a linking problem, not a code +problem, and produces no diagnostic. + +### One macro invocation per translation unit + +Each `BRIDGE_REGISTER_*` macro emits two things at namespace scope: + +1. an **explicit template specialisation** — `ModelTraits`, `ActionTraits`, + or `ActionValidator` — which has external visibility to the type system; + and +2. one or more **file-scope initialiser objects** (`[[maybe_unused]] const bool` + in an anonymous namespace) whose initialisation runs + `registerModelOnce` / `registerActionOnce` / `registerActionExecutorOnce`. + +Because of (1), a macro must be invoked in **exactly one translation unit**. +Placing a `BRIDGE_REGISTER_*` invocation in a header that is included by more +than one `.cpp` defines the same explicit specialisation in multiple translation +units — an **ODR violation** (ill-formed, no diagnostic required). The anonymous +namespace makes the *initialiser* objects internal to each TU, so it does not +save you: it would register the model once per including TU while the duplicated +specialisation remains ill-formed. Register each type in one `.cpp` (or a header +that is guaranteed to be compiled into exactly one TU). + +### Static initialisation only fires in linked translation units + +Static-init "guarantees registrations are live before `main()`" (see +[Design decisions](#design-decisions)) **only for translation units the linker +actually keeps**. The initialiser object is never referenced by name from +application code — nothing has a symbolic dependency on it. Consequences: + +- **Object files / whole executables**: an object file linked directly into an + executable contributes its static initialisers, so registration works with no + extra ceremony. This is the common case and the one the design optimises for. +- **Static libraries (`.a` / `.lib`)**: a linker pulls in an archive member + **only if some symbol in it is already referenced**. A registration-only TU + exposes no referenced symbol, so the linker drops the member and the + initialiser never runs — the model or action **silently never registers**, and + the first symptom is a runtime `std::runtime_error` ("unknown model type" / + "unknown action") far from the cause. Force the member to be retained: + `--whole-archive` / `-Wl,--whole-archive` (GNU/LLD), `-force_load` (Apple ld), + `/WHOLEARCHIVE` (MSVC), or CMake's `$`. Prefer + linking registration TUs into the executable's own object set when practical. +- **Dynamically loaded modules (`dlopen`/`LoadLibrary`)**: registrations run when + the module is loaded, i.e. **after `main` has started**, not before. Any code + path that could dispatch/create a plugin's model must run after the module is + loaded; there is no ordering guarantee relative to other TUs' static init. + +### Remotely instantiated models must be default-constructible + +`ModelRegistryFactory::create` reaches models through +`ModelFactory::create()`, which does `std::make_unique>()` +with **no constructor arguments** (see `model.hpp`). Any model registered with +`BRIDGE_REGISTER_MODEL` that will be instantiated by string id from a remote +`"register"` message must therefore be **default-constructible**. A model with no +accessible default constructor still compiles the macro (which only needs +`ModelTraits`) but fails to compile the factory instantiation. Models that need +injected dependencies are instead constructed by a custom +`HandlerBinding::modelFactory` closure and are not reachable through the string-id +factory path. + +## Customisation traits + +### `ModelTraits` + +Maps a concrete model type to its string type-id. Must be specialised (or +`BRIDGE_REGISTER_MODEL` used) before the model can be registered. The +default is a **forward declaration** — using it without a specialisation +is an incomplete-type error. + +```cpp +template +struct ModelTraits; // forward — specialize or use BRIDGE_REGISTER_MODEL +``` + +### `ActionTraits` + +Maps a concrete action type to its string id, JSON codec, result type, and +optional logging flag. The `BRIDGE_REGISTER_ACTION` macro generates a full +specialisation. Hand-written specialisations (used in tests) predate the +`loggable` member; the framework defaults to `Loggable::Yes` when it is absent +(see `detail::actionLoggable()`). The default is a **forward declaration**. + +```cpp +template +struct ActionTraits; // forward — specialize or use BRIDGE_REGISTER_ACTION +``` + +All four JSON functions throw `detail::ParseError` (a `std::runtime_error` +subclass) on glaze encode/decode failure. + +## Validation and logging policy + +### `ActionValidator` + +Decides whether an in-progress action draft is ready to execute. Resolution +order (highest priority first): + +1. **Explicit specialisation** — via `BRIDGE_REGISTER_VALIDATOR(Action, fn)`. +2. **`bool validate() const` member** on `Action` — auto-detected via the + `detail::HasValidate` concept. +3. **Default** — returns `true` (one-shot semantics: first `set<>` lands and + the action fires). + +Validation is a property of the action, not the model: different actions on the +same model have different readiness requirements. + +```cpp +template +struct ActionValidator { + static constexpr bool ready(const Action& action); +}; +``` + +### `Loggable` + +A strong enum avoiding bare `bool` arguments at registration sites: + +```cpp +enum class Loggable : std::uint8_t { No, Yes }; +``` + +### `ActionLogPolicy` + +Controls how repeated executions are checkpointed into a durable action log. +Only `coalesce` exists; every other action defaults to `false` (every execution +treated as a distinct fact). + +```cpp +template +struct ActionLogPolicy { + static constexpr bool coalesce = false; +}; +``` + +When `coalesce` is `true`, a checkpoint keeps only the most recent entry per +`(modelType, entityKey, actionType)` triple. + +## Type-erased holders and factory + +### `IModelHolder` + +Type-erased wrapper that owns a single model instance. Used by backends to store +heterogeneous models in a single map. + +```cpp +struct IModelHolder { + virtual ~IModelHolder() = default; + [[nodiscard]] virtual std::type_index type() const noexcept = 0; + template Model& into(); + void attachActionLog(std::shared_ptr, std::string contextKey); + bool hasActionLog() const noexcept; + void recordIfAttached(LogEntry entry); +}; +``` + +- `into()` down-casts to a concrete `Model&`; throws `std::bad_cast` on + mismatch. +- `attachActionLog` sets the durable log sink and the instance's stable identity + (stamped onto every `LogEntry`). +- `recordIfAttached` is called automatically by `ActionDispatcher`'s runner and + `Bridge::executeVia` — model code never calls it directly. It fills + `entityKey`, `principal` (from `session::current()`), and `timestampMs` on the + entry before forwarding. + +### `ModelHolder` + +Concrete holder that stores a `Model` by value. Inherits `BackendChangedMixin` +so that backend-change notifications are forwarded automatically when `Model` +declares `void onBackendChanged()`. + +```cpp +template +struct ModelHolder : IModelHolder, BackendChangedMixin { + Model model; + template explicit ModelHolder(Args&&... args); + std::type_index type() const noexcept override; +}; +``` + +### `ModelFactory` + +Creates default-constructed `ModelHolder` instances. If a process-wide +default action log is installed (via `morph::journal::setActionLog`), it is +attached to the new holder automatically (with an empty `entityKey`). This is the +single construction path behind every ordinary model registration, making "set +the log once in `main()`" work uniformly across topologies. + +```cpp +class ModelFactory { + template + static std::unique_ptr create(); +}; +``` + +### `IBackendChangedSink` and `BackendChangedMixin` + +Optional interface for models that need to react to backend switches. `Bridge::switchBackend` +discovers this capability via `dynamic_cast`. `ModelHolder` inherits +`BackendChangedMixin` which conditionally derives from `IBackendChangedSink` +when `M` declares `void onBackendChanged()` (detected by the +`BackendChangedNotifiable` concept). + +## Singleton registries + +`ActionDispatcher` and `ModelRegistryFactory` are both declared in +`registry.hpp` in namespace `morph::model::detail`. `ActionExecuteRegistry` lives +elsewhere — see its section below. + +### `ActionDispatcher` + +Maps `(modelId, actionId)` pairs to type-erased runner functions. Used by +`RemoteServer` to dispatch incoming JSON requests. + +```cpp +class ActionDispatcher { + using Runner = std::function; + template + void registerAction(std::string_view modelId, std::string_view actionId); + std::string dispatch(std::string_view modelId, std::string_view actionId, + IModelHolder& holder, std::string_view payload); + bool coalesce(std::string_view modelId, std::string_view actionId) const; + static ActionDispatcher& instance(); +}; +``` + +- `registerAction` registers a runner that deserialises, executes via + `Model::execute(action)`, serialises the result, and records to the attached + action log when the action is loggable and a log is attached. +- `dispatch` looks up the runner and invokes it; throws `std::runtime_error` for + unknown pairs. +- `coalesce` returns the `ActionLogPolicy::coalesce` value for the pair; + unknown pairs default to `false`. + +### `ModelRegistryFactory` + +Creates `IModelHolder` instances by string type-id. Used by `RemoteServer` to +instantiate models on demand from incoming `"register"` messages. + +```cpp +class ModelRegistryFactory { + template + void registerModel(std::string_view modelId); + std::unique_ptr create(std::string_view modelId); + static ModelRegistryFactory& instance(); +}; +``` + +- `create` throws `std::runtime_error` for unknown model types. + +### `ActionExecuteRegistry` + +Type-erased, JSON-in/JSON-out execute path for actions whose concrete C++ type +is only known by its registered string id at the call site (e.g. a schema-driven +GUI). Populated automatically by `BRIDGE_REGISTER_ACTION`. Every entry calls +through the real `BridgeHandler::execute()`, so sessions, backend +switches, and completions behave exactly as for hand-written call sites. + +Unlike `ActionDispatcher` and `ModelRegistryFactory` (which live in +`morph::model::detail` in `registry.hpp`), the whole `ActionExecuteRegistry` +class is declared in `morph/bridge.hpp` in namespace `morph::bridge` — it depends +on `BridgeHandler`, which `registry.hpp` cannot see. `Completion` here is +`morph::async::Completion`. + +```cpp +class ActionExecuteRegistry { // namespace morph::bridge, declared in bridge.hpp + using Executor = std::function<::morph::async::Completion(void*, std::string_view)>; + template + void registerAction(std::string_view modelId, std::string_view actionId); + [[nodiscard]] ::morph::async::Completion execute( + std::string_view modelId, std::string_view actionId, + void* handler, std::string_view bodyJson) const; + static ActionExecuteRegistry& instance(); +}; +``` + +- `registerAction` is only *declared* in the class body; its definition is + out-of-line in `bridge.hpp` (after `BridgeHandler` is fully defined) so the + executor can safely cast the `void*` handler and call its methods. `execute` + and `instance()` are defined inline in `bridge.hpp`. +- `execute` throws `std::runtime_error` for unknown pairs. + +### Static-init helpers + +Three `detail` functions serve as static-init helpers that the macros call: + +| Function | Purpose | +|---|---| +| `registerModelOnce(modelId)` | Registers a model factory with `ModelRegistryFactory::instance()`. Returns `true` so it can be assigned to a `const bool` in an anonymous namespace. | +| `registerActionOnce(modelId, actionId)` | Registers a runner with `ActionDispatcher::instance()`. Returns `true`. | +| `registerActionExecutorOnce(modelId, actionId)` | Registers with `ActionExecuteRegistry::instance()`. Only declared in `registry.hpp`; defined in `bridge.hpp` to avoid a `registry.hpp` → `bridge.hpp` include cycle. | + +The process-level singletons are returned by `defaultDispatcher()` and +`defaultRegistry()` (both are `inline` functions with function-local `static` +variables). + +## Registration macros + +### `BRIDGE_REGISTER_MODEL(M, NAME)` + +Specialises `ModelTraits` and registers a factory at static-init time. + +```cpp +BRIDGE_REGISTER_MODEL(AccountModel, "Account") +``` + +Expands to: +- `template <> struct morph::model::ModelTraits { static constexpr std::string_view typeId() noexcept { return NAME; } };` +- A `[[maybe_unused]] const bool` in an anonymous namespace (internal linkage, + no explicit `static`) that calls `detail::registerModelOnce(NAME)`. + +### `BRIDGE_REGISTER_ACTION(M, A, NAME, ...)` + +Variadic macro accepting 3 or 4 arguments. The 4-argument form accepts an +optional `Loggable` value (defaults to `Loggable::Yes`). + +```cpp +BRIDGE_REGISTER_ACTION(AccountModel, Deposit, "Deposit") +BRIDGE_REGISTER_ACTION(AccountModel, GetAccount, "GetAccount", morph::model::Loggable::No) +``` + +Expands to: +- `template <> struct morph::model::ActionTraits` with `Result` deduced from + `decltype(std::declval().execute(std::declval()))`, a + `static constexpr std::string_view typeId()` (no `noexcept`, unlike + `ModelTraits::typeId()`), a `static constexpr Loggable loggable`, and four JSON + codec functions using `glz::write_json` / `glz::read_json` (each throwing + `detail::ParseError` on failure). +- A `[[maybe_unused]] const bool` in an anonymous namespace calling + `detail::registerActionOnce(morph::model::ModelTraits::typeId(), NAME)` + (the model-id argument is the model's registered `typeId()`, not a raw string). +- A `[[maybe_unused]] const bool` in an anonymous namespace calling + `detail::registerActionExecutorOnce(morph::model::ModelTraits::typeId(), NAME)`. + +**Hard requirement:** Every translation unit invoking `BRIDGE_REGISTER_ACTION` +must include `` (directly or transitively) because +`registerActionExecutorOnce` is only defined there. Without it, the link fails +with an unresolved external symbol. + +### `BRIDGE_REGISTER_VALIDATOR(A, FN)` + +Specialises `ActionValidator` with a custom predicate. + +```cpp +BRIDGE_REGISTER_VALIDATOR(FormAction, [](const FormAction& a) { + return a.a != 0.0 && a.b != 0.0 && a.c != 0.0; +}) +``` + +Expands to `template <> struct morph::model::ActionValidator { static bool ready(const A& action) { return (FN)(action); } };`. + +## API reference + +### Traits and policies + +| Symbol | Kind | Purpose | +|---|---|---| +| `ModelTraits` | class template | **Customisation point.** Maps model type to `std::string_view typeId()`. | +| `ActionTraits` | class template | **Customisation point.** Maps action type to id, JSON codec, result type, and `Loggable`. | +| `ActionValidator` | class template | **Customisation point.** `static bool ready(const A&)` — built-in detection of `bool validate() const`, overridable via specialisation. | +| `ActionLogPolicy` | class template | **Customisation point.** `static constexpr bool coalesce = false` — checkpoint coalescing policy. | +| `Loggable` | enum | `{ No, Yes }` — strong boolean for action loggability. | + +### Concepts (detail) + +| Symbol | Purpose | +|---|---| +| `HasValidate` | `true` when `A` exposes `bool validate() const`. | +| `HasLoggableFlag` | `true` when `ActionTraits` exposes `static constexpr Loggable loggable`. | +| `BackendChangedNotifiable` | `true` when `M` exposes `void onBackendChanged()`. | + +### Type-erased model infrastructure + +| Symbol | Kind | Purpose | +|---|---|---| +| `IModelHolder` | abstract class | Type-erased model owner with action log slot. | +| `ModelHolder` | class template | Concrete holder storing `M` by value; conditionally inherits `IBackendChangedSink`. | +| `ModelFactory` | class | `static create()` — default-constructs `ModelHolder` and attaches the process-wide default log. | +| `IBackendChangedSink` | abstract class | Optional interface for backend-switch notification, discovered via `dynamic_cast`. | +| `BackendChangedMixin` | class template | Conditionally inherits `IBackendChangedSink` when `M` has `onBackendChanged()`. | + +### Singleton registries + +| Symbol | Kind | Purpose | +|---|---|---| +| `ActionDispatcher` | class | Maps `(modelId, actionId)` → type-erased runner; server-side dispatch. | +| `ModelRegistryFactory` | class | Maps `modelId` → factory; server-side model instantiation. | +| `ActionExecuteRegistry` | class | Maps `(modelId, actionId)` → type-erased executor through `BridgeHandler`; client/schema-driven execute. | + +### Macros + +| Macro | Arguments | Generates | +|---|---|---| +| `BRIDGE_REGISTER_MODEL` | `(M, NAME)` | `ModelTraits` specialisation + static-init factory registration. | +| `BRIDGE_REGISTER_ACTION` | `(M, A, NAME, ...)` | `ActionTraits` specialisation + static-init dispatcher and executor registration. Optional 4th arg: `Loggable`. | +| `BRIDGE_REGISTER_VALIDATOR` | `(A, FN)` | `ActionValidator` specialisation + custom predicate. | + +### Detail helpers + +| Symbol | Purpose | +|---|---| +| `PairKeyHash` | Hash functor for `std::pair` keys used by `ActionDispatcher` and `ActionExecuteRegistry`. | +| `actionLoggable()` | Returns `ActionTraits::loggable` if present, else `Loggable::Yes`. | +| `ParseError` | `std::runtime_error` subclass thrown on JSON codec failure. | +| `registerModelOnce(id)` | Static-init helper; returns `true`. | +| `registerActionOnce(id)` | Static-init helper; returns `true`. | +| `registerActionExecutorOnce(id)` | Static-init helper; only declared in `registry.hpp`, defined in `bridge.hpp`. | + +## Design decisions + +| Decision | Choice | Why | +|---|---|---| +| Trait-based registration | **Template specialisation + static-init guards** | Users never manage registry lifecycle; a macro or a hand-written specialisation is all that's needed. Static-init guarantees registrations are live before any `main()` code runs. | +| Singleton registries | **Function-local `static` in `inline` functions** | Process-level singletons with no header-level `static` ordering issues; `inline` avoids ODR violations across translation units. | +| Two registries for action execution | **`ActionDispatcher` (server) vs `ActionExecuteRegistry` (client)** | `ActionDispatcher` calls `Model::execute()` directly on an owned `IModelHolder` — the server-side path. `ActionExecuteRegistry` goes through `BridgeHandler` — sessions, backend switches, and completions work identically to hand-written call sites. Both are populated by the same macro. | +| `registerActionExecutorOnce` forward-declared in `registry.hpp` | **Defined in `bridge.hpp`** | Avoids a `registry.hpp` → `bridge.hpp` include cycle. `bridge.hpp` already includes `registry.hpp`. The cost: every translation unit that uses `BRIDGE_REGISTER_ACTION` must also include `bridge.hpp` or the link fails. | +| `HasLoggableFlag` backward compatibility | **Defaults to `Loggable::Yes` when `loggable` is absent** | Hand-written `ActionTraits` specialisations in tests predate the member; forcing them to add it would be churn. The default of `Yes` also means new actions are captured automatically — only pure queries opt out. | +| Action validation is a property of the action | **`ActionValidator`, not `ActionValidator`** | Different actions on the same model have different readiness requirements; keeping the predicate next to the action keeps the GUI side oblivious to model internals. | +| `Loggable` is a strong enum | **`Loggable::No` / `Loggable::Yes`**, not bare `bool` | Registration call sites read as intent rather than an unexplained `false`. | +| `ModelFactory::create` attaches the default log | **Single construction path for all topologies** | "Set the log once in `main()`" works uniformly across local and remote topologies. Callers that need a specific identity call `attachActionLog` again afterward. | +| `coalesce` defaults to `false` | **Every execution is a distinct, permanent fact** | The right default for anything resembling a business event. Only actions where only the latest occurrence should survive a checkpoint (e.g. a form-field edit fired repeatedly via `BridgeHandler::set`) opt in. | + +## Thread safety + +All three registries — `ActionDispatcher`, `ModelRegistryFactory`, and +`ActionExecuteRegistry` — are backed by plain `std::unordered_map` members with +**no mutex, no atomic, and no other synchronisation**. This is deliberate and +safe *only because of the registration model*: + +- **Writes happen during static initialisation**, which runs single-threaded + before `main()` (the process has not spawned worker threads yet). Every + `registerAction` / `registerModel` mutation therefore happens-before any code + that could observe the map concurrently. +- **After `main()` begins the maps are read-only.** `dispatch`, `create`, and + `coalesce` only ever call `find` on an already-populated map — concurrent + reads of a `const`-in-practice `unordered_map` are data-race-free. This is what + lets `RemoteServer` and the bridge dispatch/create/coalesce from arbitrary + threads without locking. + +The corollary is a hard constraint: **runtime registration is not thread-safe.** +Calling `registerAction` / `registerModel` after threads are running — e.g. from +a `dlopen`ed module loaded on a worker thread while another thread is +dispatching — races the map's internals against concurrent `find` calls and is +undefined behaviour. Load and register plugin modules from a single thread, +quiesced with respect to dispatch, before exposing them. + +## Failure modes + +| Situation | Behaviour | Where | +|---|---|---| +| Two registrations for the same `(modelId, actionId)` (or same `modelId`) | **Silent last-write-wins.** `ActionDispatcher::registerAction` does `_runners[key] = ...` and `_coalesce[key] = ...`; `ModelRegistryFactory::registerModel` does `insert_or_assign`. No diagnostic; the surviving entry is whichever initialiser ran last, and static-init order across TUs is unspecified. | `registry.hpp` | +| Two **distinct C++ types** registered under one string id | Same silent overwrite — the string id, not the type, is the key. The second type's runner/factory shadows the first. This is the collision hazard behind the string-vocabulary limitation below. | `registry.hpp` | +| `dispatch` / `execute` with an unknown `(modelId, actionId)` | Throws `std::runtime_error` **at runtime** (`"unknown action: …"`). The string-keyed remote path has **no compile-time completeness check** — a pair that was never registered is only discovered when a request for it arrives. | `ActionDispatcher::dispatch`, `ActionExecuteRegistry::execute` | +| `create` with an unknown model id | Throws `std::runtime_error("unknown model type: …")` at runtime. | `ModelRegistryFactory::create` | +| `coalesce` for an unknown pair | Does **not** throw — defaults to `false` (every entry kept). | `ActionDispatcher::coalesce` | +| Allocation failure inside a `register*Once` helper during static init | `registerModelOnce` / `registerActionOnce` (and `registerActionExecutorOnce`) are declared `noexcept` yet allocate (they build `std::string` keys and grow the map). An OOM there raises an exception through a `noexcept` boundary, which calls `std::terminate` — the process aborts during static init. | `registry.hpp` | + +Note the asymmetry the design accepts intentionally: the **typed local path** +(`BridgeHandler::execute()`, `Model::execute(action)`) is checked by the +compiler — an unregistered or misspelled action is a build error — whereas the +**string-keyed remote/schema path** through these registries defers every id +resolution to runtime. Registration correctness for the remote surface is a +testing obligation, not a compile-time guarantee. + +## Limitations + +- **String type-ids are an unversioned, un-namespaced global protocol + vocabulary.** Every `NAME` passed to `BRIDGE_REGISTER_MODEL` / + `BRIDGE_REGISTER_ACTION` lives in one flat namespace shared across the whole + process and, implicitly, across the wire with every peer. There is no version + tag, no module qualifier, and — per [Failure modes](#failure-modes) — collisions + are silent last-write-wins. Two independently developed subsystems that both + register `"Update"` will clobber each other with no diagnostic. Mitigations the + design does **not** yet enforce but should be adopted by convention: an + **id-namespacing convention** (e.g. `"bank.Account"` / `"bank.Account.Deposit"` + prefixes per subsystem) to make collisions structurally unlikely, and a + **startup self-check** that iterates the intended `(model, action)` set and + asserts each is present before serving traffic — there is otherwise **no + compile-time guarantee that every remotely executed pair was actually + registered** (registration is a static-init side effect that can be silently + dropped; see [Registration rules and invariants](#registration-rules-and-invariants)). +- **Global mutable singletons with no teardown or reset.** `defaultDispatcher()` + and `defaultRegistry()` (and `ActionExecuteRegistry::instance()`) are + function-local `static`s that live for the whole process and expose no clear / + reset. This hurts test isolation: registrations accumulate across test cases in + one binary, one test cannot register a fake model without leaking it into the + next, and last-write-wins means test ordering can change behaviour. Contrast + `journal::ScopedActionLog`, which deliberately provides scoped install/restore + for exactly this reason; the registries have no equivalent. +- **Per-call heap allocation on the hot path.** Every `dispatch`, `create`, + `coalesce`, and `execute` constructs a `std::string` (or a + `std::pair`) key from its `string_view` arguments + purely to probe the map — an allocation per lookup on what is the request hot + path. Heterogeneous lookup (a transparent hash/equality over `string_view`, + C++20 `unordered_map` `find` with `is_transparent`) would remove the + allocation entirely; the maps are keyed on owning `std::string` today. +- **The `ActionDispatcher` / `ActionExecuteRegistry` split can silently + diverge.** A single `BRIDGE_REGISTER_ACTION` populates both registries (one + initialiser each). But they are independent maps consulted by different + code paths — `ActionDispatcher` on the `RemoteServer` server path, + `ActionExecuteRegistry` on the `BridgeHandler` schema-driven path. If only one + registration fires (e.g. a hand-written `ActionTraits` specialisation that + registers a runner but skips the executor, or a partial refactor), one path + works and the other throws "unknown action" for the *same* logical action, with + no signal that the two are meant to stay in lockstep. + +## Cross-references + +- **[bridge.md](bridge.md)** — defines `BridgeHandler`, `Bridge`, and the + `ActionExecuteRegistry` class itself (declared in ``, not + `registry.hpp`). Explains the **hard `#include ` requirement** + for any TU using `BRIDGE_REGISTER_ACTION` (`registerActionExecutorOnce` is only + *defined* there) and the parallel executor path this spec's + `ActionExecuteRegistry` section summarises. +- **[journal.md](journal.md)** — `IActionLog`, `LogEntry`, `SessionLog`, + checkpoint coalescing, and `ScopedActionLog`. Explains how the runner's + `recordIfAttached` call and `ActionLogPolicy::coalesce` feed the + durable log, and provides the scoped-install pattern the registries lack. +- **[backend.md](backend.md)** — backends store `IModelHolder`s in a single map + and drive `IBackendChangedSink` / `BackendChangedMixin`; the model instances + created by `ModelRegistryFactory` land here. +- **[security.md](security.md)** — the `session::current()` principal stamped onto + every logged entry by `recordIfAttached`, and the trust boundary of the + string-keyed remote dispatch surface. +- **Error handling** — the `detail::ParseError` / `std::runtime_error` taxonomy + the registries raise (see [Failure modes](#failure-modes)); glaze codec errors + originate in the `ActionTraits` JSON functions. +- **Concurrency and lifetimes** — the static-init-then-read-only discipline in + [Thread safety](#thread-safety) and the process-lifetime singleton ownership in + [Limitations](#limitations). \ No newline at end of file diff --git a/docs/spec/security.md b/docs/spec/security.md new file mode 100644 index 0000000..2ba8900 --- /dev/null +++ b/docs/spec/security.md @@ -0,0 +1,214 @@ +# Security & the trust model + +Cross-cutting spec covering morph's trust boundaries and the authenticated +session layer (`session.hpp` `Context`/`IAuthorizer`, `session_auth.hpp` +`SessionToken`/`TokenIssuer`/`TokenVerifier`/`SigningAuthorizer`, and the +`RemoteServer` enforcement points in `remote.hpp`). Read this before deploying a +`RemoteServer` on anything but a trusted local socket. + +Related specs: [session.md](session.md) (the `Context`/`IAuthorizer` types), +[wire.md](wire.md) (the envelope the `session` travels in), +[backend.md](backend.md) (`RemoteServer`/`LocalBackend` dispatch), +[error_handling.md](error_handling.md) (how a rejected request surfaces). + +## Threat model — what morph does and does not defend + +morph is a typed bridge, not a security product. Its built-in guarantees are +deliberately small; everything else is delegated to the transport and the +application. Be explicit about the boundary: + +**morph provides:** +- A single choke point (`IAuthorizer`) consulted on **every** `execute` + envelope before dispatch, on the remote path. +- An opt-in, stateless **authentication** layer (`session_auth.hpp`): signed + bearer tokens whose principal the server can *verify* rather than trust, and + make authoritative for model code. +- Untrusted-wire-input hardening in the value codecs (`Rational` clamps, + `DateTime` rejects — see the respective specs), so a malformed payload is a + defined outcome rather than undefined behaviour. + +**morph does NOT provide (the application/transport must):** +- **Transport security.** There is no TLS, no encryption, no message-size cap, + and no per-request timeout at the wire layer (see [wire.md](wire.md)). A plain + `RemoteServer` transport is plaintext and has an unbounded-allocation DoS + surface. Run it behind TLS and a transport that bounds message size. +- **Authentication of the transport peer.** A bearer token proves the caller + holds a validly-signed token; it does not bind the token to a connection. + Without TLS a stolen token can be replayed. +- **Control-message authorization.** Only `execute` is passed through the + authorizer. `register`/`deregister` are **not** authorized, and model ids are + guessable sequential integers assigned from a single counter (see + [backend.md](backend.md), `RemoteServer::_nextId`). Any client that can send + envelopes can register models and deregister/execute against ids it did not + create. `RemoteServer` therefore assumes an authenticated, trusted transport + — it is not a hardened multi-tenant public-internet server as shipped. +- **Local-path authorization.** `LocalBackend::execute` installs the session + context but never calls the authorizer (the authorizer is a remote-only gate, + `remote.hpp` `dispatchExecute` is the sole call site). Security-critical checks + must be enforced inside the model so they hold in both modes. + +## The trust boundary: `Context` is untrusted input + +`session::Context` (`principal`, `token`, `requestId`, `locale`, `metadata`) is +populated by the client and, on the remote path, deserialised verbatim from the +wire envelope's `session` field. Every field is therefore **attacker-controlled +input** until something verifies it: + +- `Context::principal` on its own is a *claim*, not an identity. Model code must + not treat it as authenticated unless a verifying authorizer is installed (see + below), which makes it authoritative. +- `Context::metadata` is an unbounded map decoded from the wire; treat its size + and contents as untrusted. + +On the local path the same `Context` travels in-memory via `ActionCall` and is +whatever the caller set — trusted to the extent the process trusts itself. + +## Authentication: signed bearer tokens (`session_auth.hpp`) + +`session_auth.hpp` is an **opt-in** header (include it only when you want +authentication) that turns `Context::principal` from a claim into a verified +identity. The mechanism is a stateless signed bearer token. + +### Token format + +A token is `base64url(claimsJson) "." base64url(mac)`, where `mac = MAC(secret, +payload)` and `payload` is the base64url claims segment. The claims are a +`SessionToken`: + +| Field | Meaning | +|---|---| +| `principal` | Authenticated user/principal id. | +| `issuedAtMs` | Issue time, ms since epoch (informational). | +| `expiresAtMs` | Expiry, ms since epoch. `0` means never expires (discouraged). | +| `roles` | Coarse-grained roles an authorization policy can key on. | + +The claims are JSON (Glaze); adding application claims is compatible because +unknown fields are ignored on read. + +### The MAC primitive is pluggable + +```cpp +using MacFunction = std::function; +``` + +`MacFunction` returns the raw MAC bytes. The default is `hmacSha256`, a +self-contained reference HMAC-SHA256 (so morph has **no** crypto dependency), +verified against the FIPS 180-4 / RFC 4231 test vectors in +`tests/test_session_auth.cpp`. **The reference implementation is correct but is +not hardened** (no side-channel engineering beyond a constant-time MAC compare); +security-sensitive deployments should inject a vetted library's HMAC: + +```cpp +morph::session::MacFunction mac = + [](std::string_view key, std::string_view msg) { return myLibsodiumHmac(key, msg); }; +morph::session::SigningAuthorizer authz{sharedSecret, mac}; +``` + +### Issuing tokens — the login flow + +morph ships no `Login` action; login is an ordinary application action. The app +validates credentials however it likes, then mints a token with `TokenIssuer`: + +```cpp +// server side, inside a Login action's handler: +morph::session::TokenIssuer issuer{sharedSecret}; // default hmacSha256 +LoginResult execute(const Login& a) { + if (!checkPassword(a.user, a.password)) throw std::runtime_error("bad credentials"); + return { .token = issuer.issue({ .principal = a.user, + .issuedAtMs = nowMs(), + .expiresAtMs = nowMs() + 15 * 60'000, // 15 min + .roles = rolesFor(a.user) }) }; +} +``` + +The client attaches the returned token to every subsequent call via the default +session: + +```cpp +bridge.setDefaultSession({ .principal = user, .token = result.token }); +``` + +### Verifying tokens and making the principal authoritative + +Install a `SigningAuthorizer` on the server; it verifies the token on every +`execute`: + +```cpp +auto authz = std::make_shared(sharedSecret); +auto server = std::make_shared(pool, dispatcher, registry, authz); +``` + +`SigningAuthorizer` implements both `IAuthorizer` entry points: + +- `authorize(ctx, modelType, actionType)` returns `true` only for a token with a + valid signature and unexpired claims — and, if a `Policy` is supplied, one the + policy admits. An invalid/absent/expired token → `false` → the server replies + `err "unauthorized"`. +- `authenticate(ctx)` returns the verified `principal`. `RemoteServer` + (`dispatchExecute`) calls it after `authorize` succeeds and **overwrites** + `env.session.principal` with the verified value before building the + `ScopedContext`. So a model reading `session::current()->principal` sees the + authenticated identity, not the client's claim. + +`TokenVerifier::verify` checks the MAC **before** parsing the claims JSON, so +untrusted input is never handed to the parser until authenticity is established, +and uses a constant-time comparison (`detail::constantTimeEquals`) to avoid MAC +timing leaks. It returns `std::expected`: + +| `AuthError` | Cause | +|---|---| +| `Malformed` | Not `payload.sig`, bad base64url, or unparseable claims. | +| `BadSignature` | MAC mismatch — forged or tampered. | +| `Expired` | `expiresAtMs` is before the supplied clock. | + +The clock is injectable (`Clock`, defaulting to `systemClockMs`) so expiry is +testable without wall-clock dependence. + +### Role-based policy + +`SigningAuthorizer`'s optional `Policy` runs over the verified claims: + +```cpp +morph::session::SigningAuthorizer authz{ + secret, morph::session::hmacSha256, morph::session::systemClockMs, + [](const morph::session::SessionToken& t, std::string_view modelType, std::string_view actionType) { + return std::ranges::find(t.roles, "admin") != t.roles.end(); // admin-only + }}; +``` + +The default (no policy) admits any validly-signed, unexpired token. + +## The default is fail-open — change it in production + +`RemoteServer`'s ordinary constructor defaults to `allowAllAuthorizer()`, and the +explicit-authorizer constructor silently falls back to allow-all on a `nullptr` +argument. An unconfigured server therefore **authorizes everything**. This is +convenient for local/simulated development and wrong for production. Always +install a `SigningAuthorizer` (or a deny-by-default custom authorizer) before +exposing a server, and never pass `nullptr`. + +## Residual limitations & hardening checklist + +Even with `SigningAuthorizer` installed, the following remain the deployer's +responsibility: + +- **Use TLS.** Bearer tokens and payloads travel in plaintext otherwise, and a + captured token can be replayed until it expires. There is no envelope-level + confidentiality or replay protection. +- **Keep expiry short and rotate the secret.** A leaked secret forges any + identity; a leaked token is valid until `expiresAtMs`. +- **Bound message size and add timeouts in the transport.** The wire layer does + neither (see [wire.md](wire.md)); a hostile client can otherwise exhaust + memory or leave `Completion`s pending forever. +- **Do not rely on the authorizer for correctness inside models.** It runs only + on the remote path and only for `execute`. Enforce invariants in the model so + they also hold locally and for control messages. +- **Treat `RemoteServer` as single-trust-domain** unless you add + control-message authorization and session-scoped model ids yourself. + +## Testing + +`tests/test_session_auth.cpp` covers the SHA-256/HMAC known-answer vectors, +base64url round-tripping, token issue/verify, and rejection of tampering, wrong +secret, expiry, and malformed input, plus `SigningAuthorizer` authorization, +the no-token denial path, and role-policy enforcement. diff --git a/docs/spec/session.md b/docs/spec/session.md new file mode 100644 index 0000000..48d25c8 --- /dev/null +++ b/docs/spec/session.md @@ -0,0 +1,286 @@ +# The `session` types — design + +`morph::session` provides the per-call context that travels from the caller +through the bridge to the model, together with a pluggable authorizer that +remote servers use to gate action dispatch and, optionally, to authenticate the +caller. + +The authentication *mechanism* — signed bearer tokens, `SigningAuthorizer`, and +the `RemoteServer` enforcement points — lives in +[security.md](security.md), the primary companion to this spec. This file +documents the `session` **types and their contracts**; it cross-references +security.md for the full trust model rather than duplicating it. + +## Contents + +- [Context — an open data bag](#context--an-open-data-bag) +- [IAuthorizer — gate for action dispatch](#iauthorizer--gate-for-action-dispatch) +- [The `authenticate` hook — the authoritative principal](#the-authenticate-hook--the-authoritative-principal) +- [AllowAllAuthorizer and `allowAllAuthorizer()`](#allowallauthorizer-and-allowallauthorizer) +- [Trust boundary](#trust-boundary) +- [Thread safety — `current()` and `ScopedContext`](#thread-safety--current-and-scopedcontext) +- [API reference](#api-reference) +- [Design decisions](#design-decisions) +- [Limitations](#limitations) +- [Cross-references](#cross-references) + +## Context — an open data bag + +`Context` is a plain struct with five fields that callers populate however they +see fit. The framework only inspects `principal`, `token`, and the action ids +when consulting the configured `IAuthorizer`; everything else is passed through +verbatim from caller to model. + +| Field | Type | Purpose | +|---|---|---| +| `principal` | `std::string` | Auth principal — user/identity id. A client *claim* until a verifying authorizer overwrites it (see below); authoritative for model code only after that. | +| `token` | `std::string` | Bearer credential verified server-side. Typically a signed token minted by a login action via `session::TokenIssuer` and attached to every call. Empty when unauthenticated. | +| `requestId` | `std::string` | Stable id for distributed tracing / log correlation. Empty when unused. | +| `locale` | `std::string` | BCP-47 locale tag (`en-US`, `fr-FR`) for i18n. Empty for app default. | +| `metadata` | `std::unordered_map` | Free-form bag for feature flags, A/B buckets, app-specific metadata. | + +On remote backends the entire `Context` — including `token` — is serialised into +the wire envelope's `session` field (see [wire.md](wire.md)) so `RemoteServer` +sees the same values the GUI sent. On the local backend it travels in-memory via +`ActionCall`. + +`token` is the credential the server *verifies*; `principal` is the identity the +server *derives* from that credential. See +[the `authenticate` hook](#the-authenticate-hook--the-authoritative-principal) +for how the derived principal replaces the client's claim, and +[security.md](security.md) for the token format and login flow. + +## IAuthorizer — gate for action dispatch + +`IAuthorizer` is an abstract interface called once per `execute` envelope, +before the action is dispatched. A `false` return causes the server to reply +with `err|unauthorized` (the client surfaces the error through `.onError(...)`). + +```cpp +[[nodiscard]] virtual bool authorize( + const Context& ctx, + std::string_view modelType, + std::string_view actionType) const = 0; +``` + +The three parameters represent the caller's session, the target model **type** +string id, and the action **type** being invoked. Note that `authorize` sees +only *type* ids, never the target model **instance** id — row/instance-level +authorization (e.g. "may this principal edit *this* account?") is not +expressible here and belongs inside the model's `execute()`. + +Real deployments subclass this to check principal claims, action permissions, +rate limits, etc. `authorize` is `const` and stateless by contract; any stateful +policy (rate-limit counters, revocation lists) must live in the subclass's own +members, guarded for concurrent access — the framework calls it from the +dispatch thread and gives it no per-call mutable state. + +## The `authenticate` hook — the authoritative principal + +`IAuthorizer` has a second, **optional** virtual: + +```cpp +[[nodiscard]] virtual std::optional +authenticate(const Context& ctx) const { return std::nullopt; } +``` + +The default returns `nullopt`, meaning "I do not authenticate — leave the +client's principal untouched". `RemoteServer` calls `authenticate` **after +`authorize` has already succeeded**, and if it returns a value, the server +**overwrites `Context::principal`** with that value before building the +`ScopedContext` around dispatch. From then on, a model reading +`session::current()->principal` sees the *verified* identity the authorizer +vouched for, not the string the client sent. + +This split keeps the two concerns separate: `authorize` answers "is this call +permitted?" and `authenticate` answers "who is actually making it?". An +authorizer may implement either, both, or (via the default) only the first. + +- `AllowAllAuthorizer` uses the **default** `authenticate` — it performs no + authentication, so the client's `principal` passes through unchanged. +- `SigningAuthorizer` (`session_auth.hpp`) **overrides** it: it verifies + `Context::token` and returns the token's `principal`, so the authenticated + identity becomes authoritative for model code. + +The full mechanism — token format, verification order, `RemoteServer`'s +`dispatchExecute` call site, and the login flow — is documented in +[security.md](security.md). This spec only fixes the *contract*: `authenticate` +is consulted post-`authorize`, and its return, when present, replaces the +principal. + +## AllowAllAuthorizer and `allowAllAuthorizer()` + +The framework ships a default authorizer that permits every call and performs no +authentication (it inherits the default `authenticate`). It is used as the +default by `RemoteServer` and can also be wired explicitly. + +`allowAllAuthorizer()` returns a `std::shared_ptr` pointing at a +process-wide singleton. This avoids allocating a new shared_ptr per server +instance when the default is sufficient. + +`AllowAllAuthorizer` is **fail-open**: it is convenient for local and simulated +development and wrong for production. See [Trust boundary](#trust-boundary) and +[security.md](security.md) ("The default is fail-open") for why an exposed +server must replace it. + +## Trust boundary + +The authorizer exists because a `Context` arriving over the wire is untrusted +input. The rules that govern where trust begins: + +- **On the remote path, every `Context` field is unauthenticated wire input.** + `principal`, `token`, `requestId`, `locale`, and `metadata` are all + deserialised verbatim from the envelope the client sent. `principal` in + particular is a *claim*, not an identity, until a verifying authorizer's + `authenticate` overwrites it. Model code must not treat any field as trusted + before that point. +- **The local backend does not authorize at all.** `authorize` is a + **remote-only gate**: `LocalBackend::execute` installs the session context but + never consults the authorizer (its sole call site is `RemoteServer`'s + `dispatchExecute`, see [backend.md](backend.md)). Any security-critical check + must therefore be enforced inside the model so it holds in both local and + remote modes. +- **The `RemoteServer` default authorizer is fail-open.** An unconfigured server + uses `allowAllAuthorizer()` and permits everything. Production deployments must + install a verifying, deny-by-default authorizer. +- **`authorize` sees only type ids, not the instance id.** Instance-/row-level + authorization is not expressible at this layer and belongs in the model. + +The registry that maps those type ids to runners is described in +[registry.md](registry.md); the enforcement points and the full threat model are +in [security.md](security.md). + +## Thread safety — `current()` and `ScopedContext` + +Models that need session data can access it without changing their +`execute()` signature. The mechanism is a thread-local pointer (the +`detail::tlsCurrent()` `const Context*&`) installed by the backend for the +duration of the model invocation. + +**`current()`** returns `const Context*` — the active context for the +in-progress action, or `nullptr` when called outside a dispatch. It is declared +`noexcept`. Models that don't need session data ignore it entirely. + +**The context is thread-local and dispatch-scoped.** The backend installs the +pointer on the exact thread that runs the model call and clears it when that call +returns. It is therefore valid **only on that dispatch thread**: + +- `current()` returns `nullptr` on any thread the model *spawns*, and on async + work the model *schedules* to run later — the thread-local is not propagated + across thread boundaries. +- A model that needs session data after crossing a thread boundary must + **capture what it needs** (copy the `principal`, `locale`, or specific + metadata values) *before* handing work to another thread, rather than calling + `current()` from the other side. + +**`ScopedContext`** (in `detail` namespace) is the RAII helper that installs a +`Context` pointer for its scope and restores the **previous** one on +destruction. Because it saves and restores the prior pointer rather than +clearing to `nullptr`, nested dispatch composes correctly — an inner +`ScopedContext` shadows the outer context and the outer one is restored when the +inner scope exits. Construction is `explicit`, and copy and move are all four +deleted. The stored pointer is the address of the referenced `Context`, which +must outlive the `ScopedContext`. + +It is used by `LocalBackend::execute` to wrap `localOp(*holder)` and by +`RemoteServer::dispatchExecute` to wrap `ActionDispatcher::dispatch`, so the +context is live while the model's `execute()` runs. `ActionDispatcher::dispatch` +itself does not touch the thread-local pointer — it only looks up and invokes the +registered runner. + +A usage example from the bank example (`bank/core/principal.hpp`): + +```cpp +if (const auto* ctx = morph::session::current(); ctx != nullptr) { + // use ctx->principal, ctx->locale, etc. +} +``` + +## API reference + +### `Context` + +| Member | Signature | Notes | +|---|---|---| +| `principal` | `std::string` | Auth principal; a client claim until a verifying authorizer's `authenticate` overwrites it. | +| `token` | `std::string` | Bearer credential verified server-side; travels in the wire envelope's `session`. Empty if unauthenticated. | +| `requestId` | `std::string` | Trace id; empty when unused. | +| `locale` | `std::string` | BCP-47 locale; empty for default. | +| `metadata` | `std::unordered_map` | Free-form metadata bag. | + +### `IAuthorizer` + +| Member | Signature | Notes | +|---|---|---| +| `~IAuthorizer()` | `virtual ~IAuthorizer() = default` | Virtual destructor for polymorphic use. | +| `authorize` | `[[nodiscard]] virtual bool authorize(const Context&, std::string_view modelType, std::string_view actionType) const = 0` | Returns `true` to allow dispatch, `false` to reject. Called per `execute` envelope. Sees only type ids. | +| `authenticate` | `[[nodiscard]] virtual std::optional authenticate(const Context&) const` | Optional. Default returns `nullopt`. Called after `authorize` succeeds; a returned value overwrites `Context::principal`, making it authoritative. | + +### `AllowAllAuthorizer` + +| Member | Signature | Notes | +|---|---|---| +| `authorize` | `[[nodiscard]] bool authorize(const Context&, std::string_view, std::string_view) const override` | Always returns `true`. All parameters ignored. | +| `authenticate` | (inherited default) | Not overridden — returns `nullopt`, so the client's principal is left untouched. | + +### Free functions + +| Symbol | Signature | Notes | +|---|---|---| +| `allowAllAuthorizer()` | `std::shared_ptr allowAllAuthorizer()` | Returns a process-wide singleton `AllowAllAuthorizer`. | +| `current()` | `const Context* current() noexcept` | Returns the active context, or `nullptr` when none / on a non-dispatch thread. | + +### `detail::ScopedContext` + +| Member | Signature | Notes | +|---|---|---| +| ctor | `explicit ScopedContext(const Context& ctx)` | Saves the current thread-local context and installs `ctx`. `ctx` must outlive this object. | +| dtor | `~ScopedContext()` | Restores the **previously** active context (composes under nesting). | +| (copy/move) | deleted | Non-copyable, non-movable. | + +## Design decisions + +| Decision | Choice | Why | +|---|---|---| +| Context shape | **Plain struct, not polymorphic** | The context is a data bag, not a behaviour abstraction; a struct is simpler to construct, serialise, and consume. | +| Thread-local context | **`const Context*` TLS pointer, installed by the backend around the model call** | Models read session data without changing `execute()` signatures; the pointer is `const` (immutable during dispatch) and RAII-guarded by `ScopedContext`. | +| `ScopedContext` restores the previous pointer | **Save/restore rather than clear-to-null** | Nested dispatch (a model dispatching into another) composes: the inner context shadows the outer and the outer is restored on exit. | +| Authorizer granularity | **Per-envelope, before dispatch, type ids only** | One call per `execute`, not per sub-operation; instance-level authz is left to the model, which alone knows instance identity. | +| Authentication as a separate hook | **`authenticate` distinct from `authorize`, optional with a `nullopt` default** | Separates "is this permitted?" from "who is it?"; authorizers that don't authenticate cost nothing, and the verified principal — not the client's claim — becomes authoritative when one does. | +| Singleton authorizer | **Static local in `allowAllAuthorizer()`** | All trivial `RemoteServer` instances share one `AllowAllAuthorizer` allocation rather than each owning one. | +| Serialisation | **Entire `Context` travels on the wire** | The remote backend's `RemoteServer` sees the same `principal`, `token`, `requestId`, `locale`, and `metadata` the GUI sent; no information is stripped. | + +## Limitations + +- **`principal` carries no integrity on its own.** It is a plain string in a + serialised struct; nothing in the type prevents a client from sending any + value. It becomes trustworthy only after a verifying authorizer's + `authenticate` overwrites it. +- **Authentication depends on the transport plus a verifying authorizer.** The + `token` is a bearer credential with no envelope-level confidentiality or + replay protection; its guarantees hold only under TLS (so the token cannot be + captured and replayed) *and* with an authorizer such as `SigningAuthorizer` + installed. A plain `RemoteServer` with the default authorizer authenticates + nothing. +- **Stateful authorization policy needs the impl's own state.** `authorize` is + `const` and receives no mutable per-call state, so rate limits, revocation + lists, or lockout counters must be stored in (and synchronised by) the + authorizer subclass itself. +- **`current()` is dispatch-thread-only.** It returns `nullptr` off the dispatch + thread; session data needed across a thread boundary must be captured first. + +See [security.md](security.md) for the complete threat model and hardening +checklist (TLS, message-size bounds, control-message authorization, secret +rotation). + +## Cross-references + +- [security.md](security.md) — **the authentication subsystem**: signed bearer + tokens, `SigningAuthorizer`, the `RemoteServer` enforcement points, the threat + model, and hardening guidance. The primary companion to this spec. +- [wire.md](wire.md) — the envelope the `Context` (including `token`) is + serialised into on the remote path. +- [backend.md](backend.md) — `RemoteServer` / `LocalBackend` dispatch, and where + the authorizer is (and is not) consulted. +- [registry.md](registry.md) — the model/action type registry behind the type + ids `authorize` receives. diff --git a/docs/spec/wire.md b/docs/spec/wire.md new file mode 100644 index 0000000..b589b46 --- /dev/null +++ b/docs/spec/wire.md @@ -0,0 +1,109 @@ +# The `wire` types — design + +`morph::wire` provides the JSON wire envelope and associated helpers used +between any client and `morph::net::RemoteServer`. A single `Envelope` struct +carries all request and reply variants, discriminated by a `kind` string field. + +## Contents + +- [Envelope](#envelope) +- [Factory functions](#factory-functions) +- [Encode and decode](#encode-and-decode) +- [API reference](#api-reference) +- [Design decisions](#design-decisions) + +## Envelope + +The `Envelope` struct supersedes the legacy pipe-delimited protocol. Every field +is present in the struct so the JSON shape is fixed; callers populate only what +their `kind` needs and leave the rest as default-constructed values. + +### Discriminator values + +| `kind` | Direction | Purpose | Key fields | +|---|---|---|---| +| `"register"` | request | Client requests model creation. | `typeId`, `contextKey` (optional stable identity) | +| `"deregister"` | request | Client destroys an instance. | `modelId` | +| `"execute"` | request | Client dispatches an action. | `callId`, `modelId`, `modelType`, `actionType`, `body`, `session` | +| `"ok"` | reply | Server success. | `callId`, `result` (`body`), `modelId` (for register-replies) | +| `"err"` | reply | Server failure. | `callId`, `message` | + +### `contextKey` — stable identity + +`contextKey` is an optional stable identity for the new instance (e.g. an account +id). When present, the server-side holder gets an action log attached (if a +`LogProvider` is configured). When empty, no action log is attached. Ignored on +every kind other than `"register"`. + +### `session` — authorization context + +The `session` field carries a `morph::session::Context` for authorization and +routing. Populated on `"execute"`; ignored on every other kind. + +## Factory functions + +Four free functions construct `Envelope` instances with the correct `kind` and +relevant fields. Callers never set `kind` manually. + +| Function | `kind` | Parameters | +|---|---|---| +| `makeRegister(typeId, contextKey = {})` | `"register"` | Model type id, optional stable identity. | +| `makeDeregister(modelId)` | `"deregister"` | Instance id to destroy. | +| `makeOk(callId = 0, body = {}, modelId = 0)` | `"ok"` | Correlation id, serialized result, optional model id (for register-replies). | +| `makeErr(message, callId = 0)` | `"err"` | Error message, optional correlation id. | + +For `"execute"` there is no factory — callers construct the `Envelope` directly +and set `kind = "execute"`, or use the `Client`/`RemoteServer` APIs which handle +it internally. + +## Encode and decode + +| Function | Signature | Notes | +|---|---|---| +| `encode` | `std::string encode(const Envelope&)` | Serializes to a single JSON line via `glz::write_json`. Throws `std::runtime_error` on failure (should never happen for valid input). | +| `decode` | `Envelope decode(std::string_view)` | Deserializes from JSON via `glz::read_json`. Throws `std::runtime_error` on malformed input. | + +## API reference + +### `wire::Envelope` + +| Field | Type | Default | Used by `kind` | +|---|---|---|---| +| `kind` | `std::string` | `""` | All — the discriminator. | +| `callId` | `uint64_t` | `0` | `"execute"`, `"ok"`, `"err"` — correlation id for async matching. | +| `typeId` | `std::string` | `""` | `"register"` — model type id. | +| `contextKey` | `std::string` | `""` | `"register"` — stable identity for the new instance. | +| `modelId` | `uint64_t` | `0` | `"deregister"`, `"execute"`, `"ok"(register)` — instance id. | +| `modelType` | `std::string` | `""` | `"execute"` — routing key for `ActionDispatcher`. | +| `actionType` | `std::string` | `""` | `"execute"` — second routing key. | +| `body` | `std::string` | `""` | `"execute"`, `"ok"` — serialized JSON payload. | +| `message` | `std::string` | `""` | `"err"` — free-text error message. | +| `session` | `::morph::session::Context` | default | `"execute"` — authorization and routing context. | + +### Factory functions + +| Symbol | Signature | +|---|---| +| `makeRegister` | `Envelope makeRegister(std::string typeId, std::string contextKey = {})` | +| `makeDeregister` | `Envelope makeDeregister(uint64_t modelId)` | +| `makeOk` | `Envelope makeOk(uint64_t callId = 0, std::string body = {}, uint64_t modelId = 0)` | +| `makeErr` | `Envelope makeErr(std::string message, uint64_t callId = 0)` | + +### Serialization + +| Symbol | Signature | Throws | +|---|---|---| +| `encode` | `std::string encode(const Envelope&)` | `std::runtime_error` on serialisation failure | +| `decode` | `Envelope decode(std::string_view)` | `std::runtime_error` on malformed JSON | + +## Design decisions + +| Decision | Choice | Why | +|---|---|---| +| Single struct vs. discriminated union | **One `Envelope` struct, all fields present** | The JSON shape is fixed and predictable; callers populate only what their kind needs. Avoids a tagged-union complexity that would add no benefit over a single struct with a `kind` string. | +| `kind` as a string vs. enum | **`std::string`** | JSON naturally discriminates by string; avoids an enum-to-string mapping. The factory functions (`makeRegister`, etc.) ensure callers never set `kind` manually. | +| `"execute"` has no factory | **No factory** | `"execute"` envelopes are typically constructed by higher-level APIs (`Client`, `RemoteServer`), not by end users. Adding a factory would be dead code at the wire layer. | +| Factory functions are `inline` | **Header-only** | The entire wire module lives in the header. Wrapping each factory as a named function keeps construction safe (correct `kind`, no forgotten fields) without a separate compilation unit. | +| Serialization via glaze | **`glz::write_json` / `glz::read_json`** | glaze is the project's existing JSON library; no additional dependency. Throws on failure rather than returning error codes because encode/decode at the wire boundary should fail loud and early. | +| `contextKey` vs. `modelId` for register | **`contextKey` is a separate field, not `modelId`** | `modelId` is server-assigned (a `uint64_t` handle); `contextKey` is a client-chosen stable identity string. They are semantically different and the server treats them differently (log attachment vs. instance routing). | +| `session` as a dedicated field | **`::morph::session::Context`** | Session context is a first-class concern for authorization and routing, not an opaque sub-payload in `body`. Keeping it at the Envelope level ensures every `"execute"` carries it without caller discipline. | \ No newline at end of file diff --git a/include/morph/bridge.hpp b/include/morph/bridge.hpp index 77bba37..4fbaec1 100644 --- a/include/morph/bridge.hpp +++ b/include/morph/bridge.hpp @@ -219,8 +219,11 @@ class Bridge { _handlers.push_back(binding); } - /// @brief Installs a default session context applied to every call that does - /// not provide one explicitly via `BridgeHandler::executeWith(...)`. + /// @brief Installs a default session context that `executeVia` stamps onto the + /// `ActionCall` of every subsequent call. + /// + /// There is no per-call session override — this default is applied to all calls + /// until replaced or cleared. /// /// Typical pattern: call this once after login to bind the user's principal /// and locale, then every subsequent `handler.execute(action)` carries the @@ -258,15 +261,41 @@ class Bridge { { std::scoped_lock const lock{_mtx}; + // Phase 1 — register every live binding on the new backend WITHOUT + // mutating any `currentId` yet, staging (binding, newId) pairs. If a + // registration throws partway (a plausible remote/transport failure), + // roll back the ones already registered and rethrow, leaving the old + // backend and every `currentId` untouched — so the switch is atomic: + // it either fully succeeds or is a no-op. std::vector> live; - for (auto& weak : _handlers) { - auto binding = weak.lock(); - if (!binding) { - continue; + std::vector, uint64_t>> staged; + try { + for (auto& weak : _handlers) { + auto binding = weak.lock(); + if (!binding) { + continue; + } + auto newId = newShared->registerModelWithContext(binding->typeId, binding->modelFactory, + binding->contextKey); + staged.emplace_back(binding, newId.v); + live.push_back(weak); } - auto newId = newShared->registerModelWithContext(binding->typeId, binding->modelFactory, binding->contextKey); - binding->currentId.store(newId.v); - live.push_back(weak); + } catch (...) { + for (const auto& [binding, newId] : staged) { + try { + newShared->deregisterModel(::morph::exec::detail::ModelId{newId}); + } catch (const std::exception& exc) { + ::morph::log::logError(std::string{"[switchBackend] rollback deregister failed: "} + + exc.what()); + } + } + throw; + } + + // Phase 2 — commit. Every registration succeeded, so it is now safe to + // publish the new ids and swap the backend in. + for (const auto& [binding, newId] : staged) { + binding->currentId.store(newId); } _handlers = std::move(live); @@ -296,6 +325,10 @@ class Bridge { if (raw != 0U) { loadBackend()->deregisterModel(::morph::exec::detail::ModelId{raw}); } + // Reset to the "0 = unbound" sentinel so a late/concurrent executeVia on + // this binding fails fast on the documented guard rather than sending a + // now-destroyed ModelId to the backend. + binding->currentId.store(0); auto iter = std::ranges::find_if(_handlers, [&](auto& weak) { auto sptr = weak.lock(); return sptr && sptr.get() == binding.get(); @@ -378,6 +411,18 @@ class Bridge { } private: + template + friend class BridgeHandler; + + /// @brief Weak observer of this bridge's lifetime, handed to each handler. + /// + /// A `BridgeHandler` checks this in its destructor: if the token has expired + /// the `Bridge` is already gone, so it skips deregistration instead of + /// dereferencing a dangling `Bridge&`. The bridge must still outlive its + /// handlers for normal `execute`/`set` calls; this only makes the *teardown* + /// order-independent so a mis-ordered destruction is defined behaviour. + [[nodiscard]] std::weak_ptr liveness() const { return _liveness; } + std::shared_ptr<::morph::backend::detail::IBackend> loadBackend() const { std::scoped_lock const lock{_backendMtx}; return _backend; @@ -422,6 +467,8 @@ class Bridge { std::vector> _handlers; mutable std::mutex _sessionMtx; ::morph::session::Context _defaultSession; + // Destroyed with the Bridge; handlers hold weak_ptrs to it (see liveness()). + std::shared_ptr _liveness{std::make_shared()}; }; /// @brief RAII wrapper that binds a single model type to a `Bridge`. @@ -450,6 +497,7 @@ class BridgeHandler { /// @param guiExec Executor used to deliver `Completion` callbacks (e.g. the GUI thread). BridgeHandler(Bridge& bridge, ::morph::exec::IExecutor* guiExec) : _bridge{bridge}, + _bridgeAlive{bridge.liveness()}, _guiExec{guiExec}, _binding{bridge.template registerHandler()}, _subs{std::make_shared()} { @@ -466,6 +514,7 @@ class BridgeHandler { BridgeHandler(Bridge& bridge, ::morph::exec::IExecutor* guiExec, std::shared_ptr binding) : _bridge{bridge}, + _bridgeAlive{bridge.liveness()}, _guiExec{guiExec}, _binding{std::move(binding)}, _subs{std::make_shared()} { @@ -476,7 +525,17 @@ class BridgeHandler { } /// @brief Deregisters the binding from the bridge. - ~BridgeHandler() { _bridge.deregisterHandler(_binding); } + /// + /// If the `Bridge` has already been destroyed (liveness token expired), this + /// is a no-op: there is nothing to deregister from and dereferencing the + /// dangling `Bridge&` would be undefined behaviour. Destroying the bridge + /// before its handlers is still discouraged, but is now safe rather than a + /// use-after-free. + ~BridgeHandler() { + if (auto alive = _bridgeAlive.lock()) { + _bridge.deregisterHandler(_binding); + } + } BridgeHandler(const BridgeHandler&) = delete; BridgeHandler& operator=(const BridgeHandler&) = delete; @@ -570,7 +629,7 @@ class BridgeHandler { } std::any_cast(entry.draft).*FieldPtr = std::move(value); } - tryFireImpl(std::weak_ptr(_subs), ::morph::model::ActionTraits::typeId()); + tryFireImpl(_subs, ::morph::model::ActionTraits::typeId()); } /// @brief Discards the in-progress draft for action @p Action. @@ -640,8 +699,12 @@ class BridgeHandler { } template - static void tryFireImpl(const std::weak_ptr& weak, std::string_view typeId) { - auto state = weak.lock(); + static void tryFireImpl(const std::shared_ptr& state, std::string_view typeId) { + // Every caller holds the `SubscriberState` alive for the duration of this + // call, so no liveness check is needed on entry. The async continuations + // below may outlive the subscription, so they each re-check through this + // weak_ptr instead. + const std::weak_ptr weak{state}; using R = ::morph::model::ActionTraits::Result; Action snapshot; @@ -677,7 +740,7 @@ class BridgeHandler { outcome.sink(boxed); } if (outcome.refire) { - tryFireImpl(weak, typeId); + tryFireImpl(inner, typeId); } }) .onError([weak, typeId](const std::exception_ptr& err) { @@ -692,12 +755,13 @@ class BridgeHandler { logUnhandledError(typeId, err); } if (outcome.refire) { - tryFireImpl(weak, typeId); + tryFireImpl(inner, typeId); } }); } Bridge& _bridge; + std::weak_ptr _bridgeAlive; // expires when _bridge is destroyed ::morph::exec::IExecutor* _guiExec; std::shared_ptr _binding; std::shared_ptr _subs; diff --git a/include/morph/completion.hpp b/include/morph/completion.hpp index f147f8f..0beb868 100644 --- a/include/morph/completion.hpp +++ b/include/morph/completion.hpp @@ -61,7 +61,11 @@ struct CompletionState { auto savedFn = std::move(onErr); auto savedErr = error; callback = [savedFn = std::move(savedFn), savedErr]() mutable { savedFn(savedErr); }; - onErrAttached = true; + // Only mark the error handled (suppressing the destructor's orphan + // log) if we actually have an executor to deliver on. With a null + // executor the callback below is never posted, so the error must + // still reach the orphan logger rather than vanish silently. + onErrAttached = (cbExec != nullptr); } } if (callback != nullptr && cbExec != nullptr) { @@ -87,7 +91,10 @@ struct CompletionState { std::function fireNow; { std::scoped_lock const lock{mtx}; - onErrAttached = true; + // See setException: only suppress orphan logging when an executor + // exists to actually deliver the handler; a null executor otherwise + // drops the error and silences the orphan logger both at once. + onErrAttached = (cbExec != nullptr); if (ready && error) { auto savedErr = error; fireNow = [handler = std::move(handler), savedErr]() mutable { handler(savedErr); }; @@ -148,7 +155,9 @@ class Completion { /// @brief Constructs a completion backed by @p statePtr, delivering callbacks via @p execPtr. /// @param statePtr Shared state produced by the backend. - /// @param execPtr Executor on which callbacks are posted. May be `nullptr` (direct call). + /// @param execPtr Executor on which callbacks are posted. If `nullptr`, callbacks are + /// never delivered — they are silently dropped (see `setValue`/`setException`, + /// which post only when `cbExec != nullptr`). Completion(std::shared_ptr> statePtr, ::morph::exec::IExecutor* execPtr) : _state{std::move(statePtr)} { if (_state != nullptr) { diff --git a/include/morph/detail/quantity_equation.hpp b/include/morph/detail/quantity_equation.hpp index e84044e..42a9995 100644 --- a/include/morph/detail/quantity_equation.hpp +++ b/include/morph/detail/quantity_equation.hpp @@ -8,7 +8,7 @@ /// Included by `morph/quantity.hpp` when `MORPH_QUANTITY_PROVENANCE` is on. /// Walks the shared derivation DAG and produces the print-ready `equation()` /// lines: formula, substitution, result, and a `where` legend for reused -/// values. See `docs/quantity_type.md` for the output contract. +/// values. See `docs/spec/quantity_type.md` for the output contract. #include #include diff --git a/include/morph/executor.hpp b/include/morph/executor.hpp index 0a2f668..9e7422d 100644 --- a/include/morph/executor.hpp +++ b/include/morph/executor.hpp @@ -7,6 +7,7 @@ #include #include #include +#include #include #include @@ -35,7 +36,8 @@ struct IExecutor { /// @brief Multi-threaded executor backed by a fixed-size thread pool. /// /// Tasks are placed in a FIFO queue and consumed by worker threads. -/// Exceptions thrown by tasks are silently caught and discarded. +/// Exceptions thrown by tasks are caught and logged via `morph::log` (they do +/// not propagate out of the worker or abort sibling tasks). /// The destructor blocks until all worker threads have exited. // NOLINTNEXTLINE(cppcoreguidelines-special-member-functions) class ThreadPoolExecutor : public IExecutor { @@ -87,8 +89,12 @@ class ThreadPoolExecutor : public IExecutor { } try { task(); - // NOLINTNEXTLINE(bugprone-empty-catch) — task exceptions are intentionally discarded + } catch (const std::exception& exc) { + // A task failure must not kill the worker or unrelated tasks, but + // it must not vanish either — log and carry on. + ::morph::log::logError("[thread-pool] task threw: " + std::string{exc.what()}); } catch (...) { + ::morph::log::logError("[thread-pool] task threw unknown exception"); } } } diff --git a/include/morph/file_action_log.hpp b/include/morph/file_action_log.hpp index 30fee50..ef88834 100644 --- a/include/morph/file_action_log.hpp +++ b/include/morph/file_action_log.hpp @@ -2,6 +2,7 @@ #pragma once #include "action_log.hpp" +#include "logger.hpp" #include #include @@ -99,13 +100,30 @@ class FileActionLog : public IActionLog { [[nodiscard]] std::vector entries(std::string_view entityKey = {}) const override { std::scoped_lock const lock{_mtx}; std::ifstream in{_path}; - std::vector out; + std::vector lines; std::string line; while (std::getline(in, line)) { - if (line.empty()) { - continue; + if (!line.empty()) { + lines.push_back(line); + } + } + std::vector out; + for (std::size_t i = 0; i < lines.size(); ++i) { + LogEntry entry; + try { + entry = fromJson(lines[i]); + } catch (const std::exception& exc) { + // A crash between `append`'s `fwrite` and the next flush can leave + // a truncated final line. Tolerate exactly that — skip a malformed + // *trailing* line so the rest of the log stays readable — but a + // malformed line mid-file is genuine corruption and is re-thrown. + if (i + 1 == lines.size()) { + ::morph::log::logWarn("FileActionLog: skipping malformed trailing line in " + + _path.string() + ": " + std::string{exc.what()}); + break; + } + throw; } - auto entry = fromJson(line); if (entityKey.empty() || entry.entityKey == entityKey) { out.push_back(std::move(entry)); } diff --git a/include/morph/journal.hpp b/include/morph/journal.hpp index f8d6ed0..9ab7603 100644 --- a/include/morph/journal.hpp +++ b/include/morph/journal.hpp @@ -36,6 +36,11 @@ inline std::unique_ptr<::morph::model::detail::IModelHolder> replay( ::morph::model::detail::ModelRegistryFactory& registry = ::morph::model::detail::defaultRegistry(), ::morph::model::detail::ActionDispatcher& dispatcher = ::morph::model::detail::defaultDispatcher()) { auto holder = registry.create(modelTypeId); + // `registry.create` (via `ModelFactory::create`) auto-attaches the process + // default action log. Detach it before replaying: reconstruction re-runs the + // recorded actions, and without this each replayed dispatch would re-record + // into the live sink, corrupting the very audit trail we are reading from. + holder->attachActionLog(nullptr, {}); for (const auto& entry : entries) { dispatcher.dispatch(entry.modelType, entry.actionType, *holder, entry.payload); } diff --git a/include/morph/logger.hpp b/include/morph/logger.hpp index 7d46d2c..a613565 100644 --- a/include/morph/logger.hpp +++ b/include/morph/logger.hpp @@ -1,6 +1,7 @@ // SPDX-License-Identifier: Apache-2.0 #pragma once +#include #include #include #include @@ -47,8 +48,10 @@ using Logger = std::function; struct LogState { Logger sink = [](LogLevel lvl, std::string_view msg) { std::println(stderr, "[{}] {}", levelName(lvl), msg); }; - LogLevel minLevel = LogLevel::debug; - std::mutex mtx; + // Atomic so the level check is a lock-free fast path: `logFormat` and `log` + // can reject a suppressed message without touching `mtx` or formatting it. + std::atomic minLevel{LogLevel::debug}; + std::mutex mtx; // guards `sink` (and serialises sink invocation) }; inline LogState& logState() { @@ -57,10 +60,18 @@ inline LogState& logState() { } /// @brief Internal emit entry point used by the public level helpers. +/// +/// The sink is invoked while `mtx` is held, so sink calls are serialised. A sink +/// must therefore not call back into `morph::log` (log, `setLogger`, or +/// construct a `ScopedLoggerOverride`) — `std::mutex` is non-recursive and that +/// would self-deadlock — and should not block for long. inline void log(LogLevel level, std::string_view msg) { - std::scoped_lock const lock{logState().mtx}; auto& state = logState(); - if (state.sink && level >= state.minLevel) { + if (level < state.minLevel.load(std::memory_order_relaxed)) { + return; // lock-free reject of suppressed messages + } + std::scoped_lock const lock{state.mtx}; + if (state.sink) { state.sink(level, msg); } } @@ -84,15 +95,13 @@ inline void setLogger(std::function logger) { /// Messages below this level are silently dropped. Thread-safe. /// @param level Minimum level to emit. inline void setLogLevel(LogLevel level) { - std::scoped_lock const lock{detail::logState().mtx}; - detail::logState().minLevel = level; + detail::logState().minLevel.store(level, std::memory_order_relaxed); } /// @brief Returns the current minimum log level. Thread-safe. /// @return The active minimum level. inline LogLevel getLogLevel() { - std::scoped_lock const lock{detail::logState().mtx}; - return detail::logState().minLevel; + return detail::logState().minLevel.load(std::memory_order_relaxed); } // ── Level helpers ───────────────────────────────────────────────────────────── @@ -100,8 +109,14 @@ inline LogLevel getLogLevel() { namespace detail { /// @brief Formats @p fmt with @p args and forwards to `log()`. +/// +/// The level is checked *before* formatting, so a suppressed call pays no +/// `std::format` / allocation cost. template void logFormat(LogLevel level, std::format_string fmt, Args&&... args) { + if (level < logState().minLevel.load(std::memory_order_relaxed)) { + return; + } log(level, std::format(fmt, std::forward(args)...)); } @@ -163,10 +178,14 @@ class ScopedLoggerOverride { /// /// Use this when test code will install its own sink mid-test via /// `setLogger()` / `setLogLevel()` and just wants automatic restoration. - ScopedLoggerOverride() : _savedSink(detail::logState().sink), _savedLevel(detail::logState().minLevel) { + ScopedLoggerOverride() { + // NOLINTBEGIN(cppcoreguidelines-prefer-member-initializer) — these must be + // read while holding the lock; a member-initializer list would read the + // global state before the mutex is acquired (a data race). std::scoped_lock const lock{detail::logState().mtx}; - - + _savedSink = detail::logState().sink; + _savedLevel = detail::logState().minLevel; + // NOLINTEND(cppcoreguidelines-prefer-member-initializer) } /// @brief Installs @p sink and @p level; saves whatever was there before. @@ -174,12 +193,16 @@ class ScopedLoggerOverride { /// @param sink New logger sink. Pass a no-op lambda to suppress output entirely. /// @param level New minimum level. Defaults to `debug` so every message reaches @p sink. explicit ScopedLoggerOverride(std::function sink, - LogLevel level = LogLevel::debug) : _savedSink(std::move(detail::logState().sink)), _savedLevel(detail::logState().minLevel) { + LogLevel level = LogLevel::debug) { + // NOLINTBEGIN(cppcoreguidelines-prefer-member-initializer) — snapshot and + // install happen under one lock; a member-initializer list would read the + // previous state outside the mutex, racing a concurrent setLogger. std::scoped_lock const lock{detail::logState().mtx}; - - + _savedSink = std::move(detail::logState().sink); + _savedLevel = detail::logState().minLevel; detail::logState().sink = std::move(sink); detail::logState().minLevel = level; + // NOLINTEND(cppcoreguidelines-prefer-member-initializer) } /// @brief Restores the saved sink and level. diff --git a/include/morph/quantity.hpp b/include/morph/quantity.hpp index e906dee..d9152ba 100644 --- a/include/morph/quantity.hpp +++ b/include/morph/quantity.hpp @@ -11,7 +11,7 @@ /// of precision, and `std::nullopt` as a real "not entered" state), and **how /// it was computed** — its derivation, exposed through `equation()`. /// -/// See `docs/quantity_type.md` for the full design. The essentials: +/// See `docs/spec/quantity_type.md` for the full design. The essentials: /// /// - **Units are types.** `Quantity` and `Quantity` cannot /// be mixed; `operator*` / `operator/` deduce the result unit from the @@ -567,7 +567,7 @@ struct Quantity { return out; } - /// @brief The worked derivation as print-ready lines (see `docs/quantity_type.md`). + /// @brief The worked derivation as print-ready lines (see `docs/spec/quantity_type.md`). /// @return `[0]` formula, `[1]` substitution, `[2]` result, `[3..]` `where` /// legend; a single formatted-value element for a degenerate root /// (empty, named root, bare leaf, or tracing off). diff --git a/include/morph/remote.hpp b/include/morph/remote.hpp index b4b13e0..26eaa8b 100644 --- a/include/morph/remote.hpp +++ b/include/morph/remote.hpp @@ -92,13 +92,25 @@ class RemoteServer : public std::enable_shared_from_this { /// to call from a thread that *is* the worker pool — for example, from a /// `BridgeHandler` constructor invoked from inside an action handler. /// - /// Only safe for control messages (`register`, `deregister`) — `execute` - /// messages still post to the strand and would return before the result is - /// produced. The implementation rejects `execute` to make this explicit. + /// Only safe for control messages (`register`, `deregister`). An `execute` + /// envelope posts to the strand and produces its reply asynchronously, after + /// this synchronous call has already returned and destroyed the local reply + /// buffer the deferred callback would write into. To keep that from becoming a + /// dangling write, `execute` is rejected up front with an `err` reply. /// /// @param msg JSON-encoded `morph::wire::Envelope` (via `wire::encode`). /// @return JSON-encoded reply envelope. std::string handleInline(const std::string& msg) { + try { + auto env = ::morph::wire::decode(msg); + if (env.kind == "execute") { + return ::morph::wire::encode(::morph::wire::makeErr( + "handleInline does not support execute (reply is asynchronous)", env.callId)); + } + } catch (const std::exception&) { // NOLINT(bugprone-empty-catch) + // Malformed input: fall through so dispatchMessage produces the + // canonical decode-error reply (avoids duplicating that path here). + } std::string reply; std::function capture = [&reply](std::string out) { reply = std::move(out); }; dispatchMessage(msg, capture); @@ -182,6 +194,13 @@ class RemoteServer : public std::enable_shared_from_this { reply(::morph::wire::encode(::morph::wire::makeErr("unauthorized", env.callId))); return; } + // Make the identity authoritative: a verifying authorizer replaces the + // client-asserted principal with the one it extracted from a valid token, + // so model code reading session::current()->principal can trust it. A + // non-authenticating authorizer returns nullopt and leaves it unchanged. + if (auto verified = _authorizer->authenticate(env.session)) { + env.session.principal = std::move(*verified); + } ::morph::exec::detail::ModelId const mid{env.modelId}; std::shared_ptr<::morph::model::detail::IModelHolder> holder; { diff --git a/include/morph/session.hpp b/include/morph/session.hpp index 26cbc36..76885c8 100644 --- a/include/morph/session.hpp +++ b/include/morph/session.hpp @@ -2,6 +2,7 @@ #pragma once #include +#include #include #include #include @@ -20,9 +21,23 @@ namespace morph::session { /// the server-side `RemoteServer` sees the same values the GUI sent. On the /// local backend it travels in-memory via the `ActionCall`. struct Context { - /// @brief Auth principal (user id, JWT, session token — application-defined). + /// @brief Auth principal (user id). + /// + /// On the client this is whatever the caller sets and is **not** trustworthy + /// on its own — it is untrusted wire input. When a verifying authorizer (e.g. + /// `SigningAuthorizer`, `session_auth.hpp`) is installed, `RemoteServer` + /// **overwrites** this field with the principal extracted from a valid + /// `token` before dispatch, so `session::current()->principal` read inside a + /// model is the authenticated identity, not the client's claim. std::string principal; + /// @brief Bearer credential verified server-side (empty if unauthenticated). + /// + /// Typically a signed token minted by a login action via + /// `session::TokenIssuer` and attached to every call (see `session_auth.hpp` + /// and `docs/spec/security.md`). Travels in the wire envelope's `session`. + std::string token; + /// @brief Stable id used for distributed tracing / log correlation. Empty if unused. std::string requestId; @@ -56,6 +71,20 @@ struct IAuthorizer { // NOLINTNEXTLINE(bugprone-easily-swappable-parameters) std::string_view modelType, std::string_view actionType) const = 0; + + /// @brief Returns the authenticated principal for @p ctx, or `nullopt`. + /// + /// Called by `RemoteServer` after `authorize` succeeds. If it returns a + /// value, the server overwrites `Context::principal` with it before dispatch, + /// making the identity authoritative for model code that reads + /// `session::current()`. The default returns `nullopt` — authorizers that do + /// not authenticate (e.g. `AllowAllAuthorizer`) leave the client's principal + /// untouched. A verifying authorizer (`SigningAuthorizer`) overrides this. + /// @param ctx Per-call session attached by the client. + /// @return The verified principal to make authoritative, or `nullopt`. + [[nodiscard]] virtual std::optional authenticate([[maybe_unused]] const Context& ctx) const { + return std::nullopt; + } }; // NOLINTEND(cppcoreguidelines-special-member-functions) @@ -87,8 +116,11 @@ namespace detail { /// @brief Thread-local pointer to the `Context` for the currently dispatched action. /// -/// Set by `ActionDispatcher::dispatch` while the model's `execute()` runs, then -/// cleared. Models that opt in to session-aware logic call `current()` to read it. +/// Installed by the backend around the model call — `LocalBackend::execute` on the +/// local path and `RemoteServer::dispatchExecute` on the remote path — via a +/// `ScopedContext` that clears it again on scope exit. `ActionDispatcher::dispatch` +/// itself does not touch this pointer. Models that opt in to session-aware logic +/// call `current()` to read it. inline const Context*& tlsCurrent() { thread_local const Context* tls = nullptr; return tls; diff --git a/include/morph/session_auth.hpp b/include/morph/session_auth.hpp new file mode 100644 index 0000000..46e55d5 --- /dev/null +++ b/include/morph/session_auth.hpp @@ -0,0 +1,414 @@ +// SPDX-License-Identifier: Apache-2.0 + +#pragma once +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#include "session.hpp" + +/// @file session_auth.hpp +/// @brief Opt-in authenticated sessions: signed bearer tokens and a verifying +/// `IAuthorizer` so a `RemoteServer` can trust the caller's identity. +/// +/// Include this only if you want authentication. The token is a signed, +/// stateless bearer credential — `base64url(claimsJson) "." base64url(mac)` — +/// minted app-side by a login action via `TokenIssuer` and verified per request +/// by `SigningAuthorizer`, which also feeds the verified principal back to +/// `RemoteServer` so `session::current()->principal` is authoritative inside a +/// model. The MAC primitive is pluggable (`MacFunction`); a self-contained, +/// test-vector-verified HMAC-SHA256 (`hmacSha256`) is the default. See +/// `docs/spec/security.md` for the full trust model and its limits. + +namespace morph::session { + +// ── Cryptographic primitives (reference implementations) ───────────────────── +// +// These are self-contained so morph has no crypto dependency. They are correct +// (verified against the standard test vectors in test_session_auth.cpp) but are +// *reference* code: security-sensitive deployments should inject a vetted +// library's HMAC via `MacFunction` rather than rely on these. +namespace detail { + +// NOLINTBEGIN(readability-identifier-length,readability-function-cognitive-complexity,cppcoreguidelines-pro-bounds-constant-array-index,bugprone-easily-swappable-parameters) + +/// @brief SHA-256 (FIPS 180-4). Returns the raw 32-byte digest. +/// @param data Message bytes to hash. +/// @return 32-byte digest as a byte string. +inline std::string sha256(std::string_view data) { + static constexpr std::array kRound = { + 0x428a2f98, 0x71374491, 0xb5c0fbcf, 0xe9b5dba5, 0x3956c25b, 0x59f111f1, 0x923f82a4, 0xab1c5ed5, + 0xd807aa98, 0x12835b01, 0x243185be, 0x550c7dc3, 0x72be5d74, 0x80deb1fe, 0x9bdc06a7, 0xc19bf174, + 0xe49b69c1, 0xefbe4786, 0x0fc19dc6, 0x240ca1cc, 0x2de92c6f, 0x4a7484aa, 0x5cb0a9dc, 0x76f988da, + 0x983e5152, 0xa831c66d, 0xb00327c8, 0xbf597fc7, 0xc6e00bf3, 0xd5a79147, 0x06ca6351, 0x14292967, + 0x27b70a85, 0x2e1b2138, 0x4d2c6dfc, 0x53380d13, 0x650a7354, 0x766a0abb, 0x81c2c92e, 0x92722c85, + 0xa2bfe8a1, 0xa81a664b, 0xc24b8b70, 0xc76c51a3, 0xd192e819, 0xd6990624, 0xf40e3585, 0x106aa070, + 0x19a4c116, 0x1e376c08, 0x2748774c, 0x34b0bcb5, 0x391c0cb3, 0x4ed8aa4a, 0x5b9cca4f, 0x682e6ff3, + 0x748f82ee, 0x78a5636f, 0x84c87814, 0x8cc70208, 0x90befffa, 0xa4506ceb, 0xbef9a3f7, 0xc67178f2}; + + std::array hash = {0x6a09e667, 0xbb67ae85, 0x3c6ef372, 0xa54ff53a, + 0x510e527f, 0x9b05688c, 0x1f83d9ab, 0x5be0cd19}; + + std::vector msg(data.begin(), data.end()); + const uint64_t bitLen = static_cast(msg.size()) * 8; + msg.push_back(0x80); + while (msg.size() % 64 != 56) { + msg.push_back(0x00); + } + for (int shift = 56; shift >= 0; shift -= 8) { + msg.push_back(static_cast((bitLen >> shift) & 0xff)); + } + + auto rotr = [](uint32_t val, uint32_t bits) { return (val >> bits) | (val << (32 - bits)); }; + + for (std::size_t chunk = 0; chunk < msg.size(); chunk += 64) { + std::array w{}; + for (std::size_t i = 0; i < 16; ++i) { + w[i] = (static_cast(msg[chunk + (i * 4)]) << 24) | + (static_cast(msg[chunk + (i * 4) + 1]) << 16) | + (static_cast(msg[chunk + (i * 4) + 2]) << 8) | + static_cast(msg[chunk + (i * 4) + 3]); + } + for (std::size_t i = 16; i < 64; ++i) { + const uint32_t s0 = rotr(w[i - 15], 7) ^ rotr(w[i - 15], 18) ^ (w[i - 15] >> 3); + const uint32_t s1 = rotr(w[i - 2], 17) ^ rotr(w[i - 2], 19) ^ (w[i - 2] >> 10); + w[i] = w[i - 16] + s0 + w[i - 7] + s1; + } + + uint32_t a = hash[0]; + uint32_t b = hash[1]; + uint32_t c = hash[2]; + uint32_t d = hash[3]; + uint32_t e = hash[4]; + uint32_t f = hash[5]; + uint32_t g = hash[6]; + uint32_t h = hash[7]; + for (std::size_t i = 0; i < 64; ++i) { + const uint32_t bigS1 = rotr(e, 6) ^ rotr(e, 11) ^ rotr(e, 25); + const uint32_t choose = (e & f) ^ (~e & g); + const uint32_t t1 = h + bigS1 + choose + kRound[i] + w[i]; + const uint32_t bigS0 = rotr(a, 2) ^ rotr(a, 13) ^ rotr(a, 22); + const uint32_t maj = (a & b) ^ (a & c) ^ (b & c); + const uint32_t t2 = bigS0 + maj; + h = g; + g = f; + f = e; + e = d + t1; + d = c; + c = b; + b = a; + a = t1 + t2; + } + hash[0] += a; + hash[1] += b; + hash[2] += c; + hash[3] += d; + hash[4] += e; + hash[5] += f; + hash[6] += g; + hash[7] += h; + } + + std::string out(32, '\0'); + for (std::size_t i = 0; i < 8; ++i) { + out[(i * 4)] = static_cast((hash[i] >> 24) & 0xff); + out[(i * 4) + 1] = static_cast((hash[i] >> 16) & 0xff); + out[(i * 4) + 2] = static_cast((hash[i] >> 8) & 0xff); + out[(i * 4) + 3] = static_cast(hash[i] & 0xff); + } + return out; +} + +/// @brief HMAC-SHA256 (RFC 2104). Returns the raw 32-byte MAC. +/// @param key Secret key bytes. +/// @param msg Message bytes to authenticate. +/// @return 32-byte MAC as a byte string. +inline std::string hmacSha256Raw(std::string_view key, std::string_view msg) { + constexpr std::size_t blockSize = 64; + std::string block{key}; + if (block.size() > blockSize) { + block = sha256(block); + } + block.resize(blockSize, '\0'); + + std::string inner(blockSize, '\0'); + std::string outer(blockSize, '\0'); + for (std::size_t i = 0; i < blockSize; ++i) { + inner[i] = static_cast(static_cast(block[i]) ^ 0x36); + outer[i] = static_cast(static_cast(block[i]) ^ 0x5c); + } + const std::string innerHash = sha256(inner + std::string{msg}); + return sha256(outer + innerHash); +} + +/// @brief Constant-time byte-string equality (guards against MAC timing leaks). +/// @param lhs First string. +/// @param rhs Second string. +/// @return `true` iff the strings are byte-for-byte equal. +inline bool constantTimeEquals(std::string_view lhs, std::string_view rhs) { + if (lhs.size() != rhs.size()) { + return false; // length is not secret + } + uint8_t diff = 0; + for (std::size_t i = 0; i < lhs.size(); ++i) { + diff |= static_cast(lhs[i]) ^ static_cast(rhs[i]); + } + return diff == 0; +} + +/// @brief The url-safe base64 alphabet (RFC 4648 §5): `-` and `_` replace `+`/`/`. +inline constexpr std::string_view kB64Alphabet = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_"; + +/// @brief base64url-encodes @p in without padding (RFC 4648 §5). +/// @param in Bytes to encode. +/// @return The base64url text. +inline std::string base64UrlEncode(std::string_view in) { + std::string out; + std::size_t i = 0; + for (; i + 3 <= in.size(); i += 3) { + const uint32_t triple = (static_cast(static_cast(in[i])) << 16) | + (static_cast(static_cast(in[i + 1])) << 8) | + static_cast(static_cast(in[i + 2])); + out.push_back(kB64Alphabet[(triple >> 18) & 0x3f]); + out.push_back(kB64Alphabet[(triple >> 12) & 0x3f]); + out.push_back(kB64Alphabet[(triple >> 6) & 0x3f]); + out.push_back(kB64Alphabet[triple & 0x3f]); + } + const std::size_t rem = in.size() - i; + if (rem == 1) { + const uint32_t triple = static_cast(static_cast(in[i])) << 16; + out.push_back(kB64Alphabet[(triple >> 18) & 0x3f]); + out.push_back(kB64Alphabet[(triple >> 12) & 0x3f]); + } else if (rem == 2) { + const uint32_t triple = (static_cast(static_cast(in[i])) << 16) | + (static_cast(static_cast(in[i + 1])) << 8); + out.push_back(kB64Alphabet[(triple >> 18) & 0x3f]); + out.push_back(kB64Alphabet[(triple >> 12) & 0x3f]); + out.push_back(kB64Alphabet[(triple >> 6) & 0x3f]); + } + return out; +} + +/// @brief Decodes base64url text (no padding). Returns nullopt on invalid input. +/// @param in base64url text. +/// @return Decoded bytes, or nullopt if @p in contains an invalid character or length. +inline std::optional base64UrlDecode(std::string_view in) { + auto valueOf = [](char chr) -> int { + const auto pos = kB64Alphabet.find(chr); + return pos == std::string_view::npos ? -1 : static_cast(pos); + }; + std::string out; + uint32_t buffer = 0; + int bits = 0; + for (const char chr : in) { + const int val = valueOf(chr); + if (val < 0) { + return std::nullopt; + } + buffer = (buffer << 6) | static_cast(val); + bits += 6; + if (bits >= 8) { + bits -= 8; + out.push_back(static_cast((buffer >> bits) & 0xff)); + } + } + return out; +} + +// NOLINTEND(readability-identifier-length,readability-function-cognitive-complexity,cppcoreguidelines-pro-bounds-constant-array-index,bugprone-easily-swappable-parameters) + +} // namespace detail + +/// @brief Pluggable message-authentication primitive: `mac(secret, message)`. +/// +/// Returns the raw MAC bytes. Defaults to `hmacSha256`; inject a vetted +/// library's HMAC for production. +using MacFunction = std::function; + +/// @brief Reference HMAC-SHA256 `MacFunction` (the default). Returns raw MAC bytes. +/// @param key Secret key bytes. +/// @param message Message bytes to authenticate. +/// @return 32-byte HMAC as a byte string. +inline std::string hmacSha256(std::string_view key, std::string_view message) { + return detail::hmacSha256Raw(key, message); +} + +/// @brief Wall-clock source (milliseconds since the Unix epoch), injectable for tests. +using Clock = std::function; + +/// @brief Default clock: `std::chrono::system_clock` in milliseconds since epoch. +/// @return Current time in ms since the Unix epoch. +inline int64_t systemClockMs() { + return std::chrono::duration_cast(std::chrono::system_clock::now().time_since_epoch()) + .count(); +} + +/// @brief Verified claims carried by a session token. +/// +/// Serialised to JSON (Glaze) as the signed payload. Extend with app claims as +/// needed — unknown fields are ignored on read, so adding claims is compatible. +struct SessionToken { + /// @brief Authenticated user/principal id. + std::string principal; + /// @brief Issue time, ms since epoch (informational). + int64_t issuedAtMs = 0; + /// @brief Expiry, ms since epoch. `0` means "never expires" (discouraged). + int64_t expiresAtMs = 0; + /// @brief Coarse-grained roles the app can key authorization policy on. + std::vector roles; +}; + +/// @brief Why token verification failed. +enum class AuthError : std::uint8_t { + Malformed, ///< Not `payload.sig`, bad base64url, or unparseable claims. + BadSignature, ///< MAC did not match — forged or tampered. + Expired, ///< `expiresAtMs` is in the past relative to the supplied clock. +}; + +/// @brief Mints signed bearer tokens from claims using a shared secret. +/// +/// Wire format: `base64url(claimsJson) "." base64url(mac(secret, payload))`. +class TokenIssuer { +public: + /// @brief Constructs an issuer over @p secret using @p mac. + /// @param secret Shared secret (same value the verifier uses). + /// @param mac MAC primitive; defaults to `hmacSha256`. + explicit TokenIssuer(std::string secret, MacFunction mac = hmacSha256) + : _secret{std::move(secret)}, _mac{std::move(mac)} {} + + /// @brief Serialises @p claims and returns a signed token string. + /// @param claims Claims to embed and sign. + /// @return The signed `payload.sig` token. + [[nodiscard]] std::string issue(const SessionToken& claims) const { + std::string json; + // `SessionToken` is a flat aggregate, so writing it into a `std::string` + // cannot fail — the result is unconditional. + (void)glz::write_json(claims, json); + const std::string payload = detail::base64UrlEncode(json); + const std::string sig = detail::base64UrlEncode(_mac(_secret, payload)); + return payload + "." + sig; + } + +private: + std::string _secret; + MacFunction _mac; +}; + +/// @brief Verifies signed bearer tokens against a shared secret. +class TokenVerifier { +public: + /// @brief Constructs a verifier over @p secret using @p mac. + /// @param secret Shared secret (same value the issuer used). + /// @param mac MAC primitive; defaults to `hmacSha256`. + explicit TokenVerifier(std::string secret, MacFunction mac = hmacSha256) + : _secret{std::move(secret)}, _mac{std::move(mac)} {} + + /// @brief Verifies @p token's signature and expiry and returns its claims. + /// + /// The MAC is checked *before* the payload is parsed, so untrusted JSON is + /// never handed to the parser until authenticity is established. + /// @param token Token string (`payload.sig`). + /// @param nowMs Current time (ms since epoch) to check expiry against. + /// @return The verified claims, or an `AuthError`. + [[nodiscard]] std::expected verify(std::string_view token, int64_t nowMs) const { + const auto dot = token.find('.'); + if (dot == std::string_view::npos || token.find('.', dot + 1) != std::string_view::npos) { + return std::unexpected(AuthError::Malformed); + } + const std::string_view payload = token.substr(0, dot); + const auto sig = detail::base64UrlDecode(token.substr(dot + 1)); + if (!sig) { + return std::unexpected(AuthError::Malformed); + } + const std::string expectedMac = _mac(_secret, std::string{payload}); + if (!detail::constantTimeEquals(*sig, expectedMac)) { + return std::unexpected(AuthError::BadSignature); + } + const auto json = detail::base64UrlDecode(payload); + if (!json) { + return std::unexpected(AuthError::Malformed); + } + SessionToken claims; + if (glz::read_json(claims, *json)) { + return std::unexpected(AuthError::Malformed); + } + if (claims.expiresAtMs != 0 && nowMs > claims.expiresAtMs) { + return std::unexpected(AuthError::Expired); + } + return claims; + } + +private: + std::string _secret; + MacFunction _mac; +}; + +/// @brief `IAuthorizer` that authenticates via a signed `Context::token`. +/// +/// `authorize` returns `true` only for a token with a valid signature and +/// unexpired claims (and, if a policy is supplied, one the policy admits). +/// `authenticate` returns the verified principal so `RemoteServer` makes it +/// authoritative. Install it on the server: +/// @code +/// auto authz = std::make_shared(sharedSecret); +/// auto server = std::make_shared(pool, dispatcher, registry, authz); +/// @endcode +class SigningAuthorizer : public IAuthorizer { +public: + /// @brief Optional per-request policy over verified claims. + /// + /// Receives the verified token plus the target ids; return `false` to deny. + /// The default (empty) admits any validly-signed, unexpired token. + using Policy = + std::function; + + /// @brief Constructs the authorizer. + /// @param secret Shared secret used to verify tokens. + /// @param mac MAC primitive; defaults to `hmacSha256`. + /// @param clock Time source (ms since epoch) for expiry; defaults to system time. + /// @param policy Optional per-request policy over verified claims. + explicit SigningAuthorizer(std::string secret, MacFunction mac = hmacSha256, Clock clock = systemClockMs, + Policy policy = {}) + : _verifier{std::move(secret), std::move(mac)}, _clock{std::move(clock)}, _policy{std::move(policy)} {} + + /// @brief Allows the call iff @p ctx carries a valid token the policy admits. + /// @param ctx Per-call session (its `token` is verified). + /// @param modelType Target model type id (passed to the policy). + /// @param actionType Target action type id (passed to the policy). + /// @return `true` to allow dispatch, `false` to reject. + [[nodiscard]] bool authorize(const Context& ctx, + // NOLINTNEXTLINE(bugprone-easily-swappable-parameters) + std::string_view modelType, std::string_view actionType) const override { + const auto claims = _verifier.verify(ctx.token, _clock()); + if (!claims) { + return false; + } + return !_policy || _policy(*claims, modelType, actionType); + } + + /// @brief Returns the verified principal so the server can make it authoritative. + /// @param ctx Per-call session (its `token` is verified). + /// @return The token's principal if valid, else `nullopt`. + [[nodiscard]] std::optional authenticate(const Context& ctx) const override { + const auto claims = _verifier.verify(ctx.token, _clock()); + if (!claims) { + return std::nullopt; + } + return claims->principal; + } + +private: + TokenVerifier _verifier; + Clock _clock; + Policy _policy; +}; + +} // namespace morph::session diff --git a/include/morph/strand.hpp b/include/morph/strand.hpp index 37bf879..1d318e8 100644 --- a/include/morph/strand.hpp +++ b/include/morph/strand.hpp @@ -3,13 +3,16 @@ #pragma once #include #include +#include #include #include #include #include +#include #include #include "executor.hpp" +#include "logger.hpp" namespace morph::exec::detail { @@ -110,8 +113,13 @@ class StrandExecutor { } try { task(); - // NOLINTNEXTLINE(bugprone-empty-catch) — task exceptions are intentionally discarded + } catch (const std::exception& exc) { + // The strand is where Model::execute() actually runs; a throw + // here must not stall the strand or vanish — log and continue so + // the next queued task for this model still runs. + ::morph::log::logError("[strand] task threw: " + std::string{exc.what()}); } catch (...) { + ::morph::log::logError("[strand] task threw unknown exception"); } // Decide "keep running vs. drain-and-erase" atomically across the // map slot and the strand's pending queue. Doing it in two steps diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index 2aa2c7c..c9677c7 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -37,6 +37,7 @@ add_executable(morph_tests test_quantity.cpp test_quantity_forms.cpp test_datetime.cpp + test_session_auth.cpp ) target_link_libraries(morph_tests diff --git a/tests/test_action_log_phase2.cpp b/tests/test_action_log_phase2.cpp index dd6dfdd..3c26f09 100644 --- a/tests/test_action_log_phase2.cpp +++ b/tests/test_action_log_phase2.cpp @@ -168,6 +168,45 @@ TEST_CASE("FileActionLog: entries() skips blank lines", "[action_log][phase2][fi } } +TEST_CASE("FileActionLog: entries() tolerates a malformed trailing line (crash-truncated write)", + "[action_log][phase2][file]") { + TempFile tmp{"file_truncated_tail"}; + { + FileActionLog log{tmp.path}; + log.append(makeEntry("P2_Model", "acct-1", "P2_Deposit", "{}", "1")); + log.flush(); + } + // Simulate a crash between fwrite() and the next flush: a syntactically + // invalid, non-empty final line with no closing brace. + { + std::ofstream raw{tmp.path, std::ios::app}; + raw << R"({"seq":2,"entityKey":"acct-2","actionType":"P2_Deposit)"; + } + + FileActionLog log{tmp.path}; + auto all = log.entries(); + REQUIRE(all.size() == 1); + REQUIRE(all[0].entityKey == "acct-1"); +} + +TEST_CASE("FileActionLog: entries() rethrows on a malformed line that is NOT the last (genuine corruption)", + "[action_log][phase2][file]") { + TempFile tmp{"file_corrupt_mid"}; + { + FileActionLog log{tmp.path}; + log.append(makeEntry("P2_Model", "acct-1", "P2_Deposit", "{}", "1")); + log.flush(); + } + { + std::ofstream raw{tmp.path, std::ios::app}; + raw << "not json at all\n"; + raw << morph::journal::toJson(makeEntry("P2_Model", "acct-2", "P2_Deposit", "{}", "2")) << "\n"; + } + + FileActionLog log{tmp.path}; + REQUIRE_THROWS_AS(log.entries(), morph::journal::SerializationError); +} + // ── Save action end-to-end: SessionLog + FileActionLog, the pattern the design // doc asked for ("wire sessionLog.checkpoint(sink) into a real Save action's // completion handler") ────────────────────────────────────────────────────── diff --git a/tests/test_executor_extra.cpp b/tests/test_executor_extra.cpp index da4a047..9b1bcc2 100644 --- a/tests/test_executor_extra.cpp +++ b/tests/test_executor_extra.cpp @@ -63,3 +63,18 @@ TEST_CASE("morph::exec::ThreadPoolExecutor: exception in one task does not kill std::this_thread::sleep_for(std::chrono::milliseconds(200)); REQUIRE(afterCount.load() == afterTasks); } + +TEST_CASE("morph::exec::ThreadPoolExecutor: non-std::exception throw in one task does not kill worker", + "[executor]") { + morph::exec::ThreadPoolExecutor pool{1}; + std::atomic afterCount{0}; + constexpr int afterTasks = 5; + + pool.post([] { throw 42; }); // NOLINT(hicpp-exception-baseclass) — exercises the catch(...) arm + for (int i = 0; i < afterTasks; ++i) { + pool.post([&] { afterCount.fetch_add(1); }); + } + + std::this_thread::sleep_for(std::chrono::milliseconds(200)); + REQUIRE(afterCount.load() == afterTasks); +} diff --git a/tests/test_handler_binding.cpp b/tests/test_handler_binding.cpp index c062f22..5e8ee83 100644 --- a/tests/test_handler_binding.cpp +++ b/tests/test_handler_binding.cpp @@ -163,8 +163,9 @@ TEST_CASE("morph::bridge::detail::HandlerBinding: shared_ptr keeps binding alive } // Object is still alive and readable - no use-after-free. - // currentId retains the value it had when the model was registered; - // deregisterHandler does not reset it to 0. + // deregisterHandler resets currentId to the "0 = unbound" sentinel, so a + // late executeVia on this binding fails fast on the not-bound guard instead + // of sending a now-destroyed ModelId to the backend. REQUIRE(captured != nullptr); - REQUIRE(captured->currentId.load() != 0); + REQUIRE(captured->currentId.load() == 0); } diff --git a/tests/test_remote_extra.cpp b/tests/test_remote_extra.cpp index 3513682..4ea91e1 100644 --- a/tests/test_remote_extra.cpp +++ b/tests/test_remote_extra.cpp @@ -4,6 +4,8 @@ #include #include #include +#include +#include #include #include #include @@ -240,3 +242,98 @@ TEST_CASE("morph::backend::SimulatedRemoteBackend: register failure raises excep REQUIRE(decoded.kind == "err"); REQUIRE(decoded.message == "no models allowed"); } + +// ── morph::backend::RemoteServer::handleInline: execute is rejected up front ──────────────── + +TEST_CASE("morph::backend::RemoteServer::handleInline: execute envelope replies with err, not silently dropped", + "[remote][handleInline]") { + morph::exec::ThreadPoolExecutor pool{2}; + auto& env = sharedEnv(); + auto server = std::make_shared(pool, env.dispatcher, env.registry); + + morph::wire::Envelope req; + req.kind = "execute"; + req.callId = 5; + req.modelId = 1; + req.modelType = "RX_SquareModel"; + req.actionType = "RX_SquareAction"; + req.body = R"({"x":3})"; + + auto reply = morph::wire::decode(server->handleInline(morph::wire::encode(req))); + REQUIRE(reply.kind == "err"); + REQUIRE(reply.callId == 5U); + REQUIRE(reply.message.find("execute") != std::string::npos); +} + +TEST_CASE("morph::backend::RemoteServer::handleInline: malformed input falls through to the decode-error reply", + "[remote][handleInline]") { + morph::exec::ThreadPoolExecutor pool{2}; + auto& env = sharedEnv(); + auto server = std::make_shared(pool, env.dispatcher, env.registry); + + auto reply = morph::wire::decode(server->handleInline("not json")); + REQUIRE(reply.kind == "err"); +} + +// ── morph::backend::RemoteServer::dispatchExecute: authenticate() overrides principal ─────── + +struct WhoAmI {}; +struct PrincipalEchoModel { + std::string execute(const WhoAmI&) const { + const auto* ctx = morph::session::current(); + return (ctx != nullptr) ? ctx->principal : std::string{}; + } +}; + +template <> +struct morph::model::ModelTraits { + static constexpr std::string_view typeId() { return "RX_PrincipalEchoModel"; } +}; +template <> +struct morph::model::ActionTraits { + using Result = std::string; + static constexpr std::string_view typeId() { return "RX_WhoAmI"; } + static std::string toJson(const WhoAmI&) { return "{}"; } + static WhoAmI fromJson(std::string_view) { return {}; } + static std::string resultToJson(const std::string& res) { return "\"" + res + "\""; } + static std::string resultFromJson(std::string_view json) { + std::string out; + (void)glz::read_json(out, json); + return out; + } +}; + +TEST_CASE( + "morph::backend::RemoteServer::dispatchExecute: a verifying authorizer overrides the client-asserted principal", + "[remote][auth]") { + morph::exec::ThreadPoolExecutor pool{2}; + morph::model::detail::ModelRegistryFactory registry; + morph::model::detail::ActionDispatcher dispatcher; + registry.registerModel("RX_PrincipalEchoModel"); + dispatcher.registerAction("RX_PrincipalEchoModel", "RX_WhoAmI"); + + const std::string secret = "hmac-key"; + auto authz = std::make_shared(secret); + auto server = std::make_shared(pool, authz, dispatcher, registry); + + WaitReply reg; + server->handle(morph::wire::encode(morph::wire::makeRegister("RX_PrincipalEchoModel")), std::ref(reg)); + reg.await(); + REQUIRE(reg.env.kind == "ok"); + + morph::wire::Envelope req; + req.kind = "execute"; + req.modelId = reg.env.modelId; + req.modelType = "RX_PrincipalEchoModel"; + req.actionType = "RX_WhoAmI"; + req.body = "{}"; + req.session.principal = "spoofed-client-claim"; + req.session.token = + morph::session::TokenIssuer{secret}.issue(morph::session::SessionToken{.principal = "real-verified-user"}); + + WaitReply waiter; + server->handle(morph::wire::encode(req), std::ref(waiter)); + waiter.await(); + REQUIRE(waiter.env.kind == "ok"); + REQUIRE(waiter.env.body == "\"real-verified-user\""); +} diff --git a/tests/test_session_auth.cpp b/tests/test_session_auth.cpp new file mode 100644 index 0000000..8585f23 --- /dev/null +++ b/tests/test_session_auth.cpp @@ -0,0 +1,216 @@ +// SPDX-License-Identifier: Apache-2.0 + +#include + +#include + +#include +#include +#include +#include +#include + +using morph::session::AuthError; +using morph::session::hmacSha256; +using morph::session::SessionToken; +using morph::session::SigningAuthorizer; +using morph::session::TokenIssuer; +using morph::session::TokenVerifier; + +namespace { + +std::string toHex(std::string_view bytes) { + static constexpr std::string_view kHex = "0123456789abcdef"; + std::string out; + out.reserve(bytes.size() * 2); + for (const char chr : bytes) { + const auto val = static_cast(chr); + out.push_back(kHex.at(val >> 4)); + out.push_back(kHex.at(val & 0x0f)); + } + return out; +} + +// A fixed clock so token-expiry tests are deterministic. +constexpr int64_t kNow = 1'700'000'000'000; // ms since epoch +constexpr auto fixedClock = [] { return kNow; }; + +SessionToken sampleClaims(std::string principal, std::vector roles = {}) { + return SessionToken{.principal = std::move(principal), + .issuedAtMs = kNow, + .expiresAtMs = kNow + 60'000, + .roles = std::move(roles)}; +} + +} // namespace + +TEST_CASE("sha256 matches FIPS 180-4 known-answer vectors", "[session_auth][crypto]") { + REQUIRE(toHex(morph::session::detail::sha256("abc")) == + "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad"); + REQUIRE(toHex(morph::session::detail::sha256("")) == + "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"); + // A multi-block message (> 64 bytes) exercises the chunk loop. + REQUIRE(toHex(morph::session::detail::sha256("abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq")) == + "248d6a61d20638b8e5c026930c3e6039a33ce45964ff2167f6ecedd419db06c1"); +} + +TEST_CASE("hmacSha256 matches RFC 4231 test case 2", "[session_auth][crypto]") { + REQUIRE(toHex(hmacSha256("Jefe", "what do ya want for nothing?")) == + "5bdcc146bf60754e6a042426089575c75a003f089d2739839dec58b964ec3843"); +} + +TEST_CASE("base64url uses the url-safe alphabet without padding", "[session_auth][crypto]") { + const auto encoded = morph::session::detail::base64UrlEncode(std::string_view{"\x00\x01\x02\xff", 4}); + REQUIRE_FALSE(encoded.contains('+')); + REQUIRE_FALSE(encoded.contains('/')); + REQUIRE_FALSE(encoded.contains('=')); +} + +TEST_CASE("base64url round-trips every remainder length", "[session_auth][crypto]") { + for (const std::string_view sample : {"", "f", "fo", "foo", "foob", "fooba", "foobar"}) { + const auto decoded = morph::session::detail::base64UrlDecode(morph::session::detail::base64UrlEncode(sample)); + REQUIRE(decoded.value() == std::string{sample}); + } +} + +TEST_CASE("base64url rejects invalid characters", "[session_auth][crypto]") { + REQUIRE_FALSE(morph::session::detail::base64UrlDecode("has space").has_value()); +} + +TEST_CASE("an issued token verifies and returns its claims", "[session_auth]") { + const TokenIssuer issuer{"top-secret"}; + const std::string token = issuer.issue(sampleClaims("alice", {"admin"})); + REQUIRE_FALSE(token.empty()); + + const auto verified = TokenVerifier{"top-secret"}.verify(token, kNow); + REQUIRE(verified.has_value()); + REQUIRE(verified.value().principal == "alice"); + REQUIRE(verified.value().roles == std::vector{"admin"}); +} + +TEST_CASE("verification rejects the wrong secret", "[session_auth]") { + const std::string token = TokenIssuer{"top-secret"}.issue(sampleClaims("bob")); + const auto res = TokenVerifier{"other-secret"}.verify(token, kNow); + REQUIRE_FALSE(res.has_value()); + REQUIRE(res.error() == AuthError::BadSignature); +} + +TEST_CASE("verification rejects a tampered payload", "[session_auth]") { + std::string token = TokenIssuer{"top-secret"}.issue(sampleClaims("bob")); + token.front() = (token.front() == 'A') ? 'B' : 'A'; // flip a payload char + const auto res = TokenVerifier{"top-secret"}.verify(token, kNow); + REQUIRE_FALSE(res.has_value()); + REQUIRE(res.error() == AuthError::BadSignature); +} + +TEST_CASE("verification rejects an expired token", "[session_auth]") { + const std::string token = TokenIssuer{"top-secret"}.issue(sampleClaims("bob")); + const auto res = TokenVerifier{"top-secret"}.verify(token, kNow + 120'000); + REQUIRE_FALSE(res.has_value()); + REQUIRE(res.error() == AuthError::Expired); +} + +TEST_CASE("verification rejects a malformed token", "[session_auth]") { + REQUIRE(TokenVerifier{"top-secret"}.verify("notatoken", kNow).error() == AuthError::Malformed); +} + +TEST_CASE("verification rejects a token with more than one '.'", "[session_auth]") { + REQUIRE(TokenVerifier{"top-secret"}.verify("a.b.c", kNow).error() == AuthError::Malformed); +} + +TEST_CASE("verification rejects a signature segment with invalid base64url characters", "[session_auth]") { + const std::string token = TokenIssuer{"top-secret"}.issue(sampleClaims("bob")); + const auto dot = token.find('.'); + const std::string corrupted = token.substr(0, dot + 1) + "has space"; + REQUIRE(TokenVerifier{"top-secret"}.verify(corrupted, kNow).error() == AuthError::Malformed); +} + +TEST_CASE("verification rejects a payload segment with invalid base64url characters", "[session_auth]") { + // The MAC is computed over the raw (still-encoded) payload substring, so a + // hand-crafted payload needs its MAC recomputed to reach past BadSignature + // and into the payload-decode Malformed branch. + const std::string secret = "top-secret"; + const std::string payload = "has space"; + const std::string sig = morph::session::detail::base64UrlEncode(hmacSha256(secret, payload)); + const std::string token = payload + "." + sig; + REQUIRE(TokenVerifier{secret}.verify(token, kNow).error() == AuthError::Malformed); +} + +TEST_CASE("hmacSha256 hashes an over-long key down to the block size (RFC 4231 case 6)", "[session_auth][crypto]") { + // A key longer than the 64-byte HMAC block is first reduced with SHA-256. + const std::string longKey(131, '\xaa'); + REQUIRE(toHex(hmacSha256(longKey, "Test Using Larger Than Block-Size Key - Hash Key First")) == + "60e431591ee0b67f0d8a26aacbf5b77f8e0bc6213728c5140546040f0ee37f54"); +} + +TEST_CASE("verification rejects a signature of the wrong length", "[session_auth]") { + // A signature that base64url-decodes cleanly but is not 32 bytes must fail on + // the constant-time length check rather than the byte comparison. + const std::string secret = "top-secret"; + const std::string payload = morph::session::detail::base64UrlEncode(std::string_view{"claims"}); + const std::string shortSig = morph::session::detail::base64UrlEncode(std::string_view{"short"}); + const std::string token = payload + "." + shortSig; + REQUIRE(TokenVerifier{secret}.verify(token, kNow).error() == AuthError::BadSignature); +} + +TEST_CASE("verification rejects a correctly-signed payload that is not valid JSON", "[session_auth]") { + // Reaches past the MAC check (authentic) and the base64url decode (valid + // bytes) into the claims-parse branch: the decoded payload is not JSON. + const std::string secret = "top-secret"; + const std::string payload = morph::session::detail::base64UrlEncode(std::string_view{"not-json"}); + const std::string sig = morph::session::detail::base64UrlEncode(hmacSha256(secret, payload)); + const std::string token = payload + "." + sig; + REQUIRE(TokenVerifier{secret}.verify(token, kNow).error() == AuthError::Malformed); +} + +TEST_CASE("SigningAuthorizer authorizes a valid token and exposes its principal", "[session_auth]") { + const std::string secret = "hmac-key"; + const SigningAuthorizer authz{secret, hmacSha256, fixedClock}; + morph::session::Context ctx; + ctx.token = TokenIssuer{secret}.issue(sampleClaims("carol")); + + REQUIRE(authz.authorize(ctx, "AccountModel", "Deposit")); + REQUIRE(authz.authenticate(ctx).value() == "carol"); +} + +TEST_CASE("SigningAuthorizer denies a request with no token", "[session_auth]") { + const SigningAuthorizer authz{"hmac-key", hmacSha256, fixedClock}; + const morph::session::Context anon; + REQUIRE_FALSE(authz.authorize(anon, "AccountModel", "Deposit")); + REQUIRE_FALSE(authz.authenticate(anon).has_value()); +} + +TEST_CASE("SigningAuthorizer applies a role policy over a valid token", "[session_auth]") { + const std::string secret = "hmac-key"; + const auto requiresAdmin = [](const SessionToken& claims, std::string_view, std::string_view) { + return std::ranges::find(claims.roles, "admin") != claims.roles.end(); + }; + const SigningAuthorizer authz{secret, hmacSha256, fixedClock, requiresAdmin}; + + morph::session::Context noRole; + noRole.token = TokenIssuer{secret}.issue(sampleClaims("carol")); + REQUIRE_FALSE(authz.authorize(noRole, "AccountModel", "Deposit")); + + morph::session::Context admin; + admin.token = TokenIssuer{secret}.issue(sampleClaims("dave", {"admin"})); + REQUIRE(authz.authorize(admin, "AccountModel", "Deposit")); +} + +TEST_CASE("SigningAuthorizer with a policy still denies an invalid token before the policy runs", "[session_auth]") { + const std::string secret = "hmac-key"; + bool policyCalled = false; + const auto alwaysAllow = [&](const SessionToken&, std::string_view, std::string_view) { + policyCalled = true; + return true; + }; + const SigningAuthorizer authz{secret, hmacSha256, fixedClock, alwaysAllow}; + + morph::session::Context noToken; + REQUIRE_FALSE(authz.authorize(noToken, "AccountModel", "Deposit")); + REQUIRE_FALSE(policyCalled); + + morph::session::Context wrongSecret; + wrongSecret.token = TokenIssuer{"other-secret"}.issue(sampleClaims("eve")); + REQUIRE_FALSE(authz.authorize(wrongSecret, "AccountModel", "Deposit")); + REQUIRE_FALSE(policyCalled); +} diff --git a/tests/test_strand_extra.cpp b/tests/test_strand_extra.cpp index 6bff622..c3ac44d 100644 --- a/tests/test_strand_extra.cpp +++ b/tests/test_strand_extra.cpp @@ -25,6 +25,23 @@ TEST_CASE("morph::exec::detail::StrandExecutor: exception in task is swallowed a REQUIRE(afterRan.load()); } +TEST_CASE("morph::exec::detail::StrandExecutor: non-std::exception throw is swallowed and next task still runs", + "[strand]") { + morph::exec::ThreadPoolExecutor pool{2}; + morph::exec::detail::StrandExecutor strand{pool}; + morph::exec::detail::ModelId key{43}; + + std::atomic afterRan{false}; + + strand.post(key, [] { throw 7; }); // NOLINT(hicpp-exception-baseclass) — exercises the catch(...) arm + strand.post(key, [&] { afterRan.store(true); }); + + for (int i = 0; i < 50 && !afterRan.load(); ++i) { + std::this_thread::sleep_for(std::chrono::milliseconds(10)); + } + REQUIRE(afterRan.load()); +} + TEST_CASE("morph::exec::detail::StrandExecutor: rapid post to running strand queues correctly", "[strand]") { // Post many tasks to same key without waiting — exercises the "already running" branch morph::exec::ThreadPoolExecutor pool{4}; diff --git a/tests/test_switch_backend.cpp b/tests/test_switch_backend.cpp index 0a399ed..a4a7094 100644 --- a/tests/test_switch_backend.cpp +++ b/tests/test_switch_backend.cpp @@ -240,3 +240,133 @@ TEST_CASE("morph::backend::SimulatedRemoteBackend::notifyBackendChanged is a doc morph::backend::SimulatedRemoteBackend backend{*server}; REQUIRE_NOTHROW(backend.notifyBackendChanged()); } + +// ── switchBackend rollback on partial-registration failure ──────────────────── + +namespace { + +/// @brief Backend whose `registerModelWithContext` throws once the Nth call is +/// reached, so `Bridge::switchBackend`'s Phase-1 rollback can be exercised +/// deterministically. `deregisterModel` optionally throws too, to exercise the +/// nested rollback-failure log path. +class FlakyBackend : public morph::backend::detail::IBackend { +public: + FlakyBackend(morph::exec::IExecutor& pool, int failOnCallNumber, bool deregisterThrows = false) + : _inner{pool}, _failOnCallNumber{failOnCallNumber}, _deregisterThrows{deregisterThrows} {} + + morph::exec::detail::ModelId registerModel( + const std::string& typeId, + std::function()> factory) override { + return registerModelWithContext(typeId, std::move(factory), {}); + } + + morph::exec::detail::ModelId registerModelWithContext( + const std::string& typeId, std::function()> factory, + std::string_view contextKey) override { + ++_calls; + if (_calls == _failOnCallNumber) { + throw std::runtime_error("simulated registration failure"); + } + auto id = _inner.registerModelWithContext(typeId, std::move(factory), contextKey); + _registered.push_back(id); + return id; + } + + void deregisterModel(morph::exec::detail::ModelId mid) override { + ++(*_deregisterCalls); + if (_deregisterThrows) { + throw std::runtime_error("simulated deregister failure"); + } + _inner.deregisterModel(mid); + } + + morph::async::Completion> execute(morph::exec::detail::ModelId mid, + morph::backend::detail::ActionCall call, + morph::exec::IExecutor* cbExec) override { + return _inner.execute(mid, std::move(call), cbExec); + } + + void notifyBackendChanged() override { _inner.notifyBackendChanged(); } + void cancelPending(const std::exception_ptr& exc) override { _inner.cancelPending(exc); } + + /// @brief Shared handle to the deregister-call counter. + /// + /// `Bridge::switchBackend` takes ownership of the backend and destroys it + /// when a partial-registration failure unwinds, so a raw `this` pointer is + /// dangling by the time the test inspects the rollback. The counter lives in + /// a `shared_ptr` the test can hold independently, outliving the backend. + [[nodiscard]] std::shared_ptr deregisterCallCounter() const { return _deregisterCalls; } + +private: + morph::backend::LocalBackend _inner; + int _failOnCallNumber; + bool _deregisterThrows; + int _calls{0}; + std::shared_ptr _deregisterCalls{std::make_shared(0)}; + std::vector _registered; +}; + +} // namespace + +TEST_CASE("morph::bridge::Bridge::switchBackend - rollback on partial failure leaves old backend active", + "[bridge][switch][rollback]") { + morph::exec::ThreadPoolExecutor pool{2}; + SyncExec cbExec; + morph::bridge::Bridge bridge{std::make_unique(pool)}; + morph::bridge::BridgeHandler handler1{bridge, &cbExec}; + morph::bridge::BridgeHandler handler2{bridge, &cbExec}; + + auto const idBefore1 = handler1.binding()->currentId.load(); + auto const idBefore2 = handler2.binding()->currentId.load(); + + morph::exec::ThreadPoolExecutor pool2{2}; + // Two live handlers means two registerModelWithContext calls; fail on the 2nd + // so the 1st is already staged when the rollback runs. + auto flaky = std::make_unique(pool2, /*failOnCallNumber=*/2); + auto deregisterCalls = flaky->deregisterCallCounter(); // outlives the backend + + REQUIRE_THROWS_AS(bridge.switchBackend(std::move(flaky)), std::runtime_error); + REQUIRE(*deregisterCalls == 1); // the 1 staged registration was rolled back + + // currentId values are untouched — the switch is a no-op on failure. + REQUIRE(handler1.binding()->currentId.load() == idBefore1); + REQUIRE(handler2.binding()->currentId.load() == idBefore2); + + // The old backend is still active and functional. + std::atomic res{-1}; + handler1.execute(CountAction{3}).then([&](int val) { res.store(val); }).onError([](const std::exception_ptr&) {}); + REQUIRE(morph::testing::waitUntil([&] { return res.load() != -1; })); + REQUIRE(res.load() == 3); +} + +TEST_CASE( + "morph::bridge::Bridge::switchBackend - rollback still rethrows original error when deregister also fails", + "[bridge][switch][rollback]") { + morph::exec::ThreadPoolExecutor pool{2}; + SyncExec cbExec; + morph::bridge::Bridge bridge{std::make_unique(pool)}; + morph::bridge::BridgeHandler handler1{bridge, &cbExec}; + morph::bridge::BridgeHandler handler2{bridge, &cbExec}; + + morph::exec::ThreadPoolExecutor pool2{2}; + auto flaky = std::make_unique(pool2, /*failOnCallNumber=*/2, /*deregisterThrows=*/true); + auto deregisterCalls = flaky->deregisterCallCounter(); // outlives the backend + + REQUIRE_THROWS_AS(bridge.switchBackend(std::move(flaky)), std::runtime_error); + REQUIRE(*deregisterCalls == 1); // rollback attempted despite itself throwing +} + +// ── BridgeHandler destructor: bridge destroyed first (dead-liveness-token branch) ───────────── + +TEST_CASE("morph::bridge::BridgeHandler destructor is a no-op when the bridge is already destroyed", + "[bridge][lifetime]") { + morph::exec::ThreadPoolExecutor pool{2}; + SyncExec cbExec; + std::unique_ptr> handler; + { + auto bridge = std::make_unique(std::make_unique(pool)); + handler = std::make_unique>(*bridge, &cbExec); + // bridge destroyed here while handler still lives — its liveness token expires. + } + REQUIRE_NOTHROW(handler.reset()); // handler dtor must not dereference the dangling Bridge& +}