v1 Mantis: the sequencer, and events read by their content

docs/api/pylabrobot.formulatrix.rst

@@ -24,6 +24,7 @@ Mantis
    driver.features.rfid.Rfid
    driver.features.led_bar.LedBar
    driver.configuration.DeviceConfiguration
+   driver.events
    driver.errors.MantisError
    driver.errors.FrameError
    driver.errors.ChecksumError

docs/user_guide/formulatrix/mantis/hello-world.ipynb

@@ -41,6 +41,7 @@
     "| Generations | v3 and v4, told apart at setup |\n",
     "| Reads | firmware versions, device name, motor limits and state, the arm's absolute encoders (v4), line pressures and the pressure board's state |\n",
     "| Interface | USB, an FTDI bridge (VID 0x0403 / PID 0x6010): interface B for commands, interface A for the RFID reader |\n",
+    "| Sequencer | queues PPI writes and waits until each is played; internal, for the features that act |\n",
     "| Plate clamp | lock at one of the unit's levels, waiting for clamping done; unlock |\n",
     "| RFID | the chip in each input slot: serial, chip type, date of manufacture |\n",
     "| LED bar (v4) | one colour, one per LED, or an animation; shows setup, off at stop |\n",
@@ -55,7 +56,7 @@
    "source": [
     "## How it talks\n",
     "\n",
-    "The Mantis connects over USB through a two-channel FTDI bridge. Interface B carries binary frames to the device's boards: 0 for motion, 10 for pressure, 11 for the absolute position sensors (v4) and 12 for the LC3 board. The driver sends one command at a time and logs every event the device sends while connected. Interface A carries the RFID reader's commands.\n",
+    "The Mantis connects over USB through a two-channel FTDI bridge. Interface B carries binary frames to the device's boards: 0 for motion, 10 for pressure, 11 for the absolute position sensors (v4) and 12 for the LC3 board. The driver sends one command at a time and logs every event the device sends while connected, by what it carries. Interface A carries the RFID reader's commands.\n",
     "\n",
     "At setup the driver reads each board's firmware version. A device whose board 11 answers is a v4; one whose board 11 does not is taken to be a v3."
    ]
@@ -144,7 +145,7 @@
    "source": [
     "### Simulation\n",
     "\n",
-    "`Mantis(simulation=True)` builds a Mantis with nothing behind it; nothing reaches a wire. It takes its configuration, lock levels and RFID antennas from a Mantis v4's recording (`RECORDING_MANTIS_V4`, or a file given as `declared_configuration_json`), its reads answer from it, its plate clamp's state changes at once, every declared RFID slot reads empty, and its LED bar takes every command. Its motors read idle at 0; its pump reads off, every line at 0, with the recorded sensor calibration. The device name is not simulated."
+    "`Mantis(simulation=True)` builds a Mantis with nothing behind it; nothing reaches a wire. It takes its configuration, lock levels and RFID antennas from a Mantis v4's recording (`RECORDING_MANTIS_V4`, or a file given as `declared_configuration_json`), its reads answer from it, its plate clamp's state changes at once, every declared RFID slot reads empty, and its LED bar takes every command. Its motors read idle at 0; its pump reads off, every line at 0, with the recorded sensor calibration. Its sequencer, once started, plays each queued item at once. The device name is not simulated."
    ]
   },
   {
@@ -706,6 +707,41 @@
     "await mantis.led_bar.turn_off()"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "mantis-sequencer-00",
+   "metadata": {},
+   "source": [
+    "## Events and the sequencer\n",
+    "\n",
+    "The device reports events unasked: a motor starting or ending a move, a motor error, the sequencer playing a queued item, an input changing. The driver logs each by what it carries. A motor error fails at once every wait for a motion event, with `MantisMotionError`.\n",
+    "\n",
+    "The sequencer on board 0 plays queued items in order, each held for its duration in the sequencer's time steps, 1 ms on the v4. It is internal: the features that act queue through it. Here it holds the valve bank (PPI address 40) at 31, every valve closed, for about half a second: clear and start the sequencer, queue the write, then wait until the device reports it played. A write the device answers as not queued, its queue full, is sent again under a new item for at most `default_queue_full_timeout` seconds; no write is resent otherwise."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "mantis-sequencer-01",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "await mantis.driver._unchecked_fw_clear_sequencer()\n",
+    "await mantis.driver._unchecked_fw_start_sequencer()\n",
+    "item = await mantis.driver._queue_ppi_write(duration=500, start_address=40, values=[31])\n",
+    "await mantis.driver._wait_for_queue_item(item=item, timeout=10)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "mantis-sequencer-02",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "mantis.driver.default_queue_full_timeout"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "mantis-20",

pylabrobot/formulatrix/mantis/design.md

@@ -10,8 +10,10 @@ last section lists what is open and where the code does not follow a rule yet.
 device.py                    MantisDevice, a Resource, and the Mantis() factory. setup() / stop(),
                              the driver, the parts.
 driver/master.py             MantisDriver. Owns the link, the reader, reply matching, events,
-                             discovery and the saved configuration; reads the motors; shows
-                             setup and stop on the LED bar.
+                             the sequencer, discovery and the saved configuration; reads the
+                             motors; shows setup and stop on the LED bar.
+driver/events.py             The events the device sends unasked: their codes, what each carries
+                             and how it is logged.
 driver/simulator.py          MantisSimulationDriver, MantisSimulationPlateClamp,
                              MantisSimulationPump, MantisSimulationRfid, MantisSimulationLedBar.
                              Answer from the declared configuration; nothing reaches a wire.
@@ -103,9 +105,13 @@ which follows the software's chip catalogue, not the unit. A dispense takes one
 explicit arguments win. No lookup by viscosity: the bands leave gaps (5.5 cP is in none).
 
 **M5. One command in flight; replies match on packet number and board.** The reader runs from
-setup to stop. A frame with a code from 0x0200 is an event: it goes to every waiter registered for
-its board and code (`_expect_event`, before the command that causes it is sent, then
-`_wait_for_event`), and is logged either way. A reply nobody is waiting for is logged and dropped.
+setup to stop. A frame with a code from 0x0200 is an event: it is logged by its content
+(`describe_event`; a code not decoded, such as 0x7FFF, by its number) and goes to every waiter
+registered for its board and code whose `match` accepts its payload (`_expect_event`, before the
+command that causes it is sent, then `_wait_for_event`). A motor-error event fails every waiter for
+a motion event (`MOTION_EVENTS`) at once with `MantisMotionError`. An event whose payload does not
+match its layout is dropped before any waiter sees it. A reply nobody is waiting for is logged and
+dropped.
 A size field no frame can have (`MAXIMUM_FRAME_SIZE`) drops one byte at a time until a frame
 starts. A read that times out is resent with the same packet number, up to `attempts`. A command
 that writes or moves is sent once (`attempts=1`), never resent. A failed read ends the reader: the
@@ -114,6 +120,24 @@ already waiting for the lock, raises until setup runs again. `stop` fails the co
 every event waiter. After a failed read, `stop` (and `setup`, which runs it first) sends nothing:
 it closes the dead link with one warning.
 
+**M5a. The sequencer plays queued items; a queue write is checked by its reply.** Clear (61),
+start (62), stop (63) and abort (64) are `_unchecked_fw_*` calls, sent once, returning the reply's
+word. `_queue_ppi_write` numbers each item from 1 to `LAST_QUEUE_ITEM` per opening of the link and
+sends it once (`_unchecked_fw_queue_write_ppi`, command 65). A reply of -1 free slots means the
+queue was full and the item not queued: it is sent again under the next item, for at most
+`default_queue_full_timeout`; a non-zero error word raises `InvalidQueueItemError`.
+`_wait_for_queue_item` waits for the item's progress event with nothing left queued; the reader
+keeps the last such item, so a report that came with the reply is not missed.
+As seen on a v4: a duration is in steps of 1 ms. A sequencer not started takes 200 items
+(`SEQUENCER_QUEUE_SIZE`), then answers -1; a clear frees the slots. A frame written twice under
+one packet number is queued twice, so a queue write is never resent. A clear empties the queue
+but does not stop a running sequencer. Stop lets the queue play out, then reports it stopped;
+abort finishes the item playing, reports it stopped and keeps the rest for the next start. Only a
+clear drops items. On an idle sequencer, stop and abort keep the queue and report nothing. Clear,
+stop and abort answer 2 when sent while the sequencer runs, 0 when it is idle.
+Every progress event from board 0 sets `_sequence_progress`, the item played last, which the LED
+bar's dispense progress follows; opening the link clears it.
+
 **M6. The wire speaks the device's units; reads return them.** (STAR P4.) Every read returns what
 the device sent, decoded but not converted, until the feature that owns it knows the conversion.
 A feature then converts in one place, on its configuration, and speaks PyLabRobot's units. No unit
@@ -175,7 +199,9 @@ above the wire: `MantisSimulationPlateClamp` reads 0 until a command sets clampi
 unclamping done; `MantisSimulationRfid` reads every slot empty; `MantisSimulationPump` reads off,
 every line at 0, with the recorded calibration; `MantisSimulationLedBar` takes every command and
 waits no hold. The motors read idle at 0, as after power-on; a motor read the configuration does not
-hold is not simulated.
+hold is not simulated. The sequencer, on the driver, holds `SEQUENCER_QUEUE_SIZE` items,
+stopped and empty after setup, and once started plays each item at once; stop and abort keep what
+is queued.
 
 **M16. Locking is by subsystem.** (STAR P11.) One lock serializes the link today. When features
 add commands that the device runs together, each subsystem gets its own lock, taken in declaration
@@ -258,3 +284,11 @@ on the recorded v4.
 22. **The dispense progress has no caller yet.** `_show_fill` (`show_progress`), `_follow_fill` and
     `_show_dispense_done` wait for the dispense; `_follow_fill` reads `_sequence_progress`, which
     the sequencer sets.
+23. **A clear's words 8 and 9 are not explained.** A clear answered 8 once, right after Z homed,
+    and 9 has been seen; 2 and 0 mean running and idle. The word is returned, not checked.
+24. **`InvalidQueueItemError` is untested.** No queue reply seen has a non-zero error word.
+25. **A motor error is not kept for a later wait.** One that comes after an item is queued but
+    before `_wait_for_queue_item` begins leaves that wait to time out; the sequencer stops, so the
+    item is never reported played.
+26. **A sequence-stopped event's second word is not confirmed.** It read 0 after a stop that let
+    the queue play out and 2 after an abort that left two items queued, as if it counts them.

pylabrobot/formulatrix/mantis/driver/events.py

@@ -0,0 +1,166 @@
+"""The events a Mantis sends unasked, and what each carries.
+
+An event is a device frame with a code from ``FIRST_EVENT_CODE`` to 0x7FFF; every one seen came
+from board 0. Its packet number echoes a recent command and matches nothing. Payloads are
+little-endian. A code not listed here, such as 0x7FFF with no payload, is logged by its number.
+"""
+
+import struct
+from typing import Callable, Dict, Tuple
+
+from .errors import MOTOR_ERROR_CODE_DESCRIPTIONS, MOTOR_STATUS_ERROR_BITS, FrameError
+
+# A motor started moving: the motor.
+MOTION_STARTED_EVENT = 0x0200
+# A motor's move ended: the motor, its status word (``MOTOR_STATUS_ERROR_BITS``) and three values,
+# of which only the first, where it stopped, has been seen other than 0.
+MOVE_DONE_EVENT = 0x0201
+# A motor's homing ended: the motor and two values.
+HOME_DONE_EVENT = 0x0202
+# A motor reported an error: the motor and the code (``MOTOR_ERROR_CODE_DESCRIPTIONS``).
+MOTOR_ERROR_EVENT = 0x0203
+# The sequencer played a queue item: the item, the items still queued and the free slots.
+SEQUENCE_PROGRESS_EVENT = 0x0310
+# The sequencer stopped: the last item and a second word, 0 after a stop and 2 after an abort that
+# left two items queued.
+SEQUENCE_STOPPED_EVENT = 0x0311
+# A digital input changed: the input and whether it is on.
+INPUT_CHANGED_EVENT = 0x0321
+
+# The events a motor error ends: a waiter for one fails with the error at once.
+MOTION_EVENTS = (MOTION_STARTED_EVENT, MOVE_DONE_EVENT, HOME_DONE_EVENT, SEQUENCE_PROGRESS_EVENT)
+
+# Each event's payload layout.
+EVENT_PAYLOAD_FORMATS: Dict[int, str] = {
+  MOTION_STARTED_EVENT: "<H",
+  MOVE_DONE_EVENT: "<hHddd",
+  HOME_DONE_EVENT: "<hdd",
+  MOTOR_ERROR_EVENT: "<HH",
+  SEQUENCE_PROGRESS_EVENT: "<hhh",
+  SEQUENCE_STOPPED_EVENT: "<hh",
+  INPUT_CHANGED_EVENT: "<HH",
+}
+
+
+def _unpack_event(*, code: int, payload: bytes) -> Tuple:
+  """Split an event's payload into its fields.
+
+  Raises:
+    FrameError: if the payload's size does not match the event's layout.
+  """
+  layout = EVENT_PAYLOAD_FORMATS[code]
+  if len(payload) != struct.calcsize(layout):
+    raise FrameError(
+      f"event 0x{code:04x} carries {struct.calcsize(layout)} bytes, got {payload.hex()}"
+    )
+  return struct.unpack(layout, payload)
+
+
+def decode_motion_started(payload: bytes) -> int:
+  """The motor a motion-started event names."""
+  (motor,) = _unpack_event(code=MOTION_STARTED_EVENT, payload=payload)
+  return int(motor)
+
+
+def decode_move_done(payload: bytes) -> Tuple[int, int, float, float, float]:
+  """A move-done event's motor, status word and three values, in the device's units."""
+  motor, status, first, second, third = _unpack_event(code=MOVE_DONE_EVENT, payload=payload)
+  return motor, status, first, second, third
+
+
+def decode_home_done(payload: bytes) -> Tuple[int, float, float]:
+  """A home-done event's motor and two values, in the device's units."""
+  motor, first, second = _unpack_event(code=HOME_DONE_EVENT, payload=payload)
+  return motor, first, second
+
+
+def decode_motor_error(payload: bytes) -> Tuple[int, int]:
+  """A motor-error event's motor and code."""
+  motor, code = _unpack_event(code=MOTOR_ERROR_EVENT, payload=payload)
+  return motor, code
+
+
+def decode_sequence_progress(payload: bytes) -> Tuple[int, int, int]:
+  """A sequence-progress event's item, the items still queued and the free slots."""
+  item, queued, free = _unpack_event(code=SEQUENCE_PROGRESS_EVENT, payload=payload)
+  return item, queued, free
+
+
+def decode_sequence_stopped(payload: bytes) -> Tuple[int, int]:
+  """A sequence-stopped event's last item and its second word."""
+  item, unprocessed = _unpack_event(code=SEQUENCE_STOPPED_EVENT, payload=payload)
+  return item, unprocessed
+
+
+def decode_input_changed(payload: bytes) -> Tuple[int, bool]:
+  """An input-changed event's input and whether it is on."""
+  input_number, on = _unpack_event(code=INPUT_CHANGED_EVENT, payload=payload)
+  return input_number, bool(on)
+
+
+def _describe_motion_started(payload: bytes) -> str:
+  """A motion-started event, in words."""
+  return f"motor {decode_motion_started(payload=payload)} started moving"
+
+
+def _describe_move_done(payload: bytes) -> str:
+  """A move-done event, in words."""
+  motor, status, first, second, third = decode_move_done(payload=payload)
+  faults = [meaning for bit, meaning in MOTOR_STATUS_ERROR_BITS.items() if status & bit]
+  fault_text = f" ({', '.join(faults)})" if faults else ""
+  return f"motor {motor} moved, status 0x{status:04x}{fault_text}: {first}, {second}, {third}"
+
+
+def _describe_home_done(payload: bytes) -> str:
+  """A home-done event, in words."""
+  motor, first, second = decode_home_done(payload=payload)
+  return f"motor {motor} homed: {first}, {second}"
+
+
+def _describe_motor_error(payload: bytes) -> str:
+  """A motor-error event, in words."""
+  motor, code = decode_motor_error(payload=payload)
+  meaning = MOTOR_ERROR_CODE_DESCRIPTIONS.get(code, "a code not seen before")
+  return f"motor {motor} reported error {code}: {meaning}"
+
+
+def _describe_sequence_progress(payload: bytes) -> str:
+  """A sequence-progress event, in words."""
+  item, queued, free = decode_sequence_progress(payload=payload)
+  return f"item {item} played; {queued} queued, {free} free"
+
+
+def _describe_sequence_stopped(payload: bytes) -> str:
+  """A sequence-stopped event, in words."""
+  item, unprocessed = decode_sequence_stopped(payload=payload)
+  return f"sequencer stopped at item {item}, second word {unprocessed}"
+
+
+def _describe_input_changed(payload: bytes) -> str:
+  """An input-changed event, in words."""
+  input_number, on = decode_input_changed(payload=payload)
+  return f"input {input_number} {'on' if on else 'off'}"
+
+
+# How each event is said in a log message.
+_EVENT_DESCRIPTIONS: Dict[int, Callable[..., str]] = {
+  MOTION_STARTED_EVENT: _describe_motion_started,
+  MOVE_DONE_EVENT: _describe_move_done,
+  HOME_DONE_EVENT: _describe_home_done,
+  MOTOR_ERROR_EVENT: _describe_motor_error,
+  SEQUENCE_PROGRESS_EVENT: _describe_sequence_progress,
+  SEQUENCE_STOPPED_EVENT: _describe_sequence_stopped,
+  INPUT_CHANGED_EVENT: _describe_input_changed,
+}
+
+
+def describe_event(*, code: int, payload: bytes) -> str:
+  """An event in words, for the log; a code not decoded here by its number and payload.
+
+  Raises:
+    FrameError: if a known event's payload does not match its layout.
+  """
+  describe = _EVENT_DESCRIPTIONS.get(code)
+  if describe is None:
+    return f"event 0x{code:04x}, not decoded: {payload.hex()}"
+  return describe(payload=payload)