diff --git a/docs/api/pylabrobot.formulatrix.rst b/docs/api/pylabrobot.formulatrix.rst index 51f608179db..544915f0a9b 100644 --- a/docs/api/pylabrobot.formulatrix.rst +++ b/docs/api/pylabrobot.formulatrix.rst @@ -24,6 +24,7 @@ Mantis driver.features.rfid.Rfid driver.features.led_bar.LedBar driver.configuration.DeviceConfiguration + driver.events driver.errors.MantisError driver.errors.FrameError driver.errors.ChecksumError diff --git a/docs/user_guide/formulatrix/mantis/hello-world.ipynb b/docs/user_guide/formulatrix/mantis/hello-world.ipynb index 4667bf4896d..050340130ce 100644 --- a/docs/user_guide/formulatrix/mantis/hello-world.ipynb +++ b/docs/user_guide/formulatrix/mantis/hello-world.ipynb @@ -41,6 +41,7 @@ "| Generations | v3 and v4, told apart at setup |\n", "| Reads | firmware versions, device name, motor limits and state, the arm's absolute encoders (v4), line pressures and the pressure board's state |\n", "| Interface | USB, an FTDI bridge (VID 0x0403 / PID 0x6010): interface B for commands, interface A for the RFID reader |\n", + "| Sequencer | queues PPI writes and waits until each is played; internal, for the features that act |\n", "| Plate clamp | lock at one of the unit's levels, waiting for clamping done; unlock |\n", "| RFID | the chip in each input slot: serial, chip type, date of manufacture |\n", "| LED bar (v4) | one colour, one per LED, or an animation; shows setup, off at stop |\n", @@ -55,7 +56,7 @@ "source": [ "## How it talks\n", "\n", - "The Mantis connects over USB through a two-channel FTDI bridge. Interface B carries binary frames to the device's boards: 0 for motion, 10 for pressure, 11 for the absolute position sensors (v4) and 12 for the LC3 board. The driver sends one command at a time and logs every event the device sends while connected. Interface A carries the RFID reader's commands.\n", + "The Mantis connects over USB through a two-channel FTDI bridge. Interface B carries binary frames to the device's boards: 0 for motion, 10 for pressure, 11 for the absolute position sensors (v4) and 12 for the LC3 board. The driver sends one command at a time and logs every event the device sends while connected, by what it carries. Interface A carries the RFID reader's commands.\n", "\n", "At setup the driver reads each board's firmware version. A device whose board 11 answers is a v4; one whose board 11 does not is taken to be a v3." ] @@ -144,7 +145,7 @@ "source": [ "### Simulation\n", "\n", - "`Mantis(simulation=True)` builds a Mantis with nothing behind it; nothing reaches a wire. It takes its configuration, lock levels and RFID antennas from a Mantis v4's recording (`RECORDING_MANTIS_V4`, or a file given as `declared_configuration_json`), its reads answer from it, its plate clamp's state changes at once, every declared RFID slot reads empty, and its LED bar takes every command. Its motors read idle at 0; its pump reads off, every line at 0, with the recorded sensor calibration. The device name is not simulated." + "`Mantis(simulation=True)` builds a Mantis with nothing behind it; nothing reaches a wire. It takes its configuration, lock levels and RFID antennas from a Mantis v4's recording (`RECORDING_MANTIS_V4`, or a file given as `declared_configuration_json`), its reads answer from it, its plate clamp's state changes at once, every declared RFID slot reads empty, and its LED bar takes every command. Its motors read idle at 0; its pump reads off, every line at 0, with the recorded sensor calibration. Its sequencer, once started, plays each queued item at once. The device name is not simulated." ] }, { @@ -706,6 +707,41 @@ "await mantis.led_bar.turn_off()" ] }, + { + "cell_type": "markdown", + "id": "mantis-sequencer-00", + "metadata": {}, + "source": [ + "## Events and the sequencer\n", + "\n", + "The device reports events unasked: a motor starting or ending a move, a motor error, the sequencer playing a queued item, an input changing. The driver logs each by what it carries. A motor error fails at once every wait for a motion event, with `MantisMotionError`.\n", + "\n", + "The sequencer on board 0 plays queued items in order, each held for its duration in the sequencer's time steps, 1 ms on the v4. It is internal: the features that act queue through it. Here it holds the valve bank (PPI address 40) at 31, every valve closed, for about half a second: clear and start the sequencer, queue the write, then wait until the device reports it played. A write the device answers as not queued, its queue full, is sent again under a new item for at most `default_queue_full_timeout` seconds; no write is resent otherwise." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "mantis-sequencer-01", + "metadata": {}, + "outputs": [], + "source": [ + "await mantis.driver._unchecked_fw_clear_sequencer()\n", + "await mantis.driver._unchecked_fw_start_sequencer()\n", + "item = await mantis.driver._queue_ppi_write(duration=500, start_address=40, values=[31])\n", + "await mantis.driver._wait_for_queue_item(item=item, timeout=10)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "mantis-sequencer-02", + "metadata": {}, + "outputs": [], + "source": [ + "mantis.driver.default_queue_full_timeout" + ] + }, { "cell_type": "markdown", "id": "mantis-20", diff --git a/pylabrobot/formulatrix/mantis/design.md b/pylabrobot/formulatrix/mantis/design.md index 619ed12bb49..4554c42545b 100644 --- a/pylabrobot/formulatrix/mantis/design.md +++ b/pylabrobot/formulatrix/mantis/design.md @@ -10,8 +10,10 @@ last section lists what is open and where the code does not follow a rule yet. device.py MantisDevice, a Resource, and the Mantis() factory. setup() / stop(), the driver, the parts. driver/master.py MantisDriver. Owns the link, the reader, reply matching, events, - discovery and the saved configuration; reads the motors; shows - setup and stop on the LED bar. + the sequencer, discovery and the saved configuration; reads the + motors; shows setup and stop on the LED bar. +driver/events.py The events the device sends unasked: their codes, what each carries + and how it is logged. driver/simulator.py MantisSimulationDriver, MantisSimulationPlateClamp, MantisSimulationPump, MantisSimulationRfid, MantisSimulationLedBar. Answer from the declared configuration; nothing reaches a wire. @@ -103,9 +105,13 @@ which follows the software's chip catalogue, not the unit. A dispense takes one explicit arguments win. No lookup by viscosity: the bands leave gaps (5.5 cP is in none). **M5. One command in flight; replies match on packet number and board.** The reader runs from -setup to stop. A frame with a code from 0x0200 is an event: it goes to every waiter registered for -its board and code (`_expect_event`, before the command that causes it is sent, then -`_wait_for_event`), and is logged either way. A reply nobody is waiting for is logged and dropped. +setup to stop. A frame with a code from 0x0200 is an event: it is logged by its content +(`describe_event`; a code not decoded, such as 0x7FFF, by its number) and goes to every waiter +registered for its board and code whose `match` accepts its payload (`_expect_event`, before the +command that causes it is sent, then `_wait_for_event`). A motor-error event fails every waiter for +a motion event (`MOTION_EVENTS`) at once with `MantisMotionError`. An event whose payload does not +match its layout is dropped before any waiter sees it. A reply nobody is waiting for is logged and +dropped. A size field no frame can have (`MAXIMUM_FRAME_SIZE`) drops one byte at a time until a frame starts. A read that times out is resent with the same packet number, up to `attempts`. A command that writes or moves is sent once (`attempts=1`), never resent. A failed read ends the reader: the @@ -114,6 +120,24 @@ already waiting for the lock, raises until setup runs again. `stop` fails the co every event waiter. After a failed read, `stop` (and `setup`, which runs it first) sends nothing: it closes the dead link with one warning. +**M5a. The sequencer plays queued items; a queue write is checked by its reply.** Clear (61), +start (62), stop (63) and abort (64) are `_unchecked_fw_*` calls, sent once, returning the reply's +word. `_queue_ppi_write` numbers each item from 1 to `LAST_QUEUE_ITEM` per opening of the link and +sends it once (`_unchecked_fw_queue_write_ppi`, command 65). A reply of -1 free slots means the +queue was full and the item not queued: it is sent again under the next item, for at most +`default_queue_full_timeout`; a non-zero error word raises `InvalidQueueItemError`. +`_wait_for_queue_item` waits for the item's progress event with nothing left queued; the reader +keeps the last such item, so a report that came with the reply is not missed. +As seen on a v4: a duration is in steps of 1 ms. A sequencer not started takes 200 items +(`SEQUENCER_QUEUE_SIZE`), then answers -1; a clear frees the slots. A frame written twice under +one packet number is queued twice, so a queue write is never resent. A clear empties the queue +but does not stop a running sequencer. Stop lets the queue play out, then reports it stopped; +abort finishes the item playing, reports it stopped and keeps the rest for the next start. Only a +clear drops items. On an idle sequencer, stop and abort keep the queue and report nothing. Clear, +stop and abort answer 2 when sent while the sequencer runs, 0 when it is idle. +Every progress event from board 0 sets `_sequence_progress`, the item played last, which the LED +bar's dispense progress follows; opening the link clears it. + **M6. The wire speaks the device's units; reads return them.** (STAR P4.) Every read returns what the device sent, decoded but not converted, until the feature that owns it knows the conversion. A feature then converts in one place, on its configuration, and speaks PyLabRobot's units. No unit @@ -175,7 +199,9 @@ above the wire: `MantisSimulationPlateClamp` reads 0 until a command sets clampi unclamping done; `MantisSimulationRfid` reads every slot empty; `MantisSimulationPump` reads off, every line at 0, with the recorded calibration; `MantisSimulationLedBar` takes every command and waits no hold. The motors read idle at 0, as after power-on; a motor read the configuration does not -hold is not simulated. +hold is not simulated. The sequencer, on the driver, holds `SEQUENCER_QUEUE_SIZE` items, +stopped and empty after setup, and once started plays each item at once; stop and abort keep what +is queued. **M16. Locking is by subsystem.** (STAR P11.) One lock serializes the link today. When features add commands that the device runs together, each subsystem gets its own lock, taken in declaration @@ -258,3 +284,11 @@ on the recorded v4. 22. **The dispense progress has no caller yet.** `_show_fill` (`show_progress`), `_follow_fill` and `_show_dispense_done` wait for the dispense; `_follow_fill` reads `_sequence_progress`, which the sequencer sets. +23. **A clear's words 8 and 9 are not explained.** A clear answered 8 once, right after Z homed, + and 9 has been seen; 2 and 0 mean running and idle. The word is returned, not checked. +24. **`InvalidQueueItemError` is untested.** No queue reply seen has a non-zero error word. +25. **A motor error is not kept for a later wait.** One that comes after an item is queued but + before `_wait_for_queue_item` begins leaves that wait to time out; the sequencer stops, so the + item is never reported played. +26. **A sequence-stopped event's second word is not confirmed.** It read 0 after a stop that let + the queue play out and 2 after an abort that left two items queued, as if it counts them. diff --git a/pylabrobot/formulatrix/mantis/driver/events.py b/pylabrobot/formulatrix/mantis/driver/events.py new file mode 100644 index 00000000000..35c05d51e9e --- /dev/null +++ b/pylabrobot/formulatrix/mantis/driver/events.py @@ -0,0 +1,166 @@ +"""The events a Mantis sends unasked, and what each carries. + +An event is a device frame with a code from ``FIRST_EVENT_CODE`` to 0x7FFF; every one seen came +from board 0. Its packet number echoes a recent command and matches nothing. Payloads are +little-endian. A code not listed here, such as 0x7FFF with no payload, is logged by its number. +""" + +import struct +from typing import Callable, Dict, Tuple + +from .errors import MOTOR_ERROR_CODE_DESCRIPTIONS, MOTOR_STATUS_ERROR_BITS, FrameError + +# A motor started moving: the motor. +MOTION_STARTED_EVENT = 0x0200 +# A motor's move ended: the motor, its status word (``MOTOR_STATUS_ERROR_BITS``) and three values, +# of which only the first, where it stopped, has been seen other than 0. +MOVE_DONE_EVENT = 0x0201 +# A motor's homing ended: the motor and two values. +HOME_DONE_EVENT = 0x0202 +# A motor reported an error: the motor and the code (``MOTOR_ERROR_CODE_DESCRIPTIONS``). +MOTOR_ERROR_EVENT = 0x0203 +# The sequencer played a queue item: the item, the items still queued and the free slots. +SEQUENCE_PROGRESS_EVENT = 0x0310 +# The sequencer stopped: the last item and a second word, 0 after a stop and 2 after an abort that +# left two items queued. +SEQUENCE_STOPPED_EVENT = 0x0311 +# A digital input changed: the input and whether it is on. +INPUT_CHANGED_EVENT = 0x0321 + +# The events a motor error ends: a waiter for one fails with the error at once. +MOTION_EVENTS = (MOTION_STARTED_EVENT, MOVE_DONE_EVENT, HOME_DONE_EVENT, SEQUENCE_PROGRESS_EVENT) + +# Each event's payload layout. +EVENT_PAYLOAD_FORMATS: Dict[int, str] = { + MOTION_STARTED_EVENT: " Tuple: + """Split an event's payload into its fields. + + Raises: + FrameError: if the payload's size does not match the event's layout. + """ + layout = EVENT_PAYLOAD_FORMATS[code] + if len(payload) != struct.calcsize(layout): + raise FrameError( + f"event 0x{code:04x} carries {struct.calcsize(layout)} bytes, got {payload.hex()}" + ) + return struct.unpack(layout, payload) + + +def decode_motion_started(payload: bytes) -> int: + """The motor a motion-started event names.""" + (motor,) = _unpack_event(code=MOTION_STARTED_EVENT, payload=payload) + return int(motor) + + +def decode_move_done(payload: bytes) -> Tuple[int, int, float, float, float]: + """A move-done event's motor, status word and three values, in the device's units.""" + motor, status, first, second, third = _unpack_event(code=MOVE_DONE_EVENT, payload=payload) + return motor, status, first, second, third + + +def decode_home_done(payload: bytes) -> Tuple[int, float, float]: + """A home-done event's motor and two values, in the device's units.""" + motor, first, second = _unpack_event(code=HOME_DONE_EVENT, payload=payload) + return motor, first, second + + +def decode_motor_error(payload: bytes) -> Tuple[int, int]: + """A motor-error event's motor and code.""" + motor, code = _unpack_event(code=MOTOR_ERROR_EVENT, payload=payload) + return motor, code + + +def decode_sequence_progress(payload: bytes) -> Tuple[int, int, int]: + """A sequence-progress event's item, the items still queued and the free slots.""" + item, queued, free = _unpack_event(code=SEQUENCE_PROGRESS_EVENT, payload=payload) + return item, queued, free + + +def decode_sequence_stopped(payload: bytes) -> Tuple[int, int]: + """A sequence-stopped event's last item and its second word.""" + item, unprocessed = _unpack_event(code=SEQUENCE_STOPPED_EVENT, payload=payload) + return item, unprocessed + + +def decode_input_changed(payload: bytes) -> Tuple[int, bool]: + """An input-changed event's input and whether it is on.""" + input_number, on = _unpack_event(code=INPUT_CHANGED_EVENT, payload=payload) + return input_number, bool(on) + + +def _describe_motion_started(payload: bytes) -> str: + """A motion-started event, in words.""" + return f"motor {decode_motion_started(payload=payload)} started moving" + + +def _describe_move_done(payload: bytes) -> str: + """A move-done event, in words.""" + motor, status, first, second, third = decode_move_done(payload=payload) + faults = [meaning for bit, meaning in MOTOR_STATUS_ERROR_BITS.items() if status & bit] + fault_text = f" ({', '.join(faults)})" if faults else "" + return f"motor {motor} moved, status 0x{status:04x}{fault_text}: {first}, {second}, {third}" + + +def _describe_home_done(payload: bytes) -> str: + """A home-done event, in words.""" + motor, first, second = decode_home_done(payload=payload) + return f"motor {motor} homed: {first}, {second}" + + +def _describe_motor_error(payload: bytes) -> str: + """A motor-error event, in words.""" + motor, code = decode_motor_error(payload=payload) + meaning = MOTOR_ERROR_CODE_DESCRIPTIONS.get(code, "a code not seen before") + return f"motor {motor} reported error {code}: {meaning}" + + +def _describe_sequence_progress(payload: bytes) -> str: + """A sequence-progress event, in words.""" + item, queued, free = decode_sequence_progress(payload=payload) + return f"item {item} played; {queued} queued, {free} free" + + +def _describe_sequence_stopped(payload: bytes) -> str: + """A sequence-stopped event, in words.""" + item, unprocessed = decode_sequence_stopped(payload=payload) + return f"sequencer stopped at item {item}, second word {unprocessed}" + + +def _describe_input_changed(payload: bytes) -> str: + """An input-changed event, in words.""" + input_number, on = decode_input_changed(payload=payload) + return f"input {input_number} {'on' if on else 'off'}" + + +# How each event is said in a log message. +_EVENT_DESCRIPTIONS: Dict[int, Callable[..., str]] = { + MOTION_STARTED_EVENT: _describe_motion_started, + MOVE_DONE_EVENT: _describe_move_done, + HOME_DONE_EVENT: _describe_home_done, + MOTOR_ERROR_EVENT: _describe_motor_error, + SEQUENCE_PROGRESS_EVENT: _describe_sequence_progress, + SEQUENCE_STOPPED_EVENT: _describe_sequence_stopped, + INPUT_CHANGED_EVENT: _describe_input_changed, +} + + +def describe_event(*, code: int, payload: bytes) -> str: + """An event in words, for the log; a code not decoded here by its number and payload. + + Raises: + FrameError: if a known event's payload does not match its layout. + """ + describe = _EVENT_DESCRIPTIONS.get(code) + if describe is None: + return f"event 0x{code:04x}, not decoded: {payload.hex()}" + return describe(payload=payload) diff --git a/pylabrobot/formulatrix/mantis/driver/events_tests.py b/pylabrobot/formulatrix/mantis/driver/events_tests.py new file mode 100644 index 00000000000..092f94352af --- /dev/null +++ b/pylabrobot/formulatrix/mantis/driver/events_tests.py @@ -0,0 +1,84 @@ +"""Event decoding tests, on payloads recorded from a Mantis v4.""" + +import unittest + +from pylabrobot.formulatrix.mantis.driver.errors import FrameError +from pylabrobot.formulatrix.mantis.driver.events import ( + HOME_DONE_EVENT, + INPUT_CHANGED_EVENT, + MOTION_STARTED_EVENT, + MOTOR_ERROR_EVENT, + MOVE_DONE_EVENT, + SEQUENCE_PROGRESS_EVENT, + SEQUENCE_STOPPED_EVENT, + decode_home_done, + decode_input_changed, + decode_motion_started, + decode_motor_error, + decode_move_done, + decode_sequence_progress, + decode_sequence_stopped, + describe_event, +) + +# Event payloads a Mantis v4 sent, as hex. +RECORDED_MOTION_STARTED = "0200" +RECORDED_MOVE_DONE = "0200040000000000d0e74ec000000000000000000000000000000000" +# A move-done after a motor error, with status bit 0x0100 set. +RECORDED_MOVE_DONE_AFTER_ERROR = "00000001000000a08ac7574000000000000000000000000000000000" +RECORDED_HOME_DONE = "0200000000001148a2c0000000000078c0bf" +RECORDED_MOTOR_ERROR = "01000800" +RECORDED_SEQUENCE_PROGRESS = "03000200c600" +RECORDED_SEQUENCE_STOPPED = "19000000" +RECORDED_INPUT_CHANGED = "00000100" + + +class DecodeTests(unittest.TestCase): + def test_decodes(self): + cases = [ + (decode_motion_started, RECORDED_MOTION_STARTED, 2), + (decode_move_done, RECORDED_MOVE_DONE, (2, 4, -61.81103515625, 0.0, 0.0)), + (decode_move_done, RECORDED_MOVE_DONE_AFTER_ERROR, (0, 0x100, 95.11783599853516, 0.0, 0.0)), + (decode_home_done, RECORDED_HOME_DONE, (2, -2340.033203125, -0.128662109375)), + (decode_motor_error, RECORDED_MOTOR_ERROR, (1, 8)), + (decode_sequence_progress, RECORDED_SEQUENCE_PROGRESS, (3, 2, 198)), + (decode_sequence_stopped, RECORDED_SEQUENCE_STOPPED, (25, 0)), + (decode_input_changed, RECORDED_INPUT_CHANGED, (0, True)), + ] + for decode, payload, expected in cases: + with self.subTest(payload=payload): + self.assertEqual(decode(payload=bytes.fromhex(payload)), expected) + + def test_descriptions(self): + cases = [ + (MOTION_STARTED_EVENT, RECORDED_MOTION_STARTED, "motor 2 started moving"), + (MOVE_DONE_EVENT, RECORDED_MOVE_DONE, "motor 2 moved, status 0x0004: "), + ( + MOVE_DONE_EVENT, + RECORDED_MOVE_DONE_AFTER_ERROR, + "motor 0 moved, status 0x0100 (following error while moving)", + ), + (HOME_DONE_EVENT, RECORDED_HOME_DONE, "motor 2 homed"), + ( + MOTOR_ERROR_EVENT, + RECORDED_MOTOR_ERROR, + "motor 1 reported error 8: a skip was detected while the motor was idle", + ), + (SEQUENCE_PROGRESS_EVENT, RECORDED_SEQUENCE_PROGRESS, "item 3 played; 2 queued, 198 free"), + (SEQUENCE_STOPPED_EVENT, RECORDED_SEQUENCE_STOPPED, "sequencer stopped at item 25"), + (INPUT_CHANGED_EVENT, RECORDED_INPUT_CHANGED, "input 0 on"), + (0x7FFF, "", "event 0x7fff, not decoded: "), + ] + for code, payload, expected in cases: + with self.subTest(code=code, payload=payload): + self.assertIn(expected, describe_event(code=code, payload=bytes.fromhex(payload))) + + def test_payload_of_the_wrong_size_raises(self): + with self.assertRaises(FrameError): + decode_sequence_progress(payload=bytes.fromhex(RECORDED_SEQUENCE_PROGRESS)[:4]) + with self.assertRaises(FrameError): + describe_event(code=MOTOR_ERROR_EVENT, payload=bytes.fromhex(RECORDED_MOTOR_ERROR)[:2]) + + +if __name__ == "__main__": + unittest.main() diff --git a/pylabrobot/formulatrix/mantis/driver/master.py b/pylabrobot/formulatrix/mantis/driver/master.py index db9047c4ab9..df7baf84393 100644 --- a/pylabrobot/formulatrix/mantis/driver/master.py +++ b/pylabrobot/formulatrix/mantis/driver/master.py @@ -1,7 +1,8 @@ -"""The Mantis driver: the link on the FTDI's interface B, discovery and the device reads. +"""The Mantis driver: the link on the FTDI's interface B, discovery, the device reads, events +and the sequencer. -Every command sent here reads the device; the features in ``features/`` send the commands that -act, and the LED bar shows setup and stop. Reads return the device's units. The motor reads are +The features in ``features/`` send the commands that act; the sequencer here plays what they +queue, and the LED bar shows setup and stop. Reads return the device's units. The motor reads are here until the arm feature takes them; the pressure reads are on the pump. """ @@ -10,7 +11,8 @@ import json import logging import struct -from typing import Any, AsyncContextManager, Dict, List, Optional, Set, Tuple +import time +from typing import Any, AsyncContextManager, Callable, Dict, List, Optional, Set, Tuple from pylabrobot.io.ftdi import FTDI @@ -29,11 +31,23 @@ from .errors import ( CommandRefusedError, FrameError, + InvalidQueueItemError, MantisConfigurationError, MantisConnectionLostError, + MantisMotionError, MantisNotConnectedError, MantisTimeoutError, ) +from .events import ( + EVENT_PAYLOAD_FORMATS, + MOTION_EVENTS, + MOTOR_ERROR_EVENT, + SEQUENCE_PROGRESS_EVENT, + SEQUENCE_STOPPED_EVENT, + decode_motor_error, + decode_sequence_progress, + describe_event, +) from .features.led_bar import LedBar from .features.plate_clamp import UNCLAMPING_DONE, PlateClamp from .features.pump import Pump @@ -59,11 +73,21 @@ # How often an absolute encoder is read for a valid position; each motor's first after connecting # is not. ABSOLUTE_ENCODER_READS = 5 +# The sequencer's queue holds 200 items: a v4 not started took 200 before answering -1, and every +# progress event's queued and free counts add to it. +SEQUENCER_QUEUE_SIZE = 200 +# Queue items are numbered from 1 to 0x7FFF, the largest int16, then from 1 again. +LAST_QUEUE_ITEM = 0x7FFF +# The PPI address every queued write seen has gone to: the valve bank. +QUEUED_PPI_ADDRESSES = (40,) class MantisDriver: """Talks to a Mantis: one command in flight, events logged as they come.""" + # Where `default_queue_full_timeout` starts. + _default_queue_full_timeout: float = 60.0 + def _build_io(self, *, device_id: Optional[str], ftdi_library: Optional[str]) -> FTDI: """Build the link to the Mantis ``device_id`` names, or to the first one found.""" return FTDI( @@ -134,7 +158,12 @@ def __init__( self._packet_number = 0 self._buffer = bytearray() self._pending: Optional[Tuple[int, int, "asyncio.Future[Tuple[int, bytes]]"]] = None - self._event_waiters: List[Tuple[int, int, "asyncio.Future[bytes]"]] = [] + # Each waiter: board, code, the match its payload must pass (None: any) and its future. + self._event_waiters: List[ + Tuple[int, int, Optional[Callable[[bytes], bool]], "asyncio.Future[bytes]"] + ] = [] + self._next_item = 1 # the next queue item, per opening of the link + self._last_drained_item: Optional[int] = None # the last item played with none left queued self._reader: Optional["asyncio.Task[None]"] = None self.rfid_io: Optional[FTDI] = ( rfid_io @@ -162,6 +191,17 @@ def default_read_timeout(self, value: float) -> None: raise ValueError(f"default_read_timeout must be positive, got {value}") self._default_read_timeout = value + @property + def default_queue_full_timeout(self) -> float: + """Seconds a queue write is sent again under a new item while the sequencer's queue is full.""" + return self._default_queue_full_timeout + + @default_queue_full_timeout.setter + def default_queue_full_timeout(self, value: float) -> None: + if value <= 0: + raise ValueError(f"default_queue_full_timeout must be positive, got {value}") + self._default_queue_full_timeout = value + # -- link ----------------------------------------------------------------- async def _set_up_line(self) -> None: @@ -176,15 +216,53 @@ async def _set_up_line(self) -> None: await self.io.usb_purge_tx_buffer() def _handle_event(self, *, board: int, code: int, payload: bytes) -> None: - """Log an event and hand it to every waiter for its board and code.""" - logger.info("[%s] event 0x%04x from board %d: %s", self.name, code, board, payload.hex()) - waiting = self._event_waiters - self._event_waiters = [] - for waiter_board, waiter_code, future in waiting: - if (waiter_board, waiter_code) != (board, code): - self._event_waiters.append((waiter_board, waiter_code, future)) - elif not future.done(): - future.set_result(payload) + """Log an event by its content and hand it to every waiter for its board and code whose match + accepts it. A motor error fails every waiter for a motion event (``MOTION_EVENTS``) at once. + + A motor error, a sequencer stop and a code not decoded log a warning; the rest log at debug. + + Raises: + FrameError: if a known event's payload does not match its layout; no waiter sees it. + """ + description = describe_event(code=code, payload=payload) + failure: Optional[Tuple[str, int, int]] = None + if code == MOTOR_ERROR_EVENT: + logger.warning("[%s] board %d: %s", self.name, board, description) + motor, error = decode_motor_error(payload=payload) + failure = (f"{self.name}: {description}", motor, error) + elif code == SEQUENCE_PROGRESS_EVENT: + logger.debug("[%s] board %d: %s", self.name, board, description) + item, queued, _ = decode_sequence_progress(payload=payload) + if board == MOTION_BOARD: + self._sequence_progress = item # the LED bar's dispense progress follows it + if queued == 0: + self._last_drained_item = item + elif code in EVENT_PAYLOAD_FORMATS and code != SEQUENCE_STOPPED_EVENT: + logger.debug("[%s] board %d: %s", self.name, board, description) + else: + logger.warning("[%s] board %d: %s", self.name, board, description) + waiting, self._event_waiters = self._event_waiters, [] + for waiter in waiting: + waiter_board, waiter_code, match, future = waiter + if future.done(): + continue + if waiter_board != board: + self._event_waiters.append(waiter) + elif failure is not None and waiter_code in MOTION_EVENTS: + message, motor, error = failure + future.set_exception(MantisMotionError(message, motor=motor, code=error)) + elif waiter_code != code: + self._event_waiters.append(waiter) + else: + try: + accepted = match is None or match(payload) + except Exception as match_error: + future.set_exception(match_error) + continue + if accepted: + future.set_result(payload) + else: + self._event_waiters.append(waiter) def _handle_frame(self, *, frame: bytes) -> None: """Route one device frame: an event by its board and code, a reply to the command it echoes.""" @@ -218,7 +296,7 @@ async def _read_frames(self) -> None: if not future.done(): future.set_exception(lost) waiting, self._event_waiters = self._event_waiters, [] - for _, _, waiter in waiting: + for _, _, _, waiter in waiting: if not waiter.done(): waiter.set_exception(lost) raise @@ -314,11 +392,19 @@ async def _send_command( raise CommandRefusedError(message, board=board, command=command, status=status) return reply - def _expect_event(self, *, board: int, code: int) -> "asyncio.Future[bytes]": - """Register a waiter for the next event with ``board`` and ``code``, before the command that - causes it is sent; pass it to ``_wait_for_event``.""" + def _expect_event( + self, *, board: int, code: int, match: Optional[Callable[[bytes], bool]] = None + ) -> "asyncio.Future[bytes]": + """Register a waiter for the next event with ``board`` and ``code`` that ``match`` accepts, + before the command that causes it is sent; pass it to ``_wait_for_event``. + + Args: + board: the board the event comes from. + code: the event's code. + match: takes the event's payload and returns whether it is the one awaited; None takes any. + """ future: "asyncio.Future[bytes]" = asyncio.get_running_loop().create_future() - self._event_waiters.append((board, code, future)) + self._event_waiters.append((board, code, match, future)) return future async def _wait_for_event(self, *, waiter: "asyncio.Future[bytes]", timeout: float) -> bytes: @@ -326,6 +412,7 @@ async def _wait_for_event(self, *, waiter: "asyncio.Future[bytes]", timeout: flo Raises: MantisTimeoutError: if the event does not come within ``timeout`` seconds. + MantisMotionError: at once, if a motor reports an error while it waits for a motion event. MantisNotConnectedError: if the driver stops first. MantisConnectionLostError: if reading the link fails first. """ @@ -334,9 +421,7 @@ async def _wait_for_event(self, *, waiter: "asyncio.Future[bytes]", timeout: flo except asyncio.TimeoutError as error: raise MantisTimeoutError(f"{self.name}: no event within {timeout} s") from error finally: - self._event_waiters = [ - (board, code, future) for board, code, future in self._event_waiters if future is not waiter - ] + self._event_waiters = [entry for entry in self._event_waiters if entry[3] is not waiter] # -- device reads --------------------------------------------------------- @@ -502,6 +587,157 @@ async def request_absolute_encoder_position(self, *, motor: int) -> float: f"{ABSOLUTE_ENCODER_READS} reads" ) + # -- sequencer ------------------------------------------------------------ + + def _decode_sequencer_reply(self, *, command: int, reply: bytes) -> int: + """The int16 a sequencer command answers. + + Raises: + FrameError: if the reply is shorter than 2 bytes. + """ + if len(reply) < 2: + raise FrameError(f"{self.name}: command {command}'s reply is 2 bytes, got {reply.hex()}") + (word,) = struct.unpack_from(" int: + """Clear the sequencer's queue (board 0, command 61), sent once; return the reply's word. + + Only a clear drops queued items. A running sequencer keeps running: stop it first to halt it. + As seen on a v4, the word is 2 when sent while running and 0 when idle; 8 and 9, also seen, + are not explained. + """ + reply = await self._send_command(board=MOTION_BOARD, command=61, attempts=1) + return self._decode_sequencer_reply(command=61, reply=reply) + + async def _unchecked_fw_start_sequencer(self) -> int: + """Start the sequencer (board 0, command 62), sent once; return the reply's word, 0 as seen.""" + reply = await self._send_command(board=MOTION_BOARD, command=62, attempts=1) + return self._decode_sequencer_reply(command=62, reply=reply) + + async def _unchecked_fw_stop_sequencer(self) -> int: + """Stop the sequencer (board 0, command 63), sent once; return the reply's word. + + As seen on a v4: a running sequencer plays what is queued, then reports it stopped + (``SEQUENCE_STOPPED_EVENT``), and the word is 2. An idle one keeps its queue, reports nothing + and answers 0. + """ + reply = await self._send_command(board=MOTION_BOARD, command=63, attempts=1) + return self._decode_sequencer_reply(command=63, reply=reply) + + async def _unchecked_fw_abort_sequencer(self) -> int: + """Abort the sequencer (board 0, command 64), sent once; return the reply's word. + + As seen on a v4: a running sequencer finishes the item playing, reports it stopped + (``SEQUENCE_STOPPED_EVENT``) and keeps the items still queued, which play at the next start; + the word is 2. An idle one keeps its queue, reports nothing and answers 0. + """ + reply = await self._send_command(board=MOTION_BOARD, command=64, attempts=1) + return self._decode_sequencer_reply(command=64, reply=reply) + + async def _unchecked_fw_queue_write_ppi( + self, *, item: int, duration: int, start_address: int, values: List[int] + ) -> Tuple[int, int]: + """Queue a PPI write on the sequencer (board 0, command 65), sent once. + + Never resent: a v4 queued a frame written twice under one packet number as two items. + Returns the reply's free slots, -1 when the queue was full and the item not queued, and its + error word. + + Args: + item: the item's number, 1 to ``LAST_QUEUE_ITEM``. + duration: how long the sequencer holds the write, in its time steps of 1 ms. + start_address: the first PPI address written. + values: the values written from there. + + Raises: + FrameError: if the reply is shorter than 4 bytes. + """ + payload = struct.pack(" int: + """Number the next queue item: 1 to ``LAST_QUEUE_ITEM``, then 1 again.""" + item = self._next_item + self._next_item = item % LAST_QUEUE_ITEM + 1 + return item + + async def _queue_ppi_write(self, *, duration: int, start_address: int, values: List[int]) -> int: + """Queue a PPI write on the sequencer and return its item. + + An item the device answers -1 free slots to was not queued: the write is sent again under the + next item until one is, for at most ``default_queue_full_timeout``. No item is sent twice: a + v4 queues a write resent with its packet number as a second item, so a lost reply raises. + + Args: + duration: how long the sequencer holds the write, in its time steps of 1 ms, 1 to 0x7FFF. + start_address: the first PPI address written, one of ``QUEUED_PPI_ADDRESSES``. + values: the values written from there, each 0 to 0xFFFF; at least one. + + Raises: + ValueError: if an argument is out of range; nothing is sent. + InvalidQueueItemError: if a reply's error word is not 0. + MantisTimeoutError: if the queue stays full for ``default_queue_full_timeout``. + """ + self._check_index(name="start_address", value=start_address, allowed=QUEUED_PPI_ADDRESSES) + if not 1 <= duration <= 0x7FFF: + raise ValueError(f"duration must be 1 to 0x7FFF, an int16, got {duration}") + if len(values) == 0 or any(not 0 <= value <= 0xFFFF for value in values): + raise ValueError(f"values must be one or more uint16, got {values}") + deadline = time.monotonic() + self.default_queue_full_timeout + while True: + item = self._get_next_item() + free, error = await self._unchecked_fw_queue_write_ppi( + item=item, duration=duration, start_address=start_address, values=values + ) + if error != 0: + raise InvalidQueueItemError( + f"{self.name}: queue item {item} answered error {error}", command=65, error=error + ) + if free >= 0: + return item + if time.monotonic() >= deadline: + raise MantisTimeoutError( + f"{self.name}: the sequencer's queue stayed full for {self.default_queue_full_timeout} s" + ) + logger.debug("[%s] queue full, item %d not queued; sending it again", self.name, item) + + async def _wait_for_queue_item(self, *, item: int, timeout: float) -> None: + """Wait until the sequencer reports ``item`` played with no item left queued. + + Returns at once if it was reported already: a short item's report can come with the reply + that queued it, before the wait begins. + + Args: + item: the last item queued. + timeout: seconds to wait. + + The simulation answers at once: it returns or raises ``MantisTimeoutError``. + + Raises: + MantisMotionError: at once, if a motor reports an error first. + MantisTimeoutError: if it is not reported within ``timeout``. + """ + if self._last_drained_item == item: + return + waiter = self._expect_event( + board=MOTION_BOARD, + code=SEQUENCE_PROGRESS_EVENT, + match=lambda payload: decode_sequence_progress(payload=payload)[:2] == (item, 0), + ) + try: + await self._wait_for_event(waiter=waiter, timeout=timeout) + except MantisTimeoutError as error: + raise MantisTimeoutError( + f"{self.name}: queue item {item} not reported played within {timeout} s" + ) from error + # -- discovery and setup -------------------------------------------------- async def _request_configuration(self) -> DeviceConfiguration: @@ -749,6 +985,7 @@ async def _open(self) -> None: await self.io.setup() await self._set_up_line() self._buffer.clear() + self._next_item, self._last_drained_item, self._sequence_progress = 1, None, None self._reader = asyncio.create_task(self._read_frames()) logger.info("[%s] connected to FTDI %s", self.name, self.io.device_id) if self.rfid_io is not None: @@ -803,7 +1040,7 @@ async def stop(self) -> None: if not future.done(): future.set_exception(MantisNotConnectedError(stopped)) waiting, self._event_waiters = self._event_waiters, [] - for _, _, waiter in waiting: + for _, _, _, waiter in waiting: if not waiter.done(): waiter.set_exception(MantisNotConnectedError(stopped)) if self.plate_clamp is not None and self._connected: diff --git a/pylabrobot/formulatrix/mantis/driver/master_tests.py b/pylabrobot/formulatrix/mantis/driver/master_tests.py index 48d176bb490..afd2955ef84 100644 --- a/pylabrobot/formulatrix/mantis/driver/master_tests.py +++ b/pylabrobot/formulatrix/mantis/driver/master_tests.py @@ -10,13 +10,20 @@ from unittest import mock from pylabrobot.formulatrix.mantis.device import RECORDING_MANTIS_V4 +from pylabrobot.formulatrix.mantis.driver import master as master_module from pylabrobot.formulatrix.mantis.driver.configuration import DeviceConfiguration from pylabrobot.formulatrix.mantis.driver.errors import ( MantisConfigurationError, MantisConnectionLostError, + MantisMotionError, MantisNotConnectedError, MantisTimeoutError, ) +from pylabrobot.formulatrix.mantis.driver.events import ( + MOVE_DONE_EVENT, + SEQUENCE_STOPPED_EVENT, + decode_sequence_stopped, +) from pylabrobot.formulatrix.mantis.driver.features.led_bar import LedBar from pylabrobot.formulatrix.mantis.driver.features.pump import PressureController from pylabrobot.formulatrix.mantis.driver.features.rfid import ( @@ -103,6 +110,49 @@ "chip": (0.010661732405424118, -13.671295166015625), "vacuum": (0.010320834815502167, -38.775638580322266), } +# The sequencer's replies, by (board, command, request payload), as hex: clear, start, stop and +# abort, and queue writes to address 40 of item 1 (a hold of 10) and items 3 to 5 (holds of 500). +RECORDED_SEQUENCER_REPLIES: Dict[Tuple[int, int, str], str] = { + (0, 61, ""): "0000", + (0, 62, ""): "0000", + (0, 63, ""): "0000", + (0, 64, ""): "0000", + (0, 65, "01000a00280001001f00"): "c7000000", + (0, 65, "0300f401280001000000"): "c7000000", + (0, 65, "0400f401280001000000"): "c7000000", + (0, 65, "0500f401280001000000"): "c6000000", +} +# A clear that answered 8. +RECORDED_CLEAR_REPLY_8 = "0800" +# Item 1 reported played, sent with the reply that queued it. +RECORDED_ITEM_1_PLAYED = "12000000100100000000100301000000c800db02" +# Items 3, 4 and 5 reported played, 0.5 s apart after their replies. +RECORDED_ITEMS_3_TO_5_PLAYED = [ + "120000002d0100000000100303000200c600e802", + "120000002d0100000000100304000100c700ed02", + "120000002d0100000000100305000000c800e202", +] +# Queue writes of one hold at address 40, items 368 to 373, and their replies: the queue was full +# until item 373. +RECORDED_QUEUE_FULL_REPLIES: Dict[str, str] = { + "70010100280001001b00": "ffff0000", + "71010100280001001b00": "ffff0000", + "72010100280001001b00": "ffff0000", + "73010100280001001b00": "ffff0000", + "74010100280001001b00": "ffff0000", + "75010100280001001b00": "01000000", +} +# The sequencer reported stopped, after a stop and after an abort. +RECORDED_STOPPED_AFTER_STOP = "100000002244000000001103363200001575" +RECORDED_STOPPED_AFTER_ABORT = "100000004f0d0000000011030c000000420e" +# Motor 0 reported error 9, the sequencer stopped, then the move ended. +RECORDED_MOTOR_ERROR_EVENTS = [ + "10000000c85f00000000030200000900d25d", + "10000000c85f000000001103d9310000106d", + "28000000c85f00000000010200000001000000a08ac75740000000000000000000000000000000003c7b", +] +# A frame with code 0x7FFF and no payload, sent instead of a reply. +RECORDED_CODE_7FFF_FRAME = "0c000000000000000000ff7ff37f" # Motor 0's absolute encoder after connecting: its first read not valid, the next valid. ABSOLUTE_ENCODER_REPLIES_AFTER_POWER_ON = ["00000000000000000000", "010000000000f43f6740"] # The link reader's failure when the USB cable was pulled from a Mantis v4, through D2XX. @@ -154,17 +204,19 @@ async def read(self, num_bytes: int = 1) -> bytes: class FakeLink: - """Answers each frame written with its recorded reply, echoing packet number and board. A list - of replies is played in order and its last reply repeats. An antenna switch selects - ``reader``'s antenna.""" + """Answers each frame written with its recorded reply, echoing packet number and board, then + with the event frames recorded after it, if any. A list of replies is played in order and its + last reply repeats. An antenna switch selects ``reader``'s antenna.""" def __init__( self, answers: Dict[Tuple[int, int, bytes], List[bytes]], *, reader: Optional[FakeReader] = None, + events: Optional[Dict[Tuple[int, int, bytes], List[bytes]]] = None, ): self.answers = answers + self.events = events or {} self.reader = reader or FakeReader() self.written: List[bytes] = [] self.incoming = bytearray() @@ -184,6 +236,8 @@ async def write(self, data: bytes) -> int: self.incoming.extend( build_frame(packet_number=packet_number, board=board, command=0, payload=reply) ) + for event in self.events.get((board, command, payload), []): + self.incoming.extend(event) return len(data) async def read(self, num_bytes: int = 1) -> bytes: @@ -496,5 +550,178 @@ async def test_absolute_encoder_never_valid_raises(self): self.assertEqual(len(self.link.written) - sent_before, ABSOLUTE_ENCODER_READS) +class SequencerTests(DriverTestCase): + async def asyncSetUp(self): + answers = make_answers() + for (board, command, request), reply in RECORDED_SEQUENCER_REPLIES.items(): + answers[(board, command, bytes.fromhex(request))] = [bytes.fromhex(reply)] + for request, reply in RECORDED_QUEUE_FULL_REPLIES.items(): + answers[(0, 65, bytes.fromhex(request))] = [bytes.fromhex(reply)] + self.link = FakeLink( + answers=answers, + events={ + (0, 65, bytes.fromhex("01000a00280001001f00")): [bytes.fromhex(RECORDED_ITEM_1_PLAYED)], + (0, 63, b""): [bytes.fromhex(RECORDED_STOPPED_AFTER_STOP)], + (0, 64, b""): [bytes.fromhex(RECORDED_STOPPED_AFTER_ABORT)], + }, + ) + self.driver = await self.set_up_driver(link=self.link) + + def sent_since(self, count: int) -> List[Tuple[int, int, bytes]]: + return [parse_frame(frame=frame)[1:] for frame in self.link.written[count:]] + + async def test_sequencer_commands_are_sent_once_and_return_their_reply(self): + driver = self.driver + cases = [ + (61, driver._unchecked_fw_clear_sequencer), + (62, driver._unchecked_fw_start_sequencer), + (63, driver._unchecked_fw_stop_sequencer), + (64, driver._unchecked_fw_abort_sequencer), + ] + for command, send in cases: + with self.subTest(command=command): + sent_before = len(self.link.written) + self.assertEqual(await send(), 0) + self.assertEqual(self.sent_since(sent_before), [(0, command, b"")]) + + async def test_clear_returns_the_word_it_answers(self): + self.link.answers[(0, 61, b"")] = [bytes.fromhex(RECORDED_CLEAR_REPLY_8)] + self.assertEqual(await self.driver._unchecked_fw_clear_sequencer(), 8) + + async def test_unanswered_start_is_not_resent(self): + del self.link.answers[(0, 62, b"")] + sent_before = len(self.link.written) + with self.assertRaises(MantisTimeoutError): + await self.driver._unchecked_fw_start_sequencer() + self.assertEqual(self.sent_since(sent_before), [(0, 62, b"")]) + + async def test_stop_and_abort_are_followed_by_the_stopped_event(self): + driver = self.driver + cases = [ + (driver._unchecked_fw_stop_sequencer, (0x3236, 0)), + (driver._unchecked_fw_abort_sequencer, (12, 0)), + ] + for send, expected in cases: + with self.subTest(expected=expected): + waiter = driver._expect_event(board=0, code=SEQUENCE_STOPPED_EVENT) + await send() + payload = await driver._wait_for_event(waiter=waiter, timeout=1.0) + self.assertEqual(decode_sequence_stopped(payload=payload), expected) + + async def test_item_played_with_its_reply_is_not_missed(self): + driver = self.driver + await driver._unchecked_fw_clear_sequencer() + await driver._unchecked_fw_start_sequencer() + sent_before = len(self.link.written) + item = await driver._queue_ppi_write(duration=10, start_address=40, values=[31]) + self.assertEqual(item, 1) + self.assertEqual(self.sent_since(sent_before), [(0, 65, bytes.fromhex("01000a00280001001f00"))]) + await driver._wait_for_queue_item(item=item, timeout=1.0) + self.assertEqual(driver._event_waiters, []) + + async def test_wait_for_the_last_item_passes_over_the_earlier_ones(self): + driver = self.driver + driver._next_item = 3 + items = [ + await driver._queue_ppi_write(duration=500, start_address=40, values=[0]) for _ in "345" + ] + self.assertEqual(items, [3, 4, 5]) + wait = asyncio.ensure_future(driver._wait_for_queue_item(item=5, timeout=1.0)) + await asyncio.sleep(0.02) + for frame in RECORDED_ITEMS_3_TO_5_PLAYED[:2]: + self.link.incoming.extend(bytes.fromhex(frame)) + await asyncio.sleep(0.05) + self.assertFalse(wait.done()) + self.link.incoming.extend(bytes.fromhex(RECORDED_ITEMS_3_TO_5_PLAYED[2])) + await wait + + async def test_every_item_played_is_kept_for_the_dispense_progress(self): + driver = self.driver + self.assertIsNone(driver._sequence_progress) + driver._next_item = 3 + for _ in "345": + await driver._queue_ppi_write(duration=500, start_address=40, values=[0]) + for frame, item in zip(RECORDED_ITEMS_3_TO_5_PLAYED, [3, 4, 5]): + self.link.incoming.extend(bytes.fromhex(frame)) + await asyncio.sleep(0.02) + self.assertEqual(driver._sequence_progress, item) + self.assertEqual(driver._last_drained_item, 5) + await driver.stop() + await driver.setup() + self.assertIsNone(driver._sequence_progress) + + async def test_full_queue_is_sent_again_under_the_next_item(self): + self.driver._next_item = 368 + sent_before = len(self.link.written) + item = await self.driver._queue_ppi_write(duration=1, start_address=40, values=[27]) + self.assertEqual(item, 373) + self.assertEqual( + self.sent_since(sent_before), + [(0, 65, bytes.fromhex(request)) for request in RECORDED_QUEUE_FULL_REPLIES], + ) + + async def test_queue_staying_full_times_out(self): + self.driver._next_item = 368 + sent_before = len(self.link.written) + with mock.patch.object(master_module, "time") as clock: + clock.monotonic.side_effect = [0.0, 60.0] + with self.assertRaisesRegex(MantisTimeoutError, "stayed full for 60.0 s"): + await self.driver._queue_ppi_write(duration=1, start_address=40, values=[27]) + self.assertEqual(len(self.sent_since(sent_before)), 1) + + async def test_queue_write_refuses_out_of_range_arguments(self): + cases: List[Tuple[int, int, List[int]]] = [ + (10, 120, [31]), + (0, 40, [31]), + (0x8000, 40, [31]), + (10, 40, []), + (10, 40, [0x10000]), + ] + sent_before = len(self.link.written) + for duration, start_address, values in cases: + with self.subTest(duration=duration, start_address=start_address, values=values): + with self.assertRaises(ValueError): + await self.driver._queue_ppi_write( + duration=duration, start_address=start_address, values=values + ) + self.assertEqual(len(self.link.written), sent_before) + + async def test_default_queue_full_timeout_is_checked_when_assigned(self): + with self.assertRaises(ValueError): + self.driver.default_queue_full_timeout = 0.0 + self.assertEqual(self.driver.default_queue_full_timeout, 60.0) + + async def test_motor_error_fails_every_motion_wait_at_once(self): + driver = self.driver + stopped = driver._expect_event(board=0, code=SEQUENCE_STOPPED_EVENT) + move_done = driver._expect_event(board=0, code=MOVE_DONE_EVENT) + wait = asyncio.ensure_future(driver._wait_for_queue_item(item=0x31D9, timeout=5.0)) + await asyncio.sleep(0.02) + for frame in RECORDED_MOTOR_ERROR_EVENTS: + self.link.incoming.extend(bytes.fromhex(frame)) + with self.assertRaisesRegex(MantisMotionError, "motor 0 reported error 9") as raised: + await wait + self.assertEqual((raised.exception.motor, raised.exception.code), (0, 9)) + with self.assertRaises(MantisMotionError): + await driver._wait_for_event(waiter=move_done, timeout=1.0) + payload = await driver._wait_for_event(waiter=stopped, timeout=1.0) + self.assertEqual(decode_sequence_stopped(payload=payload), (0x31D9, 0)) + + async def test_sequencer_stop_is_a_warning_and_a_move_done_is_not(self): + with self.assertLogs(master_module.logger, level="WARNING") as logs: + for frame in RECORDED_MOTOR_ERROR_EVENTS: + self.link.incoming.extend(bytes.fromhex(frame)) + await asyncio.sleep(0.05) + output = "".join(logs.output) + self.assertIn("sequencer stopped at item", output) + self.assertNotIn("moved", output) + + async def test_event_not_decoded_is_logged_by_its_number(self): + with self.assertLogs(master_module.logger, level="WARNING") as logs: + self.link.incoming.extend(bytes.fromhex(RECORDED_CODE_7FFF_FRAME)) + await asyncio.sleep(0.05) + self.assertIn("event 0x7fff, not decoded", "".join(logs.output)) + + if __name__ == "__main__": unittest.main() diff --git a/pylabrobot/formulatrix/mantis/driver/simulator.py b/pylabrobot/formulatrix/mantis/driver/simulator.py index 5fe33512264..e7472e2fc58 100644 --- a/pylabrobot/formulatrix/mantis/driver/simulator.py +++ b/pylabrobot/formulatrix/mantis/driver/simulator.py @@ -19,7 +19,7 @@ from .features.pump import PressureController, Pump from .features.rfid import Rfid from .framing import ABSOLUTE_POSITION_SENSOR_BOARD, LC3_BOARD, MOTION_BOARD, PRESSURE_BOARD -from .master import ABSOLUTE_ENCODER_MOTORS, MOTORS, MantisDriver +from .master import ABSOLUTE_ENCODER_MOTORS, MOTORS, SEQUENCER_QUEUE_SIZE, MantisDriver logger = logging.getLogger(__name__) @@ -138,6 +138,9 @@ class MantisSimulationDriver(MantisDriver): at once, every RFID slot reads empty, and the LED bar takes every command and holds nothing. The motors read idle at 0, as after power-on. The device name and the motor reads the configuration does not hold are not simulated. + + The sequencer holds ``SEQUENCER_QUEUE_SIZE`` items, stopped and empty after setup; started, it + plays each item at once. Stop and abort keep what is queued, as on a v4; only a clear drops it. """ def __init__( @@ -162,6 +165,8 @@ def __init__( oem_config_directory=oem_config_directory, show_actions=show_actions, ) + self._queued_items: List[int] = [] + self._sequencer_started = False def _build_io(self, *, device_id: Optional[str], ftdi_library: Optional[str]) -> FTDI: """Build no link.""" @@ -247,6 +252,53 @@ async def request_absolute_encoder_position(self, *, motor: int) -> float: self._check_index(name="motor", value=motor, allowed=ABSOLUTE_ENCODER_MOTORS) raise NotImplementedError(f"{self.name}: the absolute encoders are not simulated") + def _play_queued_items(self) -> None: + """Play every queued item at once, if the sequencer is started.""" + if self._sequencer_started and self._queued_items: + self._sequence_progress = self._last_drained_item = self._queued_items[-1] + self._queued_items = [] + + async def _unchecked_fw_clear_sequencer(self) -> int: + """Empty the queue.""" + self._queued_items = [] + return 0 + + async def _unchecked_fw_start_sequencer(self) -> int: + """Start the sequencer; it plays what is queued.""" + self._sequencer_started = True + self._play_queued_items() + return 0 + + async def _unchecked_fw_stop_sequencer(self) -> int: + """Stop the sequencer; what is queued stays.""" + self._sequencer_started = False + return 0 + + async def _unchecked_fw_abort_sequencer(self) -> int: + """Stop the sequencer; what is queued stays.""" + self._sequencer_started = False + return 0 + + async def _unchecked_fw_queue_write_ppi( + self, *, item: int, duration: int, start_address: int, values: List[int] + ) -> Tuple[int, int]: + """Queue the item unless the queue is full; started, play it. Writes nothing.""" + if len(self._queued_items) >= SEQUENCER_QUEUE_SIZE: + return -1, 0 + self._queued_items.append(item) + free = SEQUENCER_QUEUE_SIZE - len(self._queued_items) + self._play_queued_items() + return free, 0 + + async def _wait_for_queue_item(self, *, item: int, timeout: float) -> None: + """Return if ``item`` was the last played, with none left queued. + + Raises: + MantisTimeoutError: at once otherwise: the sequencer is stopped or played a later item. + """ + if self._last_drained_item != item: + raise MantisTimeoutError(f"{self.name}: queue item {item} is not reported played last") + async def _request_configuration(self) -> DeviceConfiguration: """The declared configuration's ``device`` block, a field it lacks filled as for the unit blocks; the generation from its boards, as a device's is.""" @@ -295,8 +347,10 @@ def _discover_features(self) -> None: ) async def _open(self) -> None: - """Open nothing: there is no link.""" + """Open nothing: there is no link. The sequencer is stopped and empty.""" logger.info("[%s] simulated, from %s", self.name, self.declared_configuration_json) + self._next_item, self._last_drained_item, self._sequence_progress = 1, None, None + self._queued_items, self._sequencer_started = [], False async def _close(self) -> None: """Close nothing: there is no link.""" diff --git a/pylabrobot/formulatrix/mantis/driver/simulator_tests.py b/pylabrobot/formulatrix/mantis/driver/simulator_tests.py index d67b2480d6a..5083769ea0f 100644 --- a/pylabrobot/formulatrix/mantis/driver/simulator_tests.py +++ b/pylabrobot/formulatrix/mantis/driver/simulator_tests.py @@ -8,6 +8,7 @@ from pylabrobot.formulatrix.mantis import Mantis from pylabrobot.formulatrix.mantis.device import RECORDING_MANTIS_V4 from pylabrobot.formulatrix.mantis.driver.errors import MantisTimeoutError +from pylabrobot.formulatrix.mantis.driver.master import SEQUENCER_QUEUE_SIZE from pylabrobot.formulatrix.mantis.driver.simulator import MantisSimulationLedBar @@ -130,6 +131,40 @@ async def test_led_bar_takes_every_command(self): with self.assertRaises(ValueError): await bar.set_color(color="purple") + async def test_sequencer_plays_what_is_queued_once_started(self): + driver = self.mantis.driver + item = await driver._queue_ppi_write(duration=10, start_address=40, values=[31]) + self.assertEqual(item, 1) + with self.assertRaises(MantisTimeoutError): + await driver._wait_for_queue_item(item=item, timeout=1.0) + self.assertIsNone(driver._sequence_progress) + await driver._unchecked_fw_start_sequencer() + await driver._wait_for_queue_item(item=item, timeout=1.0) + item = await driver._queue_ppi_write(duration=10, start_address=40, values=[31]) + await driver._wait_for_queue_item(item=item, timeout=1.0) + self.assertEqual(driver._sequence_progress, item) + await driver._unchecked_fw_stop_sequencer() + item = await driver._queue_ppi_write(duration=10, start_address=40, values=[31]) + with self.assertRaises(MantisTimeoutError): + await driver._wait_for_queue_item(item=item, timeout=1.0) + + async def test_stopped_sequencer_fills_its_queue_and_a_clear_empties_it(self): + driver = self.mantis.driver + free_slots = [ + ( + await driver._unchecked_fw_queue_write_ppi( + item=item, duration=10, start_address=40, values=[31] + ) + )[0] + for item in range(1, SEQUENCER_QUEUE_SIZE + 2) + ] + self.assertEqual(free_slots, list(range(SEQUENCER_QUEUE_SIZE - 1, -2, -1))) + driver.default_queue_full_timeout = 0.01 + with self.assertRaisesRegex(MantisTimeoutError, "stayed full"): + await driver._queue_ppi_write(duration=10, start_address=40, values=[31]) + await driver._unchecked_fw_clear_sequencer() + await driver._queue_ppi_write(duration=10, start_address=40, values=[31]) + async def test_simulated_mantis_sends_nothing(self): with self.assertRaisesRegex(NotImplementedError, "sends nothing"): await self.mantis.driver._send_command(board=0, command=3)