USB mass storage: the BOT round trip and the SCSI bring-up are toyos-xhci machines

[Japabu]

The USB mass-storage BOT round trip and the SCSI bring-up above it become pure toyos-xhci code with host tests. kernel/src/drivers/xhci/wait/msc.rs keeps the shim: it queues the transfers the machines ask for, waits for them in place, publishes the phase to the reset path, runs the staged actuators, and writes the log lines. It also still decides the ladder's climb loop, that a disconnect goes to the port's teardown, offline: dev.failed, and data_out. A userland usbd can drive the same machines unchanged. The crate is no_std, has no unsafe code and makes no x86 assumptions.

What changed and why

toyos_xhci::bot

• cbw() builds the 31-byte CBW (USB MSC BOT 1.0 §5.1): LUN 0, direction taken from the CDB.
• RoundTrip is the round trip as a stepped machine. Its acts are Command, Data(pipe), Status and Restart { pipe, then }, and it is answered with Moved { code, residue }, Silent, Gone or Restarted.
  • CBW and CSW are fixed-length legs: a short one is a break, not a short transfer.
  • A stalled data phase is restarted and its status is read (§6.7.2). What it delivers is the smaller of what the controller's residue and the CSW's leave.
  • A stalled CSW is retried once, as §5.3.3 / Figure 2 allows, and that includes a CSW after a data-stall restart.
  • CswDue::judge checks signature and tag (§6.3.1), then applies §6.3.2's meaningful test. Status 2 is a phase error whatever the residue. Statuses 0 and 1 are refused when the residue is larger than the transfer. Statuses 3..=255 are Broke::Reserved. Delivered bytes are min(controller's count, device's count), and a controller residue larger than the transfer delivers nothing.
• Broke<W> carries left() and event() and is generic over the driver's silence reason.
  • A disconnect is its own Gone variant; it used to be Silence { why: Quiet::Gone }. The crate has to tell a disconnect apart, and the kernel's Quiet is not a type the crate has.
  • Broke<Quiet>::Silence { why: Quiet::Gone } can still be built, and only completed() in msc.rs keeps it apart from Broke::Gone. It is not made unrepresentable because the price is a second kernel enum that mirrors three of Quiet's four variants (one of them cfg'd), with conversions both ways, all to guard one match in one function. Deleting that match's Gone arm is the negative control below, run in the guest.
  • left() matches on a Phase. It used to match on a phase's name string.
• CC_STALL joins CC_SUCCESS and CC_SHORT_PACKET in toyos_xhci::job. The kernel's EP0 recovery imports it, and the kernel's own copy is deleted.

toyos_xhci::scsi

• Cdb can only be built inside this module, and it carries its own direction. The kernel's cdb_len/data_in parameters and its CDB-shape assert are gone, because a mismatch can no longer be written.
• Sense::of believes sense data only if at least 14 bytes arrived (ASCQ is byte 13; SPC-4 §4.5.3).
• Reply is the answer to one SCSI command. It is named apart from the crate root's toyos_xhci::Outcome.
• flushed(): INVALID COMMAND OPERATION CODE means the device has no cache, which is not a failure.
• Transfer does the READ/WRITE(10) batching and sector addressing. Only the first batch may answer "ask again". Geometry's fields are private and read through accessors, so only the bring-up can size a disk, and the expect in Transfer::next depends on that. Geometry::NONE fits no transfer, and a test checks it.
• BringUp is a stepped machine:
  • TEST UNIT READY on a budget, then sense or a recovery climb;
  • INQUIRY with the peripheral-type check;
  • READ CAPACITY(10), then (16) when the 10-byte answer is all ones;
  • every refusal by name (Refusal).
• The sector sizes served are 512, 1024, 2048 and 4096. This set is the driver's own and is carried over unchanged. It is not SBC-3's, which allows any logical block length. 256 divides the host block and is still refused, and the test says so.
• The kernel drives BringUp blocking, the same way the boot scan drives enumeration. A stepped kernel driver for the hot-plug bind is not built here; the renamed issue below owns that.
• Printable moved here unchanged.

toyos_xhci::ladder::Run holds a device's break count and highest rung. It used to be two loose fields. identity::first_language reads the LANGID of string descriptor zero.

Kernel shim: msc.rs drops from 2647 to 2359 lines, and kernel/src from 64984 to 64695 (find kernel/src -name '*.rs' | xargs cat | wc -l, before and after). The staged PortGone fault is now the Command act's completion, completed(Err(Quiet::Gone)), so the actuator goes through the arm that sends a disconnect to the teardown. Nothing is queued for it, as before.

src/sourcegate.rs: the : u32 = 4096 exception for msc.rs is removed, since HOST_BLOCK now lives in the crate.

Issue: the slug the-bot-scsi-machine-is-still-hand-written-in-the-kernel is no longer true. What the issue still owes is the bind, which is the one call site inside a scheduling pass. So it is renamed to issues/hardware/a-disk-plugged-in-after-boot-is-bound-inside-a-scheduling-pass.md, and both citations move with it (issues/build/the-swarm-is-not-yet-falsifiable.md, issues/kernel/the-kernel-is-small-interrupts-post-and-threads-wait.md).

No behaviour change on the wire

The commands, bytes, order, waits and recovery are the same. The two source spellings that toyos-blackbox checks against msc.rs are kept.

These things changed off the wire:

• The CBW is written as one 31-byte copy instead of field writes through Unaligned: the same bytes at the same DMA offset. The CSW is read as one 13-byte copy.
• REQUEST SENSE's response is copied out whenever the round trip completed, and then thrown away if fewer than 14 bytes arrived.
• Bring-up copies the whole allocation (36/8/32 bytes) rather than the 36/8/12 it reads.
• A status-stall retry publishes StatusOwed twice, with the same value and ring positions.
• Transfer::next now panics (expect) where the old code silently truncated a sector number to 32 bits. Bring-up's refusal of disks past 2^32 sectors and the range check already make that case unreachable.
• msc_flush's inner if !dev.no_write_cache is removed. It was always true, because the latch is checked above it.
• Two log lines change. A CSW with status 2 and a residue larger than the transfer now logs as a phase error instead of CSW claims N B unmoved, and a reserved status (3..=255) logs the CSW carries status 0x.., which the class reserves instead of a phase error. Both break in the status phase and climb the same ladder as before.
• With boot-actuators, the staged PortGone now takes a tag and writes its CBW into the pool before its Command act completes as the disconnect. Nothing reaches the controller.

Gates (this head)

gate	exit
cargo test -p toyos-build --lib	0
cargo test --workspace --exclude toyos-build	0
cargo test --test toyos-build -- --list	0
cargo run -- --clippy (x86_64 and aarch64 kernels, with and without boot-actuators)	0
cargo run -- --build-only	0
cargo test -p toyos-xhci (148 tests, 33 of them new)	0

Guest: at 627a715, the previous head, the orchestrator ran the 19 USB/storage tests and Fast, all EXIT=0. None of those runs reached the ladder. No guest run has been made at this head.

Metal is owed. The metal reading has not been made: a T14 stick boot and transport_break_on_metal (tests/common/usb.rs) at this head. The T14 is offline, so this PR claims none.

High-risk: negative controls and the independent oracle

Negative controls. Each is a checked patch: git apply --check, apply, shown to build, cargo test -p toyos-xhci, git apply -R. The tree was clean afterwards. All 23 build (EXIT=0) and all 23 go red (EXIT=101):

mutation	red test(s)
m01 CBW direction flag inverted	a_cbw_is_laid_out_as_the_class_defines_it
m02 CSW tag not checked	a_csw_is_believed_only_when_it_is_valid_and_meaningful
m03 delivered trusts the device alone	what_is_delivered_is_what_both_the_controller_and_the_device_say_arrived
m04 status stall retried without bound	a_stalled_status_is_asked_for_again_once
m05 CSW residue unbounded	a_csw_is_believed_only_when_it_is_valid_and_meaningful
m06 short CBW/CSW accepted	a_command_or_status_block_that_moved_short_breaks
m07 data stall not restarted	three bot tests
m08 sense believed at 13 bytes	sense_is_believed_only_when_its_ascq_arrived
m09 "ask again" after blocks moved	only_the_first_batch_may_answer_ask_again
m10 READ CAPACITY(16) never asked	a_disk_too_big_for_read_capacity_10_is_asked_the_16_byte_form
m11 disk past READ(10) served	same
m12 no-cache flush is a failure	a_flush_the_device_does_not_implement_is_no_failure_and_every_other_refusal_is
m13 ready budget ignored	a_disk_that_never_becomes_ready_is_given_up_on_at_its_budget
m14 sector LBA unscaled	a_transfer_is_its_batches_in_order_and_only_a_whole_one_advances
m15 peripheral qualifier read as the type	a_device_that_is_not_a_disk_this_driver_serves_is_refused_by_name
m16 Run::over keeps its rung	a_run_climbs_one_rung_per_break_until_a_round_trip_ends_it
m17 skip-the-class-reset said on every rung	two ladder tests
m18 LANGID read from an empty descriptor	a_language_is_read_only_from_a_string_descriptor_that_carries_one
m19 after a data stall, moved: self.data_len (the device's residue alone)	a_stalled_data_phase_delivers_what_both_the_controller_and_the_device_say_arrived
m20 after a data stall, .status(true) (no §5.3.3 retry)	a_status_stalled_after_a_data_stall_is_still_asked_for_again
m21 data residue wrapping_sub	what_is_delivered_is_what_both_the_controller_and_the_device_say_arrived
m22 CSW residue checked before status 2	a_csw_is_believed_only_when_it_is_valid_and_meaningful
m23 data-stall residue wrapping_sub	a_stalled_data_phase_delivers_what_both_the_controller_and_the_device_say_arrived

The disconnect seam's control is a guest mutation, because no host test drives the kernel: delete Err(Quiet::Gone) => bot::Answer::Gone, from completed(). With that patch applied the tree passes cargo run -- --clippy (EXIT=0). Its run is usb_transport_break with the arm deleted, which must go red in port_gone_is_left_to_the_teardown. That run belongs to the orchestrator.

For "no behaviour change" as a whole, the whole-change control is the guest USB suite on this head compared with its green on main. usb_transport_break is on the redlist (issues/kernel/a-held-disk-waits-for-a-pass-no-cpu-takes-when-every-cpu-is-in-a-call-on-it.md), and it drives the ladder paths this change moved most. The orchestrator forces it with the row removed at this head and at the base. Those are the orchestrator's runs.

Independent oracle. The host tests' byte layouts and verdicts are written from the specifications' tables, not from the old code:

• USB MSC BOT 1.0: §5.1 CBW, §5.2 CSW, §5.3.3 status retry, §6.3.1 valid and §6.3.2 meaningful CSW, §6.7.2 data stall.
• SPC-4: §4.5.3 fixed sense, §6.4 INQUIRY, §6.29 REQUEST SENSE, §6.37 TEST UNIT READY.
• SBC-3: §5.11 READ(10), §5.32 WRITE(10), §5.24 SYNCHRONIZE CACHE, and READ CAPACITY(10)/(16).
• USB 2.0 §9.6.7 for LANGID.

The sector-size set is the one exception. It has no oracle but the driver it was carried over from.

QEMU's usb-storage device is a second implementation of the same protocol, reached through the guest USB suite. The T14's stick is a third, reached through the metal arm that is owed above.

🤖 Generated with Claude Code

[Japabu]

Review, round 1, at 627a715

CI host SUCCESS at 627a715 (run 36473180835, --ci host and --ci gate-stage steps success). git merge-tree origin/main 627a7150 is clean (exit 0). Guest: 19 USB/storage tests and Fast EXIT=0 (orchestrator logs 588-*.log). None of those runs reaches the ladder.

Growth, from git diff --shortstat origin/main...627a7150 (+1799 −601): production is about +559 (crate +849: bot +271, scsi +527, ladder +42, identity +8, lib +1; kernel −287; sourcegate −3), tests about +641, issues −2. I accept the growth for host-testable decisions that usbd can reuse. The deletions are named below.

BLOCKER

• toyos-xhci/src/bot.rs:108, kernel/src/drivers/xhci/mod.rs:116 — CC_STALL = 6 is now declared twice, once in the crate and once in the kernel. The crate's copy sits in bot.rs rather than in job.rs beside CC_SUCCESS/CC_SHORT_PACKET, and it is a sibling of the kernel constant. Move it to job.rs, have the kernel (wait/mod.rs:485) import it, and delete mod.rs:116.
• toyos-xhci/src/bot.rs:293-296 — two mutations on the data-stall route survive every host test. (a) Self { moved: self.data_len.saturating_sub(unmoved), ..self } → Self { moved: self.data_len, ..self }: after a stall the device's residue is trusted alone. (b) .status(false) → .status(true): a CSW STALL after a data-stall restart gets no §5.3.3 retry, which is a change on the wire. Add a walk with a Data STALL at residue 100 then CSW (TAG, 0, 0), asserting Done { delivered: 412 }. Add a walk with a Data STALL, then a Status STALL, then a whole CSW, asserting the second Status act and Ok. Show both mutations red.
• kernel/src/drivers/xhci/wait/msc.rs:763, :1374 — this is the new seam that sends a disconnect to the teardown and not up the ladder. Deleting the Err(Quiet::Gone) => bot::Answer::Gone, arm passes every test in the tree. The PortGone actuator returns a ready-made Broke::Gone from bot before any transfer, so port_gone_is_left_to_the_teardown never goes through completed(). Stage the fault as the Command act's completion (Err(Quiet::Gone) fed through completed) so the actuator exercises the seam. Then show the arm-deletion mutation red in usb_transport_break.
• tests/common/usb.rs:1414 (usb_transport_break, redlisted) — the paths this change moved most have no run at 627a715. That covers Run/skips_class_reset, Broke::Gone to the teardown, Told on OutOfStep, offline after "broke N times running", abandoned_write_is_taken_offline, and the moved sticks. The pcap wire oracle (command_blocks, every_port_reset_is_followed_by_a_test_unit_ready) is the only independent check of "no change on the wire" for the ladder, and it did not run either. The orchestrator must force it by name at 627a715 and at base bc9ccad, same run count, with the row removed in a scratch checkout only. Required: the first boot (pcap order, owed-data skip, port-gone), transport_gives_up and abandoned_write_is_taken_offline are green at head. Any red at head carries a signature from one of the four redlisted issues and also occurs at the base.
• (hardware) — there is no metal reading at 627a715. This is the T14's boot-stick driver, and the PR body itself says the metal run is what would show a difference on silicon. Required: a T14 stick boot and transport_break_on_metal (tests/common/usb.rs:2273) at this head, with exit code and log. QEMU is not the hardware.

NOTE

• toyos-xhci/src/scsi.rs:15 — pub type Nanos = u64; is a fifth copy of the alias the crate root already re-exports (port::Nanos). Use crate::Nanos.
• toyos-xhci/src/scsi.rs:143-149,229 — the expect rests on Geometry coming only from geometry(), but its fields are pub and so is NONE. Make the fields private with accessors so the premise is enforced by the type rather than by a comment.
• toyos-xhci/src/bot.rs:355-363 — the checks follow the old code's order, not §6.3.1. Status 02h is meaningful whatever the residue, but here a phase error with residue > length becomes Residue. The recovery is the same (both lead to Phase::Status), but the test claims §6.3 and pins the old order. PhaseError also carries statuses 3..=255.
• toyos-xhci/src/bot.rs:278 — saturating_sub → wrapping_sub survives: no test has a controller residue larger than the transfer. One assertion closes it.
• toyos-xhci/src/bot.rs:148-150 — Broke<Quiet>::Silence { why: Quiet::Gone } can still be built beside Broke::Gone, which gives one fact two representations. Only completed() keeps them apart.
• toyos-xhci/src/scsi.rs:121 — scsi::Outcome has the same name as the crate root's toyos_xhci::Outcome (job::Outcome). Rename one.
• toyos-xhci/src/scsi.rs:497 — the 512..=4096 sector set comes from the old code, not from SBC-3, which allows any logical block length. 256 divides HOST_BLOCK and is refused. The behaviour is unchanged, so keep it, but the test claims the spec as its oracle.

REMOVE

• kernel/src/drivers/xhci/wait/msc.rs:8-11 — "Every decision is the crate's …" is false: the climb loop, the Gone-to-teardown verdict, offline: dev.failed and data_out are decided in this file.
• toyos-xhci/src/scsi.rs:10-12 — "it is driven to its end in place today" describes the tree and marks a date; it will rot.
• toyos-xhci/src/scsi.rs:387 — "A sector size that does not divide [HOST_BLOCK]; zero among them." is false: 256 divides it and lands here.
• toyos-xhci/src/bot.rs:158-160 — "The CSW named somebody else's transfer" is false for what: "signature".
• PR body "## Overlap with Storage: file servers for DATA, the log and the boot volume; the kernel's NVMe and FAT go #536" — not the merge's record.
• PR body "## What I am unsure of" — review-time text; it goes stale in main's history.
• PR body "The guest runs are queued below." — nothing is queued below.

SEND BACK

[Japabu]

Review, round 2, at f90b2d2

CI host SUCCESS at f90b2d2. git merge-tree --write-tree origin/main HEAD (origin/main 8a8fe27) exits 0. The merge 435e00b touched none of the branch's files. Guest evidence is the orchestrator's 588r2 runs at f90b2d2: 20 USB/xHCI/storage tests and Fast, all EXIT=0. Forced usb_transport_break runs: head 2/2 EXIT=0, head with the gone arm deleted EXIT=1, base e3a1cdc 2/2 EXIT=0.

Growth, from git diff --shortstat origin/main...HEAD (+1869 −607): production is about +510 and tests about +750 (bot 376, scsi 339, ladder 27, identity 10). Accepted, on round 1's grounds.

Round-1 BLOCKERs

• CC_STALL declared twice: CLOSED. toyos-xhci/src/job.rs:54 is the only declaration left. wait/mod.rs:44 imports it, and mod.rs:116 is deleted.
• Two data-stall mutations survived: CLOSED. m-a-stall-trusts-device goes red in a_stalled_data_phase_delivers_what_both_the_controller_and_the_device_say_arrived, and m-b-stall-no-status-retry goes red in a_status_stalled_after_a_data_stall_is_still_asked_for_again. Each built with EXIT=0 and tested with EXIT=101, and the tree was restored by cmp (r2/mutations.log). Each patch is exactly the round-1 line.
• The Gone seam was never exercised: CLOSED. msc.rs:1417 feeds the staged PortGone through completed(Err(Quiet::Gone)), and gone-arm-deleted.patch deletes exactly msc.rs:761.
  • The mutated run exits 1 with the driver never said "... its port's teardown takes it from here". Its log shows the disconnect climbing the ladder instead: break 1 of 3 running, then Reset Recovery took.
  • Unmutated, the head is 2/2 EXIT=0.
• Forced usb_transport_break runs: CLOSED. Round 1 required the head to be green, and any red at the head to occur at the base as well. It never required a red at the base.
  • Head 2/2 and base 2/2 print the same lines: the pcap wire oracle (4 Bulk-Only resets, each followed by both clears and a TEST UNIT READY), the owed-data skip, 3 taken offline behind a last reset, the abandoned write, and all four moved-stick boots.
• No metal reading: OPEN. No T14 stick boot and no transport_break_on_metal exist at f90b2d2. This remains a landing condition, because QEMU is not the hardware. The branch owes no code for it; it waits for the T14.

The redlist

• The PR does not claim to fix usb_transport_break, and src/redlist.rs is not in the diff. The green at base contradicts nothing the PR says.
• The row stays, and so does issues/kernel/a-held-disk-waits-for-a-pass-no-cpu-takes-when-every-cpu-is-in-a-call-on-it.md, unchanged.
  • The issue's red comes at a rate. The code it names (wait_for_return, and the disk call spinning with IF clear) is untouched by this diff. Two green runs on each side measure nothing about it.
  • The base did exercise the held-disk path. The defect the issue owes shows up identically at head and base: SameStick: the held call ended 2.000 s after the break on cpu1; the stick was bound on cpu0, after it ended. The test tolerates that shape; only the rarer interleaving that also loses the window goes red.

Mutation patches

• All seven revert what they claim: unredlist-transport-break, gone-arm-deleted, and m-a through m-e. m-d moves the residue guard ahead of every status, which is the old order in full, as the commit states.

BLOCKER

• (hardware): no metal reading at f90b2d2. Carried from round 1, not new.

NOTE

• kernel/src/drivers/xhci/mod.rs:115-116 — CC_SUCCESS and CC_SHORT_PACKET are still declared both here and in toyos-xhci/src/job.rs:51-52. At msc.rs:1416 the kernel's copy answers the crate's machine. This predates the branch, but the CC_STALL move is only complete once both kernel lines are deleted and imported from toyos_xhci::job.
• PR body, mutation table m01–m18: these were measured at 627a715, and CswDue::judge has changed since. Re-run them at the head, or state the head they were measured at.

REMOVE

• PR body: "Guest: at 627a715, the previous head, … No guest run has been made at this head." It is false at f90b2d2.
• PR body: "That run belongs to the orchestrator." This is review-time text, not main's record.
• PR body: "The orchestrator forces it with the row removed at this head and at the base. Those are the orchestrator's runs." This is review-time text as well.

SEND BACK