xHCI: every report that moves a port's belief leaves it to be read

issues/kernel/a-device-swapped-into-its-port-inside-the-port-rungs-reset-is-taken-for-the-disk-it-replaced.md

@@ -0,0 +1,36 @@
+---
+status: open
+kind: defect
+opened: 2026-09-28
+---
+
+# A device swapped into its port inside the port rung's reset is taken for the disk it replaced
+
+Derived from the code, not staged.
+
+The port rung (`port_reset_recovery` in `kernel/src/drivers/xhci/wait/msc.rs`)
+resets the port and reads it once (`reset_port`). When
+`toyos_xhci::ladder::after_reset` answers `Enumerate`, the port reads
+connected, enabled and at the speed the disk was bound at. The rung then
+addresses the disk's own slot again, sets its old configuration, adds its old
+bulk pair and asks TEST UNIT READY. **Nothing re-reads who the device is.**
+
+Take a device pulled after the reset completes and another plugged into the
+same port before that read. On a USB3 port the new device trains to Enabled by
+itself (§4.19.1.2). The CSC its arrival raised is the one
+`enumeration_ack(Some(Warm), …)` spends, because a warm reset's retrain raises
+one too (§4.19.5.1) and nothing tells the two apart. A second unit of the same
+model answers every step: every identity field is the same but its serial
+number, as with `usb_transport_break`'s `AnotherStick`. The rung then carries
+disk 0's volume on onto it.
+
+The port machine does not see the swap either. Its belief stays attached with
+disk 0's slot, and the rung's acknowledge spent the edge.
+
+`usb_transport_break` moves the stick to port 3, so the bind's serial check
+(`XhciController::adopt`) judges it there. Nothing stages the same port.
+
+**Exit**: a disk the port rung took back is the device it was. Its serial
+number is read again and judged by `toyos_xhci::identity::same` before the
+volume carries on. A `usb-reset-moves` staging that plugs another serial number
+into the same port refuses it by name.

issues/kernel/a-disk-refused-while-one-is-held-is-enumerated-again-and-no-test-reaches-that-arm.md

@@ -0,0 +1,25 @@
+---
+status: open
+kind: defect
+opened: 2026-09-27
+---
+
+# A disk refused while one is held is enumerated again, and no test reaches that arm
+
+`refuse_for_now` (`kernel/src/drivers/xhci/device.rs`) gives a disk refused as
+not ready, or for want of a pool block, a Disable Slot with
+`AfterSlot::Again` while a disk on this controller is held for its device. The
+arm in `slot_gone` (`kernel/src/drivers/xhci/mod.rs`) frees the refused
+device's block, tears its port down so it is enumerated again, and logs
+`xHCI: port N is enumerated again while a disk is held for its device`.
+
+Nothing reads that line: `rg 'enumerated again while a disk' tests/` finds
+nothing, and no run has shown whether any guest test reaches the arm. The
+decision is taken in the kernel and not in `toyos-xhci`, so no host test can
+stage it either.
+
+**Owner**: the xHCI driver, `kernel/src/drivers/xhci/`.
+
+**Exit**: a test stages a disk refused while another is held, and asserts that
+its port is enumerated again and the disk bound. With `AfterSlot::Again`
+replaced by `AfterSlot::Refused`, that test goes red.

issues/kernel/a-held-disk-waits-for-a-pass-no-cpu-takes-when-every-cpu-is-in-a-call-on-it.md

@@ -23,6 +23,25 @@ throughout and was first enumerated at 2.455 s, after both calls ended at
 untaken, so the disk is taken back and the retried operations complete (1 of 6
 runs at `f0695038` took this shape and passed).
 
+**A CPU spinning on a lock the call's caller holds counts too.**
+`usb_transport_break --nightly` at `e889d03e` (#554), the `AnotherStick` boot
+(`554r6-usb_transport_break.log` in the job scratchpad), red. cpu0 spent 0.383
+to 4.385 s in logd's create of the boot's log file, which holds `vfs::lock()`
+(`object::ops::open`): two writes of block 9351 on held disk 0, each ending
+`still held` on its bound, at 2.384 and 4.385, with no pass between them.
+cpu1 logged nothing from 0.311 to 4.390 s. Two threads resumed within a
+millisecond of that create's end: `test-runner`'s spawn of `reboot`, whose own
+`total=7ms` puts its start at about 4.383, and init's `started test-runner`
+line, stamped 4.386 for a spawn made at 0.363. An idle cpu1 kicked by
+`wait_for_return` would have torn port 1 down within its 100 ms debounce, and
+no `port 1 disconnected` line exists, so cpu1 took no pass. That it spun on the
+VFS lock is inferred, not measured. Port 3 read connected and untaken
+throughout, so the arrival rule kept disk 0 held and no `did not come back`
+line came either. The other stick was never enumerated, and the test's
+`is not disk 0 come back` line never came. From 4.424 s cpu0 spun in the
+reboot's sync and cpu1 in a shootdown
+(`a-shutdown-on-a-held-usb-disk-left-a-cpu-deaf-to-a-tlb-shootdown.md`).
+
 **What is still wrong**: the operation that waited answers `BudgetExpired`
 instead of the device's answer, and a caller that gives up on one — `logd` on a
 refused create (`issues/boot-media/logd-ends-the-boots-log-on-one-refused-create-and-nothing-durable-says-so.md`)

issues/kernel/a-poll-that-leaves-a-port-outstanding-with-no-wake-says-nothing.md

@@ -0,0 +1,20 @@
+---
+status: open
+kind: defect
+opened: 2026-09-28
+---
+
+# A poll that leaves a port outstanding with no wake says nothing
+
+The simulator's pump (`toyos-xhci/sim/src/driver.rs`) fails with
+`Stuck::Unwoken` when a pass leaves the port outstanding
+(`PortState::outstanding`) with no instant to come back at. The kernel has no
+counterpart. `XhciController::poll` (`kernel/src/drivers/xhci/mod.rs`) returns
+`None` in that state, `poll_if_pending` stores 0 in `PORT_WORK_AT`, and the port
+is not read again until some other xHCI interrupt arrives. On hardware nothing
+names the state, and the simulator catches it only on the arms it models.
+
+**Owner**: the xHCI driver, `kernel/src/drivers/xhci/`.
+
+**Exit**: `poll` logs a line naming the port whenever it returns `None` with a
+port outstanding.

issues/kernel/a-shutdown-on-a-held-usb-disk-left-a-cpu-deaf-to-a-tlb-shootdown.md

@@ -6,7 +6,7 @@ opened: 2026-09-24
 
 # A shutdown on a held USB disk left a CPU deaf to a TLB shootdown for five seconds, and the kernel panicked
 
-One sighting, `usb_transport_break` in a local `--nightly` run on the logd
+`usb_transport_break` in a local `--nightly` run on the logd
 branch at `eef19bd1` (parameter line
 `root=…,usb-transport-break,usb-reset-moves,blackbox=0x8000000`, two CPUs,
 TCG). Alone it was green.
@@ -34,6 +34,16 @@ the arithmetic for one disk operation outrunning `time::DEAF_CPU`; this is a boo
 that did outrun it, in `quiesce`, across several operations each inside its
 own budget. Whether `quiesce` holds `IF` clear between them is not measured.
 
+**Second sighting, with the roles swapped**: `usb_transport_break --nightly` at
+`e889d03e` (#554), the `AnotherStick` boot (`554r6-usb_transport_break.log` in
+the job scratchpad). cpu0 took the reboot's `Syncing filesystems` at 4.424 s
+and spent it in calls on held disk 0, each ending `still held`: at 6.427, 8.428
+and 10.430. cpu1, in its idle loop, dropped an `InboxRef` in
+`object::drain_zero_handles` and waited on cpu0:
+
+    [kernel 9.425 cpu1] PANIC: panicked at src/arch/x86_64/tlb.rs:151:42:
+    tlb: cpu 0 has not flushed for generation Generation(1) in 5000000000ns — it is not taking interrupts
+
 ## Exit condition
 
 What holds cpu1's interrupts off across that window is named from a boot, and

issues/kernel/an-acknowledge-after-an-enumeration-clears-a-replug-no-look-has-seen.md

@@ -0,0 +1,33 @@
+---
+status: open
+kind: defect
+opened: 2026-09-27
+---
+
+# An acknowledge after an enumeration clears a replug no look has seen
+
+`device::finish`, `device::refuse` and `device::refuse_for_now`
+(`kernel/src/drivers/xhci/device.rs`) report the port with
+`PortState::enumerated` and then call `acknowledge_port_read`, which writes
+back every change flag the read found, CSC included. `wait::boot::scan_ports`
+does the same for every port once the boot scan is over
+(`acknowledge_port_changes`).
+
+A replug that lands after the port's last look and before that acknowledge is
+cleared before any step reads it. One way in: its Port Status Change Event is
+drained by the same `poll` whose `advance_outstanding` ends the enumeration, so
+`service_ports` runs after `finish` has written CSC back. The port then reads
+connected with no CSC, which is what the driver believes, and the device now in
+it is never torn down or enumerated. The look `PortState::believe` owes the
+port catches a pull, because CCS reads 0, and not a replug.
+
+**Not measured.** No test stages it. The simulator's `enumerated`
+(`toyos-xhci/sim/src/driver.rs`) acknowledges nothing, so no host test can see
+it either.
+
+**Owner**: the xHCI driver, `kernel/src/drivers/xhci/`.
+
+**Exit**: the simulator's report acknowledges exactly what the kernel's does,
+and a sim test that replugs the device between an enumeration's last answer
+and the next look expects `ToreDown(Replugged)` and a second enumeration. That
+test is red on the kernel's acknowledge and green without it.

kernel/src/drivers/xhci/mod.rs

@@ -1191,11 +1191,6 @@ impl XhciController {
         }
     }
 
-    /// Record what the boot scan's enumeration left behind, so hot-plug starts from it; recorded even with no device, since a successful Enable Slot is the controller's resource regardless.
-    fn port_bound(&mut self, port_idx: u8, slot: Option<u8>) {
-        self.ports[port_idx as usize].adopt(slot.and_then(NonZeroU8::new));
-    }
-
     /// Step every port that is not where the driver left it, and say when it wants to be looked at again.
     ///
     /// One step per call, no wait; the enumeration it eventually starts is submit-and-return too.
@@ -1245,14 +1240,10 @@ impl XhciController {
                 Step::Wait(at) => return Some(at),
                 Step::GaveUp(why) => {
                     match why {
-                        GaveUp::ResetNeverFinished(kind) => log!(
-                            "xHCI: port {} never finished its {} reset (PORTSC {:#010x}); \
+                        GaveUp::ResetNeverFinished => log!(
+                            "xHCI: port {} never finished its hot reset (PORTSC {:#010x}); \
                              skipping it",
                             port_idx + 1,
-                            match kind {
-                                Reset::Hot => "hot",
-                                Reset::Warm => "warm",
-                            },
                             portsc.raw()
                         ),
                         // §4.19.1.2 has nothing further after a warm reset — this is the port's end.
@@ -1263,18 +1254,14 @@ impl XhciController {
                             portsc.raw(),
                             portsc.link_state()
                         ),
-                        GaveUp::ResetFailed(kind) => log!(
-                            "xHCI: port {} completed its {} reset without enabling \
+                        GaveUp::ResetFailed => log!(
+                            "xHCI: port {} completed its hot reset without enabling \
                              (PORTSC {:#010x}); skipping it",
                             port_idx + 1,
-                            match kind {
-                                Reset::Hot => "hot",
-                                Reset::Warm => "warm",
-                            },
                             portsc.raw()
                         ),
                     }
-                    return None;
+                    // No return: the port is left to be read, and nothing else wakes a pass for it.
                 }
                 Step::Write(write) => self.write_portsc(port_idx, write),
                 Step::Reset(kind, write) => {
@@ -1319,8 +1306,7 @@ impl XhciController {
                         log!("xHCI: port {} connected, link already trained", port_idx + 1);
                     }
                     device::begin(self, port_idx, after);
-                    // Either enumeration is under way and the port waits, or it refused before spending a command.
-                    return self.outstanding.wake_at();
+                    // No return: a `begin` that refused before Enable Slot left the port to be read, and a begun one is caught above as working.
                 }
             }
         }
@@ -1525,7 +1511,7 @@ impl XhciController {
 
         // Nothing below reads the event ring: every step `service_ports` takes is a submit, so one advance is enough.
         let mut wake_at = None;
-        if self.ports_dirty || self.ports.iter().any(PortState::outstanding) {
+        if portmachine::due(self.ports_dirty, &self.ports) {
             self.ports_dirty = false;
             wake_at = self.service_ports();
         }

kernel/src/drivers/xhci/wait/boot.rs

@@ -2,7 +2,6 @@
 //! happens before there is a scheduler; every wait here runs in place.
 
 use alloc::vec::Vec;
-use core::sync::atomic::Ordering;
 
 use crate::log;
 use crate::time::{Budget, Cadence, Duration};
@@ -21,7 +20,7 @@ use super::super::{IR0_ERDP, IR0_ERSTBA, IR0_ERSTSZ, IR0_IMAN, IR0_IMOD};
 use super::super::{OFF_CMD_RING, OFF_DCBAA, OFF_ERST, OFF_EVT_RING};
 use super::super::{OP_CONFIG, OP_CRCR, OP_DCBAAP, OP_PAGESIZE, OP_PORT_BASE, OP_USBCMD, OP_USBSTS};
 use super::super::{USBCMD_HCRST, USBCMD_RS, USBSTS_CNR, USBSTS_HCH};
-use super::super::{PORTSC_PP, PORT_REG_SIZE, PORT_WORK_AT, XHCI};
+use super::super::{PORTSC_PP, PORT_REG_SIZE, XHCI};
 use super::super::{controller_answers, PORT_DEBOUNCE_NS};
 use super::settles;
 use toyos_xhci::port::{self, GaveUp, Reset, ResetOutcome};
@@ -140,8 +139,6 @@ pub fn init(devices: &[PciDevice]) {
         }
         return;
     }
-    // Safe to zero: the boot scan acted on every port it looked at, so nothing is outstanding.
-    PORT_WORK_AT.store(0, Ordering::Relaxed);
     let hid: usize = controllers.iter().map(|c| c.devices.len()).sum();
     log!("xHCI: {} controller(s), {} HID device(s)", controllers.len(), hid);
     log!("usb-storage: {} device(s)", storage_count());
@@ -161,6 +158,7 @@ pub fn init(devices: &[PciDevice]) {
             ctrl.max_ports,
         );
     }
+    // No scheduler pass runs before `smp::set_ready`, so the scan's interrupt record is first polled with `XHCI` published.
     *XHCI.lock() = controllers;
 }
 
@@ -484,17 +482,16 @@ pub fn init_device(ctrl: &mut XhciController, port_idx: u8, protocol: Option<Pro
                             "xHCI: port {} is SuperSpeed and its link would not train, warm \
                              reset included (PORTSC {:#010x}); skipping it",
                             port_idx + 1, ctrl.read_portsc(port_idx).raw()),
-                        GaveUp::ResetFailed(k) => log!(
-                            "xHCI: port {} completed its {} reset without enabling \
+                        GaveUp::ResetFailed => log!(
+                            "xHCI: port {} completed its hot reset without enabling \
                              (PORTSC {:#010x}); skipping it",
                             port_idx + 1,
-                            match k { Reset::Hot => "hot", Reset::Warm => "warm" },
                             ctrl.read_portsc(port_idx).raw()),
-                        GaveUp::ResetNeverFinished(_) => {
+                        GaveUp::ResetNeverFinished => {
                             unreachable!("a completed reset cannot have never finished")
                         }
                     }
-                    return ctrl.port_bound(port_idx, None);
+                    return ctrl.ports[usize::from(port_idx)].gave_up(why);
                 }
             }
         }
@@ -509,7 +506,7 @@ pub fn init_device(ctrl: &mut XhciController, port_idx: u8, protocol: Option<Pro
         }
         log!("xHCI: port {} never finished its reset (PORTSC {:#010x}); skipping it",
             port_idx + 1, ctrl.read_portsc(port_idx).raw());
-        return ctrl.port_bound(port_idx, None);
+        return ctrl.ports[usize::from(port_idx)].gave_up(GaveUp::never_finished(kind));
     }
 }
 

src/redlist.rs

@@ -99,10 +99,6 @@ pub const DISABLED: &[Disabled] = &[
         test: "usb_transport_break",
         issue: "issues/kernel/a-held-disk-waits-for-a-pass-no-cpu-takes-when-every-cpu-is-in-a-call-on-it.md",
     },
-    Disabled {
-        test: "xhci_flap",
-        issue: "issues/hardware/a-collapsed-replug-is-enumerated-only-when-another-port-event-arrives.md",
-    },
 ];
 
 /// The row of `rows` that disables `test`, matched by the whole name.