netd: a listener's socket that left Listen is handed over or listens again, and a refused accept leaves its owner a wake

issues/design-debt/toyos-has-its-own-network-stack.md

@@ -16,6 +16,8 @@ netd runs smoltcp. Its replacement is ToyOS's own stack, built clean-room: reade
 - Stage 5: netd on one shard, pipe ABI unchanged; smoltcp leaves netd, `Cargo.toml` and `userland/Cargo.lock` in the same PR.
 - Then multi-core, netring (blocked on the owner's ABI ruling), TCP and IP hardening, IPv6, offloads, soak.
 
+The listener defects are this track's: `issues/hardware/a-handshake-nobody-finishes-holds-a-listeners-port-shut.md` and `issues/hardware/a-connect-between-two-accepts-is-reset.md`, on smoltcp until stage 5, and `issues/hardware/an-accept-that-never-reaches-netd-strands-its-listener.md`, in std's accept.
+
 Owed from the wire specification by stages 3–5: ETH-32, whose subnet broadcast needs the subnet `toyos-net-ip` holds; and those whose layer tags are `[ip]`, `[shell]` or `[udp]`: ETH-11, 12, 22–25, 33; ARP-16–18; IP-25, 26, 35; IPP-01–13; ICMP-32–46; IGMP-23–25, 29; UDP-17, 18; and the policy halves of ETH-10, 14, 19, IP-02, 20–23, 29, IPO-15, 16, ICMP-23, 24, 26 and UDP-22.
 
 What `toyos-net-wire` does not yet meet:

issues/hardware/a-handshake-nobody-finishes-holds-a-listeners-port-shut.md

@@ -0,0 +1,31 @@
+---
+status: open
+kind: defect
+opened: 2026-09-27
+---
+
+# A handshake nobody finishes holds a listener's port shut
+
+netd's listener is one smoltcp socket, and the port listens only while that
+socket is in `Listen` (`userland/netd/src/listen.rs`). A SYN whose sender never
+answers the SYN-ACK (a peer gone, a spoofed source) leaves it in `SynReceived`,
+and smoltcp 0.12 retransmits the SYN-ACK without end.
+
+Measured on smoltcp's interface over a wire played by hand, as
+`userland/netd/src/listen/tests.rs` plays it: after one SYN and 600 s of
+silence the socket was still `SynReceived`, having sent 72 SYN-ACKs, and
+another peer's SYN in that state was answered with a reset. With
+`set_timeout(10 s)` the socket went `Closed` at 10 s, which `listen::settle`
+turns back into `Listen`.
+
+So one packet shuts sshd's port for the rest of the boot. Nothing has chosen a
+bound on a listener's half-open handshake, and the timeout smoltcp offers also
+bounds the connection the socket becomes, so it would have to come off at the
+hand-over.
+
+**Owner**: whoever holds `issues/design-debt/toyos-has-its-own-network-stack.md`,
+which carries this and `issues/hardware/a-connect-between-two-accepts-is-reset.md`.
+
+**Exit**: a listener's half-open handshake let go within a bound, and a test
+that sends one SYN and nothing more, then finds the port answering the next
+peer with a SYN-ACK.

issues/hardware/an-accept-that-never-reaches-netd-strands-its-listener.md

@@ -0,0 +1,26 @@
+---
+status: open
+kind: defect
+opened: 2026-09-27
+---
+
+# An accept that never reaches netd strands its listener's owner
+
+std's `TcpListener::accept` (`rust/library/std/src/sys/net/connection/toyos.rs`)
+reads netd's wake byte first, and only then calls `toyos::net::tcp_accept`,
+which reaches netd (`NetdConn::connect`) and makes the data path
+(`DataPath::create`) before it sends the request. If either fails, or the send
+does, the accept returns an error with the wake spent and no request made. netd
+spends a wake only on an accept it answers (`userland/netd/src/listen.rs`), so
+it still counts the owner as woken, writes no second wake, and the owner's next
+`accept` blocks for the rest of the boot while the connection holds the port.
+
+Read from the code and not reproduced. `NetdConn::connect` fails with
+`ResourceExhausted` when the kernel's port queue refuses it, and
+`DataPath::create` with `Io` when a pipe cannot be made.
+
+**Owner**: whoever holds `issues/design-debt/toyos-has-its-own-network-stack.md`.
+
+**Exit**: no failure between the wake and netd's answer leaves the owner
+holding a spent wake netd still counts, and a test in which that step fails and
+the next `accept` still returns the waiting connection.

tests/toyos-rust-tests/src/bin/netd_refused_accept.rs

@@ -0,0 +1,129 @@
+//! An accept netd refuses for room still spends its owner's wake, so the
+//! connection it left is announced again once room returns, and not before.
+//!
+//! The host dials this program's listener through the forward. Woken, this
+//! program fills netd's connections to the host until one is refused, and
+//! asks for the connection; netd refuses it for room. Once a request netd
+//! answered after that refusal says room is still gone, no wake may be
+//! waiting. One connection closed, the next wake is the verdict.
+//!
+//! argv[1] is the port of the harness's host server on `HOST`, and the harness
+//! forwards a host port to this guest's `FORWARDED_PORT`.
+//! `netd_refused_accept: ok` is the only success line.
+
+#[path = "../netd_stream.rs"]
+mod netd_stream;
+
+use netd_stream::{ask, Ask, FORWARDED_PORT, HOST};
+use toyos::net::{
+    MsgType, NetError, NetdConn, TcpAcceptPipedRequest, TcpAcceptPipedResponse, TcpBound, TcpConnectPipedRequest,
+    TcpConnectResponse, TcpConnection, TcpSocketId, DATA_FROM_CLIENT, DATA_HANDLES, DATA_TO_CLIENT,
+};
+use toyos_abi::syscall::SyscallError;
+
+fn main() {
+    let port: u16 = std::env::args()
+        .nth(1)
+        .and_then(|p| p.parse().ok())
+        .expect("usage: netd_refused_accept <host port>");
+    let listener = toyos::net::tcp_bind([0; 4], FORWARDED_PORT).expect("bind the forwarded port");
+
+    let dial = toyos::net::tcp_connect(HOST, port, 0).expect("connect to the host server");
+    ask(&dial.tx, Ask::Dial);
+    wake(&listener, "the host's dial");
+    let mut held = Vec::new();
+    let refused = loop {
+        match connect(port) {
+            Ok(conn) => {
+                ask(&conn.tx, Ask::Held(0));
+                held.push(conn);
+            }
+            Err(e) => break e,
+        }
+    };
+    assert_eq!(refused, NetError::ResourceExhausted, "a connect after {} held", held.len());
+    assert_eq!(
+        accept(listener.socket_id),
+        Err(NetError::ResourceExhausted),
+        "an accept with every connection taken"
+    );
+    println!("netd_refused_accept: an accept refused for room, {} connections held", held.len());
+    assert_eq!(
+        connect(port).err(),
+        Some(NetError::ResourceExhausted),
+        "a connect after an accept refused for room"
+    );
+    let mut byte = [0u8; 1];
+    assert_eq!(
+        listener.notify.read_nonblock(&mut byte),
+        Err(SyscallError::WouldBlock),
+        "netd woke its owner for a connection there is no room to take"
+    );
+    end(held.pop().expect("the cap holds at least the connection before the refusal"));
+    wake(&listener, "the connection an accept refused for room left, once room returned");
+    accept(listener.socket_id).unwrap_or_else(|e| panic!("the connection an accept refused for room left: {e:?}"));
+    end(dial);
+    held.into_iter().for_each(end);
+    toyos::net::tcp_close(listener.socket_id).expect("close the listener");
+    println!("netd_refused_accept: ok");
+}
+
+/// Wait for netd's wake on `listener`, and take it.
+fn wake(listener: &TcpBound, what: &str) {
+    println!("netd_refused_accept: waiting for a wake for {what}");
+    let mut byte = [0u8; 1];
+    match listener.notify.read(&mut byte) {
+        Ok(1) => {}
+        Ok(_) => panic!("a wake for {what}: netd closed the listener"),
+        Err(e) => panic!("a wake for {what}: {e:?}"),
+    }
+}
+
+/// netd's answer to an accept on `listener`. An accepted connection is closed
+/// at once.
+fn accept(listener: TcpSocketId) -> Result<(), NetError> {
+    let (_rx, _tx, handles) = data_path();
+    let resp: TcpAcceptPipedResponse = reach_netd()
+        .request_with_handles(&handles, MsgType::TcpAcceptPiped, &TcpAcceptPipedRequest { socket_id: listener.0 })
+        .expect("netd takes the request")
+        .response()?;
+    toyos::net::tcp_close(TcpSocketId(resp.socket_id)).expect("close the accepted connection");
+    Ok(())
+}
+
+/// netd's answer to a connect to the host server.
+fn connect(port: u16) -> Result<TcpConnection, NetError> {
+    let (rx, tx, handles) = data_path();
+    let resp: TcpConnectResponse = reach_netd()
+        .request_with_handles(
+            &handles,
+            MsgType::TcpConnectPiped,
+            &TcpConnectPipedRequest { addr: HOST, port, _pad: 0, timeout_ms: 0 },
+        )
+        .expect("netd takes the request")
+        .response()?;
+    Ok(TcpConnection { rx, tx, socket_id: TcpSocketId(resp.socket_id), local_port: resp.local_port })
+}
+
+/// A connection to netd. **Refused only by the kernel's port queue**, which
+/// `toyos::net` spells as netd's own `ResourceExhausted`, so it is no answer
+/// of netd's here.
+fn reach_netd() -> NetdConn {
+    NetdConn::connect().unwrap_or_else(|e| panic!("reach netd: {e:?}"))
+}
+
+/// The ends of a duplex data path this side keeps, and the two it hands netd.
+fn data_path() -> (toyos::Pipe, toyos::Pipe, [toyos_abi::RawHandle; DATA_HANDLES]) {
+    let (rx, to_client) = toyos::pipe_pair().expect("the pipe netd writes into");
+    let (from_client, tx) = toyos::pipe_pair().expect("the pipe netd reads from");
+    let mut handles = [toyos_abi::HANDLE_INVALID; DATA_HANDLES];
+    handles[DATA_TO_CLIENT] = to_client.into_raw();
+    handles[DATA_FROM_CLIENT] = from_client.into_raw();
+    (rx, tx, handles)
+}
+
+fn end(conn: TcpConnection) {
+    let TcpConnection { rx, tx, socket_id, .. } = conn;
+    drop((rx, tx));
+    toyos::net::tcp_close(socket_id).expect("close a connection");
+}

tests/toyos.rs

@@ -307,15 +307,16 @@ const RUST_SKIP: &[&str] = &[
     "netd_listener_forgery",
     // Needs a NIC in front of netd and a host server behind it.
     // `netd_slow_reader`, `netd_held_open`, `netd_stalled_peer`,
-    // `netd_udp_refused`, `netd_udp_any_address` and `netd_refused_pipes` run
-    // them on `tests/netcase`, and `netd_lookup_let_go` on it with its frames
-    // held.
+    // `netd_udp_refused`, `netd_udp_any_address`, `netd_refused_pipes` and
+    // `netd_refused_accept` run them on `tests/netcase`, and
+    // `netd_lookup_let_go` on it with its frames held.
     "netd_slow_reader",
     "netd_held_open",
     "netd_stalled_peer",
     "netd_udp_refused",
     "netd_udp_any_address",
     "netd_refused_pipes",
+    "netd_refused_accept",
     "netd_lookup_let_go",
     // It asserts nothing at all: it holds a `tests/lancase` boot open for
     // twenty seconds so the host can reach this machine over the cable. On a
@@ -916,6 +917,10 @@ const MACHINE_TESTS: &[(&str, Sched, Tier)] = &[
     // round trip after each case, a named line per refusal and a clean
     // console; its clocks are liveness guards.
     ("netd_refused_pipes", Sched::Parallel, Tier::Fast),
+    // The netcase boot again: an accept netd refuses for room leaves its owner
+    // a wake for the connection it left, once room returns. The verdict
+    // is the guest's wake or its absence.
+    ("netd_refused_accept", Sched::Parallel, Tier::Fast),
     // The netcase boot again: bytes held back past a full pipe move on the
     // pipe's room alone, the peer holding the connection open and silent. The
     // verdict is the guest's byte-for-byte comparison; its clocks are
@@ -1683,6 +1688,7 @@ const CARRIES: &[(&str, &[&str])] = &[
     ("netd_listener_forgery", &["test_rs_netd_listener_forgery"]),
     ("netd_slow_reader", &["test_rs_netd_slow_reader"]),
     ("netd_refused_pipes", &["test_rs_netd_refused_pipes"]),
+    ("netd_refused_accept", &["test_rs_netd_refused_accept"]),
     ("netd_held_open", &["test_rs_netd_held_open"]),
     ("netd_stalled_peer", &["test_rs_netd_stalled_peer"]),
     ("netd_udp_refused", &["test_rs_netd_udp_refused"]),
@@ -10535,6 +10541,22 @@ fn netd_refused_pipes(rust_bins: &[(String, Vec<u8>)]) -> Result<(), String> {
     Ok(())
 }
 
+/// An accept netd refuses for room leaves its owner a wake for the connection
+/// it left: the guest's wakes are the verdict. This side carries that netd named the
+/// refusal for room and that no program panicked.
+fn netd_refused_accept(rust_bins: &[(String, Vec<u8>)]) -> Result<(), String> {
+    let HostRun { result, console, .. } = netcase_against_host(rust_bins, "netd_refused_accept", true, "")?;
+    if !result.stdout.lines().any(|l| l.trim_end().ends_with("netd_refused_accept: ok")) {
+        return Err(format!("the guest never said it was done:\n{}", result.stdout));
+    }
+    if !console.contains("netd: refusing accept, ") {
+        return Err(format!("netd refused an accept for room without saying so:\n{console}"));
+    }
+    serial::Serial::named("boot console", console.as_str()).must_be_clean()?;
+    eprintln!("  [netcase] an accept refused for room left a wake once room returned");
+    Ok(())
+}
+
 /// Ctrl+Alt+D at a live desktop: every CPU answers, and the two halves of the
 /// report agree.
 ///
@@ -16065,6 +16087,7 @@ fn run_machine_test(
         }
         "netd_slow_reader" => netd_slow_reader(rust_bins),
         "netd_refused_pipes" => netd_refused_pipes(rust_bins),
+        "netd_refused_accept" => netd_refused_accept(rust_bins),
         "netd_held_open" => netd_held_open(rust_bins),
         "netd_stalled_peer" => netd_stalled_peer(rust_bins),
         "netd_udp_refused" => netd_udp_refused(rust_bins),

userland/netd/src/listen.rs

@@ -0,0 +1,88 @@
+//! Whether a listener's owner is owed a wake, and what its accept finds.
+//!
+//! **A listener is one smoltcp socket that becomes the connection it
+//! accepts**, so the port listens only while that socket is in `Listen`: one
+//! that left it is handed to its owner or listens again, or the port answers
+//! every other peer with a reset for the rest of the boot.
+//!
+//! **An accept spends the owner's wake whatever it answers, a refusal
+//! included, and a wake is owed only for a connection there is room to
+//! take.** An owner refused holds no wake, so the connection it left is
+//! announced again, and an owner refused for room is not woken until room
+//! returns.
+
+use smoltcp::socket::tcp;
+
+/// A listener's port, and whether its owner holds a wake it has not spent on
+/// an accept.
+pub struct Listening {
+    port: u16,
+    woken: bool,
+}
+
+/// What an accept finds, handed the pipes `P` its request carried.
+#[derive(Debug, PartialEq, Eq)]
+pub enum Accept<P> {
+    /// A connection, to hand over on the pipes.
+    Take(P),
+    /// A request that carried no pipes, whatever waits.
+    NoPipes,
+    /// A connection, and no room to take it.
+    NoRoom,
+    /// No connection.
+    Nothing,
+}
+
+impl Listening {
+    pub fn new(port: u16) -> Self {
+        Self { port, woken: false }
+    }
+
+    pub fn port(&self) -> u16 {
+        self.port
+    }
+
+    /// The bytes to write the owner for `socket`: one wake if a connection
+    /// waits, there is `room` to take it, and the owner holds no wake, and
+    /// none otherwise. The wake is held from here on, so the caller ends the
+    /// listener if the owner is not handed it.
+    pub fn wake(&mut self, socket: &mut tcp::Socket, room: bool) -> &'static [u8] {
+        let owed = settle(socket, self.port) && room && !self.woken;
+        self.woken |= owed;
+        if owed { &[1] } else { &[] }
+    }
+
+    /// An accept, with `room` for another connection or not, and the pipes
+    /// its request carried.
+    pub fn accept<P>(&mut self, socket: &mut tcp::Socket, room: bool, pipes: Option<P>) -> Accept<P> {
+        self.woken = false;
+        match (settle(socket, self.port), room, pipes) {
+            (_, _, None) => Accept::NoPipes,
+            (false, _, Some(_)) => Accept::Nothing,
+            (true, false, Some(_)) => Accept::NoRoom,
+            (true, true, Some(pipes)) => Accept::Take(pipes),
+        }
+    }
+}
+
+/// Puts `socket` back to listening on `port` if its peer reset it before its
+/// owner took it, and says whether it holds a connection: a handshake
+/// finished, whatever the peer did since. Its FIN included, which can land in
+/// the same pass as the handshake's last ACK, so no pass ever sees the socket
+/// `Established`.
+fn settle(socket: &mut tcp::Socket, port: u16) -> bool {
+    match socket.state() {
+        tcp::State::Listen | tcp::State::SynReceived => false,
+        tcp::State::Established | tcp::State::CloseWait => true,
+        tcp::State::Closed => {
+            socket
+                .listen(port)
+                .unwrap_or_else(|e| panic!("netd: a closed socket refused to listen on {port}: {e:?}"));
+            false
+        }
+        other => panic!("netd: a listener's socket is {other:?}, which only netd closing or connecting it reaches"),
+    }
+}
+
+#[cfg(test)]
+mod tests;