Skip to content

toyos-transport: the one transport core; blockring's rings are rebuilt on it - #552

Merged
Japabu merged 12 commits into
mainfrom
wt/toyos-transport
Sep 27, 2026
Merged

Japabu merged 12 commits into
mainfrom
wt/toyos-transport

Conversation

@Japabu

@Japabu Japabu commented Sep 27, 2026 •

Copy link
Copy Markdown
Collaborator

Stage T1 of the unified IPC transport: the pure core every client/server session will run over, with blockring and blockd converted onto it. No ABI change.

What changed, per decision

toyos-transport, a new crate. It is no_std, has no alloc, is forbid(unsafe_code), and depends only on toyos-untrusted.

  • Word, the port: load and store.
  • Producer/Consumer<E, D, N>: SPSC queues of E-word entries, D deep, among a region's N words.
    • Place<E, D, N> says where one queue is. It is made only by Place::new::<HEAD, TAIL, ENTRIES>(), whose inline const refuses any word at or past N. A place outside its words therefore does not compile (E0080), and each end's new cannot fail. publish answers bool and release answers nothing.
    • The one index into the words is word, under that invariant. The crate forbids the arithmetic, unwrap, expect and panic lints and denies indexing_slicing and as_conversions. The allows are at word, at the ring slot's u32→usize, and at the place check's D as usize.
    • pop answers [Untrusted<u32>; E], and push answers Ok(false) on a full queue.
    • A tail is bounded against what this end released, and a head against what it published. Anything past that is TailPastDepth or HeadPastTail.
    • Each end looks at the peer's cursor only when what it last saw is spent.
  • Geometry/Run: the arena, declared once. Run::decode bounds a peer's two words, and Geometry::run cuts this side's. A run's Debug is its slots; the span follows from them.
  • Inflight<T, D>: a tag is a slot index under that slot's own sequence. The index takes the bits the table needs (6 for 64 slots) and the sequence the rest (26). An answered, ended or replayed tag is Violation::Tag, and end answers every tag in flight once. tags() walks the tags in flight, and Debug shows only filled slots.

blockring on the core.

  • layout::ARENA is the block geometry, and SESSION_BYTES derives from Geometry::BYTES.
  • layout::REQUESTS and COMPLETIONS are checked Place constants, so layout::client and layout::server cannot fail.
  • A request's run and lba belong to its op: Op::Read { run, lba } | Op::Write { run, lba } | Op::Flush. A read or write without a run, and a flush with an lba, cannot be written.
  • Client<D>'s wire is an Inflight as deep as the session's rings (D defaults to DEPTH). on_the_wire() gives its tags.
  • ServerSession keeps its requests in the order it took them, and abort_all answers a reset in that order. Nothing in the protocol compares a tag except for equality, which is what makes the model's tag renaming exact (below). inflight() gives (tag, op) in take order.

blockd ends that session when a completion finds no room, or when a head cannot be believed. Both cases stay one arm at main.rs:281. Before this, the old push asserted and blockd panicked. Every expect left in blockd is one main has, message for message. Region::words is &[AtomicU32; RING_WORDS].

The block model (toyos-blockring/src/model.rs) runs the transport's rings beside its VecDeque queues and holds them to each other in both directions:

  • every entry the ring gives equals the queue's;
  • a ring whose queue is empty pops nothing.

next is the one function for every event the search can take. It names each event and gives the world after it, or the law it broke. dfs walks it, and a test walks named paths through the same function.

A state's key renames tags by first appearance, the client's tags first and in slot order, and renders the client's table by its filled slots. states_named_alike_act_alike stages the pair that a key ordering tags by value merges. Both states have F2 on the device and slot 1 free:

  • in one, W1 was asked after W0 was answered;
  • in the other, W0 and W1 were in flight together.

The test demands that the two are named alike, and that they are still named alike after W3 is taken and the device reset.

Its rings are 4 deep and its client's table is too. With no failure, both rings reach 4 entries (the_model_reaches_the_end_it_should), and one session carries all 5 of SCRIPT's requests, so each ring wraps. A const assertion keeps SCRIPT longer than the depth. The bounds are main's.

Gates, each the command's own exit code, at 0a117e1

Gate Exit
cargo test --workspace --exclude toyos-build (transport: 10 unit, 3 loom, 4 doc; blockring: 19) 0
cargo test --lib (379 passed, 1 ignored; also at 87bef93) 0
cargo run -- --clippy (10 invocations clean) 0
cargo check --target aarch64-unknown-none-softfloat, transport and blockring 0, 0
cargo test --test toyos-build -- --nightly blockd_survives_its_death, run by the orchestrator (PASS, 60s) 0
cargo test --test toyos-build -- --nightly blockd_serves_partitions, run by the orchestrator (PASS, 49s) 0

The blockd fix: blockd_io hostile-head

This role runs first in blockd_survives_its_death. blockd starts with --silence-write 1, and the client puts one Request::encoded write on the ring by hand. After blockd says WITHHELD, the client stores CQ_HEAD := CQ_TAIL − DEPTH and rings. The role then waits for blockd to say the session closed after, and writes, flushes and reads back through a new session.

Measured at b69deab:

Arm Run exit What reds it
main's toyos-blockring and userland/blockd (plus the new case) 1 panicked at toyos-blockring/src/ring.rs:93:9: a push into a full ring
main.rs:281 patched to panic! instead of ending the session 1 panicked at blockd/src/main.rs:281:35
head 0

Negative controls at 0a117e1, each the CONTROLS command run by hand

Control Exit Verdict line
publish-relaxed (loom) 101 a_published_entry_is_read_whole ... FAILED
no-clamp (loom) 101 a_hostile_producer_yields_entries_or_a_violation ... FAILED
end-keeps-inflight 101 inflight::tests::an_end_answers_every_tag_once_and_a_late_completion_nothing ... FAILED
mutate-session-end-forgets 101 every_request_is_answered_exactly_once ... FAILED: "ticket 3 ended answered None"
mutate-abort-keeps-inflight 101 every_request_is_answered_exactly_once ... FAILED: "the client met a second completion for tag 7"
mutate-no-reissue-after-loss 101 what_a_flush_calls_durable_is_on_the_medium ... FAILED: "flush 4 was answered durable with block 0 holding 1"

Mutations at 0a117e1

Each is a checked patch, shown to build (exit 0), run, and restored, and the tree is clean after.

  • ServerSession on a BTreeMap keyed by tag, answering a reset in key order: -p toyos-blockring --lib gives 101.
    • a_reset_answers_in_the_order_taken gets [4, 9].
    • states_named_alike_act_alike passes its two assertions before the reset and fails at the one after it: renamed, the completion queues come out [1, 0] and [0, 1].
    • The other 17 tests pass. The model's end states under this patch are 494, 461 and 1311 (165 given up), where take order gives 492, 460 and 1300 (162).
  • queue.rs:181, self.ready = self.ready.wrapping_sub(1); deleted:
    • -p toyos-blockring --lib gives 101, and all four model tests fail with "the request ring gave what the queue does not hold";
    • -p toyos-transport --lib gives 101, with three queue tests failing.
  • The tail bounded against local instead of released: 101. a_tail_stepped_past_each_pop_is_refused_at_the_depth fails with "a published entry was not taken".
  • The head bounded against local instead of published: 101. a_head_stepped_past_each_push_is_refused_at_the_depth fails with "a ring with nothing published was full", and the_ring_wraps_and_counts_its_space fails too.
  • Inflight::INDEX_BITS set to 16: 101. a_slot_refilled_past_sixteen_bits_refuses_its_first_tag answers the first tag Ok(()).
  • A place past its words: each of the three bad places in Place's doctests was compiled standalone against this crate. The good place builds (0). Each bad one exits 101 with error[E0080]: evaluation panicked: a place names a word past its words. The entries case also exits 101 under cargo check.
    • rustdoc on stable 1.98 does not check the code in compile_fail,E0080: E0277 in its place stays green.
    • So the standalone compiles are what prove the code, and the compiling twin beside the doctests guards against a typo.

Oracles

  • loom's C11 model judges the publication ordering.
  • blockring's model holds the transport's rings to VecDeque queues in both directions: what the ring gives, and that it gives nothing when the queue is empty. The rings are 4 deep, so they fill and wrap within the model's bounds.
  • rustc's const evaluator refuses a place outside its words.
  • The block-fix arms ran on QEMU's NVMe.

Model time, main against the head

Main is c551894's blockring and blockhold, built in a scratch tree with the host workspace's [profile.dev] opt-level = 2. Each test binary was run by exact test name, interleaved main then head, three rounds on the same loaded host. User seconds, with real seconds in brackets for the two long tests:

Test main head
every_request_is_answered_exactly_once 5.20, 5.61, 4.95 (6.99, 10.26, 5.61) 4.99, 5.15, 4.78 (6.26, 7.08, 4.96)
what_a_flush_calls_durable_is_on_the_medium 5.42, 5.36, 5.52 (7.65, 6.23, 8.28) 4.82, 5.04, 4.70 (5.50, 6.36, 4.81)
the_model_reaches_the_end_it_should 0.01, 0.01, 0.01 0.01, 0.01, 0.01
the_model_reaches_a_write_given_up 0.03, 0.03, 0.03 0.03, 0.03, 0.03
states_named_alike_act_alike — 0.00, 0.00, 0.00
control mutate-session-end-forgets (whole lib) 8.65, 8.03, 8.40 8.36, 8.15, 7.74
control mutate-abort-keeps-inflight (whole lib) 0.46, 0.46, 0.51 0.65, 0.64, 0.56
control mutate-no-reissue-after-loss (whole lib) 0.46, 0.46, 0.51 0.63, 0.65, 0.56

End states per bound, main against head:

(resets, crashes, errors) main head
0,0,0 28 28
1,0,0 606 492
0,1,0 329 244
0,0,1 260 236
1,1,0 826 460
2,2,0 505 363
0,1,5 481 (45 given up) 373 (32 given up)
1,1,4 1643 (237 given up) 1300 (162 given up)

Main's key carried the client's next_tag, a count of every request ever issued, and the head's does not.

Lines

These are against the merge base c551894 and exclude lockfiles. model.rs, loom.rs, tests/ and each file from its #[cfg(test)] on count as tests.

  • Production: net +338. That includes the two issue files (42 lines) and Place's 22-line doctest pair.
  • Tests: net +526.
  • git diff --shortstat c5518949...1a3fc019: 28 files, +1629/−733.

Issues filed

  • issues/design-debt/the-transport-heads-orderings-have-no-oracle.md
  • issues/filesystem/a-read-or-write-answered-lost-is-never-answered.md. This is pre-existing on main and held by the orchestrator.

🤖 Generated with Claude Code

Japabu and others added 2 commits September 27, 2026 19:04
…t on it

Stage T1 of the unified transport. A new pure crate, `toyos-transport`
(no_std, no alloc, forbid(unsafe_code), rustc-dep-of-std, the domain lint
line, one dependency: toyos-untrusted), holds what every client/server
session will run over:

- `Word`, the port: load, store, and a SeqCst fence in the word's own
  memory model, so loom sees the fence.
- `Producer`/`Consumer<E, D>`: SPSC queues of E-word entries. Entries come
  out as `[Untrusted<u32>; E]`; a peer's cursor is bounded against what this
  end released or published (`Violation::TailPastDepth`/`HeadPastTail`), and
  one moved back within bounds costs only its owner. Each end re-reads the
  peer's cursor only when what it saw is spent.
- `StreamTx`/`StreamRx`: byte rings that answer `Span`s and never touch a
  byte, with a producer-written `end` word stored after the tail and loaded
  before it.
- `Wake`/`before_sleep`: the sleeper stores `sleep`, fences, looks again;
  the publisher stores its tail, fences, loads `sleep`, and answers
  `Wake::Peer` only then.
- `Geometry::decode`, `Run::decode`, and `Own`/`Lent`/`Held`, none Clone.
- `Inflight<T, D>`: a tag is a slot index under the slot's own sequence, so
  an answered, ended or replayed tag is `Violation::Tag`; `end` answers every
  tag in flight, once, eagerly.
- `Schema<SQE, CQE>`: LAYOUT, decode_request, decode_reply.

blockring's `ring.rs` is deleted. Its places, `client`/`server` and the
ring types live in `layout.rs` over the core; a consumer's `sleep` word
takes the free word after its head (1 and 33), the page otherwise as it
was. `Request::decode`/`Completion::decode` decode `Untrusted` words, and
`entry::Block` is the protocol as a schema. `ServerSession::take_entry`
takes the popped words; the model's `take` wraps its raw words as a peer's,
so `src/model.rs` is unchanged byte for byte and its end-state counts match
main's for every bound.

blockd compiles against the rebuilt rings with its doorbells as they were:
it rings on any publish, whatever the wake says. One difference: a
completion that finds no room, or a client head past what was posted, now
ends that session instead of panicking blockd, whose old `push` asserted.

The loom publication check moves into the core (`tests/loom.rs`) beside a
lost-wake model (futex as park/unpark) and two hostile-peer models; an
exhaustive session model (crash, reconnect, replayed tags) holds Inflight
and the rings together. Four mutation features, each a CONTROLS row that
reds: publish-relaxed, no-sleep-fence, no-clamp, end-keeps-inflight;
blockring's mutate-ring-publish-relaxed goes with the file it reverted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu
Japabu marked this pull request as ready for review September 27, 2026 17:08
@Japabu

Japabu commented Sep 27, 2026

Copy link
Copy Markdown
Collaborator Author

Review r1 at d56fc31

CI host run 36335828317 finished success at d56fc31. The four transport controls reached their verdicts in that run's log: publish-relaxed, no-sleep-fence, no-clamp and end-keeps-inflight. No guest ran, because ci.yml boots none.

Lines: git diff --shortstat origin/main...HEAD gives 24 files, +1803/−417. The core's production code is 761 lines, the src of each file before its test module; that agrees with the body's production +955/−270 and tests +811/−146.

BLOCKER

  • The stream loses the close wake. At toyos-transport/src/stream.rs:114, close with no new bytes leaves the tail unchanged, but before_sleep at :183 makes that tail the futex word. So close → Wake::Peer → futex_wake can land before the reader's futex_wait(tail, v). The tail still equals v, and the reader sleeps for good. tests/loom.rs cannot see this: loom's unpark leaves a token, which a futex does not, and no loom model touches the stream at all. Stream* has no caller before T6 and T9 (design §3 and §8), so delete stream.rs, End, StreamPlace and Violation::End (−283 lines). The stream lands with its first user and a model that is faithful to the futex.
  • The tighter clamps have no test that can fail (PR "unsure 2"). Apply this in toyos-transport/src/queue.rs:117-119: published(page, self.local, D)?, self.ready = published, Ok(self.local.wrapping_add(published)). Also change :61-63 to unreleased(page, self.local, D) without pending. Every test in queue.rs and tests/loom.rs stays green, because the hostile producer only ever stores tails D+1 and u32::MAX at local == released == 0. a_hostile_producer_yields_entries_or_a_violation needs a producer that steps its tail one past each pop while the consumer never releases, and it must go red under that patch (taken > D). Add the same for the head against published.
  • The blockd fix has no test, and blockd never ran. userland/blockd/src/main.rs:284 ends a session instead of panicking. No test turns red with Ok(false) | Err(_) => panic!() there, and on origin/main nothing shows the panic. The body shows only cargo check for a device driver; CI boots no guest, and blockd_* are Nightly. Two things are needed:
    • blockd_serves_partitions and blockd_survives_its_death at this head, with exit codes and logs.
    • A blockd_io case where a client, with a request on the device, stores CQ_HEAD := CQ_TAIL − DEPTH. The case asserts that blockd serves the next session. It must go red on origin/main and red under the panic!() patch.
  • Second copies of rules blockring still owns, and API with no caller:
    • The arena is declared twice: once in layout::ARENA_BLOCKS/arena_byte and the bound at toyos-blockring/src/entry.rs:89-90, and again in Geometry/Run::decode/Run::span. the_schemas_arena_is_the_sessions (entry.rs:191) exists only to hold the two equal.
    • The tag table is declared twice: Inflight against Client.wire/next_tag/session_ended (client.rs:129,206,299). Inflight cannot host that client as written, because the client iterates its in-flight entries (client.rs:167,185) and Inflight has no way to do that.
    • Schema::decode_request/decode_reply and entry::Block have no caller at all.
    • Fix: either convert blockring onto the core here, or move arena.rs, inflight.rs, model.rs, Schema, entry::Block and Violation::{Entry,Run,Tag} to T2, where the design converts blockd and its client.

NOTE

  • src/ci.rs:376 — the no-sleep-fence verdict is a generic loom deadlock line, not a test name, so any deadlock in tests/loom.rs satisfies it. The control also removes both fences at once (lib.rs:175,186); removing one side alone is unmeasured.
  • toyos-transport/src/queue.rs:146, lib.rs:164 — nothing tests the head edge's Release/Acquire. The loom README lists load buffering as unmodelled, so loom is the oracle for the tail edge and the fences only. This gap already existed in ring.rs.
  • userland/blockd/src/main.rs:367, session.rs:237 — every publish now pays a SeqCst fence and a load, and its Wake is discarded. No client calls before_sleep, so the answer is always Busy. This is unmeasured: measure it, or record it in issues/ with T2 as the exit.
  • toyos-blockring/src/server.rs:94-99 — the #[cfg(test)] take shim exists only to keep model.rs byte-identical. The differential is recorded, so delete the shim and rename take_entry back.
  • toyos-blockring/src/entry.rs:100-101 — opaque launders with map_or(0, …) and an as cast, a default on a branch that cannot run.
  • toyos-transport/src/arena.rs:49,117, inflight.rs:11 — three compromises are named only in PR prose, not in issues/:
    • Geometry::own mints any Own, so being not Clone protects nothing.
    • Lent::back needs no completion.
    • The 16-bit sequence wraps, although D ≤ 64 needs only 6 index bits.
  • toyos-blockring/src/model.rs — it still models the rings as VecDeque, a second copy of the queue's semantics. The core's model.rs shows the real Producer/Consumer run single-threaded over Cell words.
  • aarch64 is not this branch's doing. The shared toolchain's rust/build/aarch64-apple-darwin/stage2/lib/rustlib holds only x86_64-unknown-{toyos,none,uefi}, although GUEST_TARGETS (src/toolchain.rs:88-95) names aarch64. issues/kernel/toyos-runs-on-arm64.md stage 1 tracks it; its exit is cargo +toyos build --target aarch64-unknown-toyos. The cheap check left unrun is cargo check -p toyos-transport --target aarch64-unknown-none.

REMOVE

  • toyos-transport/tests/loom.rs:10 — "every word it can reach, at every step": the hostile peer runs a fixed script of five stores.
  • PR body, "Models in the core" — "store to every word they can reach": the same falsehood.
  • PR body, "Oracle" — the sentence that makes blockring's model the oracle. It never reaches the queue, as the body's own "unsure 1" says.
  • toyos-untrusted/Cargo.toml:2, toyos-untrusted/src/lib.rs:42 — "no dependencies" is false once the optional core dependency lands.
  • src/ci.rs:371-372 — a second copy of the four feature comments in toyos-transport/Cargo.toml.
  • toyos-transport/src/arena.rs:3-8, lib.rs:7 — "none of them Clone" and "lending consumes" claim a uniqueness that Geometry::own does not give.

SEND BACK

Japabu and others added 3 commits September 27, 2026 19:24
…nly what it has a user for

Blockers of review r1:

- The stream is deleted (stream.rs, End, StreamPlace, Violation::End): nothing
  used it before T6/T9, and its close left the futex word unchanged, a lost
  wake. That defect is recorded for T6 in the transport design, not the tree.
- The clamps have tests that can fail: a producer that steps its tail one
  past each pop with nothing released, and a consumer that steps its head onto
  each unpublished push. Each goes red under the reviewer's mutation (tail
  bounded against `local`, head bounded against `local` without `pending`).
- blockd_io gains `hostile-head`: with a write on the device, the client moves
  CQ_HEAD to CQ_TAIL - DEPTH; blockd must end that session and serve the next.
  blockd_survives_its_death runs it first.
- The conversion is finished here rather than in T2:
  - The arena is declared once, as `Geometry`/`Run`. `layout::ARENA` is the
    block geometry; SESSION_BYTES derives from `Geometry::BYTES`;
    ARENA_BLOCKS, ARENA_OFFSET and arena_byte are gone. A request carries the
    `Run` its words decode to, so blockd's device address and the client's
    arena window come from its span, and region.rs's own bound check goes.
  - The tag table is declared once: the client's wire is an `Inflight`, which
    gains `values()` for the two places the client looks across what is in
    flight. next_tag, the BTreeMap and the client's own Violation go; a tag
    table that is full holds the next request back.
  - Schema, entry::Block, Layout and Geometry::decode had no caller and go.
  - Own/Lent/Held protected nothing (anyone could mint an Own, and Lent::back
    took no completion) and go; `Run::decode` answers a bounded `Run`.
  - The test-only `take` shim goes and `take_entry` is `take` again.
  - blockring's model keeps its VecDeque queues as a reference and drives the
    session page's own rings beside them, holding every entry either end takes
    equal to the queue's. A pop that reads the wrong slot reds all four model
    tests.

Notes:

- The no-sleep-fence control is split: no-wake-fence takes the producer's
  fence away and no-sleep-fence the consumer's, each red on its own. The wake
  model no longer parks; the consumer reports whether it slept, so the verdict
  is the test's FAILED line and not loom's deadlock.
- `opaque` fails fast instead of defaulting to 0 on a branch that cannot run.
- The 16-bit tag sequence's wrap is stated at the site as the invariant it is:
  a replay a wrap later answers what a server could answer by name anyway.
- rustc-dep-of-std leaves the transport and toyos-untrusted: nothing builds
  either under std before T2, which adds it with its user.
- Two issues filed: a block publish pays a fence and a load for a wake nobody
  asks for, and the head's Release/Acquire has no oracle.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The panic arm showed the race: the client saw its connection close while
`try_wait` still found blockd running, so the role went on to open a session
nobody would answer and ended by the host's timeout instead of by name. The
supervisor now hands every line blockd says to the role, which waits for
`WITHHELD` and then for the session's `closed after`; a blockd that ends first
disconnects the channel and the role fails saying so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu

Japabu commented Sep 27, 2026

Copy link
Copy Markdown
Collaborator Author

Review r2 at b69deab

CI host run 36340643834 finished success at b69deab. Its log reaches every transport control's own verdict:

  • publish-relaxed: a_published_entry_is_read_whole ... FAILED.
  • no-wake-fence and no-sleep-fence: each a_publish_and_a_sleep_cannot_both_miss ... FAILED.
  • no-clamp: a_hostile_producer_yields_entries_or_a_violation ... FAILED, "took 3 entries from a ring of 2".
  • end-keeps-inflight: every_tag_is_answered_exactly_once ... FAILED.

The guest arms are the body's: blockd_io hostile-head is red on main, red under the panic patch and green at the head, and both nightly blockd suites exit 0.

Lines: git diff --shortstat origin/main...HEAD gives 29 files, +1807/−599. Less 32 lockfile lines, that is the body's production +403 and tests +773.

Round-1 BLOCKERs

  • The stream loses the close wake: CLOSED. stream.rs, End, StreamPlace and Violation::End are gone from the tree.
  • The tighter clamps have no test that can fail: CLOSED. queue.rs:250 and :270 step a cursor one past each pop and each push. The body's two mutations give 101: "took 9 entries from a ring of 8" and "pushed 9 into a ring of 8".
  • The blockd fix has no test: CLOSED. hostile-head is red on main (ring.rs:93) and under main.rs:281 → panic!, and green at the head. blockd_serves_partitions and blockd_survives_its_death exit 0 at b69deab.
  • Second copies and API with no caller: CLOSED. layout::ARENA is the only arena and Client.wire is an Inflight. Schema, entry::Block, Own/Lent/Held, the take shim and rustc-dep-of-std are deleted.

Round-1 NOTEs:

  • The fence controls are split and name their tests: closed.
  • The head orderings are recorded in issues/design-debt/the-transport-heads-orderings-have-no-oracle.md.
  • The wake's cost is recorded; see the wake NOTE below.
  • The shim is deleted.
  • opaque is now an expect on a branch that cannot run.
  • The wrap is stated at inflight.rs:7-10.
  • aarch64-unknown-none-softfloat checks with exit 0.

All six round-1 REMOVEs are gone.

BLOCKER

None.

NOTE

  • toyos-blockring/src/model.rs — the model takes 91.3 s in this run (18:32:16→18:33:47) against 13.1 s in run 36340721499, whose blockring is main's, in the same window. mutate-session-end-forgets went from 8.4 s to 88.3 s. The new states are the same histories under other tag names: Client.wire holds per-slot sequences, and main's one next_tag never split on them. Every key also renders all 64 slots. The protocol only compares tags for equality, and a fresh tag never equals one already present, so the fix is in the key, not the bounds: relabel tags by first appearance and print only occupied slots, keep main's bounds, and measure the time again.
  • toyos-blockring/src/model.rs:446,497 — the differential checks one direction only. take and read run only while the VecDeque is non-empty, so an extra entry from the ring is never seen. Patch toyos-transport/src/queue.rs:139 by deleting self.ready = self.ready.wrapping_sub(1);: all four model tests stay green, and only an_entry_is_nobodys_before_it_is_published goes red. Assert that the ring pops None whenever the reference is empty. The mutated-pop arm proves the queue→ring comparison can fail. The model still never wraps or fills the 64-deep ring.
  • toyos-transport/src/model.rs — 328 lines of a second session model beside blockring's, which now runs the same Inflight, Producer and Consumer through crash and reconnect. end-keeps-inflight already turns inflight::tests::an_end_answers_every_tag_once_and_a_late_completion_nothing red in run 36340643834. Delete the model and point the control at that test.
  • toyos-transport/src/lib.rs:88-104,148-169, queue.rs:101-102,125-128,152-168, toyos-blockring/src/layout.rs:32,35 — the wake has no production caller. Nobody calls before_sleep, and both publish sites throw the Wake away.
    • Deleting it removes about 95 production lines (with issues/design-debt/a-block-publish-pays-for-a-wake-nobody-asks-for.md), about 49 test lines and two controls. It also takes a SeqCst fence and a load off every blockd publish.
    • The wake then lands in T2 with its first caller. That issue file is load-bearing only while the dead wake stays; the head-orderings issue is load-bearing.
  • toyos-transport/src/inflight.rs:14 — 16 index bits for a table of 64. Insertion takes the first free slot, so slot 0 takes most inserts and its sequence wraps every 65 536. A buggy server's duplicate answer (the mutate-abort-keeps-inflight class) then answers the current request instead of being refused.
    • A hostile peer gains nothing from the wrap. The client's table answers only the server, and the server already holds every current tag. blockd's own table in ServerSession is keyed by the client's raw tag and has no sequence.
    • Fix: size the index to D, so 6 bits, which leaves the sequence 26.
  • userland/blockd/src/main.rs:281 — only the Ok(false) half is measured. Split the arm into Ok(false) => closing and Err(_) => panic!() and every test stays green. The Err half is reachable: a client moves its head back, so pull caches room 0, then moves it past the tail before a completion. I found no deterministic guest path to test it, so keep the two halves in one arm.
  • toyos-blockring/src/entry.rs:42, client.rs:208, userland/blockd/src/main.rs:339 — run: Option<Run> beside op makes a write without a run representable, and expects guard that. Op::Read(Run) | Op::Write(Run) | Op::Flush makes it unrepresentable.
  • userland/blockd/src/main.rs:355,365, session.rs:238,305 — each call after new can still return Violation::Region for a place new already checked, so blockd carries four expects that cannot fire. Ends built over a &[W; N] fixed at construction would make these calls infallible.
  • tests/toyos-rust-tests/src/bin/blockd_io.rs:552 — the request words are encoded by hand. The one encoder is Request { op: Op::Write, tag: 1, lba: 0, run: ARENA.run(0, 1) }.encode().
  • toyos-blockring/src/client.rs:221,232 — pre-existing on main. A user read or write answered Lost is taken off the wire and then refused as Violation::Entry, so session_ended never answers it. Its ticket waits forever in blockd's pending, against the module's "answered exactly once". File it.

REMOVE

  • toyos-blockring/Cargo.toml:41 — "and the schema the protocol is to them": Schema is deleted.
  • tests/toyos-rust-tests/src/bin/blockd_io.rs:102-103 — "past blockd's ten seconds of silence" restates blockd's COMMAND_SILENCE.
  • PR body, "Oracles" — "so it is a differential oracle for the production queue": it checks one direction, and the ring is never wrapped or filled.

LAND AFTER NAMED CHANGES

Japabu and others added 3 commits September 27, 2026 20:58
… back to main's speed, held to its queues both ways

The wake had no caller: nobody called `before_sleep`, and both publish
sites threw the `Wake` away. `Wake`, `Asleep`, the `sleep` words and
their fences, `Word::fence`, the two fence controls, their loom test and
issues/design-debt/a-block-publish-pays-for-a-wake-nobody-asks-for.md are
deleted; the wake lands in T2 with its first caller.

toyos-transport/src/model.rs was a second session model beside
blockring's, which runs the same `Inflight`, `Producer` and `Consumer`
through crash and reconnect. It is deleted, and `end-keeps-inflight`
names `inflight::tests::an_end_answers_every_tag_once_and_a_late_completion_nothing`.

Ring ends are made over `&[W; N]`: `new` holds the place to `N`, so
`publish` and `release` cannot fail, and blockd's four `expect`s on them
go. The one index into the words is `word`, under the invariant `new`
checked.

A request's run is its op's: `Op::Read(Run) | Op::Write(Run) | Op::Flush`.
`run: Option<Run>` and the `expect`s that guarded it go.

A tag's index takes the bits the table needs (6 for 64 slots) and the
sequence the rest (26), so one slot's sequence no longer wraps every
2^16 fills.

The block model: its state key names tags by first appearance and
renders the client's table by its filled slots; its rings are four
deep, so each fills and wraps within the bounds; the client's table is
as deep as the rings; and a ring whose queue is empty must pop nothing.
Deleting `Consumer::pop`'s `ready` decrement now reds all four model
tests. The bounds are main's.

blockd_io's hostile request is `Request::encode`d.

Filed issues/filesystem/a-read-or-write-answered-lost-is-never-answered.md,
pre-existing on main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s page inline

A run's span follows from its first slot and count under the one
geometry, so `Run`'s Debug prints only those. The block model keys every
state by a rendering full of runs, and the shorter one takes its two law
tests from about 6.2 s to about 5.5 s of user time on this host.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu

Japabu commented Sep 27, 2026

Copy link
Copy Markdown
Collaborator Author

Review r3 at 64d7149

CI host run 36345209293 at 64d7149: success. In its log each transport and blockring control reaches its own FAILED line: publish-relaxed, no-clamp, end-keeps-inflight, mutate-session-end-forgets, mutate-abort-keeps-inflight and mutate-no-reissue-after-loss. The nightly blockd suites passed at 64d7149: qemu-final-death.log PASS blockd_survives_its_death (68s) and qemu-final-partitions.log PASS blockd_serves_partitions (43s).

Round 2 had no BLOCKER. This round's asks:

  • The wake is gone. No Wake, sleep word or fence is left in toyos-transport, so a sleep word waiting on its event is moot.
  • toyos-transport/src/model.rs is deleted.
  • The empty-queue direction is reached: under mut-ready.patch, all four model tests red at model.rs:184 (final-arm-ready.log).
  • The 6-bit index is red under INDEX_BITS = 16 (final-arm-index16.log).
  • The Lost-answer issue is filed.

Model time, from ab-head.log, three interleaved rounds:

  • head: 4.71–5.11 s real;
  • main: 5.01–5.85 s.

Lines:

  • git diff --shortstat origin/main...HEAD: 28 files, +1519/−644. The body splits that into production +341 and tests +502.
  • This round, d1bb6b1d..HEAD: +438/−771.

BLOCKER

  • toyos-blockring/src/model.rs:449-473 and server.rs:122-126: the key merges states whose reset answers come out in different orders.
    • Why. abort_all answers in tag order, and a tag sorts by (slot sequence, slot). The key names tags by slot rather than by value, and it drops free slots' sequences.
    • A pair it merges. Both runs end with F2 on the device and slot 1 free, so their keys are alike:
      • (a) W0 is answered before W1 is asked. The slot sequences are then (3, 0).
      • (b) W0 and W1 are in flight together. The sequences are then (2, 1).
    • Where they part. W3 then takes slot 1:
      • in (a), W3's tag is 5 and sorts before F2's 12, so a reset answers [W3, F2];
      • in (b), W3's tag is 9 and sorts after F2's 8, so a reset answers [F2, W3].
      • The search prunes the second of the pair it reaches. Two tags still in the SQ are merged the same way.
    • Main. Main's key carried the raw tags and was exact.
    • Fix. ServerSession keeps its requests in take order (a Vec of at most DEPTH), and abort_all answers in that order. Tags are then compared only for equality, first-appearance naming is an isomorphism, and the key renders inflight() in take order.
    • Red arm. A server.rs test takes tag 9, then tag 4, and asserts abort_all() answers [9, 4]. The patch that restores self.inflight.keys() over a BTreeMap must turn it red.
    • If tag order stays. The key carries every slot's sequence less the least, and the model time is measured again.

NOTE

  • toyos-transport/src/queue.rs:35-44, layout.rs:61-69, userland/blockd/src/main.rs:248, session.rs:138,359: the places are constants, yet building every end is fallible. blockd gained three expects over main, whose ring::client/server could not fail. Check the places at compile time and have new take the checked place. Violation::Region, the four Results, the three blockd expects and the model's RING expects then go.
  • toyos-transport/src/lib.rs:75-114: Cursors is only ever a field of Place, and published, unreleased, clamp, word and PUBLISH serve only queue.rs. Fold head/tail into Place and move those items into queue.rs: one public type fewer.
  • toyos-blockring/src/entry.rs:30-36: lba beside op makes a flush with a nonzero lba representable, and encode writes it. Op::Read { run, lba } | Op::Write { run, lba } | Op::Flush makes that unrepresentable and removes the lba: 0 at every flush site.
  • toyos-blockring/src/model.rs:753: assert!(SCRIPT.len() > DEPTH as usize) compares two constants and cannot fail at run time. The wrap already follows from (0,0,0) ending with all five answered in one session. Delete it, or make it a const _ assertion.

REMOVE

  • toyos-blockring/src/model.rs:446-448: "A fresh tag equals none present, … orders only in a reset's answers". The order it names is exactly what the key loses.
  • PR body, negative controls: "The three block controls red on the model's laws under the renamed key. That is the measurement that the key hides none of them." Three reds show the key keeps three bugs visible, not that it merges only equivalent states.
  • PR body, "What I am unsure of": review conversation, not main's record.
  • PR body, Lines: "This round, against d1bb6b1: production −62, tests −271." This is review chronology.
  • PR body, Gates: "origin/main moved to c551894 … touches none of …". The merge queue decides this, and the line rots at the next push.
  • toyos-transport/Cargo.toml:35: "Already resolved in this workspace for kernel-loom and toyos-sched/loom." It describes the lockfile and rots when either of those moves.

SEND BACK

Japabu and others added 3 commits September 27, 2026 21:57
…le time; lba in the op

The model's key renamed tags by first appearance while `ServerSession`
kept its requests in a `BTreeMap` and `abort_all` answered in tag order.
Two states that differ only in free slots' sequences then got one name
and answered a later reset in different orders, and the search pruned
the second of them. `ServerSession` now keeps its requests in take order
and answers a reset in that order. A tag is now compared only for
equality, so renaming by first appearance is exact.

- `server::tests::a_reset_answers_in_the_order_taken`: takes 9, then 4,
  and demands [9, 4].
- `model::tests::states_named_alike_act_alike`: stages the two states,
  shows the key names them alike, then takes W3 and resets both and
  demands they are still named alike. To walk named paths the search's
  events are now one function, `next`, that `dfs` and the test both call.
  The law checks after an event return a `Result` instead of writing
  into the run.
- Under `ServerSession` restored to round 3's `BTreeMap`, both tests red
  and the search reproduces that version's end-state counts exactly.

`Place<E, D, N>` is now made by `Place::new::<HEAD, TAIL, ENTRIES>()`,
whose inline const refuses a word at or past `N` (E0080). `new` of each
end is infallible: `Violation::Region`, the `Result`s of the ends'
`new` and of `layout::client`/`server`, blockd's three `expect`s and the
model's `RING` expects are gone. `Cursors` folds into `Place`; `word`,
`clamp` and `PUBLISH` move to `queue.rs`, their one user.

`Op::Read { run, lba } | Op::Write { run, lba } | Op::Flush`: a flush
can no longer carry an `lba`, and `Request` and `Client::submit` lose
their separate one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…used forever

Found running this branch's guest gates: sysroot 5dc157f7fac727be, the key
main's sources name, was cloned from the primary's stage2 while its bin/
had no cargo, then marked finished. `ensure` refuses it on every use and
nothing rebuilds it, so no guest gate runs in a worktree naming that key.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu
Japabu marked this pull request as draft September 27, 2026 20:22
@Japabu

Japabu commented Sep 27, 2026

Copy link
Copy Markdown
Collaborator Author

Review r4 at 87bef93

Tests at this head

  • CI host run 36347603028 at 87bef93 concluded success.

  • In that run's log, states_named_alike_act_alike and a_reset_answers_in_the_order_taken pass, and the four Place doctests pass (three of them compile_fail).

  • Each of the six transport and blockring controls reaches its own FAILED line.

  • The orchestrator ran the guest gates at 87bef93, clean tree (orch-runs/summary.txt):

    • blockd_survives_its_death: EXIT=0, PASS (60s);
    • blockd_serves_partitions: EXIT=0, PASS (49s).

    So this round's edits to blockd and blockd_io are built and run.

Lines. git diff --shortstat origin/main...HEAD gives 29 files, +1659/−733. The body splits that into production +368 and tests +526. This round, f42ea5d9..87bef932, is +340/−289.

Round 3's BLOCKER is CLOSED. The key merged states whose reset answers came out in different orders.

  • The fix. ServerSession keeps its requests in a Vec in take order, and abort_all and inflight() answer in that order.
  • The red arm. Under tag-order.patch (BTreeMap restored, reset answered in key order), -p toyos-blockring --lib fails 2 of 19:
    • a_reset_answers_in_the_order_taken gets [4, 9];
    • states_named_alike_act_alike fails at "a reset answered the two in different orders". Its completion queues come out [1, 0] and [0, 1] (arm-tag-order.run.log).
  • The staged pair is the one round 3 named: W1 asked after W0 was answered, against the two in flight together; then W3 is taken and the device reset.
  • Is the key exact? I checked every other consumer of a tag. The server compares tags for equality only. The client orders them by slot, and the key renders the wire in slot order. The device queue, posted and the CQ are held in event order. A fresh tag equals nothing present, because its slot's sequence advanced and 26 bits do not wrap within the model's bounds. No place is left where a tag's value orders anything.

Is the state-space change explained by take order alone? Yes.

  • The patch run on this round's tree prints 28/494/244/236/461/363/373(32)/1311(165) (arm-tag-order-counts.run.log). Round 3's head printed exactly those eight counts (transport-r3/head-ends-final.log).
  • So moving to next/settle and putting lba in the op changed no state.
  • Take order moves only three bounds, all with a reset: 494→492, 461→460, 1311(165)→1300(162). The five other bounds, including (2,2,0), are unchanged.

The compile_fail doctests prove the right error.

  • On stable, rustdoc passes any compile failure: wrongcode.log shows E0277 in the fence staying green. The error code in the fence is unchecked.
  • The reason is pinned by the standalone compiles (placecheck-{head,tail,entries}.log). Each gives error[E0080]: evaluation panicked: a place names a word past its words for its own new::<…> instance, and the good twin builds.
  • Each boundary is pinned, so an off-by-one reds a doctest:
    • head = N and tail = N;
    • entries at N+1 (3 + 4·2 = 11 > 10);
    • the twin ends exactly at N (2 + 8 = 10), so tightening <= to < reds the twin.
  • D = 4 and E = 2 in all four, so no other assertion in the const block can be what fires.

Round 3's NOTEs are all done.

  • The places are checked at compile time. Violation::Region, the four Results and the model's RING are gone.
  • Cursors is folded into Place, and word/clamp/PUBLISH moved to queue.rs.
  • lba is in the op.
  • model.rs:805 is a const _.
  • blockd's expect/unwrap/panic! sets in main.rs and session.rs are identical to origin/main's: I diffed the sorted extractions, exit 0.
  • Round 3's REMOVEs are gone from source and body.

Other mutations of this round's production changes are covered.

  • complete removing index 0 instead of the matched tag leaves a request unanswered, which red every_request_is_answered_exactly_once.
  • encode writing 0 for a read's lba reds a_request_survives_its_words (lba 999).
  • Swapping the read and write closures in decode reds the same test.

Is anything left to delete? Nothing in the code. Every item of this round has a caller.

BLOCKER

None.

NOTE

  • transport-r4/arm-tag-order*.log: the arms' records have gaps.
    • The arm ran on the tree before commit, which differs from 0a117e1 only by the After alias (I compared the hunks).
    • arm.sh echoes RUN EXIT= to stdout, which was not kept, so no log holds the exit code.
    • The counts arm's build step failed (error: Unrecognized option: 'no-run'), so "shown to build (exit 0)" does not hold for that arm. Its run log stands on its own.
    • Keep the exit line in the log next time.

REMOVE

  • issues/build/a-sysroot-cloned-while-stage2-lacks-cargo-is-finished-and-refused-forever.md: the toolchain-fix PR owns this defect. It comes off this branch.
  • PR body, Gates row "cargo test --test toyos-build -- --nightly blockd_survives_its_death | 101, before any guest ran (below)": false at this head. The orchestrator's run gives 0.
  • PR body, "The guest gates did not run. …": false now.
  • PR body, "The same sysroot is what blockd_serves_partitions … not compiled here:" and its two bullets: false now.
  • PR body, Issues filed, the third bullet (a-sysroot-cloned-…): the file leaves the branch.
  • PR body, Lines: "That includes the three issue files (72 lines)": false once the file leaves.
  • PR body, blockd paragraph: "Its four expects on publish/release are gone." Against main nothing is gone, since blockd's expects equal main's. The sentence is branch chronology.

LAND AFTER NAMED CHANGES

…here

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu
Japabu marked this pull request as ready for review September 27, 2026 20:39
@Japabu
Japabu enabled auto-merge September 27, 2026 20:39
@Japabu
Japabu added this pull request to the merge queue Sep 27, 2026
Merged via the queue into main with commit 1808fb8 Sep 27, 2026
2 checks passed
Japabu added a commit that referenced this pull request Sep 27, 2026
#552 rebuilt blockd's rings on toyos-transport. In blockd's main.rs its side
is kept whole — `Op::Read { run, lba } | Op::Write { run, lba }` with
`run.span().offset` and `run.count()`, `let write`, `inflight().len()`,
`post()` and `release(s.region.words())` — and every controller use goes
through this branch's `Drive::up()`. In blockd_io.rs both import edits, both
`main` arms (`nothing`, `hostile-head`) and both module-doc bullets are kept;
the `bench` bullet is this branch's, since the kernel's driver it compared
against is gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu
Japabu deleted the wt/toyos-transport branch September 28, 2026 09:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant