toyos-transport: the one transport core; blockring's rings are rebuilt on it - #552
Conversation
…t on it Stage T1 of the unified transport. A new pure crate, `toyos-transport` (no_std, no alloc, forbid(unsafe_code), rustc-dep-of-std, the domain lint line, one dependency: toyos-untrusted), holds what every client/server session will run over: - `Word`, the port: load, store, and a SeqCst fence in the word's own memory model, so loom sees the fence. - `Producer`/`Consumer<E, D>`: SPSC queues of E-word entries. Entries come out as `[Untrusted<u32>; E]`; a peer's cursor is bounded against what this end released or published (`Violation::TailPastDepth`/`HeadPastTail`), and one moved back within bounds costs only its owner. Each end re-reads the peer's cursor only when what it saw is spent. - `StreamTx`/`StreamRx`: byte rings that answer `Span`s and never touch a byte, with a producer-written `end` word stored after the tail and loaded before it. - `Wake`/`before_sleep`: the sleeper stores `sleep`, fences, looks again; the publisher stores its tail, fences, loads `sleep`, and answers `Wake::Peer` only then. - `Geometry::decode`, `Run::decode`, and `Own`/`Lent`/`Held`, none Clone. - `Inflight<T, D>`: a tag is a slot index under the slot's own sequence, so an answered, ended or replayed tag is `Violation::Tag`; `end` answers every tag in flight, once, eagerly. - `Schema<SQE, CQE>`: LAYOUT, decode_request, decode_reply. blockring's `ring.rs` is deleted. Its places, `client`/`server` and the ring types live in `layout.rs` over the core; a consumer's `sleep` word takes the free word after its head (1 and 33), the page otherwise as it was. `Request::decode`/`Completion::decode` decode `Untrusted` words, and `entry::Block` is the protocol as a schema. `ServerSession::take_entry` takes the popped words; the model's `take` wraps its raw words as a peer's, so `src/model.rs` is unchanged byte for byte and its end-state counts match main's for every bound. blockd compiles against the rebuilt rings with its doorbells as they were: it rings on any publish, whatever the wake says. One difference: a completion that finds no room, or a client head past what was posted, now ends that session instead of panicking blockd, whose old `push` asserted. The loom publication check moves into the core (`tests/loom.rs`) beside a lost-wake model (futex as park/unpark) and two hostile-peer models; an exhaustive session model (crash, reconnect, replayed tags) holds Inflight and the rings together. Four mutation features, each a CONTROLS row that reds: publish-relaxed, no-sleep-fence, no-clamp, end-keeps-inflight; blockring's mutate-ring-publish-relaxed goes with the file it reverted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review r1 at d56fc31CI Lines: BLOCKER
NOTE
REMOVE
SEND BACK |
…nly what it has a user for
Blockers of review r1:
- The stream is deleted (stream.rs, End, StreamPlace, Violation::End): nothing
used it before T6/T9, and its close left the futex word unchanged, a lost
wake. That defect is recorded for T6 in the transport design, not the tree.
- The clamps have tests that can fail: a producer that steps its tail one
past each pop with nothing released, and a consumer that steps its head onto
each unpublished push. Each goes red under the reviewer's mutation (tail
bounded against `local`, head bounded against `local` without `pending`).
- blockd_io gains `hostile-head`: with a write on the device, the client moves
CQ_HEAD to CQ_TAIL - DEPTH; blockd must end that session and serve the next.
blockd_survives_its_death runs it first.
- The conversion is finished here rather than in T2:
- The arena is declared once, as `Geometry`/`Run`. `layout::ARENA` is the
block geometry; SESSION_BYTES derives from `Geometry::BYTES`;
ARENA_BLOCKS, ARENA_OFFSET and arena_byte are gone. A request carries the
`Run` its words decode to, so blockd's device address and the client's
arena window come from its span, and region.rs's own bound check goes.
- The tag table is declared once: the client's wire is an `Inflight`, which
gains `values()` for the two places the client looks across what is in
flight. next_tag, the BTreeMap and the client's own Violation go; a tag
table that is full holds the next request back.
- Schema, entry::Block, Layout and Geometry::decode had no caller and go.
- Own/Lent/Held protected nothing (anyone could mint an Own, and Lent::back
took no completion) and go; `Run::decode` answers a bounded `Run`.
- The test-only `take` shim goes and `take_entry` is `take` again.
- blockring's model keeps its VecDeque queues as a reference and drives the
session page's own rings beside them, holding every entry either end takes
equal to the queue's. A pop that reads the wrong slot reds all four model
tests.
Notes:
- The no-sleep-fence control is split: no-wake-fence takes the producer's
fence away and no-sleep-fence the consumer's, each red on its own. The wake
model no longer parks; the consumer reports whether it slept, so the verdict
is the test's FAILED line and not loom's deadlock.
- `opaque` fails fast instead of defaulting to 0 on a branch that cannot run.
- The 16-bit tag sequence's wrap is stated at the site as the invariant it is:
a replay a wrap later answers what a server could answer by name anyway.
- rustc-dep-of-std leaves the transport and toyos-untrusted: nothing builds
either under std before T2, which adds it with its user.
- Two issues filed: a block publish pays a fence and a load for a wake nobody
asks for, and the head's Release/Acquire has no oracle.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The panic arm showed the race: the client saw its connection close while `try_wait` still found blockd running, so the role went on to open a session nobody would answer and ended by the host's timeout instead of by name. The supervisor now hands every line blockd says to the role, which waits for `WITHHELD` and then for the session's `closed after`; a blockd that ends first disconnects the channel and the role fails saying so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review r2 at b69deabCI
The guest arms are the body's: Lines: Round-1 BLOCKERs
Round-1 NOTEs:
All six round-1 REMOVEs are gone. BLOCKERNone. NOTE
REMOVE
LAND AFTER NAMED CHANGES |
… back to main's speed, held to its queues both ways The wake had no caller: nobody called `before_sleep`, and both publish sites threw the `Wake` away. `Wake`, `Asleep`, the `sleep` words and their fences, `Word::fence`, the two fence controls, their loom test and issues/design-debt/a-block-publish-pays-for-a-wake-nobody-asks-for.md are deleted; the wake lands in T2 with its first caller. toyos-transport/src/model.rs was a second session model beside blockring's, which runs the same `Inflight`, `Producer` and `Consumer` through crash and reconnect. It is deleted, and `end-keeps-inflight` names `inflight::tests::an_end_answers_every_tag_once_and_a_late_completion_nothing`. Ring ends are made over `&[W; N]`: `new` holds the place to `N`, so `publish` and `release` cannot fail, and blockd's four `expect`s on them go. The one index into the words is `word`, under the invariant `new` checked. A request's run is its op's: `Op::Read(Run) | Op::Write(Run) | Op::Flush`. `run: Option<Run>` and the `expect`s that guarded it go. A tag's index takes the bits the table needs (6 for 64 slots) and the sequence the rest (26), so one slot's sequence no longer wraps every 2^16 fills. The block model: its state key names tags by first appearance and renders the client's table by its filled slots; its rings are four deep, so each fills and wraps within the bounds; the client's table is as deep as the rings; and a ring whose queue is empty must pop nothing. Deleting `Consumer::pop`'s `ready` decrement now reds all four model tests. The bounds are main's. blockd_io's hostile request is `Request::encode`d. Filed issues/filesystem/a-read-or-write-answered-lost-is-never-answered.md, pre-existing on main. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s page inline A run's span follows from its first slot and count under the one geometry, so `Run`'s Debug prints only those. The block model keys every state by a rendering full of runs, and the shorter one takes its two law tests from about 6.2 s to about 5.5 s of user time on this host. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review r3 at 64d7149CI Round 2 had no BLOCKER. This round's asks:
Model time, from
Lines:
BLOCKER
NOTE
REMOVE
SEND BACK |
…le time; lba in the op
The model's key renamed tags by first appearance while `ServerSession`
kept its requests in a `BTreeMap` and `abort_all` answered in tag order.
Two states that differ only in free slots' sequences then got one name
and answered a later reset in different orders, and the search pruned
the second of them. `ServerSession` now keeps its requests in take order
and answers a reset in that order. A tag is now compared only for
equality, so renaming by first appearance is exact.
- `server::tests::a_reset_answers_in_the_order_taken`: takes 9, then 4,
and demands [9, 4].
- `model::tests::states_named_alike_act_alike`: stages the two states,
shows the key names them alike, then takes W3 and resets both and
demands they are still named alike. To walk named paths the search's
events are now one function, `next`, that `dfs` and the test both call.
The law checks after an event return a `Result` instead of writing
into the run.
- Under `ServerSession` restored to round 3's `BTreeMap`, both tests red
and the search reproduces that version's end-state counts exactly.
`Place<E, D, N>` is now made by `Place::new::<HEAD, TAIL, ENTRIES>()`,
whose inline const refuses a word at or past `N` (E0080). `new` of each
end is infallible: `Violation::Region`, the `Result`s of the ends'
`new` and of `layout::client`/`server`, blockd's three `expect`s and the
model's `RING` expects are gone. `Cursors` folds into `Place`; `word`,
`clamp` and `PUBLISH` move to `queue.rs`, their one user.
`Op::Read { run, lba } | Op::Write { run, lba } | Op::Flush`: a flush
can no longer carry an `lba`, and `Request` and `Client::submit` lose
their separate one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…used forever Found running this branch's guest gates: sysroot 5dc157f7fac727be, the key main's sources name, was cloned from the primary's stage2 while its bin/ had no cargo, then marked finished. `ensure` refuses it on every use and nothing rebuilds it, so no guest gate runs in a worktree naming that key. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review r4 at 87bef93Tests at this head
Lines. Round 3's BLOCKER is CLOSED. The key merged states whose reset answers came out in different orders.
Is the state-space change explained by take order alone? Yes.
The compile_fail doctests prove the right error.
Round 3's NOTEs are all done.
Other mutations of this round's production changes are covered.
Is anything left to delete? Nothing in the code. Every item of this round has a caller. BLOCKERNone. NOTE
REMOVE
LAND AFTER NAMED CHANGES |
…here Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#552 rebuilt blockd's rings on toyos-transport. In blockd's main.rs its side is kept whole — `Op::Read { run, lba } | Op::Write { run, lba }` with `run.span().offset` and `run.count()`, `let write`, `inflight().len()`, `post()` and `release(s.region.words())` — and every controller use goes through this branch's `Drive::up()`. In blockd_io.rs both import edits, both `main` arms (`nothing`, `hostile-head`) and both module-doc bullets are kept; the `bench` bullet is this branch's, since the kernel's driver it compared against is gone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Stage T1 of the unified IPC transport: the pure core every client/server session will run over, with blockring and blockd converted onto it. No ABI change.
What changed, per decision
toyos-transport, a new crate. It isno_std, has no alloc, isforbid(unsafe_code), and depends only ontoyos-untrusted.Word, the port:loadandstore.Producer/Consumer<E, D, N>: SPSC queues of E-word entries,Ddeep, among a region'sNwords.Place<E, D, N>says where one queue is. It is made only byPlace::new::<HEAD, TAIL, ENTRIES>(), whose inline const refuses any word at or pastN. A place outside its words therefore does not compile (E0080), and each end'snewcannot fail.publishanswersboolandreleaseanswers nothing.word, under that invariant. The crate forbids the arithmetic, unwrap, expect and panic lints and deniesindexing_slicingandas_conversions. The allows are atword, at the ring slot'su32→usize, and at the place check'sD as usize.popanswers[Untrusted<u32>; E], andpushanswersOk(false)on a full queue.TailPastDepthorHeadPastTail.Geometry/Run: the arena, declared once.Run::decodebounds a peer's two words, andGeometry::runcuts this side's. A run'sDebugis its slots; the span follows from them.Inflight<T, D>: a tag is a slot index under that slot's own sequence. The index takes the bits the table needs (6 for 64 slots) and the sequence the rest (26). An answered, ended or replayed tag isViolation::Tag, andendanswers every tag in flight once.tags()walks the tags in flight, andDebugshows only filled slots.blockring on the core.
layout::ARENAis the block geometry, andSESSION_BYTESderives fromGeometry::BYTES.layout::REQUESTSandCOMPLETIONSare checkedPlaceconstants, solayout::clientandlayout::servercannot fail.lbabelong to its op:Op::Read { run, lba } | Op::Write { run, lba } | Op::Flush. A read or write without a run, and a flush with anlba, cannot be written.Client<D>'s wire is anInflightas deep as the session's rings (Ddefaults toDEPTH).on_the_wire()gives its tags.ServerSessionkeeps its requests in the order it took them, andabort_allanswers a reset in that order. Nothing in the protocol compares a tag except for equality, which is what makes the model's tag renaming exact (below).inflight()gives(tag, op)in take order.blockd ends that session when a completion finds no room, or when a head cannot be believed. Both cases stay one arm at
main.rs:281. Before this, the oldpushasserted and blockd panicked. Everyexpectleft in blockd is one main has, message for message.Region::wordsis&[AtomicU32; RING_WORDS].The block model (
toyos-blockring/src/model.rs) runs the transport's rings beside itsVecDequequeues and holds them to each other in both directions:nextis the one function for every event the search can take. It names each event and gives the world after it, or the law it broke.dfswalks it, and a test walks named paths through the same function.A state's key renames tags by first appearance, the client's tags first and in slot order, and renders the client's table by its filled slots.
states_named_alike_act_alikestages the pair that a key ordering tags by value merges. Both states have F2 on the device and slot 1 free:The test demands that the two are named alike, and that they are still named alike after W3 is taken and the device reset.
Its rings are 4 deep and its client's table is too. With no failure, both rings reach 4 entries (
the_model_reaches_the_end_it_should), and one session carries all 5 ofSCRIPT's requests, so each ring wraps. Aconstassertion keepsSCRIPTlonger than the depth. The bounds are main's.Gates, each the command's own exit code, at 0a117e1
cargo test --workspace --exclude toyos-build(transport: 10 unit, 3 loom, 4 doc; blockring: 19)cargo test --lib(379 passed, 1 ignored; also at 87bef93)cargo run -- --clippy(10 invocations clean)cargo check --target aarch64-unknown-none-softfloat, transport and blockringcargo test --test toyos-build -- --nightly blockd_survives_its_death, run by the orchestrator (PASS, 60s)cargo test --test toyos-build -- --nightly blockd_serves_partitions, run by the orchestrator (PASS, 49s)The blockd fix:
blockd_io hostile-headThis role runs first in
blockd_survives_its_death. blockd starts with--silence-write 1, and the client puts oneRequest::encoded write on the ring by hand. After blockd saysWITHHELD, the client storesCQ_HEAD := CQ_TAIL − DEPTHand rings. The role then waits for blockd to say the sessionclosed after, and writes, flushes and reads back through a new session.Measured at b69deab:
toyos-blockringanduserland/blockd(plus the new case)panicked at toyos-blockring/src/ring.rs:93:9: a push into a full ringmain.rs:281patched topanic!instead of ending the sessionpanicked at blockd/src/main.rs:281:35Negative controls at 0a117e1, each the
CONTROLScommand run by handpublish-relaxed(loom)a_published_entry_is_read_whole ... FAILEDno-clamp(loom)a_hostile_producer_yields_entries_or_a_violation ... FAILEDend-keeps-inflightinflight::tests::an_end_answers_every_tag_once_and_a_late_completion_nothing ... FAILEDmutate-session-end-forgetsevery_request_is_answered_exactly_once ... FAILED: "ticket 3 ended answered None"mutate-abort-keeps-inflightevery_request_is_answered_exactly_once ... FAILED: "the client met a second completion for tag 7"mutate-no-reissue-after-losswhat_a_flush_calls_durable_is_on_the_medium ... FAILED: "flush 4 was answered durable with block 0 holding 1"Mutations at 0a117e1
Each is a checked patch, shown to build (exit 0), run, and restored, and the tree is clean after.
ServerSessionon aBTreeMapkeyed by tag, answering a reset in key order:-p toyos-blockring --libgives 101.a_reset_answers_in_the_order_takengets[4, 9].states_named_alike_act_alikepasses its two assertions before the reset and fails at the one after it: renamed, the completion queues come out[1, 0]and[0, 1].queue.rs:181,self.ready = self.ready.wrapping_sub(1);deleted:-p toyos-blockring --libgives 101, and all four model tests fail with "the request ring gave what the queue does not hold";-p toyos-transport --libgives 101, with three queue tests failing.localinstead ofreleased: 101.a_tail_stepped_past_each_pop_is_refused_at_the_depthfails with "a published entry was not taken".localinstead ofpublished: 101.a_head_stepped_past_each_push_is_refused_at_the_depthfails with "a ring with nothing published was full", andthe_ring_wraps_and_counts_its_spacefails too.Inflight::INDEX_BITSset to 16: 101.a_slot_refilled_past_sixteen_bits_refuses_its_first_taganswers the first tagOk(()).Place's doctests was compiled standalone against this crate. The good place builds (0). Each bad one exits 101 witherror[E0080]: evaluation panicked: a place names a word past its words. The entries case also exits 101 undercargo check.compile_fail,E0080: E0277 in its place stays green.Oracles
VecDequequeues in both directions: what the ring gives, and that it gives nothing when the queue is empty. The rings are 4 deep, so they fill and wrap within the model's bounds.Model time, main against the head
Main is c551894's blockring and blockhold, built in a scratch tree with the host workspace's
[profile.dev] opt-level = 2. Each test binary was run by exact test name, interleaved main then head, three rounds on the same loaded host. User seconds, with real seconds in brackets for the two long tests:every_request_is_answered_exactly_oncewhat_a_flush_calls_durable_is_on_the_mediumthe_model_reaches_the_end_it_shouldthe_model_reaches_a_write_given_upstates_named_alike_act_alikemutate-session-end-forgets(whole lib)mutate-abort-keeps-inflight(whole lib)mutate-no-reissue-after-loss(whole lib)End states per bound, main against head:
Main's key carried the client's
next_tag, a count of every request ever issued, and the head's does not.Lines
These are against the merge base c551894 and exclude lockfiles.
model.rs,loom.rs,tests/and each file from its#[cfg(test)]on count as tests.Place's 22-line doctest pair.git diff --shortstat c5518949...1a3fc019: 28 files, +1629/−733.Issues filed
issues/design-debt/the-transport-heads-orderings-have-no-oracle.mdissues/filesystem/a-read-or-write-answered-lost-is-never-answered.md. This is pre-existing on main and held by the orchestrator.🤖 Generated with Claude Code