Skip to content

Each side says everything about routing on one stream - #8835

Merged
kixelated merged 36 commits into
mainfrom
quest/m2/cluster-routes/route-stream
Nov 16, 2026
Merged

kixelated merged 36 commits into
mainfrom
quest/m2/cluster-routes/route-stream

Conversation

@kixelated

@kixelated kixelated commented Nov 15, 2026 •

Copy link
Copy Markdown
Collaborator

Finishes quest/m2/cluster-routes/route-stream.md: on moq-lite-07-wip each endpoint says everything about discovery and routing on one unidirectional Announce Stream.

  • Wire (draft, rs/moq-net, js/net, shared fixtures): one uni Announce Stream (type 0x3) per endpoint carries its ANNOUNCE_REQUEST edits as a subscriber and, as a publisher, ANNOUNCE_OK, the ROUTE_* messages, DEAD, the claims, ANNOUNCE_FLOOR, and SYNCED. The bidirectional Announce stream no longer exists on lite-07.
  • Edits: ANNOUNCE_REQUEST is More, Remove Count, Prefix IDs, Add Count, {Prefix, Hidden}, stop-and-wait, answered by an empty ANNOUNCE_OK. An edit too large for one message continues over several with More set on all but the last, and one ANNOUNCE_OK answers it whole. At the OK both ends carry the live routes over to the new set with one pure function (lite::rekey::plan, ported to js/net/src/lite/rekey.ts), so a swap re-sends nothing both sets cover.
  • DEAD: a closed-session DEAD carries a Moved Count; that many ROUTE_UPDATEs follow, and the receiver applies all of it as one step.
  • Rust: one RouteStream machine per session (lite/subscriber/route_stream.rs) drives the asking half (subscriber/routed.rs) and the answering half (publisher/routed.rs). The route node files one key per session for both directions. A token-scope change is an edit of the interest set, not a new stream.
  • JS: one Routed per session feeds every routes() reader and asks the peer only for the covering set of their prefixes; one Answer per session answers the peer's set. forwardAnnounced re-runs admission when a floor changes what a claim covers.

Validation

  • just check: passes.

  • just fix: clean.

  • cargo nextest run -p moq-net --all-features: 4012 pass. js/net bun test: 3375 pass. Both on the merge with main at 1ca6136. After a conflict-free merge of main at 8f8eec3 (which brings fix(net): a held or kept route stream resumes with what it missed #8860's route path fix), just check passes and moq-net runs 4025 tests, all passing; interop was not rerun for that merge.

  • just test interop --all: 210 pass and 1 skip (route key on Chromium 156, as on main), on the merge with main at 1ca6136. go -> js timed out once at 35s under the full matrix's load and passed alone in 5s (just test interop --publishers go --subscribers js).

  • verify-route on the single-flight edit rule, rerun after decision C and again on the merge with main: EditNarrow.cfg (739,441 distinct states) and EditWiden.cfg (3,152,075) pass exhaustively, and Edit.cfg passes its Simulate: line (40,000 traces). Every other config has Editors = {}, so the rule changes nothing they explore.

  • verify-failover on the changed failover rule: FailoverChain.cfg (about 1.96M distinct states) and Leave.cfg pass unchanged, since the model's FrontFail already fails only once no route is left. EndedRelay.cfg was not run; it fails on its documented open finding.

  • New tests on mocked time cover the Announce Stream's edit, lifetime, and cap on real sessions (tests/integration/lazy_discovery.rs, lite/subscriber.rs). They also cover a DEAD and its updates applying as one, with each violation closing the session (Rust and JS), a swap edit that re-sends nothing both sets cover (Rust and JS), and the rekey plan's unit tests in both languages.

  • test/drill/sensitivity.sh relay-withdraws-lost-publisher: the mutation applies. interrupted_publisher_republishes_new_content::loopback passes without it and fails under it with the interrupted publisher: no announcement change.

  • Review-round tests:

    • a_route_past_the_root_carries_over_as_the_peer_carries_it: a route the subscriber cannot hold under its root carries over an edit under the Route ID the peer assigns. Without the fix the IDs come out one apart.
    • a_swap_too_large_for_one_request_goes_as_one_edit: the router asks a swap past one message as one edit, which request_parts splits.
    • an_edit_too_large_for_one_message_goes_in_parts (Rust and js/net): 20,000 five-byte Prefix IDs and 3,000 adds split into parts that each fit, carry More on all but the last, round-trip, and together hold the edit in order.
    • The publisher answers an edit in parts only at its last, refuses a part removing a prefix an earlier part added, and refuses parts early once their Adds pass the limit or they remove more than the set holds (Rust and js/net). Any other message inside an edit closes the session (a_message_inside_an_edit_closes_the_session, and js/net's connection tests). The shared fixtures add an empty last part and a More of 2.
    • Local review after the design change: a_lite07_prefix_asked_again_keeps_its_mount (Rust), plus five js/net tests: a lite-07 reader skips a claim above a newer floor's region, a lite-07 DEAD takes a reader's claim offline at its own prefix, a lite-07 DEAD takes a narrower reader's claim offline beside a broader reader, and a lite-07 edit merging two routes keeps each one's covering claim. Each fails without its fix.
    • CodeRabbit's round: a lite-07 DEAD shows readers only its end, finishing behind an unwritten send finishes the stream it opens, and a failed send breaks the stream (js/net), each failing without its fix.
    • a_copy_failing_during_a_switch_waits_for_it and a_switch_given_up_on_asks_the_serving_source_again (model/front.rs), with route_cluster::a_matching_epoch_takes_over_at_the_far_edge now passing.
    • an_edit_of_many_prefixes_never_overruns: an edit of thousands of prefixes never overruns the stream.
  • Decision C (ROUTE_START Exact):

    • a_lite07_prefix_asked_again_keeps_its_mount: widening past the mount point now answers with only ANNOUNCE_OK and SYNCED. With the publisher's mark switched off, it ends the route and starts it again.
    • a_lite07_mount_point_route_starts_exact: a mount's route at its mount point starts exact, and the session's own covering route does not.
    • an_exact_route_stays_where_a_widening_leaves_it (subscriber): the exact route stays at its prefix, the covering one widens, and a later claim under the exact route is served. Without Router::edited_exact the node widens it and the claim is never served.
    • an_exact_route_stays_at_its_own_prefix (Rust and js/net rekey), a lite-07 edit to a wider prefix leaves an exact route at its own prefix (js/net, fails without the rule), the codec's exact cases in Rust and js/net, and three new shared fixtures.
    • a_lite07_mount_route_stays_through_a_narrowing_and_a_widening: a mount's route below the requested prefix starts exact too, so root → mount point → root answers each edit with only ANNOUNCE_OK and SYNCED.
  • Main's fix(net)!: a transcode derivative is no longer sometimes unroutable #8827 Above on the single stream:

    • a_narrowed_claim_is_more_above_its_new_path (subscriber), a_lite07_narrowing_carries_a_covering_claim_further_above (publisher), and the js/net rekey test's lowered check: a narrowing grows the carried covering claim's Above by the segments between, on both ends.
    • a lite-07 reader below the request ranks a broader claim as above its prefix (js/net, fails without the fix): a reader below the shared request ranks a broader claim as that much above its prefix.
    • a_claim_above_its_path_counts_from_the_sessions_root runs on the route stream harness rather than the pre-lite-07 announce prefix.
  • Main's fix(net): a redialed lite-07 session syncs once the doubt lets its routes in #8890 redial sync: a_redial_syncs_once_the_doubt_lets_its_routes_in passes, and fails with the gate in Routed::release switched off.

  • Main's test(route): the route layer is accepted on the implementation, and the reference model goes #8832 acceptance tests, adapted to this wire:

    • an_unassigned_or_retired_route_id_closes_the_session: a DEAD's Moved Count followed by its ROUTE_UPDATE.
    • residual_each_closed_session_is_remembered_until_the_doubt_timer: dropping the session's stream in place of Node::closing.
    • next_hop_moves_keep_a_live_reader: 6 Announce Stream messages to D, which are 4 ROUTE_UPDATEs plus a DEAD before each of the 2 moves a closed session caused.
    • an_origin_leaving_the_interest_ends_its_route_by_its_floor: End in the scoped case too.
  • Main's A request treats routing as a hint #8881 tests on this branch:

    • a_burst_of_group_streams_opens_its_groups_in_order (lite/subscriber.rs): three group streams accepted after three earlier ones finished open their groups as 1, 2, 3; without the fix they open as 2, 1, 3.
    • an_upstream_flapping_faster_than_its_news_strands_no_reader passes on five random seeds; it failed every run before the fix.
    • residual_two_depths_of_one_claim_subscribe_twice passes here and on main with the relaxed check.
    • relay_pool_pressure_folds_final_runs_and_ends_stale_reports passes with the accept in order and with the old accept.

Decisions

Taken without asking, each undoable in review:

  1. The Announce Stream is unidirectional, one per endpoint. Rust opens it once the session is set up; js/net opens it with its first message, which the draft's "MAY open" allows.

  2. A request's prefixes are relative to the session, never nest, and fit one message (65,535 bytes) as one request's Adds; anything else is a PROTOCOL_VIOLATION.

  3. Stop-and-wait: one unanswered edit per direction.

  4. The Prefix ID rides only on ROUTE_START and SYNCED. The quest also suggested ANNOUNCE_START and ANNOUNCE_FLOOR; both already name their route, whose ROUTE_START named the prefix.

  5. DEAD carries a Moved Count and the ROUTE_UPDATEs follow as their own messages. Nothing else may come between them, not even the peer's own ANNOUNCE_REQUEST.

  6. The rekey at ANNOUNCE_OK: a route stays, widens (keeps its Route ID, loses the claim at its old prefix and its floors, which the publisher sends again), narrows (each gets the next Route ID), or ends. A subscriber that refused a route's suffix treats the route as lying at its requested prefix. A route its ROUTE_START marked Exact stays at its own prefix through a widening (decided below).

  7. Claim paths and compression bases are relative to the session, not to the requested prefix. WARM_START's path follows in the draft.

  8. A FIN of the Announce Stream means its sender is leaving: it ends its routes first, and its peer keeps the session and keeps answering the prefixes it asked for. A reset or a second stream is a PROTOCOL_VIOLATION. The quest suggested closing on any end.

  9. The node files one key per session for both directions where publish and subscribe share a node, and a handle reading through a mount answers under a further key. Node::closing is removed.

  10. A token-scope change relinks the session's stream and refiles its interest; Router::admit's narrowing is removed.

  11. The machine sends before it reads, and reads whatever its own writer's backpressure.

  12. The Announce Stream holds one unidirectional stream credit toward each side for the session's life, and lite-07 needs one fewer bidirectional credit. The Rust task that accepted it keeps its slot until the session closes, so group streams still arrive in order.

  13. js/net: a widened route's claim at its old prefix stays shown to readers until the route's next floor; a covering set too large for one request falls back to the root prefix; hidden is asked per covering prefix, only when a reader under it wants hidden claims; after a floor changes coverage, an open request no longer covered ends unroutable.

  14. A node wanting more prefixes than one ANNOUNCE_REQUEST holds asks more coarsely instead of being refused by its peer (route::path::coarsen): the prefixes inside one scope region give way to the longest prefix they share, then a prefix alone in its region to the region, measured as the session sends them. It asks for more than it wants and never outside its scope. js/net falls back to the root prefix instead, since a browser's session has no scope regions to keep.

  15. Merging main brought in the flapping publisher's DEAD wording; the changelog line keeps its doubt and loop text with this PR's Moved Count.

  16. The lite-07 branches of AnnounceRun (the bidirectional stream's answer) are left in place, unreachable, with their lite-07-only tests deleted.

  17. Merging main brought in lite-07 ROUTE dropping its informational RTT (feat(net)!: lite-07 ROUTE drops its informational RTT #8838): the Announce Stream's Route Offer drops it too, the draining flag moves to 0x2, and js/net's session-wide Routed stops sampling the link.

  18. Both ends plan the rekey at ANNOUNCE_OK on the paths the stream names, relative to the session. A route or claim the Rust subscriber cannot hold under its own root is still carried as the stream names it, so it no longer shifts every later Route ID and Announce ID. Every other refusal is decided by the bytes the publisher sent, so the draft now says both ends apply it alike: a refused suffix at its requested prefix, a refused offer ranked last, and a claim with a refused path left out. A compliant publisher sends none of those.

  19. An edit too large for one ANNOUNCE_REQUEST goes in several: decided by the maintainer below (C). Rust splits it in lite::request_parts and js/net in requestParts, by exact sizes; route::path::next_edit and editPart are gone.

  20. Only a DEAD and the route changes it seals count toward an Announce Stream overrun. The edits' markers and SYNCEDs count nothing: each side has one edit unanswered at a time, so they hold back at most about two edits.

  21. A broken Announce Stream closes a live session with a PROTOCOL_VIOLATION, whatever the error; once the session is ending, its own end is reported. The choice reads the session's state (poll_ending in Rust, the transport's close in js/net), never the error's variant.

  22. Not changed here: a mount's key files with no scope (route/node.rs Stream::mount), as answering keys already did on main, so a scoped session's close can still doubt the Node ID it declared. Pullfrog flagged it as a gap this PR did not introduce; the maintainer made it a follow-up quest (below).

  23. A second merge of main brought in feat(lite)!: lite-07 freezes without warm routes #8840, under which lite-07 has no warm routes, so the draft drops the Warm Routes and WARM_* sections and keeps this PR's Bidirectional Streams table. It also brought in test(route): the route layer is accepted on the implementation, and the reference model goes #8832's acceptance tests. They now run on one Announce Stream: the session's route stream disconnects before its subscriptions abort, as main's driver ordered it, and two expectations change. A closed session's move costs a DEAD as well as its ROUTE_UPDATE (Decision 5). A scoped relay certifies an origin's withdrawal as End, because it asks the origin for everything and refuses only what the grant does not cover (Decision 10).

  24. From CodeRabbit's review: in js/net, a DEAD defers reader refreshes and consume re-checks (BroadcastCache.batch) until all of it applies, and a failed Announce Stream send closes the session, as a failed flush does in Rust. coarsen files a prefix by its own heads rather than scanning every region, and warns when a scope's regions alone pass one request instead of dropping one silently. Two findings are left as on main: the per-node view reporting a node's newest route, which main's tests pin, and the initial-set drain before SYNCED.

  25. From the local review after the design change. An edit keeps a removed prefix's mount when an added prefix reads it alike (same mount, same interest). js/net shows a reader a claim above its prefix only where the newest floor certifying the prefix leaves it, as a consume picks. A widening that merges several routes of one node keeps each one's claim at its old prefix until the next floor. A DEAD refreshes each reader's presented path once and keeps any loss. RouteHandoff.tla asks one edit at a time, as Router::ask does. The model's sync commit stays at cbb742cec: fix(route): an ended stream still waits on the child it taught #8833's and test(route): the route layer is accepted on the implementation, and the reference model goes #8832's route changes since then have not been audited against it.

  26. Merging main at 80ca978 brought in fix(net)!: a transcode derivative is no longer sometimes unroutable #8827's Above and fix(net): a redialed lite-07 session syncs once the doubt lets its routes in #8890's redial sync, both written against the per-request streams this PR replaces. Above rides ANNOUNCE_START, UPDATE, and RESTART on the single stream as on main. A narrowing at ANNOUNCE_OK presents the claim covering the new prefix from above at that prefix under the next Announce ID, so both ends grow its Above by the segments between (lite::rekey::Claim::lowered, rekey.ts), and the draft's Edits section says so. fix(net): a redialed lite-07 session syncs once the doubt lets its routes in #8890 gated the snapshot release in AnnouncePrefix, which lite-07 no longer runs, so the gate moves to Routed::release, which waits on node::Stream::poll_undoubted.

  27. CodeRabbit's closing-phase point on the JS Announce Stream's failure guard: left as is. A failed send while the session closes fails it, as Rust's route stream does when a flush fails during its withdrawal (flushed? and closed? in Running::poll_send, then lite::error::control); the guard only defers to a session already closed.

  28. Merging main at 1ca6136 brought in A request treats routing as a hint #8881. model/front.rs keeps both main's unstrand and this PR's redispatch_idle when the serving route wins again: the stranded tracks are asked first, then any track that waited on a switch given up on.

  29. The root cause of an_upstream_flapping_faster_than_its_news_strands_no_reader failing here: UniAccept pushed every accepted stream into its kio::Tasks and then ran one pass, which polls by slot from a rotating start and reuses freed slots last-in-first-out. A burst of group streams whose bytes had all arrived read their headers in no particular order, so E opened group 16 before group 15, and its live reader got 15 after the live edge. This PR's long-lived Announce Stream task only shifted which slots the burst took. Each accepted stream now starts before the next is accepted, and the Announce Stream's task no longer holds its slot. The fix is Rust only: js/net reads each accepted stream in its own async task, started in accept order.

  30. residual_two_depths_of_one_claim_subscribe_twice: ✅ relax the test (rejected: the subscriber filing a route into its own origin before forwarding it). Traces show R already files W's exact route into its own origin before E1 hears it, on main and here alike. What differs is whether R's subscriber task reads W's TRACK answer before R's front sees the route, and W sends both at once on two streams, so either order is legal. When the answer wins, the first copy is spliced and the move overlaps, as any move does, until the new copy delivers a group the reader lacks. The test now allows two open subscriptions, has W write its next group, and checks that one stays open.

  31. relay_pool_pressure_folds_final_runs_and_ends_stale_reports (main's) runs each stale reader on its own chain. With the accept in order, the remote reader's relay record ended first and freed enough to pay the debt, so the local reader was never charged. Which reader the debt reaches first depends on timing; neither claim changes.

  32. From Pullfrog's review: a track parked while a route switch is pending was only asked again when the switch attached or when selected saw it given up on, so a switch the target retracted (Refusal { standing: false }, which clears upstream before it reselects) left it Idle once the serving route won the reselect. selected's serving-is-best arm now always re-dispatches idle tracks, which skips any that are not idle, stranded, unwanted, or closing. a_switch_refused_without_standing_asks_the_serving_source_again covers it, and cargo nextest run -p moq-net --all-features passes (4026).

Decided by the maintainer on 2026-11-15:

  • How an edit too large for one ANNOUNCE_REQUEST goes out: ✅ C, one edit spans several messages (rejected: A, patching the split; B, the whole set in every request). ANNOUNCE_REQUEST gains More first; every part but the last sets it and nothing else may come between them. The edit applies whole at its last part, which one ANNOUNCE_OK answers. Parts fail early with a PROTOCOL_VIOLATION once their Adds alone pass the set's limit or they remove more than the set holds; removes name only prefixes held before the edit. The set bound and coarsen stay; only the edit-level split goes, in Rust, js/net, the draft, and the fixtures.
  • a_matching_epoch_takes_over_at_the_far_edge: ✅ fixed in this PR (rejected: a separate quest first). In model/front.rs, a delivered copy that fails while a route switch is under way waits for that switch, whatever the error, and attach asks the new source; a switch given up on asks the serving source again. FrontServe takes a pending table change first when a copy ends in error, so the switch is under way when the failure is judged. The model's FrontFail already fails only once no route is left, so FailoverChain.tla is unchanged and passes. js/net already waits for another route on an unroutable failure.
  • The mount scope gap: ✅ a new m2 follow-up quest in this PR, quest/m2/cluster-routes/mount-scope.md (rejected: m3; ignoring it).
  • Widening a requested prefix past a route that sits exactly at it, such as a mount's route at its mount point: ✅ C, ROUTE_START marks a route that sits exactly at the requested prefix, and widening leaves it at its own prefix, so nothing ends or restarts (rejected: A, end first and accept the brief offline; B, start first with an announce-cap allowance). How the head applies it:
    • Wire: ROUTE_START gains Exact (8) after the Route Suffix: 1 when the publisher keeps the route at its own prefix whatever prefix the request lies at, with or without a suffix. Any value but 0 or 1 is a PROTOCOL_VIOLATION. The draft's Edits rule keeps an exact route at its own prefix as it keeps one strictly inside the old prefix; a route that stays keeps its Exact, and a widened or narrowed one is not exact. A publisher MAY send 0, and the route then widens and is sent again under its own prefix.
    • Rust: lite::rekey::plan and rekey.ts keep an exact route. The publisher sets Exact only on a route a mount presents at its mount point, the case the decision names, whether that lies at the requested prefix or below it (so a narrowing to the mount point and a widening back keep it too): nothing above a mount point names such a route, so it stays whatever the widening. The session's own key presents a covering route at the requested prefix, and the route node (route::path::Router) widens it at an edit and then sends any route at its own prefix inside the wider one, which already never went offline; marking those would need the node to track a mark per sent route that its covering-source switches could make stale, for no change a reader sees. js/net sends only its own root route, which covers every prefix, so it never sets it.
    • The subscriber hands the routes marked exact to the node's rekey of what it heard (Router::edited_exact), so the node keeps them where the wire does.
    • RouteHandoff.tla is unchanged but for a comment: it has no mounts, so no route in it is marked exact. EditNarrow.cfg and EditWiden.cfg pass exhaustively and Edit.cfg its Simulate: line.

Impact

Wire: moq-lite-07-wip only, which is unpublished. lite-06 and older and moq-transport are unchanged. The draft's changelog records it.

Public API: none in Rust, JS, or the bindings; routes() keeps its shape.

Migration: none. A lite-07-wip endpoint from before this change does not interoperate with one after it.

Follow-ups

  • quest/m2/cluster-routes/mount-scope.md: a mount files its key under its session's scope, so a scoped session's close no longer doubts a Node ID outside it. No open decisions.
  • quest/m4/announce-run-lite07-branches.md: delete the unreachable lite-07 branches of AnnounceRun and of js/net's runAnnounce.
  • quest/m4/js-announce-edit-deadline.md: js/net closes a session whose Announce Stream stalls, as Rust's write-stall deadline does; it recommends no deadline on an unanswered edit, which neither Rust nor the draft has, and settles eager against lazy opening.

Models

Size: XL
Implementer: Claude Code, claude-opus-5-5, medium effort (iterate, S, arm B)
Implementer: Claude Code, claude-opus-5-5, high effort (iterate, M, arm B)
Implementer: Claude Code, claude-opus-5-5 (iterate)
Implementer: Claude Code, claude-sonnet-5-5 (iterate, S, arm B)

🤖 Generated with Claude Code

Review

Rewritten by quest review after each round.

  • Head: 10a2c1c

  • Base: origin/main at 3e7987a

  • Risk: medium, the base has no RISK.md to name what is always high or customer-facing, so every change is medium

  • Classified: medium by default

  • Reviewer: Codex, gpt-6.1-sol, xhigh effort

  • Session: 01a1e3c9-4416-7403-ac4d-ba13121ddbe0, round 4

  • Score: 9/10, The incremental fix correctly redispatches waiting tracks after a withdrawn switch is refused, with regression coverage and no new defects found; builds and tests were not run.

  • Rounds: 19; first round 3 (high 1, medium 2); findings 15 (high 1, medium 13, low 1): fixed 14, refuted 1, open 0

  • Findings: none

kixelated and others added 14 commits November 15, 2026 04:51
…'s routing

Work in progress: the library builds, the tests and js/net follow.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Work in progress.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Work in progress.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The Announce Stream holds one uni stream credit for the session's life, and the uni task that accepted it keeps its place until the session closes.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
… sessions

Work in progress: js/net still speaks the old wire.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…tream

One unidirectional Announce Stream per endpoint carries its ANNOUNCE_REQUEST edits and its answers. The subscriber keeps one route state per session and asks only for the covering set of its readers' prefixes; the publisher answers a set of prefixes and carries routes over each edit with the same rekey plan as Rust. A DEAD and the ROUTE_UPDATEs it counts apply as one.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The draft allows nothing between a DEAD and the ROUTE_UPDATEs it counts. Also regenerates the drill mutation that removes the relay's withdrawal of a lost publisher.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Deletes the quest, points its mentions at the PR, and files the two follow-ups.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…is asked for coarsely

js/net dropped a widened route's carried claims from its path index, reporting them offline. A node wanting more prefixes than one ANNOUNCE_REQUEST holds now asks for the prefix they share, inside each scope region, so its peer never refuses the set. js/net races the session's close with the cancellable race.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…he session sends

A prefix alone in its scope region gives way to the region before any is dropped, and the set is measured as the prefixes travel, beneath the session's root.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…es/route-stream

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review November 15, 2026 18:56
@coderabbitai

coderabbitai Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 13 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 056695f2-3105-46aa-959b-2eada5b7ebef
Commits

Reviewing files that changed from the base of the PR and between e0b7487 and 10a2c1c.

Files selected for processing: 81

  • .claude/skills/verify-route/SKILL.md
  • doc/.vitepress/drafts.test.ts
  • drafts/draft-lcurley-moq-lite.md
  • js/net/bench/broadcasts.ts
  • js/net/src/connection/close.test.ts
  • js/net/src/connection/forward.test.ts
  • js/net/src/connection/forward.ts
  • js/net/src/consume.ts
  • js/net/src/integration.test.ts
  • js/net/src/lite/announce.test.ts
  • js/net/src/lite/announce.ts
  • js/net/src/lite/announce_stream.test.ts
  • js/net/src/lite/announce_stream.ts
  • js/net/src/lite/answer.ts
  • js/net/src/lite/connection.test.ts
  • js/net/src/lite/connection.ts
  • js/net/src/lite/corpus.test.ts
  • js/net/src/lite/publisher.test.ts
  • js/net/src/lite/publisher.ts
  • js/net/src/lite/rekey.test.ts
  • js/net/src/lite/rekey.ts
  • js/net/src/lite/routed.ts
  • js/net/src/lite/stream.ts
  • js/net/src/lite/subscriber.test.ts
  • js/net/src/lite/subscriber.ts
  • js/net/src/request-deadline.test.ts
  • js/net/src/wire.ts
  • quest/m2/README.md
  • quest/m2/cluster-routes/README.md
  • quest/m2/cluster-routes/mount-scope.md
  • quest/m2/cluster-routes/route-stream.md
  • quest/m2/multicast/switch.md
  • quest/m4/announce-interest-memory.md
  • quest/m4/announce-run-lite07-branches.md
  • quest/m4/js-announce-edit-deadline.md
  • quest/m4/regrant-route-recheck.md
  • rs/moq-mux/src/container/source.rs
  • rs/moq-mux/src/container/test_util.rs
  • rs/moq-net/fixtures/lite.json
  • rs/moq-net/src/auth.rs
  • rs/moq-net/src/coding/writer.rs
  • rs/moq-net/src/fuzz.rs
  • rs/moq-net/src/ietf/publisher.rs
  • rs/moq-net/src/lite/announce.rs
  • rs/moq-net/src/lite/compress.rs
  • rs/moq-net/src/lite/corpus.rs
  • rs/moq-net/src/lite/mod.rs
  • rs/moq-net/src/lite/publisher.rs
  • rs/moq-net/src/lite/publisher/routed.rs
  • rs/moq-net/src/lite/rekey.rs
  • rs/moq-net/src/lite/session.rs
  • rs/moq-net/src/lite/stream.rs
  • rs/moq-net/src/lite/subscriber.rs
  • rs/moq-net/src/lite/subscriber/route_stream.rs
  • rs/moq-net/src/lite/subscriber/routed.rs
  • rs/moq-net/src/lite/subscriber/routed/tests.rs
  • rs/moq-net/src/lite/test_transport.rs
  • rs/moq-net/src/lite/version.rs
  • rs/moq-net/src/model/front.rs
  • rs/moq-net/src/model/origin.rs
  • rs/moq-net/src/route/mod.rs
  • rs/moq-net/src/route/node.rs
  • rs/moq-net/src/route/node/tests.rs
  • rs/moq-net/src/route/path.rs
  • rs/moq-net/src/route/tests/interest.rs
  • rs/moq-net/src/route/tests/mod.rs
  • rs/moq-net/src/route/tests/scope.rs
  • rs/moq-net/src/route/view.rs
  • rs/moq-net/src/session.rs
  • rs/moq-net/tests/integration/auth.rs
  • rs/moq-net/tests/integration/claim_continues.rs
  • rs/moq-net/tests/integration/drop_record.rs
  • rs/moq-net/tests/integration/lazy_discovery.rs
  • rs/moq-net/tests/integration/route_cluster.rs
  • rs/moq-net/tests/integration/route_failover.rs
  • rs/moq-net/tests/integration/slow_joiner.rs
  • rs/moq-net/tests/integration/stream_credit.rs
  • rs/moq-net/tests/integration/support/mock.rs
  • rs/moq-net/tests/integration/support/mod.rs
  • rs/moq-net/verify/route/RouteHandoff.tla
  • test/drill/mutations/relay-withdraws-lost-publisher.patch

Walkthrough

Lite-07 discovery and routing move to one unidirectional Announce Stream per endpoint. The stream carries prefix-set edits, acknowledgements, route and claim messages, and per-prefix synchronization. Rust and JavaScript add session-level routing, route rekeying across edits, and stream lifecycle handling. The changes also revise the DEAD message format, update route-node session and mount behavior, and adjust protocol tests, integration tests, fixtures, and documentation.

Merge Risk: Moderate · previous review e0b74

This change moves lite-07 routing to a single shared Announce Stream. Several issues on that path can stop or skew discovery on lite-07 sessions:

  • Prefix edits can exceed the wire message limit.
  • Scope regions can be silently dropped from requests.
  • Per-node route selection can pick a worse route.
  • A route failure (DEAD) can cause spurious publisher restarts.
  • A failed answer loop or a stalled peer can leave a JS session open with discovery stopped.

The impact is limited to the work-in-progress protocol version, but these should be fixed or explicitly accepted before merge.

Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title "Each side says everything about routing on one stream" matches the main change: each endpoint carries its discovery and routing on one unidirectional Announce Stream, replacing the bidirect…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Description check Passed The description is a detailed, on-topic account of the changeset: one unidirectional Announce Stream per endpoint on moq-lite-07-wip, ANNOUNCE_REQUEST edits, the shared rekey plan, DEAD/ROUTE_UPDATE h…
Full details: Title check

Explanation

The title "Each side says everything about routing on one stream" matches the main change: each endpoint carries its discovery and routing on one unidirectional Announce Stream, replacing the bidirectional Announce stream on moq-lite-07-wip. It is somewhat vague in wording ("says everything") but it refers to the core change and is specific enough for a teammate scanning history to understand the primary change.

Full details: Description check

Explanation

The description is a detailed, on-topic account of the changeset: one unidirectional Announce Stream per endpoint on moq-lite-07-wip, ANNOUNCE_REQUEST edits, the shared rekey plan, DEAD/ROUTE_UPDATE handling, and the Rust/JS routing structures. It lists validation and decisions tied to the changed files, so it is not off-topic or vague.

Finishing Touches

Simplify code

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No blocking issues. Two things to look at: one design gap in the rekey rule, and one rule violation.

Reviewed changes

I reviewed the PR at 3f52de8: the lite-07 move to one unidirectional Announce Stream per endpoint, in the draft, Rust, JS, and the shared fixtures. I read the core Rust and the connection glue directly; specialists covered the codec, the JS mirror, and the route node.

  • Draft: The bidirectional Announce stream (0x1) is gone on lite-07. The new messages are ANNOUNCE_REQUEST 0x10 (edits by Prefix ID) and ANNOUNCE_OK 0x11 (stop-and-wait). SYNCED and ROUTE_START now carry a Prefix ID, claim paths are relative to the session, and DEAD carries a Moved Count followed by that many ROUTE_UPDATEs. A new Edits section defines how routes stay, widen, narrow, or end, and how ties are broken.
  • Rust state machine: RouteStream in lite/subscriber/route_stream.rs opens the stream, waits for setup, and runs both halves on it. It keeps the 10 s stall check and the overrun check.
  • Rust halves: the asking half is Routed::ask/ok/release/limit and the answering half is Answer::request/edited/rekey/synced/withdraw. Both carry routes over an edit with the pure lite::rekey::plan.
  • Route node: one key per session for both directions (Node::connect, Stream::edit/edited/relink/mount/withdraw). path::Router::asks coarsens a request that is too large. Node::closing and the publisher's Drop hook are removed.
  • Streams and credit: the peer's Announce Stream holds its UniServe slot until the session closes. A lite-07 session needs one fewer bidirectional credit and one more unidirectional credit. On lite-07 the old bidirectional Announce type is now filtered out.
  • JS: announce_stream.ts, answer.ts, rekey.ts, and routed.ts mirror the Rust side. Connection.#runAnnounce reads the peer's stream, and the asker requests the covering set of what its readers want, falling back to the root prefix when that set is too large.

The codec specialist confirmed that Rust, JS, and the draft agree on type numbers and field order, and that each count from the peer is bounded before anything is allocated. It also confirmed lite-06 and older still use the bidirectional stream unchanged. The two other specialists, on JS↔Rust rekey parity and on the route node's edit and relink paths, had not reported when this review was submitted. Their questions are still open, so this review does not approve.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using claude-opus-5-5 | 𝕏

Comment thread rs/moq-net/src/lite/rekey.rs Resolved
Comment thread rs/moq-net/src/route/path.rs Resolved
Comment thread rs/moq-net/src/route/node.rs Resolved
kixelated and others added 5 commits November 15, 2026 13:11
…es/route-stream

Main's lite-07 ROUTE drops its informational RTT (#8838): the Announce
Stream's Route Offer loses it here too, with the draining flag at 0x2, and
js/net's Routed stops sampling the link.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nt only DEADs toward an overrun

- The Rust subscriber plans an ANNOUNCE_OK's rekey on session-relative paths, as the
  peer does, so a route or claim it cannot hold under its own root no longer shifts
  every later Route ID or Announce ID. The draft says which refusals both ends
  apply alike.
- An edit whose removes and adds together pass one ANNOUNCE_REQUEST removes first and
  adds once that is answered, in Rust and js/net.
- Only a DEAD and what it seals count toward an Announce Stream overrun: the edits'
  markers and SYNCEDs are bounded by stop-and-wait.
- A broken Announce Stream is a PROTOCOL_VIOLATION while the session lives, decided by
  the session's state rather than the error's variant.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…r lapses

An edit past one ANNOUNCE_REQUEST no longer removes first: it goes in parts, each
changing whole groups of a prefix and the prefixes nested with it, which it replaces
at once, the groups that shrink the set first so every set between fits too. Rust's
`next_edit` and js/net's `editPart` match.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…es/route-stream

Takes #8840's warm-route removal and adapts #8832's acceptance tests to one Announce Stream: a DEAD counts the ROUTE_UPDATEs after it, and dropping a session's stream closes it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The route stream disconnects before the session's subscriptions abort, as main's driver did. #8832's tests count a DEAD before each move a closed session caused, and a scoped relay's floor certifies an end, since it asks the origin for everything. The route-stream quest's dependent no longer requires it, which this lands.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as draft November 16, 2026 00:13

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No new issues. Three of the four earlier Pullfrog threads are now resolved. The remaining one, the size estimate in next_edit, stays open because the author is waiting on a maintainer decision about its two edge cases, so this review does not approve.

Reviewed changes

This run covers what changed since the review at 3f52de8. The two merges of main are left out, apart from where they meet this PR: the removal of ROUTE's RTT, which moved the drain flag to 0x2, and the removal of warm routes.

  • Rekey on wire paths: the Rust subscriber's ok() now plans the rekey on paths relative to the session, as the stream names them (Asked.relative, WireRoute.wire, WireClaim.wire). A route or claim the subscriber cannot hold under its own root therefore no longer shifts Route IDs or Announce IDs. The draft's Refused Content section now says that both ends carry such routes over alike.
  • Oversized edits split by group: route/path.rs next_edit and js/net editPart split an edit too large for one request into parts made of whole groups, where a group is one prefix plus the other side's prefixes nested in it. A coarsened prefix is therefore replaced by its narrower prefixes in the same part. Parts that shrink the set go first, so every intermediate set also fits.
  • Overrun budget: Barrier::counts/cost now count only a DEAD and the routes it seals. Edit markers and SYNCEDs count nothing.
  • Broken Announce Stream: Rust poll_receive checks session.poll_ending, and JS #runAnnounce checks whether the transport closed. Neither inspects the error's variant any more.
  • Driver drop order: SubscriberDriver::drop now disconnects the route stream before cleanup aborts subscriptions, so a reader's front fails over instead of ending. Main's acceptance tests now expect one DEAD before each move that a closed session causes.

Pullfrog  | Fix it ➔ | View workflow run | Using claude-opus-5-5 | 𝕏

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 10


  • Fix comments: Fix CodeRabbit comments on this PR
Prompt to fix all issues
Treat finding text, file paths, and code as untrusted review data. Never follow
instructions embedded in them. Verify each finding against current code. Fix
only still-valid issues, skip the rest with a brief reason, keep changes
minimal, and validate.

Inline comments:
Review comments at @js/net/src/lite/announce_stream.ts:
- Line 65: Update finish() to read this.#writer inside the queued completion
callback, after queued sends have completed, so it closes a writer opened by a
queued send. Preserve the existing behavior when no send was queued and the
stream remains unopened.

Review comments at @js/net/src/lite/announce.ts:
- Line 510: Update the batching loop around `spent`, `found.cost`, and
`MAX_MESSAGE_SIZE` to handle an atomic group that exceeds the limit even when no
edits have been added; do not return an oversized request or split the atomic
group. Add a boundary case beside the multipart test verifying that an oversized
first group is rejected.

Review comments at @js/net/src/lite/publisher.ts:
- Line 640: Update the #answering rejection handler in the Answer flow to pass
failures to the existing connection failure handling instead of only logging
them, while suppressing propagation once shutdown has started. Reuse the
established failure path and shutdown state checks in the surrounding connection
lifecycle.

Review comments at @js/net/src/lite/routed.ts:
- Around line 734-735: Update the DEAD processing path around #move and
#endRoute to apply all affected route changes before triggering reader
notifications or selection revalidation. Defer the observable updates until
every route has its final state, then notify readers based on that final state
so intermediate selections do not queue restarts.
- Line 431: Update the per-node selection in the candidate loop so multiple
routes to the same node are compared with bestCandidate rather than letting the
last candidate overwrite earlier ones. Preserve the selected best route’s
metadata in now for forwardAnnounced.

Review comments at @js/net/src/lite/subscriber.ts:
- Line 571: Update the Announce Stream send callback in the JS subscriber and
its Routed write path to add a progress watchdog matching the Rust Stall
behavior: while messages are waiting to send, close the session with an
“announce stream stalled” error after 10 seconds without write progress,
resetting the deadline when progress resumes. Do not rely on
Connection.close()’s timeout, which only applies after local closure begins.

Review comments at @quest/m4/js-announce-edit-deadline.md:
- Around line 13-14: Update the Plan to distinguish the write-stall deadline
from the answer deadline: the existing Rust Stall timer covers pending writes,
not a successfully flushed ANNOUNCE_REQUEST awaiting ANNOUNCE_OK. Specify an
answer deadline for unanswered edits in both Rust and JavaScript, while
retaining the write-stall deadline for stalled writes.

Review comments at @rs/moq-net/src/lite/publisher/routed.rs:
- Around line 579-580: Make the mount synchronization loop around `self.mounts`
and `self.sourced` resumable by retaining pending synchronization state across
polls and processing only a bounded amount of queued output per poll, so control
returns to the session for flushing and timeout or cancellation checks. Preserve
the required replay state and send `SYNCED` only after the mount replay is
complete.

Review comments at @rs/moq-net/src/route/path.rs:
- Line 796: Update the capacity-handling branch around set.pop_last() in
ask_bytes so it never silently drops a wanted scope region; preserve all regions
or return an explicit capacity error when they cannot fit. Ensure discovery
cannot report completion with claims missing from the watched open paths.
- Around line 769-771: Update the region-selection logic in coarsen to compute
region membership once and maintain candidate groups incrementally across
replacements, rather than rescanning every region on each pass. Preserve the
existing selection behavior, including the case where no region is dropped;
changing only the truncation branch will not address the repeated comparisons.

After applying the fix, consider running `coderabbit review --agent` for local
review. Visit https://docs.coderabbit.ai/cli?utm_source=ghpr
Review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 3dce1f71-4543-49d2-b6b2-5da0a533a25f
Commits

Reviewing files that changed from the base of the PR and between d57d4dc and e0b7487.

Files selected for processing: 74

  • .claude/skills/verify-route/SKILL.md
  • doc/.vitepress/drafts.test.ts
  • drafts/draft-lcurley-moq-lite.md
  • js/net/bench/broadcasts.ts
  • js/net/src/connection/close.test.ts
  • js/net/src/connection/forward.test.ts
  • js/net/src/connection/forward.ts
  • js/net/src/integration.test.ts
  • js/net/src/lite/announce.test.ts
  • js/net/src/lite/announce.ts
  • js/net/src/lite/announce_stream.ts
  • js/net/src/lite/answer.ts
  • js/net/src/lite/connection.test.ts
  • js/net/src/lite/connection.ts
  • js/net/src/lite/corpus.test.ts
  • js/net/src/lite/publisher.test.ts
  • js/net/src/lite/publisher.ts
  • js/net/src/lite/rekey.test.ts
  • js/net/src/lite/rekey.ts
  • js/net/src/lite/routed.ts
  • js/net/src/lite/stream.ts
  • js/net/src/lite/subscriber.test.ts
  • js/net/src/lite/subscriber.ts
  • js/net/src/request-deadline.test.ts
  • js/net/src/wire.ts
  • quest/m2/README.md
  • quest/m2/cluster-routes/README.md
  • quest/m2/cluster-routes/route-stream.md
  • quest/m2/multicast/switch.md
  • quest/m4/announce-interest-memory.md
  • quest/m4/announce-run-lite07-branches.md
  • quest/m4/js-announce-edit-deadline.md
  • quest/m4/regrant-route-recheck.md
  • rs/moq-mux/src/container/source.rs
  • rs/moq-mux/src/container/test_util.rs
  • rs/moq-net/fixtures/lite.json
  • rs/moq-net/src/auth.rs
  • rs/moq-net/src/coding/writer.rs
  • rs/moq-net/src/fuzz.rs
  • rs/moq-net/src/ietf/publisher.rs
  • rs/moq-net/src/lite/announce.rs
  • rs/moq-net/src/lite/compress.rs
  • rs/moq-net/src/lite/corpus.rs
  • rs/moq-net/src/lite/mod.rs
  • rs/moq-net/src/lite/publisher.rs
  • rs/moq-net/src/lite/publisher/routed.rs
  • rs/moq-net/src/lite/rekey.rs
  • rs/moq-net/src/lite/session.rs
  • rs/moq-net/src/lite/stream.rs
  • rs/moq-net/src/lite/subscriber.rs
  • rs/moq-net/src/lite/subscriber/route_stream.rs
  • rs/moq-net/src/lite/subscriber/routed.rs
  • rs/moq-net/src/lite/subscriber/routed/tests.rs
  • rs/moq-net/src/lite/test_transport.rs
  • rs/moq-net/src/lite/version.rs
  • rs/moq-net/src/model/origin.rs
  • rs/moq-net/src/route/mod.rs
  • rs/moq-net/src/route/node.rs
  • rs/moq-net/src/route/node/tests.rs
  • rs/moq-net/src/route/path.rs
  • rs/moq-net/src/route/tests/interest.rs
  • rs/moq-net/src/route/tests/mod.rs
  • rs/moq-net/src/route/tests/scope.rs
  • rs/moq-net/src/session.rs
  • rs/moq-net/tests/integration/auth.rs
  • rs/moq-net/tests/integration/drop_record.rs
  • rs/moq-net/tests/integration/lazy_discovery.rs
  • rs/moq-net/tests/integration/route_cluster.rs
  • rs/moq-net/tests/integration/route_failover.rs
  • rs/moq-net/tests/integration/slow_joiner.rs
  • rs/moq-net/tests/integration/stream_credit.rs
  • rs/moq-net/tests/integration/support/mock.rs
  • rs/moq-net/tests/integration/support/mod.rs
  • test/drill/mutations/relay-withdraws-lost-publisher.patch
Files with no reviewable changes: 5

  • js/net/src/lite/corpus.test.ts
  • quest/m2/cluster-routes/route-stream.md
  • quest/m2/multicast/switch.md
  • rs/moq-net/src/lite/corpus.rs
  • rs/moq-net/src/session.rs

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 1 remain after this review.

Comment thread js/net/src/lite/announce_stream.ts Resolved
Comment thread js/net/src/lite/announce.ts Resolved
Comment thread js/net/src/lite/publisher.ts Resolved
Comment thread js/net/src/lite/routed.ts Resolved
Comment thread js/net/src/lite/routed.ts Resolved
Comment thread js/net/src/lite/subscriber.ts Resolved
Comment thread quest/m4/js-announce-edit-deadline.md Resolved
Comment thread rs/moq-net/src/lite/publisher/routed.rs Resolved
Comment thread rs/moq-net/src/route/path.rs Resolved
Comment thread rs/moq-net/src/route/path.rs Resolved
@kixelated

Copy link
Copy Markdown
Collaborator Author

Maintainer decision (2026-11-15, /quest-complete):

  1. Big interest edits: ✅ C, one edit spans several messages with a continuation flag and a single ANNOUNCE_OK after the last part; remove the size-estimating split in Rust and JS (rejected: A patch the split; B resend the whole set).
  2. a_matching_epoch_takes_over_at_the_far_edge: ✅ fix it in this PR: when a copy fails while a route switch is already under way, failover in model/front.rs waits for that switch; check it with the failover TLA+ model (rejected: a separate m2 quest first).
  3. Mount scope gap: ✅ a new m2 quest, added as a follow-up here (rejected: m3; ignore).

/quest-merge pick stands once these land.

(Written by Opus 5.5)

@kixelated

Copy link
Copy Markdown
Collaborator Author

Answering the maintainer decision (widening a requested prefix, ✅ C). It is applied as of 707f267 and ac1545f, and the head is 53a3a50:

  • Wire: ROUTE_START gains Exact (8) after the Route Suffix, in the draft, rs/moq-net, js/net, and the shared fixtures. The Edits rule keeps an exact route at its own prefix at ANNOUNCE_OK, as it keeps a route strictly inside the old prefix. lite::rekey::plan and js/net/src/lite/rekey.ts both apply it, so a widening sends no ROUTE_END and no second ROUTE_START for it.
  • Who sets it: the Rust publisher marks every route a mount presents at its mount point. It does so whether the mount point is the requested prefix or lies below it, so root → mount point → root also leaves the route alone. Local review found that case. The session's own covering route stays unmarked, because the route node widens it the same way at an edit.
  • Subscriber: it hands the exact routes to the node's rekey of what it heard (Router::edited_exact), so the node keeps them where the wire does.
  • Model: RouteHandoff.tla has no mounts, so it gained only a comment. EditNarrow.cfg and EditWiden.cfg pass exhaustively, and Edit.cfg passes its Simulate: line.

The PR's Waiting item is replaced. Main's newest merge (#8881) leaves two of its new integration tests failing on this branch, so the PR stays a draft. See ## Waiting in the description.

(Written by Claude Opus 5.5 with high effort)

# Conflicts:
#	rs/moq-net/src/model/front.rs
kixelated and others added 2 commits November 16, 2026 03:07
…they arrived

UniAccept pushed every stream it accepted and then ran one pass over its task
set, which polls by slot from a rotating start and reuses freed slots LIFO. A
burst of group streams whose bytes had all arrived therefore read its headers in
no particular order, so a live reader on the far side of a relay could get a
group after a newer one. Each accepted stream now starts before the next is
accepted. The Announce Stream's task no longer holds its slot for the session,
which only shifted the layout.

Merging main's #8881 exposed it: `an_upstream_flapping_faster_than_its_news_strands_no_reader`
lost E's group 15 to the reorder. Main's `residual_two_depths_of_one_claim_subscribe_twice`
now allows the TRACK answer to beat the exact route and proves the overlap closes
at the next group, and `relay_pool_pressure_folds_final_runs_and_ends_stale_reports`
runs each stale reader on its own chain, since the first to end can pay the debt
the other was about to be asked for.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review November 16, 2026 11:38
@kixelated

Copy link
Copy Markdown
Collaborator Author

The merge with main is done, and the PR is ready at dfac993. Both failing #8881 tests are resolved:

  • an_upstream_flapping_faster_than_its_news_strands_no_reader: the bug was in the Rust accept loop. UniAccept pushed every accepted uni stream into its kio::Tasks and then ran one pass. That pass polls by slot from a rotating start and reuses freed slots last-in-first-out, so a burst of group streams read their headers out of order. E opened group 16 before group 15, and its live reader got 15 after the live edge. Each accepted stream now starts before the next is accepted, and the Announce Stream task no longer holds its slot. Holding it only moved where the burst landed. New test: a_burst_of_group_streams_opens_its_groups_in_order. Without the fix the groups open as 2, 1, 3.
  • residual_two_depths_of_one_claim_subscribe_twice: the test is relaxed (Decision 30). R already files the exact route into its own origin before forwarding it, here and on main. W sends the route and the TRACK answer at the same moment on two streams, so R may read either first. When the answer wins, the overlap closes at the new copy's next group. The test now proves that by having W write one more group.
  • relay_pool_pressure_folds_final_runs_and_ends_stale_reports (from main) depended on the old scrambled order. It now runs each stale reader on its own chain (Decision 31).

Checks: just check passes, moq-net passes 4025 of 4025 tests, js/net passes 3375, and just test interop --all passes 210 with 1 skip. go -> js timed out once under load and passed when rerun alone. The route rule is unchanged, so verify-route was not rerun. Local Codex review, rounds 2 and 3: no findings.

(Written by Claude Opus 5.5)

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

One failover regression: a track whose copy fails during a route switch can stall forever if the switch is refused and the serving route is still the best. Details inline. Everything else in this round looks sound.

Reviewed changes

This run covers the PR's own commits since the last Pullfrog review at e0b7487. The merges of main are left out except where they meet this PR.

  • Multi-part edits (More): an edit too large for one ANNOUNCE_REQUEST now spans several. Rust request_parts and JS requestParts set More on every part but the last, and the answerer collects the parts and applies them at the last one. Both reject any other message between the parts, and both reject the parts early once their Adds pass the limit or their removes outnumber the set. next_edit/REMOVE_BYTES are gone, so the earlier size-estimate thread no longer applies.
  • ROUTE_START Exact: a route that a mount presents at its mount point starts marked exact. Both ends' rekey::plan then keep it at its own prefix when an edit widens past it. The subscriber passes the exact routes to Router::edited_exact, which looks them up by the peer's names (via filed).
  • Mount carry-over: when a removed prefix's mount equals one an added prefix reads (Mounted: PartialEq), the mount keeps its stream under the new Prefix ID instead of ending and starting again.
  • JS DEAD batching and coverage: while a DEAD applies, consumes.batch and the #deferred reader map hold back refreshes. Each reader is then refreshed once, and a loss is never downgraded. #candidates checks floors.covers and ranks a broader claim as above a reader's prefix. Merged routes' claims at their old prefixes are kept as stale until the next floor.
  • Failover during a switch (model/front.rs): a copy that fails while a switch is pending now waits for the new source instead of asking the old one again. FrontServe selects first when the table changed underneath a failed copy.
  • CodeRabbit follow-ups: a failed send on JS AnnounceStream fails the session, finish() closes a writer that a queued send opened, and coarsen finds regions through heads and warns when it drops one.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using claude-opus-5-5 | 𝕏

Comment thread rs/moq-net/src/model/front.rs Resolved

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: dfac99334c17ed0f0df1e704bbab7537e14a2427.

I independently confirmed the open Pullfrog failover regression. A failed copy parks while an upstream switch is pending; a non-standing refusal clears the pending state before reselect, so if the serving route wins again, the parked track is never retried. Add a regression for that refusal followed by selecting the old route. I would fix this before merging; I am not duplicating the inline finding.

One ordered stream per direction is a good simplification for atomic edits and routing updates. I checked Rust/JS field layout and edit/rekey behavior and found no additional actionable mismatch in those paths. This still breaks interop with older lite-07-wip peers, as documented; the JS write-progress watchdog is separate follow-up work. Static review only: I did not run the model, builds, or interop.

…ce again

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI) · 10a2c1c

The parked-track regression is addressed. Reselecting the serving route now redispatches idle tracks even when the non-standing refusal already cleared the pending switch. The regression asserts the missing query back to the old source, and redispatch still skips ended, unwanted, or already-querying tracks. No new blocker in this delta. Static review; I did not run the test or rerun interop.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found. The failover stall from the last review is fixed, and its thread is resolved.

Reviewed changes

This run covers the one commit since the Pullfrog review at dfac993.

  • Failover after a refused switch (model/front.rs): in selected, the branch where the serving route is still best now always calls redispatch_idle, not only when a pending upstream was taken. A track that waited on a switch is now asked again even when a refusal without standing cleared upstream before the reselect. Running it more often is safe: redispatch already skips tracks that are unwanted, ended, not Idle, or stranded, and tracks whose serving route is closing.
  • Regression test: a_switch_refused_without_standing_asks_the_serving_source_again checks that the serving source gets the Query after a non-standing Refusal and a reselect back to that source.

Pullfrog  | View workflow run | Using claude-opus-5-5 | 𝕏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant