Skip to content

feat(moq-net)!: lite-07 sessions run the new route rule - #8987

Merged
kixelated merged 16 commits into
mainfrom
quest/m2/multicast/switch
Nov 18, 2026
Merged

kixelated merged 16 commits into
mainfrom
quest/m2/multicast/switch

Conversation

@kixelated

@kixelated kixelated commented Nov 17, 2026 •

Copy link
Copy Markdown
Collaborator

Claims quest/m2/multicast/switch.md.

Every lite-07 Rust session now drives route::relay::Router (#8901) on one Announce Stream per side, the draft specifies it, and the old route rule is deleted: ROUTE_HOLD, doubt, DEAD's moved list, the Route Key with its proof and model, qualified Node IDs, routes per prefix, path::Router, and RouteHandoff.tla. js/net implements the endpoint half only. lite-03 to lite-06 sessions are untouched on the wire.

What changed

  • Wire (lite-07, -wip): one unidirectional Announce Stream per side carries everything about discovery and routing. Message types are dense: 0 ANNOUNCE_START, 1 ANNOUNCE_END (id, lost), 2 ANNOUNCE_UPDATE, 3 ANNOUNCE_FLOOR, 4 SYNCED, 5 ANNOUNCE_REQUEST, 6 ANNOUNCE_OK, 7 ROUTE, 8 ROUTE_END, 9 DEAD, 10 DRAIN. A claim is cost, above, flags (unavailable, draining), priority, and a chain. SETUP carries a Node ID (relay and start) only from a routing node; the Route Key and Lite-06 Chain parameters are gone. Requests name the node announcing their claim. Routing-only messages on a session that does not route are a PROTOCOL_VIOLATION.
  • Rust: two sessions route when both declare a Node ID, which an origin does only when it forwards, the link is a peer's, and neither grant confines it. Every other session exchanges claims and adopts its peer's as its own.
  • JS: @moq/net is an endpoint. It names no node, ignores routing messages it would never be sent, and ranks claimed routes by the same rendezvous hash as Rust.
  • Draft: drafts/draft-lcurley-moq-lite.md rewrites Announce, Claims, Floors, Routing, Closed Sessions and Drains, and Resolution, and drops Holds, Refused Content, and the Route Key.
  • Deleted: rs/moq-net/verify/route, verify/route-key, their skills and nightly legs, moq_crypto::endpoint, the relay's cluster.route_key, the interop route-key lane, and four quests about the old rule.

Router and origin fixes

Driving the router from sessions showed these, fixed here:

  • Router::ways serves a node's own (adopted) claim through every neighbour that gave it, and another node's claim through every usable route, so a reader already holds another copy when the one it follows closes.
  • Every giver of a node's own claim ranks alike, so the origin's order picks the cheapest and then the newest: a reconnect serves the moment it lands, and a GOAWAY handover empties the old session.
  • An endpoint link costs nothing; a claim costs what the links it crossed do.
  • Router::withheld: a node's own claim that crossed a marked link is never told to a marked routing neighbour.
  • A claim its publisher ended ends at once; only a lost one is held for its route to return.
  • The origin reads both of its announce cursors before feeding its route node. Read one at a time, an endpoint that closed or replaced a broadcast briefly filed the old claim as one that crossed a marked link, and its reader was shown it lost and held it for 800 ms.
  • A routing peer granted an explicit everything pattern (**) is whole. It was refused as confined, which closed every real cluster link.

Found by the local review and fixed:

  • A prefix removed and added again in one edit, as a JS client changing hidden sends, is told its claims and SYNCED again. The router compared the sets, saw no change, and the peer was left with nothing.
  • A routing neighbour's claim changes since the last floor keep one per claim, so a peer that withholds the floor holds no more than its live claims.
  • A session sends one interest edit at a time and the next when it is answered, so a peer slow to answer holds one.

Found by the local review of the iterate and fixed:

  • A subscriber sends one interest edit at a time: the draft now says it MUST NOT send an edit before its last is answered, and both publishers refuse a second with PROTOCOL_VIOLATION. Before, a peer whose side of the stream was blocked could queue edits on the publisher without limit. Both subscribers already waited.
  • A session that does not route, whose readers want more prefixes than one ANNOUNCE_REQUEST holds (17 prefixes of 4 KB, or a few thousand short ones), asks for them more coarsely inside its grant, as the path rule on main did. Without it the peer closed the session.
  • The route node publishes its next deadline before it releases its lock. Two sessions on different threads could leave the timer disarmed with a deadline pending, so a lost claim stayed unavailable.

Found by Pullfrog's review of the ready head and fixed:

  • A claim whose path the receiver could not resolve was dropped whenever any prefix was removed, though its publisher kept it, so the publisher's later update or end of it closed the session. It now goes only when no earlier prefix is left for it to lie in, in Rust and JS.
  • A claim's chain names at most 31 relays on the wire. The decoder took 32 and the router then refused it as malformed.
  • The draft now says what the router does: a routing neighbour is sent routes only while its set is the empty prefix and may ask for nothing else, nothing but an answer precedes the first ANNOUNCE_OK, an unresolved claim keeps its Announce ID across an unrelated removal, and a node may cut off a peer that reads too slowly.

Found answering the maintainer's question about a clock that goes back:

  • A routing neighbour whose Start is before a session of its relay already known closed is refused at setup with route::Error::Regressed, and the refusing side logs why. Before, nothing refused or logged it: a node holding such a floor silently ignored every route through the restarted relay's sessions until its clock passed the old Start. moq-relay is not exposed, since its identity is random per start; only a library user that pins origin::Config::node across restarts is. The config's docs and doc/bin/relay/cluster.md say so.

Validation

On the head d98a191b42:

  • just check: 12,566 Rust tests pass (161 skipped), 3,314 @moq/net tests pass, drafts, drills, and the rest green, run inside nix develop.
  • just test interop --all: 208 checks pass, none fail or skip. This runs the built relay against Rust, Node, Bun, and Python clients over WebSocket and WebTransport on lite-05, lite-06, lite-07, and moq-transport, including the claims and socket lanes on lite-07.
  • Real sockets: moq-relay upstream_links builds two-tier clusters of real relays over TLS on TCP speaking only lite-07 and checks tiered regions, that an edge never transits between cores, and an uplink transiting both ways. moq-tokio migrate moves live subscriptions in both directions across a GOAWAY and checks the old session closes once idle.
  • Second way: the route layer's simulation and bake-off tests (route/tests) still run the prototype (multi.rs) beside the router, and the drill mutations were regenerated and shown to fail both drills when applied.
  • New regression tests: an_endpoint_closing_its_broadcast_ends_it_for_its_reader, an_endpoint_replacing_its_broadcast_ends_the_old_one, the ** case in a_routing_neighbour_keeps_a_whole_permit, a_batch_keeps_one_change_per_claim, interest_changes_wait_for_the_unanswered_edit, the re-added prefix case in the router's edit test, a_start_that_went_back_is_refused, a_set_past_one_request_is_asked_for_more_coarsely, and the second-edit cases in a_lite07_request_naming_no_prefix_or_nested_is_a_violation and its JS twin. Also an_unresolved_claim_outlives_another_prefixs_removal with its JS twin, which fails without the fix, and a_claim_chain_at_the_hop_limit_is_refused. The deadline race has no test: it needs a thread interleaving the harness cannot force.

Not run: the nightly benches and the TLA+ models that remain (failover, resume, subscribe, reconnect).

moq-c session_tokens_and_grants and server_accepts_a_session race a client closing itself against the test closing it. They fail 15 of 100 runs at the merge base on lite-06, so they are not from this change and are not fixed here. Owner: quest/m0/test-flakes-3/c-server-accepts-session.md, which #8964 adds.

@moq/net "a reset Announce Stream closes moq-lite-07 with PROTOCOL_VIOLATION" timed out in 2 of 7 full-suite runs earlier on this branch. #9024 fixed it on main and is merged here.

Open, not root-caused: just test interop --all failed its finish lane in 3 of about 30 runs, all while the machine was loaded by other builds, and in none of 19 runs once the load fell. Each time the relay answered UNROUTABLE to the first few readers that subscribed to a broadcast about 5 ms after its publisher's session announced it, on a session that had just ended another broadcast; readers a few milliseconds later were served. Tracing ruled out the route node's own refusals, which leaves the origin beneath it. Whether main does the same was not checked. It is written up as quest/m0/test-flakes-3/interop-finish-unroutable.md, added here.

The moq-net auth/waiting/moq-lite-07-wip/1/1 bench still hangs on this head, in warm-up, as it does on main. quest/m0/nightly/lite07-route-hang.md owns it and asked for this rerun as its first step; its plan is then to fix it on top of this PR. The other three benches it lists were not run.

Impact

Breaking, wire and public API.

Wire: moq-lite-07-wip changed incompatibly under the same name. It also gains rules a third-party implementation must follow: one ANNOUNCE_REQUEST edit at a time, the next after the ANNOUNCE_OK; a claim chain of at most 31 relays; and a routing neighbour asks for the empty prefix or nothing. A build from before this PR does not interoperate with one after on lite-07. The default version set excludes lite-07, so only a deployment that enabled it by name is affected: it upgrades both ends of a lite-07 link together, or links them over lite-06 for the rollout. lite-03 to lite-06 and moq-transport are unchanged.

The Route Key never shipped: the release branch has no moq-crypto crate and no route key. The notes below about cluster.route_key, moq_crypto::endpoint, and origin::RouteKey matter only to a build from main, and the release cut should drop the ## Unreleased entry "Origins prove a route key" from rs/moq-net/MIGRATE.md rather than publish it (sh/migrate.sh refuses that edit outside a cut).

Migration notes:

  • moq-relay: the cluster.route_key setting is removed. Delete it from the config; nothing replaces it.
  • moq-crypto: moq_crypto::endpoint, its default endpoint feature, and the serde feature are removed with the Route Key. Nothing replaces them; the crate keeps provider and cpu.
  • moq-net: a lite-07 session whose routing peer declares a Start before a session of that relay known closed fails with route::Error::Regressed. It reaches only a node that keeps origin::Config::node across restarts while its wall clock goes back.
  • @moq/net: WireRoute.node and WireRoute.depth are replaced by claimed?: boolean; Floors, the coverage plumbing, and route key support are removed; a publisher's claims option is per instance.
  • moq-net: origin::RouteKey and the origin's route key setting are removed. Nothing replaces them: a routing node is one whose origin forwards (Producer::with_proxy) on a peer link with an unconfined grant.
  • A revoked grant shows the claims it no longer covers as lost (offline) on lite-07, not ended.

Decisions

Recommendations from the brief, confirmed or changed:

  • One identity per node: kept. Node ID is (Config::node, start) with the start in wall-clock microseconds.
  • Rendezvous hash basis: the standard FNV-1a basis (0xcbf29ce484222325) over the path and instance, keyed with the claiming relay in Rust. JS keeps the rendezvous hash for claimed routes, over the instance.
  • Dense message type numbers: kept, and ANNOUNCE_RESTART is removed. Another instance is another claim.
  • The recommended wire: kept, with ANNOUNCE_END carrying lost.

Taken where I would have asked:

  • Malformed forwarded content fails the stream; only a resolved path the node cannot hold is dropped and counted.
  • Producer::with_proxy stays as the switch that makes an origin a routing node.
  • A lite-07 session that only publishes declares no identity.
  • The JS endpoint always names node 0 and ignores a named one.
  • The m3 cache-routing prototype test is deleted with the rule it exercised.
  • Every giver of an adopted claim ranks alike (above), which keeps the origin's documented order: newest wins a full tie.
  • One edit at a time is a wire rule, not a queue limit: a limit would close a peer the draft allowed, and stopping the read to push back can deadlock two peers that each wait on the other's answer.

Decided by the maintainer on 2026-11-17, in this comment:

  • ✅ This PR merges without retiring the moq-cluster draft and its moq-transport binding, and without making Config::update_hold, Source, RouteInfo::source(), and Consumer::local() crate-private. Both land in the follow-up quest before the cut. Rejected: holding this PR for them.
  • ✅ The follow-up quest "What the route switch left" is kept.
  • ✅ The adopted-claim gap is required before the cut, not left to the implementer: a claim adopted over a marked link that does not route must not reach an upstream through a peer. A rollout with two releases side by side is when a relay's upstream does not route, so an edge pair could carry a broadcast between two cores. The second, marked identity stays a recommendation.

Follow-ups

The quest is rewritten as "[L] What the route switch left" and keeps:

Before the cut:

  • Retire the moq-cluster draft with its moq-transport binding (about 540 sites).
  • Make Config::update_hold, Source, RouteInfo::source(), and Consumer::local() crate-private; their callers are tests and benches outside the crate that need another way in first.
  • An adopted claim that crossed a marked link still floods to unmarked routing neighbours. Recommended: a second, marked identity for what a node adopts across a marked link.

Left for the implementer to decide:

  • The announce-churn mitigations carry no marker saying they are temporary.
  • The path-rule benches (drain, repath, follow) have no equivalent on the new router.
  • Measured costs left: re-offering a route re-sends its claims; one ANNOUNCE_UPDATE per broadcast to a non-routing peer on a cost change; a prefix swap re-tells shared claims; a relay whose only route closes ends its subscriptions.
  • The proof's binding to the router that ships, and the quests still written against the old wire.

New quest, in the open flakes epic at m0 where flakes go: "[M] A request right after its announcement is not refused unroutable" (quest/m0/test-flakes-3/interop-finish-unroutable.md), for the intermittent finish lane failure above.

The follow-up quest also gains: remove poll::Session::export_keying_material, which has no caller since the Route Key proof left and was never released.

No quest for the two test races the maintainer asked about: the moq-c race has an owner, named in Validation, and the @moq/net race is fixed on main.

Caused by

  • moq-dev/moq#8901 - the router served an adopted claim through one giver and another node's through one route, charged an endpoint link, and told a marked neighbour claims that crossed a mark (found by check)

Models

Size: XL
Implementer: Claude Code, claude-fable-5-2, unknown effort
Implementer: Claude Code, claude-fable-5-2, unknown effort (iterate)

🤖 Generated with Claude Code

Review

Rewritten by quest review after each round.

  • Head: d98a191

  • Base: origin/main at 789ece4

  • Risk: high, it changes .github/workflows/nightly.yml, which is always high: workflows run with the repository's secrets and decide what merges and ships

  • Classified: high by the always-high paths

  • Reviewer: Codex, gpt-6.1-sol, xhigh effort

  • Session: 01a1eb12-3e83-73f3-a74b-92e4f52171e3, round 4

  • Score: 10/10, Source inspection found no new issues in the announce-ID lifetime fixes, synchronized chain limits, or draft updates; builds and tests were not run.

  • Rounds: 8; first round 3 (high 2, medium 1); findings 6 (high 3, medium 3): fixed 6, refuted 0, open 0

  • Findings: none

kixelated and others added 8 commits November 16, 2026 21:49
Work in progress: Rust sessions drive route::relay::Router; the old rule's
router, proof, and tests are deleted.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…fixtures

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…route key

js/net declares no Node ID, sends and reads claims only, and refuses route
messages. moq-crypto loses its endpoint key module and the relay its
route_key option.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The moq-lite draft specifies lite-07's one announce stream per side: claims,
routes, floors, DEAD and DRAIN, with the Node ID as a relay and its start.
The route key, its proof and model, the old rule's TLA+ model, and the quests
they answered are gone. js/net keeps the rendezvous hash for claimed routes.

Fixes found driving the rule from every crate's tests:
- a routing peer granted an explicit everything pattern is whole, not confined
- the origin reads both of its cursors before feeding its route node, so an
  endpoint that closes or replaces a broadcast ends it for its reader instead
  of showing it lost
- every session that gives a node its own claim ranks alike, so a reconnect
  serves the moment it lands and a GOAWAY handover empties the old session

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…itch

The route key flags main still listed in lite/version.rs stay removed, as does
moq-crypto's endpoint test, and a publisher test main added names no node.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@kixelated kixelated changed the title lite-07 sessions run the new route rule feat(moq-net)!: lite-07 sessions run the new route rule Nov 17, 2026
kixelated and others added 3 commits November 17, 2026 04:53
…r holds

Review findings:
- a prefix removed and added again in one edit is told its claims and SYNCED
  again, as the peer dropped them
- a routing neighbour's claim changes since the last floor keep one per claim
- a session sends one interest edit at a time, the next when it is answered

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…itch

The draft keeps neither side's dropped references: main's WebRTC binding and
Padding Stream, and this branch's Route Key.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…itch

The nightly model matrix keeps main's warm leg and drops the route legs, whose
model this branch deletes.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Maintainer decision (2026-11-17, asked in the coordinating session with this PR's report):

  1. Picked to merge without the two deferred items. Retiring the moq-cluster draft with its moq-transport binding, and making Config::update_hold, Source, RouteInfo::source(), and Consumer::local() crate-private, land in the follow-up quest before the cut. Rejected: holding this PR until both are in it.
  2. The follow-up quest "What the route switch left" is kept.
  3. The adopted-claim gap is required before the cut, not left to the implementer: a claim adopted over a marked link that does not route must not reach an upstream through a peer. The quest's recommended fix (a second, marked identity for what a node adopts across a marked link) stays a recommendation; that the gap is closed before the cut is the decision.

(Written by Claude Fable 5.2 with high effort)

kixelated and others added 2 commits November 17, 2026 11:18
…d the maintainer's decisions

A relay keeping its identity across restarts with its clock behind its last
start peered and routed nothing: every node holding a closed session of the
later run ignores the new run's sessions. The session is refused at setup
instead, with the reason logged, as the draft already said.

The quest records the 2026-11-17 decisions and moves the adopted-claim gap to
what lands before the cut.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…itch

moq-crypto keeps only the provider and the CPU check: main moved e2ee to
moq-e2ee and left the endpoint key, whose one caller was the Route Key this
branch removes.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
kixelated and others added 2 commits November 17, 2026 12:54
…est, timer published under the lock

A lite-07 subscriber sends one edit and waits for its ANNOUNCE_OK, so a
publisher holds one unanswered edit however slowly its peer reads. The
draft requires it and both publishers refuse a second.

A non-routing session whose wanted prefixes pass one ANNOUNCE_REQUEST
asks for them more coarsely inside its grant, as the path rule did,
instead of sending a set the peer closes the session on.

The route node publishes its next deadline before releasing its lock, so
a deadline read earlier on another thread cannot replace a later one.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…itch

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review November 17, 2026 21:49

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No blocking defects in the code I read. The draft misstates two router rules a third-party implementation would hit, and some release notes for the deleted Route Key are stale.

Approach: sound. Running every lite-07 session on the one relay::Router, and deleting the path rule, holds, doubt and the Route Key instead of keeping them alongside, is the right call for a -wip wire that isn't in the default version set.

Reviewed changes I read route/relay.rs, the rewritten route/node.rs, model/origin.rs, lite/setup.rs, lite/session.rs, and the coarsening and one-edit fixes in 7c4c1532e. I compared the rewritten draft against the codec and router, and looked for stale references to the deleted rule across the tree.

  • Router session API: ask/asking/answered keep a queue of unanswered edits, and Msg::Edited applies the oldest one. edit now applies removes and adds as the edit names them, so removing a prefix and adding it again re-tells its claims.
  • Serving every way: ways() serves an adopted claim through every giver and another node's claim through every usable route. withheld/crossed_mark keep a claim that crossed a mark away from marked neighbours.
  • Error::Regressed: connect refuses a routing neighbour whose start is below a session of the same relay already known closed.
  • route/node.rs: the session node is rewritten around Outbox { wire, serve, limit }, with limit = held*2 + 1024, plus per-claimant Slots with Orphan hand-off. The timer deadline is now published under the lock.
  • Origin: Consumer::node_id declares a Node ID only for a proxying, peer-linked, whole-tree pair. OwnClaims::poll drains both cursors before feeding. Claimed routes rank by the standard-basis rendezvous hash, the same as JS routeKey.
  • SETUP: lite-07 PARAM_HOP is now {relay, start} bytes, and a relay of 0 or trailing bytes is refused.
  • Non-routing asks: coarsen widens a want set until it fits one ANNOUNCE_REQUEST. Each step merges a group into its longest common prefix or its grant region, otherwise it drops the last prefix and logs a warning, so the loop always ends.
  • Draft: the message type numbers, field orders, flag bits, SETUP parameter IDs, the ROUTE/DEAD/DRAIN validation, and the UNAUTHORIZED rule for a confined grant all match the Rust codec. The two mismatches are inline.

ℹ️ The Route Key never shipped, but a release note for it would

The release branch has no moq-crypto crate and no route_key anywhere, so the Route Key only ever existed on main. Even so, rs/moq-net/MIGRATE.md still has a ## Unreleased entry, "Origins prove a route key (#7937)", which the next release cut would publish for an API that no longer exists. The Impact section's migration notes for cluster.route_key, moq_crypto::endpoint/serde, and origin::RouteKey likewise describe removing things no release ever had.

Technical details
# Stale release notes for the never-released Route Key

## Affected sites
- rs/moq-net/MIGRATE.md:38-41 — `### Origins prove a route key (#7937)` under `## Unreleased` describes `origin::Config::route_key` and `poll::Session::export_keying_material` proofs, which this PR deletes.
- PR body, Impact → Migration notes — the `moq-relay` `cluster.route_key`, `moq-crypto` `endpoint`/`serde`, and `moq-net` `origin::RouteKey` bullets.

## Required outcome
- The release cut drops the `Origins prove a route key` entry rather than shipping it.
- The Impact section says so, since `sh/migrate.sh` rejects editing `MIGRATE.md` outside a cut, and notes that the route-key bullets need no user migration because none was released (`git show release:rs/` has no `moq-crypto`).
- Check whether `export_keying_material` survives this PR. If it was only for the proof, the second sentence of that note is stale too.

ℹ️ Docs still point to the deleted verify-route skill

Two unchanged files say "verify-route and the route proof cover how routes settle", but this PR deletes that skill and its TLA+ model. A reader following them looks for a check that no longer exists, when only the Agda proof (#8904) is left.

Technical details
# Stale `verify-route` references

## Affected sites
- .claude/skills/verify-warm/SKILL.md:63 — "`verify-route` and the route proof cover how routes settle"
- rs/moq-net/verify/warm/WarmSettle.tla:38 — "(`verify-route` and the route proof cover how routes settle)"

## Required outcome
- Both point to what now covers route convergence (the Agda proof from #8904), or drop the reference.

ℹ️ Two checks did not finish

I asked two other checks whether a peer can make the new Rust session wiring panic or cost unbounded work, and whether JS and Rust agree byte for byte on the lite-07 wire. Neither had reported when this review was submitted, so this review does not cover those two questions.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using claude-opus-5-5 | 𝕏

Comment thread drafts/draft-lcurley-moq-lite.md Resolved
Comment thread drafts/draft-lcurley-moq-lite.md Resolved
@pullfrog

pullfrog Bot commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

Follow-up to my review above. The two checks I flagged as unfinished have now reported. One found a real bug this PR introduces.

⚠️ Removing any prefix drops every claim whose path couldn't be resolved

At ANNOUNCE_OK, Rust Routed::ok (rs/moq-net/src/lite/subscriber/routed.rs:482-492) ends every claim with relative: None. That covers every claim whose path this node couldn't hold, for whatever prefix was removed. It calls decoder.end(id) and removes the claim from claims.

The peer drops only the claims inside the removed prefix, so for any other unresolvable claim it still holds the Announce ID. If it later sends ANNOUNCE_UPDATE or ANNOUNCE_END for that ID, the Change/End arms (routed.rs:586-596, compress.rs:134-141) return PROTOCOL_VIOLATION, and the session closes even though the peer behaved correctly. The draft says the opposite: an unresolvable claim's ANNOUNCE_START "still assigns its Announce ID, [and] the messages naming it stay valid".

The JS twin does the same thing: js/net/src/lite/routed.ts:419, path === undefined || removed.some(..) → #end(id, true).

One way to reach it: the peer announces a claim whose path exceeds 32 segments or 4 KB through Compression. A reader then stops wanting some unrelated prefix, and the peer later ends the long claim.

Technical details
# Unresolvable claims dropped on unrelated prefix removal

## Affected sites
- rs/moq-net/src/lite/subscriber/routed.rs:482-492 — `within` is `relative.is_none_or(..)`, so every unresolvable claim is ended on any removal.
- js/net/src/lite/routed.ts:419 — same rule.

## Required outcome
- An honest peer's later ANNOUNCE_UPDATE or ANNOUNCE_END for an unresolvable claim outside the removed prefixes never closes the session.
- Regression test in Rust and JS: an unresolvable START, removal of a disjoint prefix, then END for that id. The session stays up.

## Suggested approach
- Keep unresolvable claims (and their `refused` decoder ids) across removals: the receiver can't tell which prefix they lie in, and keeping one costs only its slot until the peer ends it.

ℹ️ Smaller items

  • A 32-relay chain. Rust's decoder accepts a claim chain of exactly 32 relays (announce.rs:244), but well_built (route/relay.rs:2428, chain.len() < MAX_HOPS) rejects the whole batch with Malformed, which closes the session. The draft says only that the receiver MUST NOT take such a claim. JS accepts it. Either the decoder or the draft should state the 31-relay limit, and dropping the claim fits the draft better than closing the session.
  • Nothing else confirmed. Message type numbers, field order, flag bits, ANNOUNCE_END lost, the Node ID parameter, the rendezvous hash basis, and the one-edit-at-a-time refusal agree between JS and Rust. The robustness check found no panic, expect, or overflow a peer can reach in route/node.rs or the lite-07 session code.

Pullfrog  | View workflow run | via Pullfrog | Using claude-opus-5-5 | 𝕏

…l, cap a claim chain at 31

A claim whose path a receiver could not resolve was dropped whenever any
prefix was removed, while its publisher kept it, so the publisher's later
update or end of it closed the session. It now goes only when no earlier
prefix is left for it to lie in. Rust and JS.

A claim's chain names at most 31 relays on the wire: 32 was decoded and
then refused by the router as malformed.

The draft says what the router does: a routing neighbour is sent routes
only while its set is the empty prefix and may ask for nothing else,
nothing but an answer precedes the first ANNOUNCE_OK, and a node may cut
off a peer that reads too slowly.

Adds a quest for the finish interop lane's intermittent UNROUTABLE.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Answers to the findings in the review summary and its follow-up, all in d98a191:

  • Unresolvable claims dropped on an unrelated prefix removal: fixed in Rust (Routed::ok) and JS (#ok). Such a claim now goes only when no prefix that was in effect before the edit survives it, since then every earlier claim is dropped by both ends; otherwise it keeps its Announce ID for the peer to end. The draft's Edits section says so. Regression tests: an_unresolved_claim_outlives_another_prefixs_removal in Rust and "a lite-07 unresolved claim outlives another prefix's removal" in JS, which fails without the fix.
  • A 32-relay chain: the wire limit is now 31 in the draft and in both codecs, since a chain that would reach 32 is never taken and so never sent. That keeps the router's rule that a batch the route rule could not have built closes the session, and makes the decoder agree with it. Test: a_claim_chain_at_the_hop_limit_is_refused.
  • Stale Route Key release notes: the PR's Impact section now says the Route Key never shipped, that its notes matter only to a build from main, and that the release cut drops the "Origins prove a route key" entry from rs/moq-net/MIGRATE.md, which sh/migrate.sh refuses to edit outside a cut. poll::Session::export_keying_material survives this PR with no caller; removing it across moq-net, moq-quic, moq-tokio, and moq-uring is listed in the follow-up quest (quest/m2/multicast/switch.md), recommended before the cut.
  • The two inline draft findings are answered in their threads.

(Written by Claude Fable 5.2)

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes The single commit since the prior pullfrog review (d98a191) fixes that review's two follow-up findings and brings the draft in line with the router.

  • An unresolved claim now survives an unrelated prefix removal: Routed::ok and its JS twin #ok() drop a claim whose path could not be resolved only when the edit leaves no prefix that was in effect before it, instead of on any removal, so an honest peer's later ANNOUNCE_END or ANNOUNCE_UPDATE for that id no longer closes the session. Both sides gained a regression test that fails without the fix.
  • A claim's chain now caps at 31 relays on the wire: encode_claim refuses a chain at route::MAX_HOPS, decode_claim refuses a count at it, and JS CHAIN_MAX drops to 31 with its > comparisons unchanged. The new codec test round-trips 31 and refuses 32 in both directions.
  • The draft states what the router does: a routing neighbour is sent routes exactly while its set is the empty prefix, and an edit that empties it drops every route and its claims at the ANNOUNCE_OK with no ROUTE_END; a set that is neither {} nor {""} and any message before the first ANNOUNCE_OK (other than ANNOUNCE_REQUEST) are PROTOCOL_VIOLATIONs; a node may cut off a peer whose reads fall behind the changes sent to it.
  • Two quests record what is left: quest/m0/test-flakes-3/interop-finish-unroutable.md writes up the intermittent finish lane UNROUTABLE, and quest/m2/multicast/switch.md adds removing poll::Session::export_keying_material before the cut alongside the stale MIGRATE.md release note.

I checked the survives computation against the publisher's drop rule for plain removals, a remove plus re-add of one path, and the {""}→{} routing edit, and confirmed a 31-relay chain is the longest a conforming origin can advertise (Origin::forwardable filters at MAX_HOPS), so the tightened encoder never fires on a legitimate send. 16 moq-net tests (lite::announce::chain_tests, lite::subscriber::routed::tests) and 208 @moq/net tests in subscriber.test.ts and announce.test.ts pass.

Pullfrog  | View workflow run | Using deepseek-v4.1-flash | 𝕏

@kixelated

Copy link
Copy Markdown
Collaborator Author

Maintainer decision (2026-11-18, asked in the coordinating session with this PR's report):

  • Land now, and root-cause the intermittent finish lane failure next. quest/m0/test-flakes-3/interop-finish-unroutable.md owns it and an agent starts it on main once this lands. Rejected: holding this PR until the cause is found; checking main first.

Still open with the maintainer, and not blocking this PR since lite-07 stays wip until the cut: whether "a subscriber MUST NOT send an edit before its last is answered" stays a MUST. They asked whether it is enforceable and for a fuller explanation.

(Written by Claude Fable 5.2 with high effort)

@kixelated

Copy link
Copy Markdown
Collaborator Author

Answering the decision in #8987 (comment): it asks for no change to this head. The quest it names is already in this PR, and the open question about the edit rule is recorded for a later PR before the cut.

(Written by Claude Fable 5.2 with high effort)

@kixelated
kixelated added this pull request to the merge queue Nov 18, 2026
Merged via the queue into main with commit 61f8956 Nov 18, 2026
6 checks passed
@kixelated

Copy link
Copy Markdown
Collaborator Author

Maintainer decision (2026-11-18, in the coordinating session), closing the question left open in #8987 (comment): the edit rule stays a MUST. A lite-07 subscriber sends no announce edit before its last is answered, and a publisher closes on a second. Rejected: dropping the rule and having the publisher fold unanswered edits into one.

(Written by Claude Fable 5.2 with high effort)

kixelated added a commit that referenced this pull request Nov 18, 2026
main's new route rule (#8987) replaced the code this PR's route tests
targeted. cargo-mutants over the route code it changed left 31 survivors;
each now has a test, an exclusion with its reason, or is gone with the
dead code it mutated. Exclusions naming deleted code are dropped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kixelated added a commit that referenced this pull request Nov 18, 2026
#8987 deleted the old rule's route tests, so code it kept lost the tests
that killed its mutants. A sweep of the rest of the route code found 27
more survivors: each gets a test or an exclusion with its reason, and the
dead code behind four goes (the reader's unread change list, a floor guard
that always holds, and a check another check covers).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kixelated added a commit that referenced this pull request Nov 18, 2026
The switch (#8987, #9052, #9053) replaced lite-07's path rule. Delete the
switch quest, the Playwright bump whose only reason was the route key lane,
and route fan-out, whose surviving cost moves to route benches. Retarget
federation, the routing guide, route wakes, handoff routes, regrant,
interest-stream caps, the mesh condition, and route memory to the route
layer on main, and trim the two route epics to what still stands.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant