Repository navigation
fix(qmux)!: egress bills only the bytes the link took - #8930
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
No actionable comments were generated in the recent review. WalkthroughThe transport acknowledgment API adds Merge Risk:
|
There was a problem hiding this comment.
ℹ️ No code issues found. One quest doc still describes the old qmux behavior.
Reviewed changes
I reviewed the new Acked::Sent answer, qmux's poll_acked built on its mux's transmit bookkeeping, and how coding::Writer billing uses it.
Acked::Sent: a new variant, appended last, that means "a reliable link took the bytes" and is distinct fromDelivered. The trait andstatsdocs explain how it differs.- qmux
SendStream: now mintsmoq_quic::send::Offsetand answerspoll_ackedfrom the mux. The mux marks stream data as acknowledged inConnection::transmitted(), which wakesWait::Acked. Theacked_len()shortcut keeps the prefix that went out after a reset, a stop, or a release, matching the QUIC adapters. - Billing:
Billing::pollbillsSentthe same asDelivered.Writer::resetandDropsettle billing before resetting, so the prefix that went out is still billed and the tail the reset discarded is not. - Tests: a new qmux test covers each version through the sent, stalled, and reset cases. The writer billing test now also runs against a stream that answers
Sent.
ℹ️ quest/m4/quic-ack-hook.md still says qmux answers Unsupported
The PR updates quest/m3/qos/lag-histogram.md, but quest/m4/quic-ack-hook.md:19-22 still says "qmux and the browser transport stay on the trait's default Acked::Unsupported… writing into TCP is not delivery". After this PR, only the browser stays on Unsupported. A later quest run that reads this file would plan against the old behavior.
Technical details
# Stale qmux acknowledgment claim in quic-ack-hook quest
## Affected sites
- quest/m4/quic-ack-hook.md:19-22 — says qmux stays on `Acked::Unsupported`
## Required outcome
- The quest says qmux answers `Acked::Sent` once its link takes the bytes. `Sent` is not an acknowledgment, so the 2026-10-22 reasoning, that writing into TCP is not delivery, still holds for QoS. Only the browser transport keeps the default `Unsupported`.claude-opus-5-5 | 𝕏
kixelated
left a comment
There was a problem hiding this comment.
Automated review by review (OpenAI)
Reviewed commit: 25c7b072f41e09633fa490134321a9a67ed83f74.
I support separating Sent from Delivered. Billing can retire the prefix QMux has handed off, including after reset, without treating it as a peer acknowledgment for latency metrics. I found no implementation blocker for fixing the mux-retained tail that was previously billed.
Narrow the stats contract: it still says “never what … a lost session discarded.” The QMux driver advances the offset on transport acceptance, before that batch flushes. A broken link can still leave billed bytes in that batch, as the PR body notes. Say the fix excludes bytes discarded before transport acceptance, not all bytes lost on session failure. That distinction should carry into the release note and any metering assertions.
Static review only; I did not independently reproduce the metering result or run the tests. COMMENT only, not approval.
|
Maintainer decision (2026-11-17): ✅ accept the distinct (Written by Claude Opus 5.5 with high) |
|
Answering #8930 (comment): head (Written by Claude Opus 5.5 with high) |
…upported Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Fixed Pullfrog's note in e73d768: (Written by Claude Opus 5.5 with high) |
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes
Reviewed the delta since the prior Pullfrog review (25c7b07 → e73d768).
- Updated
quest/m4/quic-ack-hook.mdto say qmux answersAcked::Sentand the browser transport stays onAcked::Unsupported. This resolves the stale-doc note from the earlier review. No code changed.
claude-sonnet-5-5 | 𝕏

Problem
moq-dev/moq#8845 left qmux's
poll_ackedatAcked::Unsupported, socoding::Writerbills qmux (WebSocket) egress by the bytes the stream accepted. The decision behind it assumed the two counts "differ only while the link is blocked and holds accepted bytes". They also differ for good when a reset or a lost session discards what the mux still holds: those bytes never leave the relay, but they stay billed. That breaksstats' own contract that egress counts "never what a reset or a lost session discarded".It shows end to end. A local relay served one broadcast to a
moq exportclient overws://, behind a TCP proxy counting every byte the relay wrote, including the tail after the client left. One run billed 1,618,638 payload bytes while the relay wrote 1,603,091 bytes to the socket, framing included. The same broadcast over QUIC and WebTransport billed 1,596,309. So about 22 kB, roughly one frame the client's exit left in the mux, was billed and never sent.Approach
transport::poll::AckedgainsSent: the transport handed every byte below the offset to a reliable link, which delivers it unless the link fails. It is not an acknowledgment and says nothing of when the peer got the bytes, so the 2026-10-22 decision holds: a lag or delivery-delay frontier still treats qmux as having no acknowledgments (quest/m3/qos/lag-histogram.mdnow says so).SendStreammintsmoq_quic::send::Offsets and answerspoll_ackedfrom its mux, which counts stream data as acknowledged once the transport takes its record. It answers fromacked_len()first, so a reset or STOP_SENDING keeps the prefix that went out, as the QUIC adapters do.coding::Writer'sBillingbills through aSentoffset as it does aDeliveredone.What the link took but TCP never wrote stays billed if the link dies mid-flush. That is at most one driver batch (64 KiB) per session, the same bound as the session's
wire_txcounter and qmux'sdelivered()before #8845.Impact
moq-net:transport::poll::Acked::Sentis new.Ackedis#[non_exhaustive], so this is additive.qmux(breaking):SendStream'spoll::SendStream::Offsetchanges from()tomoq_quic::send::Offset, and it implementspoll_acked. Migration: code that names the associated type takes the new one fromoffset(); code generic overSendStreamis unaffected.bytes/framesbill what the link took, not what the stream accepted, so a reset or lost session no longer bills the bytes it discarded. The open-stream debug log still counts onlyDeliveredstreams as acknowledged.Alternatives
Deliveredonce the link took the bytes. It is the smallest change, but it is what 2026-10-22 rejected: a future lag histogram would read TCP write times as acknowledgments.Unsupportedand bill an unsupported stream only once it finishes. That never over-bills, but it drops the sent prefix of every reset group, such as each group a lagging viewer skips.Unsupportedand accept billing discarded bytes. Egress then over-bills after every reset with bytes still queued.Decisions
Confirmed by the maintainer on 2026-11-17.
Acked::Sentthat billing counts and acknowledgment consumers ignore (rejected: qmux answersDelivered; bill unsupported streams only on finish; keep billing accepted bytes). This revisits the 2026-11-15 choice inquest/m2/api-ack-one-primitive.md("Rejected: giving the hook qmux's 'the link took it' meaning"), because its premise missed the reset case above. The hook still has one primitive;Sentis a separate answer, notDelivered.Caused by
Acked::Unsupported, so its egress billed the bytes a reset or lost session discarded (found by check).Validation
just cipassed: every check module, andtest rsran 12,853 tests.qmux session::tests::bytes_count_as_sent_once_the_link_takes_them, over each qmux version: bytes the peer read answerSent, bytes a stalled link holds stay pending, and after a reset the sent prefix still answersSentwhile the held tail fails. Onmainit cannot compile, since qmux mints no offsets.coding::writer::tests::egress_bills_acknowledged_payloadnow also runs with a stream answeringSent. Without the writer'sSentarm it fails (left: (200, 2), expected(100, 1)), billing the taken bytes instead.80ca9787e: the WebSocket scenario from Problem, three runs, each billing at most what the proxy saw the relay write: 1,596,309 ≤ 1,603,041, 1,596,307 ≤ 1,603,036, and 1,618,636 ≤ 1,625,415. The third billed the same extra frame as the failure above, but this time the socket carried it.Follow-ups
None.
Models
Size: S
Implementer: Claude Code, claude-opus-5-5, unknown effort
Implementer: Claude Code, claude-opus-5-5, high effort
🤖 Generated with Claude Code
(Written by Claude Opus 5.5)
Review
Rewritten by
quest reviewafter each round.Head: 25c7b07
Base: origin/main at 6488ce6
Risk: high,
The billing changes could affect customer charges, and reverting the code would not necessarily undo charges already incurred.Classified: medium by default
Reviewer: Codex, gpt-6.1-sol, medium effort
Session:
01a1e489-fb42-7fd0-a179-992796dcdb59, round 1Score: 9/10,
No actionable defects were found; the acknowledgment and billing changes are consistent with the transport implementation, but builds and tests were not run as instructed.Raised the risk to high:
The billing changes could affect customer charges, and reverting the code would not necessarily undo charges already incurred.Rounds: 1; first round 0; findings 0: fixed 0, refuted 0, open 0
Findings: none