Skip to content

fix(watch): release the audio backlog a ring trailing the clock will skip - #8893

Merged
kixelated merged 3 commits into
mainfrom
quest/m0/interop-green/replacement-audio
Nov 16, 2026
Merged

kixelated merged 3 commits into
mainfrom
quest/m0/interop-green/replacement-audio

Conversation

@kixelated

@kixelated kixelated commented Nov 16, 2026 •

Copy link
Copy Markdown
Collaborator

Claims quest/m0/interop-green/replacement-audio.md. Leaves work: the quest is rewritten to what remains, and the case stays quarantined.

Problem

The replacement media case's audio stalled past its 1039 ms switch bound under CPU load. Traced with batched in-page instrumentation (not in this PR) on failing runs, every slow switch had the same shape:

  1. The decoder resets the ring and the shared clock on the new instance. The incumbent's audio subscription replays its retained backlog (up to delay + 2 s) oldest first, so the first stale audio frame anchors the fresh clock.
  2. The live picture lands and pulls the clock 1 to 2 s ahead. The ring now trails the clock past its 200 ms stretch bound and skips onto it.
  3. The decode gate (Backpressure) metered the backlog by the ring's own playhead: one floor plus a 30 ms window per worklet state report. The ring skipped each window as late on arrival. Catching up 1.7 s took about 20 round trips, which a starved main thread stretched past the bound. One failing lite-07 fallback: the clock jumped 1663 ms ahead, and the playhead then gained about 260 ms per 65 ms of wall time.

Approach

Backpressure now counts from where the ring plays next: the clock once the playhead trails it past the bound, since the ring then skips all the way onto the clock, and the playhead otherwise, since within the bound the ring stretches onto the clock playing everything. What the ring will skip is released at once. AudioBuffer passes it the clock as the ring rendered the reported playhead (offset + state.time, the same reading it sends the worklet) and the ring's bound, floored at the stretch window as the ring's own threshold is. The replay harness (replay.ts) drives the gate the same way.

The case stays quarantined. Under the same load a second cause remains, and the quest is rewritten to it. Every group the viewer skips restarts the decoder and trims its 80 ms Opus warmup, so one lost 20 ms group costs 100 ms of audio. When groups go missing more often than every 80 ms, nothing plays at all: one fallback stayed silent past its bound. The rest are underruns of the case's fixed 50 ms delay on a saturated host, which its sibling cases share.

Impact

  • No public API or wire change. Backpressure (@internal) gains an optional bound on its constructor and setLatency, and an optional clock on observe.
  • Behavior: after a re-anchor, a ring trailing the clock catches up as fast as the decoder can work through the backlog.

Alternatives

  • Drop frames behind the clock before decoding instead of decoding them for the ring to skip. It saves a little decode work, but every dropped frame becomes a gap, and today a gap trims 80 ms of warmup (the remaining cause).
  • Start the audio consumer at the live edge after an instance change. It cannot know the edge before the newest group lands, and the backlog is how the arrival estimator learns.
  • A larger switch bound with a load term. That loosens the assertion rather than fixing the defect.

Validation

  • just test media --media-cases replacement on fedora, pinned with busy loops to 6 or 8 cores (taskset), and unpinned with other agents holding the host near saturation. Each round ran the old and new buffer.ts back to back.
    • Before: 15 of 161 switches took 600 ms or more, and 1 missed the bound with audioStalled:true.
    • After: 1 of 165 took 600 ms or more, and none missed it.
    • Both still fail the steady-state audio tone check about one run in ten at this load. Under the same pinned load, the default media run without replacement failed audio rendition switch back 1 of 4. Main's own scheduled run 43729752212 failed audio tone: after resume (69/81).
  • New unit tests (Backpressure, and AudioBuffer's clock wiring) fail without the change. One covers a floor short of the bound (50 ms against 200 ms), where an earlier head counting from clock - bound released only through 1860 ms of a 2000 ms clock, so the skip landed on an empty ring and stalled (Pullfrog's finding).
  • After that fix, unloaded just test media --media-cases replacement: passes. The loaded before/after numbers above were measured on the earlier clock - bound head; the fix releases strictly more after a skip.
  • just test audio-quality --replays --enforce: within budget. Counting from the clock moves the two Opus relay-mic rows, re-budgeted: relay-mic lag p50 -0.3 to 0.2 ms and p95 9 to 7.7 ms, relay-mic-stall p50 1.7 to 2.3 ms. Per-sample traces show the difference is the startup catch-up from the retained backlog, which converges faster (lag at 1.75 s: 46.7 against 74.8 ms; accelerated audio 233 against 250 ms) and settles a fraction of a millisecond apart; trims, skips, underruns, and stalls are unchanged.
  • just check, just ci: pass. The final build passed 3 of 3 loaded runs of the case.

Decisions

Taken on the maintainer's behalf, for review:

  1. Keep replacement quarantined, with the quest rewritten to the gap-warmup cause, instead of lifting it now. Lifting now would add about a 10% flake on a loaded host. On an idle CI host it would likely pass, as it did on 11-14 and 11-15.
  2. In the rewritten quest, recommend re-priming the restarted decoder on the frames before the gap (restamped to end at the first frame after it) over dropping the warmup trim after a gap (that reverses moq-dev/moq#5569) or decoding across gaps with output restamped by input order (AAC priming breaks the one-to-one mapping).
  3. Release what the ring will skip and let it decode and drop, rather than drop it encoded (see Alternatives).
  4. Re-budget the relay-mic and relay-mic-stall replay lag p50 rather than treat the 0.6 ms rise on the stall row as a regression: the README calls a rising steady-state lag a regression, but the shift is the residual after a faster startup catch-up, and relay-mic moved toward zero.

Follow-ups

None new: quest/m0/interop-green/replacement-audio.md is rewritten to the remaining work, with the recommendation in its Plan.

Seen on main, not this quest's: scheduled run 43729752212 (11-15) failed datagram_tracks_javascript (panic at datagram_tracks.rs:33), go -> js timeouts (36 to 37 s of 30) in both matrices, datagram rust -> js-native-node/bun (missing groups or clean end) on moq-transport, and the after resume tone above.

Caused by

  • moq-dev/moq#8507 - made a started ring's decode gate release only a floor and window past its own playhead, including while stalled or trailing the clock, so a backlog the ring skips drains one state report at a time (found by check).

Models

Size: M
Implementer: Claude Code, claude-opus-5-5 (iterate, S, arm A)

🤖 Generated with Claude Code

(Written by Claude Opus 5.5 with medium effort)

Review

Rewritten by quest review after each round.

  • Head: ef3b7ab

  • Base: origin/main at 3e7987a

  • Risk: medium, the base has no RISK.md to name what is always high or customer-facing, so every change is medium

  • Classified: medium by default

  • Reviewer: Codex, gpt-6.1-sol, high effort

  • Session: 01a1e42e-11dd-74b1-9bc2-17934c06ab6f, round 2

  • Score: 8/10, No actionable regressions found: the updated decode gate matches the ring’s skip destination and adds targeted regression coverage, though builds and tests were not run as instructed.

  • Rounds: 2; first round 0; findings 0: fixed 0, refuted 0, open 0

  • Findings: none

kixelated and others added 2 commits November 16, 2026 01:45
…skip

After a route switch the audio ring re-anchors on the incumbent's retained
backlog while the live picture moves the shared clock ahead. The ring skips
onto the clock, but the decode gate metered the backlog by the ring's own
playhead, one floor and window per worklet state report, so on a starved page
the audio stayed stalled past the replacement case's switch bound. The gate
now counts from the bound behind the clock when the playhead trails it.

The replacement case stays quarantined: its quest is rewritten to the
remaining cause, a decoder restart at each skipped group trimming 80 ms of
warmup.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title Restore the replacement media case: audio after a route switch fix(watch): release the audio backlog a ring trailing the clock will skip Nov 16, 2026
@kixelated
kixelated marked this pull request as ready for review November 16, 2026 11:49
@coderabbitai

coderabbitai Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 12 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 17eeef5d-838f-4c6e-bb61-c59438e29ae0
Commits

Reviewing files that changed from the base of the PR and between 0019174 and ef3b7ab.

Files selected for processing: 8

  • js/watch/src/audio/buffer.test.ts
  • js/watch/src/audio/buffer.ts
  • js/watch/src/audio/replay.ts
  • quest/m0/interop-green/README.md
  • quest/m0/interop-green/replacement-audio.md
  • test/audio-quality/budgets.json
  • test/interop/README.md
  • test/interop/clients/js/media.ts
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: 2363828bee89adc17dac6c0f4990c5c7f96009fb.

No actionable defect found in this static pass. Traced the decode gate's new max(playhead, clock - bound) accounting, the 30 ms minimum shared with the ring's skip threshold, reset clearing, and the AudioBuffer clock projection onto the worklet report's context time. The gate keeps its existing playhead behavior without a clock and leaves the pre-start admission path unchanged. The replay harness uses the corresponding end-of-quantum clock reading, and the new tests distinguish within-bound lag from a large re-anchor.

Keeping replacement quarantined is appropriate: this change addresses backlog admission, not the separately documented repeated-gap/warmup loss or loaded-host underruns. Verification limits: GitHub-only source and test inspection; no unit tests, browser runs, replay benchmarks, or loaded-host measurements were run independently. COMMENT only, not approval or a claim that the remaining audio failures are fixed.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No blocking issues. One inline point: the gate stops short of where the ring lands.

Reviewed changes

This review covers the initial two commits: the decode-gate fix for a ring trailing the shared clock, and the quest and quarantine rewrite.

  • Backpressure counts from the skip edge: a started ring's release window now starts at max(playhead, clock - bound), with bound floored at the 30 ms stretch window to match the ring's own threshold.
  • AudioBuffer clock wiring: the clock reading is sent first. Then observe gets the clock at the reported playhead's render time (#offset + data.time) on the ring's timeline.
  • Replay harness parity: replay.ts drives the gate with sync.now() and the same bound(...) the ring uses.
  • Quest and quarantine text: replacement stays quarantined. The quest and the interop README now name the remaining cause: a group skip restarts the decoder and trims 80 ms of Opus warmup.

buffer.test.ts (35 tests) and replay.test.ts (20 tests) pass locally.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using claude-opus-5-5 | 𝕏

Comment thread js/watch/src/audio/buffer.ts Resolved
The ring skips a trailing playhead all the way onto the clock, not to the
bound behind it, so a floor short of the bound left the skip landing on an
empty ring that stalled and drained one window per report again. Count
from the clock once the playhead trails it past the bound.

The two Opus relay-mic replays catch up faster at startup and settle a
fraction of a millisecond apart; their lag budgets are re-measured.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found. The gate now stops where the ring lands.

Reviewed changes

This review covers ef3b7ab, which follows up on the prior review's finding.

  • Backpressure counts from the clock: a started ring that trails the clock by more than the bound is now metered from clock. Within the bound, it is metered from playhead. This matches RingBuffer.read, which skips the whole lag onto the clock using the same strict > check.
  • Regression test for a floor short of the bound: a new Backpressure case with a 50 ms floor and a 200 ms bound confirms that the gate releases through clock + floor + window. The existing tests in Backpressure and in AudioBuffer steering were updated to the new release edge.
  • Replay budgets re-baselined: in budgets.json, relay-mic lag p50 goes from -0.3 to 0.2 ms and p95 from 9 to 7.7 ms. relay-mic-stall p50 goes from 1.7 to 2.3 ms, in line with the faster startup catch-up the PR describes.

Pullfrog  | View workflow run | Using claude-opus-5-5 | 𝕏

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI) · ef3b7ab

No new blocker in the audio gate delta. For a running live ring beyond the bound, the skip targets the clock, limited by what PCM has arrived. The old clock-minus-bound calculation could stop decoding short of that target when the floor was smaller than the bound. The new 48 kHz regression covers that combination and confirms release stops just past the new target; within the bound the gate still uses the playhead. Keep replacement quarantined for the separate repeated-gap and warmup problems. Static review; I did not run Bun, browser interop, or replay budgets.

@kixelated
kixelated added this pull request to the merge queue Nov 16, 2026
Merged via the queue into main with commit adb8fa0 Nov 16, 2026
5 checks passed
@kixelated
kixelated deleted the quest/m0/interop-green/replacement-audio branch November 16, 2026 21:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant