Skip to content

fix(watch): audio keeps a hole a late burst fills within the stretch - #8982

Merged
kixelated merged 8 commits into
mainfrom
quest/m2/audio-playout/browser-audio-hole-skip
Nov 17, 2026
Merged

kixelated merged 8 commits into
mainfrom
quest/m2/audio-playout/browser-audio-hole-skip

Conversation

@kixelated

@kixelated kixelated commented Nov 16, 2026 •

Copy link
Copy Markdown
Collaborator

Interim rule for quest/m2/audio-playout/browser-audio-hole-skip.md.

Problem

moq-dev/moq#8799 gave @moq/watch's audio consumer the shared clock: a missing group ends once the clock reaches the group past it. Natively that rule cost bursty 155 underruns a minute against 3 (moq-dev/moq#8909), so the quest asked whether the browser regressed too.

On the live Chromium rows it did not (tables below). But #8799 never gave the replay harness the clock, so the replay rows were not grading the player as it ships. With the clock, the replays show the quest's exact shape. In the recorded bursty-aac trace, group 37668 lands 0.8 ms after 37669, and the clock has already passed 37669, so the consumer skips it. That cost bursty-aac 24 underruns a minute (66.7 ms of gap) and local-aac 16.1 (42.2 ms), where both had none.

The ring plays audio trailing the clock up to its stretch bound (200 ms by default), and only skips past that. So a hole whose successor trails the clock by less than the bound is not yet too late to play.

Approach

  • playable(sync, output) (js/watch/src/audio/playhead.ts) is the oldest media the ring still plays. That is the clock plus the render lead the ring renders ahead by (ring.lead, the context's observed lead, which follows a fall in output latency at once), less the ring's bound (the stretch, or a buffered player's lookahead when larger, and never under the ring's 30 ms window). It applies both to the signed clock position before clamping at zero, since the ring renders media zero before the clock reads it. Its anchor also changes when the bound or the output latency moves. The audio decoder gives it to the container consumer in place of the bare Sync. Video keeps the bare clock.
  • replay.ts gives its consumer the same clock (with no output latency), so the replay rows grade the player as it ships.
  • Playhead.now() and ConsumerProps.playhead docs now say "the oldest media the player still plays", which for video is the clock itself.

With a zero stretch the bound is the ring's 30 ms window and the rule is #8799's plus that window.

Validation

All on fedora, loaded by other agents, debug build, seed 7, 40 s rows. Live rows ran in three arms: main (bare clock), no clock (the estimate wait the quest's Plan recommended), and this PR (clock less the bound).

Chromium bursty: no arm regressed. Every run had 0 tap underruns but one main Opus run (56.5/min) and the last AAC run of this PR (1.7/min), and player underruns 0 to 10.3/min. Chromium's WebTransport lands a burst's out-of-order groups at most 24 ms apart (from a --capture run), against a 310 to 350 ms target, so the clock rarely passes the successor first.

Chromium congestion, player skips per minute and trimmed ms per minute, one value per run:

arm AAC skips AAC trimmed Opus skips Opus trimmed Opus tap underruns
main (bare clock) 15.4, 25.6, 12.1, 54.6 3228, 1891, 3980, 4322 137, 116, 100, 135 4523, 4076, 4269, 4195 0, 0, 179, 0
no clock 106.5, 110, 53.2 4019, 4178, 4173 124, 106, 60 4436, 3771, 3616 196, 0, 0
this PR 18.8, 68.3, 51.2, 13.8, 22.4 3232, 4069, 4017, 3226, 3175 81, 127, 119, 124, 106 4332, 4215, 3963, 3446, 3689 134, 1.7, 0, 0, 0

The first four runs of this PR predate the output-latency term (review round 1), and the fifth includes it.

Noisy. This PR sits between main and no clock on AAC skips, and level elsewhere. A/V lag p95 stayed under 23 ms in every run but one main AAC run (68.6 ms) and one no-clock Opus run (85 ms).

Replays (just test audio-quality --replays, deterministic):

row no clock (budgets) bare clock this PR
bursty-aac tap underruns/min 0 24 0
bursty-aac trimmed ms/min 312.7 0 312.7
local-aac tap underruns/min 0 16.1 0
relay-mic-rtt40 tap underruns/min 0.7 0 0.7

With this PR every replay row matches budgets.json exactly (--enforce passes, rerun after the 30 ms window and render lead changes), so no budget changes.

Firefox resume (moq-dev/moq#8799's repro, taskset -c 4-5 just test media --media-engines firefox --media-cases rename,pause, 10 runs each, interleaved). Under this host's load many runs failed an earlier window (rename or coded-size growth) for every arm. Of the runs that reached after resume:

arm reached resume window failed leading silent samples (50 ms each)
main (bare clock) 6 4 (tone) 12, 12, 6, 6
no clock 6 4 (tone) 41, 15, 17, 9
this PR 6 3 (2 tone, 1 A/V sync) 7, 8, 0

The estimate wait alone brings back #8799's long silence. This PR is no worse than main. These runs predate the output-latency term, which only ends a hole sooner. Main still fails the window on a starved host, as its Nightly does.

Unit test: replay.test.ts "keeps a hole a late burst fills once the clock passed the group past it" holds groups 250 to 259 and releases them together, with 250 landing 1 ms after the rest. The consumer keeps the hole; with the stretch off it skips it. The test fails on main's bare clock. playhead.test.ts checks the output-latency term, its wake, and the signed position before media zero.

just check passes, as do bun test src/audio/ in js/watch (320 pass) and just test audio-quality --replays --enforce.

Impact

  • Public API: none. @moq/hang's Playhead.now() doc is broadened to "the oldest media the player still plays". Its shape is unchanged, and Sync still satisfies it.
  • Behavior: browser audio waits on a missing group until the group past it trails the clock by the stretch bound, rather than until the clock reaches it. Video is unchanged.
  • Wire: none. No Rust counterpart: native audio has never taken the clock (moq-dev/moq#8909). quest/m2/delay-hole-wait.md settles one rule for both languages, and its Plan now records these measurements.

Decisions

  • Maintainer, 2026-11-17: ✅ browser audio keeps this PR's rule, the clock less the stretch bound (rejected: main's bare clock with only the replay harness fixed and re-budgeted; the estimate wait the quest recommended). It is the only option with no measured regression. Native and browser keep different rules until moq-dev/moq#8984 settles one.
  • ✅ The quest stays: main rewrote it (moq-dev/moq#8989) into the ordered-reader work, which this PR does not do, so the merge keeps that quest and records this rule in its Plan as the interim one the ordered reader replaces (rejected: deleting the quest as finished).

Caused by

  • moq-dev/moq#8799 - ended an audio hole at the bare clock, which ignores the late audio the ring still plays, and left the replay harness without the clock, so the replay rows never graded it (found by check).

Follow-ups

None.

Models

Size: M
Implementer: Claude Code, claude-sonnet-5-5, low effort (iterate, S, arm B)

🤖 Generated with Claude Code

(Written by Claude Opus 5.5)

Review

Rewritten by quest review after each round.

  • Head: 927ba35

  • Base: origin/main at 44c18eb

  • Risk: medium, the base has no RISK.md to name what is always high or customer-facing, so every change is medium

  • Classified: medium by default

  • Reviewer: Codex, gpt-6.1-sol, xhigh effort

  • Session: 01a1eb25-8a05-7c50-8a2a-b6031cc6ebe0, round 3

  • Score: 10/10, Code inspection confirms the cutoff now follows the observed render lead, with no remaining findings in the incremental changes.

  • Rounds: 6; first round 1 (medium 1); findings 4 (medium 4): fixed 4, refuted 0, open 0

  • Findings: none

  • The last round's findings, at 59a59cb:

    • Fixed: medium, js/watch/src/audio/decoder.ts:291: Use the observed render lead instead of the latency peak; decoder.ts:291 now uses ring.lead, which buffer.ts:467-470 derives from currentTime minus the same sounding position used to align the worklet clock, independently of the five-second latency peak.

kixelated and others added 2 commits November 16, 2026 15:40
Browser audio's container consumer ended a missing group as soon as the
shared clock reached the group past it. The ring still plays audio
trailing the clock within its stretch bound, so a late burst's hole that
fills a moment later was thrown away. The consumer now reads the clock
less that bound, and the replay harness gives its consumer the same
clock the decoder does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title Browser audio keeps a hole a late burst still fills fix(watch): audio keeps a hole a late burst fills within the stretch Nov 17, 2026
kixelated and others added 2 commits November 16, 2026 19:01
…io's hole clock

Review: the ring renders the output latency ahead of the shared clock, so
the oldest media it still plays is the clock plus that, less the bound.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review: before media zero's turn the clock reads zero while the ring
already renders, so the output term applies before the clamp.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Maintainer decision (2026-11-17, asked in the /quest-run m0/m1/m2 coordinating session):

✅ Browser audio keeps this PR's rule, the clock less the stretch bound (rejected: main's bare clock with only the replay harness fixed and re-budgeted; the estimate wait the quest recommended). It is the only option with no measured regression: replays are back within budget, live rows are level, and resume is no worse than main. Native and browser keep different rules until moq-dev/moq#8984 settles one.

(Written by Claude Opus 5.5 with high)

@kixelated
kixelated marked this pull request as ready for review November 17, 2026 20:29
@coderabbitai

coderabbitai Bot commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 9 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: f34d4451-3389-47b2-b795-66127fc831c5
Commits

Reviewing files that changed from the base of the PR and between 44c18eb and 3ef3d8e.

Files selected for processing: 11

  • doc/concept/audio-jitter.md
  • js/hang/src/container/consumer.ts
  • js/watch/src/audio/buffer.test.ts
  • js/watch/src/audio/buffer.ts
  • js/watch/src/audio/decoder.ts
  • js/watch/src/audio/playhead.test.ts
  • js/watch/src/audio/playhead.ts
  • js/watch/src/audio/replay.test.ts
  • js/watch/src/audio/replay.ts
  • quest/m2/audio-playout/browser-audio-hole-skip.md
  • quest/m2/delay-hole-wait.md
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Approach: sound. The consumer now skips a hole only once the ring would skip its audio anyway, which fixes the actual mismatch between the hole rule and what the ring still plays. It also gives the replay harness the same clock, so the replay rows grade the player as it ships.

Reviewed changes

The whole diff at 3ef3d8e was checked against the ring's real skip and clock-reading logic.

  • playable(sync, output): Gives the audio consumer a clock that reads the signed position, plus the render lead, minus max(bound, WINDOW). This matches ring-buffer.ts's threshold max(#bound, stretch.window) and the clock reading in buffer.ts#read (position + currentTime − heard). The decoder's ring bound comes from the same sync.out.delay / maxDelay / in.stretch inputs.
  • AudioBuffer.lead: Tracks the context's observed render lead. It ignores moves under 5 ms, measured against the last reported value, so slow drift still adds up to an update. It falls back to the output peak before the context reports an output position.
  • anchor: A Derived of sync.anchor and the trail, so a change to the bound or the lead wakes a consumer waiting on a hole. Derived only subscribes to its sources once something subscribes to it, so building a playable for each consumer leaks no subscriptions.
  • Replay harness: replay.ts hands its consumer playable(sync). The new test fails on main's bare clock, and when the stretch is off it still skips the hole.
  • Docs and quests: Playhead.now() is now documented as "the oldest media the player still plays". The quest's ## Waiting entry on #8910 is gone, and #8910 has merged.

Pullfrog  | View workflow run | Using claude-opus-5-5 | 𝕏

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)
Reviewed commit: 3ef3d8e4e37e874b7c7d41cc23a3520df4852d86

The stretch-aware cutoff is a better fit than the bare clock, and using observed render lead is right once the clocks are mapped. One gap at js/watch/src/audio/buffer.ts:467–470,494–500: without a valid output timestamp, #read clears the ring's clock mapping, but #observeLead supplies the estimated output latency to the consumer anyway. The worklet then has no shared-clock cutoff while the consumer can discard a still-playable hole. For example, with zero stretch and a 120 ms fallback output estimate, the consumer ends the hole 90 ms before the bare clock reaches it. Gate the lead-adjusted cutoff on a valid mapping, and test the missing-to-valid timestamp transition. Keep the ordinary wall-time hole budget while no mapped clock can judge the audio stale.

Static review; I did not run browser audio or replay tests.

@kixelated
kixelated added this pull request to the merge queue Nov 17, 2026
Merged via the queue into main with commit 262f10d Nov 17, 2026
7 checks passed
@kixelated
kixelated deleted the quest/m2/audio-playout/browser-audio-hole-skip branch November 17, 2026 21:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant