Skip to content

fix(cli): moq import capture reopens a failed microphone and keeps the video - #9060

Merged
kixelated merged 3 commits into
mainfrom
quest/m1/precut-fixes/capture-audio-reopen
Nov 18, 2026
Merged

kixelated merged 3 commits into
mainfrom
quest/m1/precut-fixes/capture-audio-reopen

Conversation

@kixelated

@kixelated kixelated commented Nov 18, 2026 •

Copy link
Copy Markdown
Collaborator

Claims quest/m1/precut-fixes/capture-audio-reopen.md.

Problem

When a microphone fails partway through moq import capture (a USB mic re-enumerating, another app grabbing the input, a sample-rate change), the whole command fails and the video goes with it. In rs/moq-cli/src/publish.rs the audio future returned the capture error, and tokio::try_join! failed both media. moq-dev/moq#7706 took the automatic reopen out of moq-audio on purpose, but nothing moved it into the CLI.

Approach

This follows the maintainer's decision in #9010: the CLI reopens the device, and moq-audio still does not.

  • publish_audio reopens the capture and a new publication in a loop. Each reopen calls publish_capture again, so it gets a new unique track name (0.opus, 1.opus, ...). Content under the failed track's name never changes.
  • Reopen holds the policy. It waits 0.5 s before the first reopen, doubles the wait up to 4 s (release's numbers), and gives up after 10 s of failures with no healthy capture between them. A capture counts as healthy once it has run for 1 s after opening.
  • If it gives up, only the audio track ends, carrying its error. The video keeps publishing. When the video stops, the command exits with the audio error, so a partial failure is still reported. With --video off, giving up fails the command as soon as the budget runs out.
  • If the very first open or publication fails, the import ends at once, as before. That kind of failure is a setup problem (wrong device id, denied permission) and should fail early.
  • A failed track still serves its retained audio until it is served or expires (up to --max-age, 30 s by default), as moq-audio documents. That draining runs in the background so it delays neither the reopen nor the give-up report: the audio error reaches the command as soon as the budget runs out, and any draining tracks are dropped when the command exits.
  • Updated doc/bin/cli.md, which said a failed device ends the import.

Decisions

Choices I made that the maintainer should check:

  • 10 s give-up budget and 1 s health threshold. 10 s matches the workspace's fail-fast budget from #2647. The quest's "persists past the backoff" didn't give a number. With these values a device that stays missing gets about 6 reopens over roughly 11.5 s.
  • No jitter. feat: fail-fast retries: jittered backoff bounded by time, not error type #2647 jitters its loops so a fleet doesn't retry in lockstep. A single local device has no fleet, so I left jitter out.
  • The first open still fails the import. release retried even its first open for errors it classified as retryable. This repo forbids matching on errors, so the CLI can't tell a transient failure from a permanent one. Failing fast at startup seemed the better default.
  • The give-up doesn't wait on draining tracks. It used to join them, which delayed the report (and the --video off exit) by up to --max-age. Per the maintainer's call to abort aggressively, the drains are detached: they keep serving while video runs, the last failed track still ends with its error if it finishes draining, and they are dropped at exit. (Pullfrog r4609581892.)
  • A brief second audio rendition. While a failed track drains its retained audio, the catalog lists it next to the reopened track. If nobody is subscribed, that overlap can last up to --max-age. The alternative was to drop the retained audio on failure, or to make the reopen wait for the drain, which can take 30 s by default.

Impact

  • Public API: none (CLI behavior only). Wire: none.

Validation

  • Unit tests for Reopen, using the kio clock instead of wall time: the backoff sequence and give-up, a healthy capture resetting the backoff, and a device that keeps failing within 1 s still giving up.
  • Real hardware on Linux: moq import capture --display x11:0 --software --microphone alsa:sysdefault:CARD=USB -- export h264, with the AT2020 USB mic deauthorized through /sys/bus/usb/devices/*/authorized.
    • Down 3 s: reopens logged at 0.5, 1, 2 and 4 s waits (not found, then busy while PipeWire grabbed the returning device). The mic reopened about 8 s after the fault. The H.264 output kept growing the whole time (44 KB → 67 KB → 111 KB → 178 KB).
    • Down 16 s: after 5 reopens, audio capture ended; video keeps publishing error=audio capture kept failing for 10s. The H.264 output kept growing until the command was stopped.
  • cargo clippy -p moq-cli --all-targets --features capture: clean. cargo test -p moq-cli --features capture: passes. just check: passes.

Alternatives

The maintainer rejected these in #9010: putting the reopen back in moq-audio, and ending the audio track without trying to reopen.

Follow-ups

  • quest/m2/export-replaced-rendition.md: moq export ts (and the other container exports) end with the failed audio track's error even though the publisher reopens it. Validation hit this. It was hidden before because the whole publish died. No open decisions.

Caused by

  • moq-dev/moq#7706 - removed moq-audio's device reopen without moving it into the CLI (found by the 2026-11-17 pre-cut correctness audit)

Models

Size: S
Implementer: Claude, claude-opus-5-5, unknown effort (arm A)
Implementer: Claude Code, claude-opus-5-5, medium effort (iterate, XS, arm B)

🤖 Generated with Claude Code

Review

Rewritten by quest review after each round.

  • Head: 5a6c8b4

  • Base: origin/main at ef0de9c

  • Risk: medium, the base has no RISK.md to name what is always high or customer-facing, so every change is medium

  • Classified: medium by default

  • Reviewer: Codex, gpt-6.1-sol, medium effort

  • Session: 01a1ed16-3c81-7803-a591-6b4406bc455c, round 3

  • Score: 10/10, The delta reports exhausted audio recovery promptly while allowing failed tracks to finish draining, with no new blocking issues found.

  • Rounds: 3; first round 1 (high 1); findings 1 (high 1): fixed 0, refuted 1, open 0

  • Findings: none

kixelated and others added 2 commits November 17, 2026 19:49
…e video

The audio future returned the capture error and try_join! failed the whole
command, video included. The CLI now reopens the capture and a new audio
track with release's 0.5 s to 4 s backoff, gives up after 10 s of failures
with no healthy capture between them, and then ends only the audio track
while the video keeps publishing. A first open that fails still ends the
import at once. A failed track drains its retained audio in the background so
the reopen is not held up.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title moq import capture keeps publishing when its microphone fails fix(cli): moq import capture reopens a failed microphone and keeps the video Nov 18, 2026
@kixelated
kixelated marked this pull request as ready for review November 18, 2026 05:19
@coderabbitai

coderabbitai Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 1 minute.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: f65c73fa-4a50-44ad-9a53-d06cc2359373
Commits

Reviewing files that changed from the base of the PR and between d8ec56d and 5a6c8b4.

Files selected for processing: 4

  • doc/bin/cli.md
  • quest/m1/precut-fixes/capture-audio-reopen.md
  • quest/m2/export-replaced-rendition.md
  • rs/moq-cli/src/publish.rs
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No critical issues — two notes: one on what the follow-up quest leaves out, one on how long the give-up takes to surface.

Approach: sound — the reopen policy lives in the CLI where the maintainer put it, each reopen publishes a fresh uniquely named track, and splitting audio failure out of try_join! is what stops a microphone from taking the video with it.

Reviewed changes

  • Audio reopen loop (publish_audio): opens the capture, publishes it, and select!s capture against driver; a first open or publish failure still fails the import, later ones go through the reopen policy, and each failed driver is spawned so its retained-audio drain does not hold up the reopen.
  • Failure split in run(): a video error still ends everything, while an audio error that outlasted its reopens is logged and awaited so the command exits with the audio error once the video ends. --video off lands on the same error through the video branch.
  • Reopen policy: 0.5 s doubling to 4 s, a 1 s health threshold that resets the run, and a 10 s give-up budget, covered by three deterministic unit tests (backoff sequence, reset on health, flapping still gives up).
  • Docs and quests: doc/bin/cli.md now describes the reopen, the completed m1 quest is removed, and quest/m2/export-replaced-rendition.md records the export fallout found during validation.

ℹ️ The replaced-rendition follow-up leaves out playback

Before this change an audio failure ended the publish, so no consumer ever saw a failed rendition replaced by a live one. Now they can, and the new quest scopes that to container exports, but moq play ends the same way: a track error that is not a Decode error propagates out of the audio task and takes the whole player down, video included, even though the catalog lists the reopened rendition. Not a defect in this diff, but the quest that is supposed to cover the consumer side would land with playback still broken on the exact scenario the publish half now makes possible.

Technical details
# Playback also ends on a rendition that fails and is replaced

## Affected sites
- `rs/moq-cli/src/play/media.rs:746` — a non-`Decode` read error returns `Err`, so a track that ended with its source error ends `play_audio`.
- `rs/moq-cli/src/play/media.rs:172` — `joined(...)?` propagates that error out of `play_broadcast`; only an `Ok` end reaches `playback.ended`.
- `rs/moq-cli/src/play/playback.rs:176` — `joined` turns a task error into `Err`.
- `rs/moq-cli/src/play/headless.rs:101` — the media task error breaks the run with `audio playback task failed`.
- `rs/moq-mux/src/container/consumer.rs:974` — a failed track's reader returns the bare error after its pending frames drain.
- `quest/m2/export-replaced-rendition.md` (added in this PR) — Goal names only `export ts`, `fmp4`, `mkv`, and `flv`.

## Required outcome
- Decide whether playback belongs in this follow-up: if `moq play` should follow the replacement, widen the quest's Goal to say so rather than tracking the export path alone; if it should stay out of scope, say why so the omission reads as deliberate.

## Open questions for the human
- Does the player deliberately treat a failed rendition as fatal (rather than a retirement it follows), and does the JS player behave the same way?

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using deepseek-v4.1-flash | 𝕏

Comment thread rs/moq-cli/src/publish.rs Resolved
@kixelated

Copy link
Copy Markdown
Collaborator Author

Decision from the maintainer:

  • ✅ Merge, after the open Pullfrog finding (r4609581892) is answered.

(Written by Claude Opus 5.5 with high)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

Reviewed the delta since the prior pullfrog review (d32ff0340..5a6c8b46d).

  • Detached drains on give-up: publish_audio now calls draining.detach_all() instead of joining every drain task, so the audio error is reported as soon as the 10 s budget runs out (and --video off exits promptly) rather than up to --max-age later. This resolves the delayed give-up noted in the earlier review.

Pullfrog  | View workflow run | Using claude-sonnet-5-5 | 𝕏

@kixelated
kixelated added this pull request to the merge queue Nov 18, 2026
Merged via the queue into main with commit c422cfe Nov 18, 2026
7 checks passed
@kixelated
kixelated deleted the quest/m1/precut-fixes/capture-audio-reopen branch November 18, 2026 06:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant