Skip to content

fix(moq-webrtc): a WHEP pull announces once its catalog exists and survives a mid-group join - #8997

Merged
kixelated merged 6 commits into
mainfrom
quest/m2/whep-pull-announce
Nov 17, 2026
Merged

kixelated merged 6 commits into
mainfrom
quest/m2/whep-pull-announce

Conversation

@kixelated

@kixelated kixelated commented Nov 17, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

moq import webrtc --connect <whep> -- export rtmp --connect <rtmp> (a WHEP pull chained to another gateway in one process) failed at startup: init FLV export: moq: not found. The CLI announced the pulled broadcast before moq_webrtc::Client::pull had created its catalog track, so an export or viewer resolving it early found no catalog.

With that fixed, the same chain still failed most starts, inside the WHEP ingest itself (a lone import webrtc --connect failed 4 of 8 starts on main):

  • missing keyframe: a group must open on a keyframe: the RTP stream joined mid-group, and the first video frame could not open a group.
  • frame timestamp 0 µs is below the previous group's start 20000 µs: the egress's held-before-connect burst arrives reordered (seen in a trace: the first ~34 Opus frames arrive scrambled, then in order), and a late Opus frame rewinds a published group.

Each of these alone ends the session, so the chain from the quest only restreams once all three are fixed.

Approach

  • moq_webrtc::Client::pull takes the catalog producer, matching moq_rtsp::Client::pull. The CLI creates the route hidden and announces it after the catalog's first update, as rs/moq-cli/src/rtsp.rs does. A viewer that arrives first waits for the announcement instead of failing.
  • The ingest session (WHEP client and WHIP server) now filters frames per media line (IngestOrder) before they reach the codec bridges:
    • A video line drops frames until its first keyframe. Meanwhile it sends a PLI where the publisher negotiated one, at once and then at most once a second.
    • A late frame is dropped where it would rewind a published group: for audio, any frame not after the last one (each Opus frame opens a group); for video, any frame before the last keyframe. The filter compares each line's own RTP time, before the session clock rebases it.
  • When Client::pull fails before its session starts (an HTTP error, bad SDP), it aborts the caller's catalog with that error, so readers see the failure instead of waiting.
  • IngestSink::new takes a catalog producer; IngestSink::routed (WHIP server) still builds one from a config.

Impact

  • moq_webrtc::Client::pull(url, broadcast, catalog) now takes &broadcast::Producer and &moq_mux::catalog::Producer instead of an owned producer and a catalog::Config. Migrate: build the catalog with moq_mux::catalog::Producer::new(&mut broadcast, config) and pass references to both. moq-webrtc has not been published to crates.io, so the title takes no !.
  • Behavior: moq import webrtc --connect announces its broadcast only once the catalog names a track.
  • Behavior: a WHEP pull or WHIP publish whose video starts mid-group, or that delivers a late frame, drops those frames instead of ending the session. Where the publisher negotiated PLI, a video line asks for a keyframe until its first one arrives.
  • No wire change.

Validation

  • Real binaries, three processes on loopback: an ffmpeg H.264+Opus source through moq import fmp4 -- export webrtc --listen (WHEP server), the chain under test moq import webrtc --connect http://…/src -- export rtmp --connect rtmp://…/live/out, and a sink moq import rtmp --listen -- export rtmp --listen that ffprobe -rtmp_enhanced_codecs avc1,Opus plays.
    • main: the chain exited at startup in all 3 runs (init FLV export: moq: not found twice, the timestamp rewind once), and the sink never got a stream.
    • This branch: the chain kept running in all 16 runs. With 40 ms source fragments, ffprobe read both h264 320x240 and opus from the sink in the last 4 runs. (Earlier runs used 1 s fragments, which burst audio and made the export skip it, so only Opus showed in the probe.)
  • WHEP pull alone (moq import webrtc --connect, 5 s per run): main failed 4 of 8 runs (missing keyframe ×1, timestamp rewind ×3); this branch kept all 8 running.
  • Unit tests: webrtc::tests::a_whep_pull_hides_its_broadcast_until_the_catalog_exists (moq-cli) points the pull at an endpoint that accepts the offer but never answers. It checks that nothing is announced while the session negotiates, nor after the pull fails. session::tests::video_waits_for_its_first_keyframe and late_frames_are_dropped cover the per-line filter, including PLI pacing on a mocked clock.
  • After the review fixes, the same binary runs again: 6 of 6 WHEP pulls kept running, and 3 of 3 chain runs played h264 and opus from the sink.
  • just check passes. just rs test -p moq-cli -p moq-webrtc: 320 passed. client::tests::http_error_redacts_url now also checks that a refused dial aborts the catalog.
  • just test interop --all: 209 passed. One case failed: CLI FETCH rust -> rust (group 1 of tail not found through the relay). That is moq-lite FETCH, which this PR does not touch, and it passed when just test interop was rerun.

Caused by

  • No single PR. Announcing up front is as old as the WHEP pull in moq-cli (moq-dev/moq#7806 kept the older behavior), and the ingest has fed every RTP frame straight to the bridges since the session driver was written.

Alternatives

  • Skip MissingKeyframe and TimestampRewind in the bridges by matching on the mux errors, as some TS and FLV paths do. Rejected: the repository forbids matching on errors, and the frame's keyframe flag and timestamp are already known before the bridge sees it.
  • Fix the WHEP egress so it never sends a reordered burst. That is worth doing as well, but an ingest must tolerate reordered and mid-group RTP from any third-party publisher anyway.

Follow-ups

  • A WHEP pull chained into an export stops cleanly (new, m2, S): on SIGTERM, the chained export rtmp --connect sometimes exits with the source's clock runs +59.8 ppm off ours, past the 30 ppm the output's may follow, and the WHEP ingest's tracks log track::Producer dropped without finish() or abort(). No open decisions.

Decisions

Models

Size: S
Implementer: Claude Code, claude-opus-5-5, medium effort

(Written by Claude Opus 5.5 with medium)

Review

Rewritten by quest review after each round.

  • Head: 7d7ede9

  • Base: origin/main at 8fbbb2a

  • Risk: medium, the base has no RISK.md to name what is always high or customer-facing, so every change is medium

  • Classified: medium by default

  • Reviewer: Codex, gpt-6.1-sol, medium effort

  • Session: 01a1e927-cb8f-7762-9864-f9c905fe1a21, round 3

  • Score: 10/10, The incremental change only documents a shutdown follow-up and links it from the roadmap, with no new findings.

  • Rounds: 3; first round 2 (medium 2); findings 2 (medium 2): fixed 2, refuted 0, open 0

  • Findings: none

@kixelated kixelated changed the title A WHEP pull announces its broadcast once the catalog exists fix(moq-webrtc): a WHEP pull announces once its catalog exists and survives a mid-group join Nov 17, 2026
… failed dial

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review November 17, 2026 12:01
@coderabbitai

coderabbitai Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 7 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 2a2e9672-a3fa-4e79-bb8d-67b1f402b7dc
Commits

Reviewing files that changed from the base of the PR and between dac066a and 7d7ede9.

Files selected for processing: 9

  • quest/m2/README.md
  • quest/m2/whep-chain-shutdown.md
  • quest/m2/whep-pull-announce.md
  • rs/moq-cli/src/webrtc.rs
  • rs/moq-webrtc/src/client/mod.rs
  • rs/moq-webrtc/src/client/whep.rs
  • rs/moq-webrtc/src/ingest.rs
  • rs/moq-webrtc/src/lib.rs
  • rs/moq-webrtc/src/session.rs
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

pullfrog[bot]
pullfrog Bot previously approved these changes Nov 17, 2026

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

This review covers the full PR at 185d0ac. The PR fixes the startup race in the WHEP pull's announcement and makes the WebRTC ingest tolerate a mid-group join and late frames.

  • Caller-owned catalog for Client::pull: pull now borrows a broadcast::Producer and a catalog::Producer, matching moq_rtsp::Client::pull. A dial that fails before the session starts aborts the catalog. catalog::Producer::abort returns early when the catalog is already closed, so a cause the session set first is kept.
  • Hidden-until-catalog announce in moq import webrtc --connect: The route is created hidden and announced on the catalog's first update, the same way rs/moq-cli/src/rtsp.rs does it. The announce future ignores catalog errors, so the pull reports its own failure.
  • IngestOrder per-line admission: Frames are filtered on each line's raw RTP time before IngestClock::normalize. Dropped frames never move the session clock. A video line waits for str0m's is_keyframe and paces PLI to once a second, and only on lines that negotiated fb_pli. str0m's keyframe flags agree with what moq-mux accepts as a group opener, including H.265 IRAP. Audio drops anything that is not strictly later than its last frame, and video drops anything before its last keyframe.
  • IngestSink::new is infallible: IngestSink::routed still creates its catalog from a config. The tests and the lib.rs integration tests use the new signatures.

Pullfrog  | View workflow run | Using claude-opus-5-5 | 𝕏

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed 185d0ac53e01b227071c3f060ca9b83f962df0ab. No blocker found. Deferring the route announcement until a catalog update closes the early-export race, and aborting the catalog on a failed dial makes the wait terminate. Filtering against the original per-line RTP time before normalization is the right place to drop late frames; the video gate leaves in-group delta reordering alone. The hidden-route failure fixture and admission tests cover the important cases in code. Static review only; I didn’t rerun WHEP/WHIP or the gateway chain.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

This review covers what changed since 185d0ac. The delta is documentation only.

  • Follow-up quest added: quest/m2/whep-chain-shutdown.md records the clean-shutdown work for a chained WHEP pull (clock drift on stop, unfinished tracks).
  • Roadmap link: quest/m2/README.md links the new quest and drops a duplicated moq-vaapi entry.

Pullfrog  | View workflow run | Using claude-sonnet-5-5 | 𝕏

@kixelated
kixelated added this pull request to the merge queue Nov 17, 2026
Merged via the queue into main with commit 6d7aff3 Nov 17, 2026
10 checks passed
@kixelated
kixelated deleted the quest/m2/whep-pull-announce branch November 17, 2026 12:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant