Skip to content

feat: the program reclaims the sources' replay encoders' sessions - #189

Merged
kixelated merged 9 commits into
mainfrom
quest/m0/program-encoder-priority
Nov 16, 2026
Merged

kixelated merged 9 commits into
mainfrom
quest/m0/program-encoder-priority

Conversation

@kixelated

@kixelated kixelated commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Implements quest/m0/program-encoder-priority.md using the option the maintainer settled on, (a): when the program's encoder cannot open, the studio closes the replay encoders of the sources on program, opens the program's encoder in their sessions, and lets each source reopen when a session frees up. Its replay starts over.

Summary

  • Sessions (moq-scene) now also counts spare sessions: those held by a source's replay encoder, meaning a source shown on program whose track nothing reads. Sessions::reclaim() returns a guard. While the guard lives, replay encoders close and none reopen. Reclaim::closed() resolves once the last spare session has closed.
  • Sources (publish::Video) close a replay encoder while a reclaim is pending, even when no picture is coming in (a still screen or an ended clip): settle() now also wakes on session changes. A source whose encoder was taken may then be refused when it reopens. In that case it waits until any of the studio's sessions closes after its attempt, instead of failing its track. A source whose track is read (by a recording, or by forwarding while published) keeps its encoder.
  • A playing media file gives its replay encoder back as soon as a reclaim asks, not at its next picture: play_clip waits through Driver::serve, which ended() now shares, so sparse video or a sound tail past the last picture cannot hold going live.
  • Encoding::drain closes the encoder before its session counts as given back, so a replay woken by the count finds the GPU's session free.
  • Encoding::hold(spare) counts a session before its encoder asks the GPU, as spare for a replay, and refuses a spare one with Error::Reclaimed while a reclaim is under way, all in one state update; Encoding::open(session, encoder) takes it. So the program, on its own thread, always sees a replay that is still opening, and none starts opening once a reclaim is admitted. Only a session an encoder opened in counts as given back, so a refused attempt does not wake itself.
  • Program side (moq-studio) reclaims in two places:
    • Probe: when the go-live probe (or a published scene's, or a canvas change's) fails, or falls back to openh264 even though launch found a hardware encoder, it reclaims and probes again. It holds the claim until the full-size encoder opens, so no source takes the session back in between. This is the path a real NVENC cap takes, because Kind::Auto falls back to software instead of failing.
    • Encode thread: when an encoder fails to open and spare sessions exist, the thread reclaims, waits for the replays to close, and tries once more, gated like the probe: only when launch found a hardware encoder or the sessions are capped (Pullfrog's finding). This covers the canvas size, the half size when a viewer arrives, published scenes, and the recording.
  • Sessions::capped(n) refuses an open past n, standing in for a capped GPU, and so lets the program reclaim as on hardware. Launch::sessions lets a test pass one in.
  • README: the encode-sessions paragraph no longer says the program opens before any source, which stopped being true with go-live. It now describes the reclaim.
  • Deletes the quest.

Validation

Real GeForce check on nixos (RTX 3070 Ti, driver 580.196.07, NVENC cap 8, with 4 sessions already held by other processes on the host). A scratch, uncommitted copy of going_live_takes_the_replays_sessions with uncapped Sessions, Vulkan on the card, --features nvidia,openh264, N looping 1280x720 H.264 clips on program, the real Engine::go_live to a local relay, then end_stream, printing the studio's counts, nvidia-smi's session count, the program's encoder, and each source's track at every stage:

  • With 4 clips, on the PR's original head (471374e):
    • Off air: 4 replays live, card 8/8.
    • Live: log closing the sources' replay encoders for the program's hardware encoder, program encoder nvenc, no video error, card 8/8. One clip logged waiting for an encode session (driver error IncompatibleClientKey, which is how NVENC refuses past the cap), and no track failed.
    • After the stream ended, the waiting clip never reopened (20 s; studio 3 open, card 7/8 with one session free). Two bugs, fixed in this PR:
      1. Encoding::drain dropped the Session before the encoder, so the count said "given back" while NVENC still held it, and the woken replay was refused again. Seen after the first fix alone: the clip retried at end of stream and was refused once more.
      2. A refused replay waited for fewer sessions open than at its refusal, but a sibling replay's session the driver already held is counted only once its open returns, so the threshold could be too low to ever be crossed. It now waits for any session closed after it tried.
  • With both fixes, 4 clips, run twice: off air 4 live (card 8/8); live: the reclaim log line, program on nvenc, no video error, no failed track, card 8/8; after the stream ended, all 4 replays reopened (studio 4 open, card 8/8); after close the card was back to its 4.
  • After the session reservation (from an online review), the first card run passed but a refused replay retried at every picture: 302 waiting for an encode session lines in one run, each a refused NVENC open, because its own failed reservation counted as a session given back. With a close counted only for a session an encoder opened in, at head 4f12117, run twice: off air 4 live (card 8/8); live: the reclaim log line, program on nvenc, no video error, no failed track, exactly one waiting line; after the stream ended all 4 replays reopened (card 8/8); after close the card was back to its 4.
  • With 8 clips (4 over the free sessions): the reclaim, nvenc, and no new failures behaved as above, but the 4 clips beyond the cap had already failed off air, before going live, because Kind::Auto does not fall back to openh264 on a cap refusal. That predates this PR; see Follow-ups.
  • nixos: built under quest gate with nice and -j 8. The scratch checkout was removed afterwards, nothing was left running, and CARLA and the other GPU process were untouched.

Simulated cap (Sessions::capped):

  • publish::tests::the_program_reclaims_a_replays_session (moq-scene, real openh264, paused clock), with a cap of 1: a source on program holds the session as spare; a reclaim closes it without a picture, and none reopens while held; refused once the program holds the session, it waits without failing; a session given back and taken again before it looks still wakes it (new; fails on the old open < snapshot rule), and it waits again; once a session is free it reopens; a subscribed source keeps its encoder.
  • publish::tests::an_encoder_closes_before_its_session_is_given_back (new): a fake encoder records the count at its own drop. It reads 1 with the fix; reversing the two drops makes it read 0 and fail.
  • source::media_tests::a_playing_media_file_gives_its_session_back_at_once (new, from the review's finding): a clip on program with a cap of 1; a reclaim closes its replay within 10 ms of paused time, before its next picture 100 ms out. Reverting the wait between pictures to a bare ahead(due) makes it fail.
  • The render-sw engine suite (lavapipe) passes 15/15, including going_live_takes_the_replays_sessions and ending_the_stream_delivers_its_end_and_goes_live_again.
  • Earlier mutation check: with the reclaim branch disabled, the engine test fails with go-live erroring all 1 encode sessions are open.
  • publish::tests::a_replay_is_counted_before_it_opens (new): a replay's held session counts as spare before any encoder opens; a reclaim refuses a new spare one with Error::Reclaimed but lets the program's through, and closed() waits for the opening replay; a session no encoder opened in gives nothing back.
  • just ci passed on the head: lints, including clippy with every feature, and 376 tests. The last GeForce runs include every fix.

Decisions

  • Land on the simulated-cap proof, and turn the GeForce check into a follow-up: ✅ merge (rejected: keep it a draft until a GeForce run; skip). Maintainer, 2026-11-16. Superseded in part: an agent ran the GeForce check on nixos (above), so no follow-up quest is needed for it.
  • Close only replay encoders; a source whose track something reads (a recording, a source published on its own) keeps its encoder: ✅ keep (rejected: close those too). Same comment.
  • A reclaimed source that is refused on reopen waits and logs a warning; it does not fail its track. It retries once any of the studio's sessions closes after its attempt, so it never retries in a busy loop. A source that was never reclaimed and gets refused still fails loudly, as before.
  • The reclaim covers every program-side encoder (the canvas size, the half size when watched, published scenes, the recording), since each one is "the program's".
  • The probe path decides a refusal by comparing against the tier the launch probe found: if the canvas falls back to openh264 where launch found hardware, that counts as a refusal. On a machine without hardware and with uncapped sessions, neither path ever reclaims.

Caused by

  • moq-dev/studio#175 - moved the program's encoder open from launch to GO LIVE, after the sources' replay encoders already hold their sessions (found by reviewer)

Public API

  • moq-scene (unpublished; moq-studio is its only consumer):
    • Sessions::capped, Sessions::is_capped, Sessions::spare, Sessions::reclaim, and Reclaim are new.
    • Encoding::hold(spare) -> Result<Session, Error> is new, failing with the new Error::Sessions(cap) when capped sessions are all open, or the new Error::Reclaimed for a spare one during a reclaim. Session is now public, opaque.
    • Breaking: Encoding::open(encoder) is now Encoding::open(session, encoder). Migrate by calling hold(false) before building the encoder and passing its session.
    • Sessions no longer derives Default (it implements it by hand).
  • moq-studio: engine::Launch gains sessions.
  • No wire impact.

Follow-ups

  • moq-dev/moq (for /quest-plan there): moq_video::encode::Kind::Auto fails rather than falling back to openh264 when NVENC refuses an open, seen on the 3070 Ti both past the session cap (IncompatibleClientKey) and for a frame below NVENC's minimum (a 64x64 media source: InvalidParam: Frame Dimension less than the minimum supported value). So on an NVIDIA machine a small source, or a source past the cap off air, fails its track where any other machine encodes it in software. Recommendation: have Kind::Auto try the next backend on those refusals, with the warning it already logs for an unavailable backend, and a moq-side test for each.

Models

Size: S
Implementer: Claude Code, claude-opus-5-5, unknown effort (arm A)
Implementer: Claude Code, claude-opus-5-5, unknown effort (iterate, S, arm A)

🤖 Generated with Claude Code

(Written by Claude Opus 5.5)

Review

Rewritten by quest review after each round.

  • Head: 11c9f7f

  • Base: origin/main at d47ed50

  • Risk: medium, the base has no RISK.md to name what is always high or customer-facing, so every change is medium

  • Classified: medium by default

  • Reviewer: Codex, gpt-6.1-sol, medium effort

  • Session: 01a1e61b-173a-79f3-86ab-f0e3a540f3ee, round 5

  • Score: 10/10, Code inspection found no new issues in the merge integration, excluding changes inherited from the base branch; builds and tests were not run, as instructed.

  • Rounds: 5; first round 1 (medium 1); findings 1 (medium 1): fixed 1, refuted 0, open 0

  • Findings: none

  • Reviewer: Codex, gpt-6.1-sol, low effort, shadow

  • Session: 01a1e61b-1732-73f3-a3b5-0929e342e7e2

  • Rounds: 2; first round 1 (medium 1); findings 1 (medium 1): fixed 1, refuted 0, open 0

kixelated and others added 2 commits November 16, 2026 08:05
When the GPU refuses the program, a published scene, or the recording an
encoder while sources on program hold sessions only so a replay has their
recent past, those replay encoders close and the program's opens in their
sessions. Each such source reopens once a session is given back, its replay
starting over; a source whose track is read keeps its encoder.

Sessions::capped stands in for a session-capped GPU in tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title The program's encoder before the sources' feat: the program reclaims the sources' replay encoders' sessions Nov 16, 2026
@kixelated

Copy link
Copy Markdown
Contributor Author

Maintainer decision (2026-11-16, /quest-run):

  • ✅ /quest-merge this PR on the simulated-cap proof (rejected: keep it a draft until a GeForce run; skip).
  • The GeForce check (the PR's three steps: 8 sessions counted off air, go live, the log says the replay encoders closed and the program is on nvenc; no source fails, sources reopen after the stream ends) becomes a follow-up Maintainer: try-quest, not a blocker. No reachable agent host currently has an NVIDIA card.
  • Only replay encoders are closed; a source whose track something reads (a recording, a source published on its own) keeps its encoder: ✅ keep (rejected: close those too).

(Written by Claude Opus 5.5)

kixelated and others added 3 commits November 16, 2026 12:23
On a GeForce card, a replay the GPU refused after the program's reclaim
never reopened after the stream ended. Two causes, both found on an RTX
3070 Ti:

- An encoder's session counted as given back before the encoder closed
  it on the GPU, so a replay woken by the count was refused again.
  Drain now closes the encoder first.
- A refused replay waited for fewer sessions open than at its refusal,
  but a sibling's session the GPU already held was not counted yet, so
  it could wait for fewer than ever were. It now waits for any session
  closed after it tried.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A media source on program released its replay encoder for the program's
reclaim only at its next picture, so a sparse video, or a clip whose
sound outlasts its pictures, held going live until then. Playing now
serves the reclaim while it waits, as an ended file already did.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review November 16, 2026 21:00
@coderabbitai

coderabbitai Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 11 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: d00df5b6-fcfb-4008-b003-a09d497f1f53
Commits

Reviewing files that changed from the base of the PR and between d47ed50 and 11c9f7f.

Files selected for processing: 9

  • README.md
  • quest/m0/program-encoder-priority.md
  • rs/moq-scene/src/lib.rs
  • rs/moq-scene/src/publish.rs
  • rs/moq-scene/src/source.rs
  • rs/moq-studio/src/engine.rs
  • rs/moq-studio/src/engine/air.rs
  • rs/moq-studio/src/shots.rs
  • rs/moq-studio/src/show.rs
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@kixelated kixelated left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI) · 01534d8

[P2] Count replay opens before asking the GPU for a session. In Video::frame, a replay checks whether it's wanted, opens the hardware encoder, and only then marks the session spare. The program runs on a different thread. If a replay takes the last GPU slot and is preempted before set_spare, the program's open can fail with spare() == 0 and skip reclaim altogether. An open admitted just before reclaim can also slip past closed() while it isn't counted.

Reserve a replay open under the same state that admits a reclaim, and include pending opens in the reclaim's drain condition. The reservation should clear after the hardware handle drops, including on a failed open. A barrier-backed fake encoder can hold one replay between hardware admission and publication and verify that the program still wins.

Keeping watched source encoders and prioritizing the program over replay-only encoders is the right policy. Static review of this interleaving; I did not run the GPU or simulated-cap tests.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No blocking issues. One minor suggestion inline.

Reviewed changes

This review covers the whole PR at 01534d8: when the GPU refuses the program an encoder, the program now takes back the encode sessions held by the sources' replay encoders.

  • Sessions state and Reclaim guard: the open count is now a watch over open, spare, closed, reclaims, and an optional cap. reclaim() closes the replay-only encoders and stops them reopening while the guard lives.
  • Replay side (publish::Video): wanted() folds in the reclaim and a refusal snapshot. settle() also wakes on session changes. A reclaimed source whose reopen is refused waits instead of failing its track.
  • Encoding::drain order: the encoder now drops before its Session, so a woken replay finds the GPU session free.
  • Media playback (Driver::serve): a playing clip services settle() between pictures, through its sound tail, and up to its end, so it hands its session back at once.
  • Program side (engine.rs): Probe::new reclaims when the probe errors or falls back to openh264, and the canvas Job keeps the claim until its encoder opens. Encoder::open reclaims and retries once when an open fails.
  • Tests and README: three new paused-clock or fake-encoder tests and a render-sw engine test. The README paragraph on encode sessions now describes the reclaim.

I checked that reclaim.closed() cannot hang. Only the capture and media drivers create a publish::Video, and both service settle() whenever an encoder is open. I also checked that the new sessions.changed() wake in settle() cannot spin: set_spare and hold only notify on a real change. The race between a replay's open and its set_spare is already raised in the earlier automated review, so I have not repeated it here. quest context ran in this review environment.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using claude-opus-5-5 | 𝕏

Comment thread rs/moq-studio/src/engine.rs Resolved
kixelated and others added 2 commits November 16, 2026 13:16
A replay opened its encoder on the GPU and only then counted its
session, so the program on another thread could find no spare session
and skip the reclaim, or see the reclaim close while a replay was still
opening. Encoding::hold now counts the session first, spare for a
replay and refused while a reclaim is under way, and Encoding::open
takes it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Counting a replay's session before it asks the GPU made a refused
attempt drop a counted session, which woke the same replay to try again
at every picture: 300 refused NVENC opens in 20 s on an RTX 3070 Ti.
Only a session an encoder opened in now counts as given back.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Contributor Author

Re the review on 01534d8 ("Count replay opens before asking the GPU for a session"): fixed in 98af64a and 4f12117.

  • Encoding::hold(spare) counts the session before the encoder asks the GPU, as spare for a replay, and refuses a spare one with Error::Reclaimed while a reclaim is under way, in the same state update that admits it. Encoding::open(session, encoder) takes that session, and the program's open holds its own the same way. So the program always sees an opening replay as spare, and Reclaim::closed() waits for it.
  • The reservation drops after the encoder, including on a failed open. Only a session an encoder opened in counts as given back, so a refused replay does not wake itself. The first GeForce run after the reservation showed exactly that (a retry at every picture), and the second commit fixes it.
  • Test: publish::tests::a_replay_is_counted_before_it_opens. It checks the invariants directly instead of using a barrier-backed fake, since Video opens a real moq_video::encode::Encoder.
  • Re-checked on an RTX 3070 Ti at 4f12117, twice: program on nvenc, no failed track, one refused replay that reopens after the stream ends. Details are under Validation.

(Written by Claude Opus 5.5)

@kixelated kixelated left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI) · 4f12117

The in-flight replay race is addressed. A replay now counts as spare before invoking the encoder constructor. The same lock rejects new spare reservations once reclaim starts, and the drain waits for ones already in progress. A failed reservation/open stops blocking reclaim but does not increment the close counter, so it cannot make a refused replay retry on its own failure. The new fixture checks those transitions. No new blocker in this delta; static review, not a GPU or simulated-cap test run.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No new issues in this delta. The earlier minor suggestion on engine.rs:2793 (the encode-thread reclaim isn't gated on Renditions::reclaim) is unchanged and still open.

Reviewed changes

This review covers 01534d8..4f12117, which closes the race the OpenAI review found between a replay's open and the program's reclaim.

  • Session reservation before the GPU open: Encoding::hold(spare) now counts the session, as spare when only a replay uses it, in the same send_if_modified that admits a reclaim. It refuses a spare session with the new Error::Reclaimed while a reclaim is under way. Encoding::open(session, encoder) takes that session.
  • Only opened sessions count as given back: Session.opened is set in open, so a reservation that fails in Encoder::new releases its spare count, which unblocks Reclaim::closed(), without bumping closed. A refused replay therefore can't wake on its own failure.
  • Replay path (Video::frame): reserves before Encoder::new. On Error::Reclaimed it marks itself reclaimed and stays closed until the reclaim ends, when the reclaims change wakes settle().
  • Program path (Encoder::try_open): holds a non-spare session before building the encoder. hold drains first, but Encoder::open runs only when the track is closed, so behavior there is unchanged.
  • Tests: added a_replay_is_counted_before_it_opens, and moved existing tests to hold/open(session, …) with an opened() helper for sessions that stand in for a GPU-held one.

I checked the transitions: a replay still inside Encoder::new counts as spare, so the program's spare() > 0 check sees it and closed() waits for it. No new spare reservation starts once reclaims > 0. A watched source (spare = false) still opens during a reclaim, which matches the "keep read tracks" decision.

Pullfrog  | Fix it ➔ | View workflow run | Using claude-opus-5-5 | 𝕏

The program's encode thread reclaimed the sources' replay encoders on
any failed open while they held a session, even on a machine with no
hardware encoder, where freeing GPU sessions cannot help and only
throws away each source's replay. It now reclaims only where the probe
would: when launch found a hardware encoder, or the sessions stand in
for a capped one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@kixelated kixelated left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI) · 21bd7be

No new blocker. The encode-thread retry now uses the same reclaim policy as the probe: software-only production runs do not discard replay history after an unrelated open error, while a simulated cap can still exercise reclaim without a GPU. The non-owning open reservation from the previous round is unchanged. Static review; I did not rerun the software or GPU fixtures.

pullfrog[bot]
pullfrog Bot previously approved these changes Nov 16, 2026

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found. The encode thread now uses the same reclaim gate as the probe.

Reviewed changes

This review covers 4f12117..21bd7be, which addresses the earlier finding that the encode thread's reclaim had no gate.

  • Shared reclaim gate: Renditions::reclaim is now Some when launch found Kind::Hardware or when the sessions are Sessions::capped. The new Sessions::is_capped reports the second case.
  • Encode thread (Encoder::open): the reclaim now goes through reclaim_from, which carries Renditions::reclaim on the Job. The always-present sessions field is gone. The Job built in Renditions is the only construction site, so no encoder skips the gate. An uncapped software machine therefore never closes replays on either path.

A side effect, for tests only: with capped software sessions, openh264 never counts as hardware in Probe::new. The probe path now reclaims whenever spare sessions exist, so the capped(1) engine test probably exercises the probe path more than Encoder::open's retry. Production never caps sessions, so its behavior does not change.

Pullfrog  | View workflow run | Using claude-opus-5-5 | 𝕏

# Conflicts:
#	rs/moq-studio/src/show.rs

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found. The only new commit merges main, and the merge resolves cleanly.

Reviewed changes

This review covers 21bd7be..11c9f7f, a merge of origin/main that brings in #188 (close the show and return to the library).

  • Merge resolution in show.rs: the Launch in Opener keeps room: room.clone() from #188 and adds this PR's sessions: Default::default(). Every other Launch construction (shots.rs, including the ..launch(&gpu) spread, and the engine tests) already sets sessions, so the earlier approval of the reclaim logic still holds.

Pullfrog  | View workflow run | Using claude-opus-5-5 | 𝕏

@kixelated
kixelated merged commit 303b14f into main Nov 16, 2026
4 checks passed
@kixelated
kixelated deleted the quest/m0/program-encoder-priority branch November 16, 2026 22:41
kixelated added a commit that referenced this pull request Nov 16, 2026
Brings in the program reclaiming replay encoders' sessions (#189). Launch
keeps its new sessions and no room, which Engine::join follows here; the
README points a published scene's refusal at the Health window, where the
encoders are now.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant