Skip to content

fix(obs): reconnect tests drop a session only once it is healthy - #9016

Draft
kixelated wants to merge 1 commit into
mainfrom
fix/obs-reconnect-healthy
Draft

kixelated wants to merge 1 commit into
mainfrom
fix/obs-reconnect-healthy

Conversation

@kixelated

@kixelated kixelated commented Nov 17, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

Nightly OBS and OBS TSan fail in test/moq-output-test.cpp (lines 526, 999-1003, 1020-1021): after the test relay drops a session, the output never reconnects and OBS gets a stop signal.

Since moq-dev/moq#8817, a session its peer closes before the 1 s healthy bar ends the client instead of redialing, the way a refused session does. The OBS tests dropped the session as soon as it went live, so they now hit the fail-fast path. The tests are out of date, not the product: a relay that closes a session within a second of accepting it looks like a refusal, and OBS still gets OBS_OUTPUT_DISCONNECTED, which lets its own reconnect retry.

Approach

A new OutlastHealthy() helper in moq-test-relay.h waits out moq-ffi's healthy bar after the test has seen the session go live. Because the client went live before the test observed it, waiting 1 s from that point always gets past the bar. The three output-test drops and the source test's "follows a session reconnect" call it before closing the session. moq-ffi doesn't let a caller set the healthy bar, so the wait has to be on the real clock. #8817 already did the same in the Go, Python, and moq-c tests and missed OBS, because the PR gate's OBS filter doesn't watch rs/moq-net.

Impact

  • Tests only. No public API or wire change.
  • moq-output-test takes about 3 s longer, and moq-source-test about 1 s longer. Both run nightly, and in just obs ci when OBS paths change.

Alternatives

  • Expose healthy in MoqBackoff so the test could set a 10 ms bar. That adds API to every binding just for this test, and the test would still have to wait out the bar.
  • End the session with a GOAWAY, which establishes the session right away. moq-ffi's server has no GOAWAY, and a GOAWAY isn't the hard drop these tests mean to cover.
  • Assert the new fail-fast behavior instead. That would drop coverage of the reconnect count, of instance reuse across a reconnect, and of the dock naming the failure while it redials.

Follow-ups

Caused by

  • moq-dev/moq#8817 - made a session closed before it was healthy end the client, and updated the Go, Python, and moq-c tests for it but not the OBS tests (found by check: nightly OBS / OBS TSan)

Validation

  • Before the fix, nix develop -c bash sh/obs/unit.sh plain reproduced the failures at lines 526 and 999-1003.
  • After the fix, sh/obs/unit.sh plain and sh/obs/test.sh (ThreadSanitizer, the same as nightly OBS TSan) both pass every test binary, including moq-source-test's "follows a session reconnect", which never ran before because unit.sh stops at the first failing binary.
  • just obs check and just ci pass.

Models

Size: S
Implementer: Claude Code, claude-opus-5-5, unknown effort

(Written by Claude Opus 5.5)

🤖 Generated with Claude Code

Review

Rewritten by quest review after each round.

  • Head: 2e3dba7

  • Base: origin/main at bc66d3e

  • Risk: medium, the base has no RISK.md to name what is always high or customer-facing, so every change is medium

  • Classified: medium by default

  • Reviewer: Codex, gpt-6.1-sol, medium effort

  • Session: 01a1ea8e-5ae4-7c30-af1e-5200040c0099, round 1

  • Score: 9/10, The narrowly scoped waits match the reconnect policy by letting real-QUIC sessions reach the one-second healthy threshold; no actionable defects were found, but builds and tests were not run as instructed.

  • Rounds: 1; first round 0; findings 0: fixed 0, refuted 0, open 0

  • Findings: none

A session its peer closes before moq-ffi's 1s healthy bar now ends the
client rather than redialing (#8817), so the OBS tests that dropped a
freshly live session saw the stream fail instead of reconnect.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review November 17, 2026 17:29
@coderabbitai

coderabbitai Bot commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 36 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 58bb5e34-4000-4577-a549-b51a9487d91e
Commits

Reviewing files that changed from the base of the PR and between bc66d3e and 2e3dba7.

Files selected for processing: 3

  • cpp/obs/test/moq-output-test.cpp
  • cpp/obs/test/moq-source-test.cpp
  • cpp/obs/test/moq-test-relay.h
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes: the single commit adding a healthy-bar wait to the OBS reconnect tests.

  • OutlastHealthy() helper: a 1 s real-clock sleep in moq-test-relay.h. <thread> and <chrono> are already included.
  • Output and source tests: moq-output-test.cpp (3 sites) and moq-source-test.cpp (1 site) call it after the session is seen live and before the drop, so the drop lands past moq-ffi's healthy bar and the client redials.

Pullfrog  | View workflow run | Using claude-sonnet-5-5 | 𝕏

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)
Reviewed commit: 2e3dba76b6599eeccbf809f7c1a9b9ff687c2bea

No blocker. Each hard drop waits after the client has shown a live session (or received both video and audio), rather than sleeping from the dial or server accept. Rust's reconnect policy uses tenure >= healthy, so a one-second wait from those observations is sufficient at the current default. This preserves the reconnect assertions without adding a public FFI knob just for tests. As a separate CI follow-up, consider having reconnect-policy changes trigger the OBS test lane, so the next cross-layer mismatch is caught before nightly. Static review; I did not rerun OBS or TSan.

@kixelated

Copy link
Copy Markdown
Collaborator Author

Maintainer decision (2026-11-17): ✅ redesign the healthy signal so a session counts as established on an explicit signal instead of 1 s of uptime (rejected: keep the real 1 s waits in the OBS tests; expose Backoff.healthy in the bindings for tests). This PR waits on that quest, and the OBS tests should then drop the session right after the new signal. Moving to draft.

(Written by Claude Opus 5.5)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant