[gareth-morgan-nv]

Summary

The Orin camera producer sends H.264 straight to a PC worker over a direct link, with no LiveKit on the Orin; the PC worker publishes to LiveKit so the egocap consumers (backend, recorder, page) are unchanged. Also adds a management channel and fault diagnosis to the producer, and a mock producer for testing without cameras.

Orin to PC link

• The producer's DirectSender sends AVS1 H.264 records over a DATA connection and exchanges keyframe requests and a four-timestamp clock exchange over a CTRL connection. It has a bounded queue (size and age), reconnects, and shuts down promptly.
• Each encoded unit is matched to its capture by the stamp the encoder returns (the encoder is asked to copy input timestamps). Inputs the encoder skipped are counted, and a unit that matches nothing is sent without a capture time. --frame-correlation=fifo keeps an approximate in-order mode as an explicit choice.
• The producer lowers its bitrate while its direct link drops frames (not while no destination is connected), raises it again after a calm stretch, and spaces forced keyframes at least a second apart.

PC worker (video_stage_worker)

• Default mode forwards access units untouched; no CUDA context is created. --operation=sine-warp runs NVDEC, an example GPU operation and NVENC; a VUI-less Orin stream's SPS is rewritten to declare no reordering.
• It never withholds video for a timing problem: when the clock mapping is missing or over --max-upstream-uncertainty-ms, or the record has no source stamp, the frame is published without a capture timestamp and counted. A session reset rebuilds the decoder; a stream with non-default chroma siting is refused.
• Counters are atomic; the receiver's descriptors are guarded against concurrent shutdown and replacement. Output is fixed-rate (a downstream bitrate request is logged once).

Management channel and fault diagnosis (producer)

• Optional authenticated WebSocket control server (--control-token-file, off by default; port 5610) with get_status, set_direct_target and ping, plus tools/orin_control.py. One thread owns the camera session; retargets are recorded and applied by that thread.
• A stall watchdog restarts a session after 5 s without frames. Camera start failures are matched against a table of known failure signatures (including driver log lines, captured around camera start) and reported in plain language, for example a camera power or cable fault.
• sensing_mock_producer runs the same server, handler, supervisor and DirectSender over looped H.264 files.

Measurements (real Orin, Wi-Fi, two cameras)

Orin-to-publish p50 about 45-78 ms on a good link; clock mapping within about 4.5 ms. On a loaded Wi-Fi link round trips reached 150-300 ms, which is what produced mapping outside the budget; those frames now flow without timestamps instead of being dropped. With the timestamp copy enabled, the Orin reported every unit matched and none skipped. These are not webpage latency.

Validation

• Run on the PC: unit tests for the correlator, AVS1, clock exchange, SPS parsing, the receiver, DirectSender, and the control server/handler/supervisor (the control server under sanitizers); a mock-producer end-to-end test; pre-commit. These were last run before the final encoder edit, which cannot be built on the PC.
• Run on a real Orin: the producer builds, streams both cameras to the PC worker, and the stats report stamps matched N, unmatched 0, encoder skipped 0.
• Not run: the full test suite after the final rebase; sine-warp mode in this revision; the management channel against a real client over a long session.

Base is gmorgan/TimestampAlignment.

🤖 Generated with Claude Code

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

• Source mode: Auto
• Repository: NVIDIA/IsaacCapture
• Branch: gmorgan/TimestampAlignment
• File: .coderabbit.yaml
• Commit: 30c47c3
• Layers: Repository YAML
• Review profile: CHILL
• Plan: Enterprise
• Run ID: 888abd8e-43a1-4f70-b64e-a90b173731eb

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• Trigger review

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts