Skip to content

feat(ffi)!: append_group takes a required timestamp start in every binding - #9002

Merged
kixelated merged 6 commits into
mainfrom
quest/m2/time-addressing/bindings
Nov 17, 2026
Merged

kixelated merged 6 commits into
mainfrom
quest/m2/time-addressing/bindings

Conversation

@kixelated

@kixelated kixelated commented Nov 17, 2026 •

Copy link
Copy Markdown
Collaborator

Finishes quest/m2/time-addressing/bindings.md (deleted here): every binding's append_group takes a required Timestamp start, the way Rust's track::Producer::append_group(start: Timestamp) does, so no binding invents "one past the newest start" anymore.

Problem

Rust's append_group takes a required start (moq-dev/moq#8961), but moq-ffi's MoqTrackProducer::append_group() and moq-c's moq_publish_track_group took none and each computed one past the newest start in a private next_start. On a timed track that meant a group's start was a tick count unrelated to its media time (a microsecond track's groups started at 0, 1, 2 us whatever their frames said).

Approach

  • moq-ffi: append_group(start: MoqTimestamp), converted to the track's timescale as a frame's timestamp is; an untimed track takes MoqTimestamp::unscaled(key). There is no Option: the start is required. next_start is deleted. create_group's doc now says how it differs (it declares nothing below it skipped).
  • moq-c: moq_publish_track_group(track, const moq_timestamp *start), NULL failing with MOQ_ERROR_INVALID_POINTER like every other timestamp argument; its next_start is deleted.
  • Wrappers follow with the same shape: Python append_group(start: Timestamp), Go AppendGroup(start Timestamp), Swift appendGroup(start:); Kotlin, Dart, and C++ call the generated method directly.
  • Every test, bench, example, and the Python interop socket client passes an explicit start. Timed call sites start the group at its first frame's time (so the backlog tests now read starts 0, 1000, 2000 instead of 0, 1, 2); the Python clock examples start each minute's group at that minute.
  • New tests: moq-ffi append_group_starts_at_its_timestamp (a millisecond start lands at its microsecond key, a rewound or unscaled start is refused) and the moq-c raw track test (NULL refused, rewind refused, a millisecond start reads back from moq_publish_group_start in microseconds). Python's test_raw_append_group_start_increments, which asserted the deleted counter, became the conversion check.
  • Docs: doc/lib/c/index.md, doc/lib/dart/index.md, and the moq-c README's signature list.

Impact

Public API: breaking in every binding. Wire: none.

Migration (for each package's MIGRATE.md at the release cut):

  • moq-ffi (UniFFI): MoqTrackProducer::append_group() is append_group(start: MoqTimestamp). Pass the group's first frame time; on an untimed track pass MoqTimestamp::unscaled(key), choosing a key past the newest group (the old implicit value was largest().group + 1, or 0 on an empty track). A start at or before a group the track holds is refused with TimestampMismatch.
  • Python: track.append_group() is track.append_group(start), with a moq.Timestamp (moq.Timestamp.unscaled(key) on an untimed track).
  • Go: AppendGroup() is AppendGroup(start moq.Timestamp) (moq.Unscaled(key) on an untimed track).
  • Swift: appendGroup() is appendGroup(start:) (.unscaled(key) on an untimed track).
  • Kotlin: appendGroup() is appendGroup(start) (Timestamp.unscaled(key)).
  • Dart: appendGroup() is appendGroup(start: ...) (Timestamp.unscaled(key)).
  • C++: append_group() is append_group(moq::Timestamp{value, scale}) (std::nullopt scale on an untimed track).
  • C (moq-c): moq_publish_track_group(track) is moq_publish_track_group(track, const moq_timestamp *start); an ABI break. On an untimed track pass has_scale = false and the key in value.

Alternatives

  • Keep a start-less convenience next to the required one: rejected by the maintainer (feat(net)!: writing a group needs no invented start, and its end has one word #8961, and the brief for this PR): there is no Option<Timestamp> anywhere, and an untimed track uses a timestamp with no timescale.
  • Pass the C start by value (as moq_publish_media_frame does) instead of by pointer: kept the pointer, matching moq_publish_track_frame and moq_publish_group_frame, the raw-track writes it sits beside.

Validation

  • just check: passes for rs, c, cpp (bench compile), obs, py, go, kt, dart, quest, docs. Swift was skipped (no Swift toolchain on this host); its change is the labeled start: argument the other UniFFI wrappers already compile against.
  • After merging main (to clear the merge-queue conflict in doc/lib/c/index.md, kept both sides): just check passes, and just test interop --all passes every cell. The merge also needed main's new track_abort_unwanted_keeps_a_late_reader to pass a start to moq_publish_track_group.
  • moq-c parity::session_tokens_and_grants flaked in that just check (moq_session_close(client) returned -21). Reproduced with the built test binary, 400 runs 64 at a time: 123 failed before the fix, 0 of 400 after it.
  • just test interop --all: 201 PASS. The 9 finish rust -> * cells failed in the full run (the Rust publisher, which uses moq-net directly and nothing this PR touches); just test interop --all --finish --publishers rust alone passes all 9. Python/Rust sockets (which now call append_group(Timestamp.unscaled(key))) pass in both roles.
  • Real surface, second implementation: served py/moq-rs/examples/serve_clock.py (each minute's group appended at its own timestamp) and read it with the Rust moq-tokio clock example over WebTransport; the Rust reader printed every second (2026-11-17 11:04:12 through 11:04:19) from a group whose start is the minute's time.
  • The generated moq.h declares int32_t moq_publish_track_group(uint32_t track, const struct moq_timestamp *start);.

Caused by

  • moq-dev/moq#8817 - made a dialed session closed by its peer before it is healthy end on its own, and reordered the other moq-c session tests to close their client first, but missed parity::session_tokens_and_grants, which then raced the client's own ending (found by check)

Decisions

  • The merge with main fixed parity::session_tokens_and_grants here, because it failed this PR's just check and would flake its landing gate: the test now closes the client first, then waits for the accepted session to end and checks its handle is gone, as announce_route_crosses_a_relay_session and a_minted_instance_pins_request_and_follow already do. moq-dev/moq#8964 adds quest/m0/test-flakes-3/c-server-accepts-session.md, which plans this fix along with server_accepts_a_session; that test closes the server side first on purpose (its tail must drain), so it is left to that quest, which can drop session_tokens_and_grants once this lands. Recommendation: keep the fix here.

Follow-ups

None.

Models

Size: M
Implementer: Claude Code, claude-opus-5-5, high effort
Implementer: Claude Code, claude-opus-5-5, unknown effort

Review

Rewritten by quest review after each round.

  • Head: 5d0829f

  • Base: origin/main at 0edf6e2

  • Risk: high, The branch changes public group-creation signatures, including the C ABI, so downstream binding users must update and rebuild their callers, and reverting after adoption requires coordinated rollback.

  • Classified: medium by default

  • Reviewer: Codex, gpt-6.1-sol, high effort

  • Session: 01a1e92f-5522-7f50-acac-5198849023d8, round 2

  • Score: 10/10, The incremental review found no actionable defects in the test fixes or merge resolution; builds and tests were not run as instructed.

  • Raised the risk to high: The branch changes public group-creation signatures, including the C ABI, so downstream binding users must update and rebuild their callers, and reverting after adoption requires coordinated rollback.

  • Rounds: 6; first round 0; findings 0: fixed 0, refuted 0, open 0

  • Findings: none

  • Reviewer: Codex, gpt-6.1-sol, low effort, shadow

  • Session: 01a1e46e-249e-7792-8710-e97fe1a3af2a

  • Rounds: 1; first round 0; findings 0: fixed 0, refuted 0, open 0

kixelated and others added 4 commits November 17, 2026 02:03
…start

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title Bindings start a group at a given timestamp feat(ffi)!: append_group takes a required timestamp start in every binding Nov 17, 2026
@kixelated
kixelated marked this pull request as ready for review November 17, 2026 11:08
@coderabbitai

coderabbitai Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 26 seconds.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 57bba7e4-3a69-4ce1-a007-8ca00746e48a
Commits

Reviewing files that changed from the base of the PR and between 1399add and 5d0829f.

Files selected for processing: 33

  • cpp/moq/bench/main.cpp
  • dart/moq/bench/ffi.dart
  • dart/moq/test/moq_test.dart
  • dart/moq_ffi/test/leak_test.dart
  • dart/moq_ffi/test/smoke_test.dart
  • doc/lib/c/index.md
  • doc/lib/dart/index.md
  • go/wrapper/errors_test.go
  • go/wrapper/moq_test.go
  • go/wrapper/publish.go
  • go/wrapper/runtime_bench_test.go
  • kt/moq/src/jvmAndAndroidTest/kotlin/dev/moq/SmokeTest.kt
  • kt/moq/src/jvmTest/kotlin/dev/moq/FfiBench.kt
  • py/moq-rs/benches/ffi.py
  • py/moq-rs/examples/clock.py
  • py/moq-rs/examples/serve_clock.py
  • py/moq-rs/moq/publish.py
  • py/moq-rs/tests/test_local.py
  • py/moq-rs/tests/test_testing_clock.py
  • quest/m2/time-addressing/bindings.md
  • rs/moq-c/README.md
  • rs/moq-c/src/api.rs
  • rs/moq-c/src/publish.rs
  • rs/moq-c/src/test.rs
  • rs/moq-c/src/test/parity.rs
  • rs/moq-ffi/src/bench.rs
  • rs/moq-ffi/src/origin.rs
  • rs/moq-ffi/src/producer.rs
  • rs/moq-ffi/src/test.rs
  • swift/Sources/Moq/Track.swift
  • swift/Tests/MoqTests/FfiBench.swift
  • swift/Tests/MoqTests/SmokeTests.swift
  • test/interop/clients/python/sockets.py
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed a1b5a7986baa612d680ad9c62cad2e485ac097b5. Explicit group starts are the right API and the timescale tests exercise the main behavior. One C release concern: moq_publish_track_group keeps its old link symbol but adds a pointer that the implementation dereferences. An object built with the old header can silently link to the new library and pass garbage in the new argument slot. I'd give the new signature a different C symbol so stale callers fail to link. That doesn't require keeping any start-less API. If shipping the unchanged symbol is intentional, package the header/library as a pair and require a clean rebuild of consumers. I didn't run the bindings matrix.

pullfrog[bot]
pullfrog Bot previously approved these changes Nov 17, 2026

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

This review covers the whole PR: in every non-JS binding, append_group now takes a required Timestamp start, matching Rust's track::Producer::append_group(start: Timestamp).

  • moq-ffi append_group(start: MoqTimestamp): it converts the start with try_into and hands it to moq_net's append_group, which rescales it to the track's timescale and refuses a rewind with TimestampMismatch. The private next_start is deleted.
  • moq-c moq_publish_track_group(track, const moq_timestamp *start): this is now unsafe extern "C" and parses the start with ffi::parse_timestamp. NULL fails with MOQ_ERROR_INVALID_POINTER. Its next_start is also deleted.
  • Wrappers: Python append_group(start), Go AppendGroup(start Timestamp), and Swift appendGroup(start:) each forward to the FFI. Kotlin, Dart, and C++ call the generated method directly.
  • Call sites: every test, bench, example, and the Python interop socket client now passes an explicit start. Timed backlog tests expect starts 0, 1000, 2000.
  • New coverage: moq-ffi append_group_starts_at_its_timestamp checks ms→µs conversion and refuses rewound or unscaled starts. The moq-c raw-track test checks that NULL and rewound starts are refused and that a ms start reads back in µs. The Python counter test became a conversion check.
  • Docs: doc/lib/c/index.md, doc/lib/dart/index.md, and the moq-c README signature list are updated.

I also checked that no non-JS caller still uses the old zero-arg append_group() or the one-arg moq_publish_track_group. JS appendGroup() is the separate js/net API. Every wrapper helper the new call sites rely on exists: Go Unscaled/TimestampFromScale, Python Timestamp._ffi, Dart Timestamp.unscaled and Duration.toTimestamp, Kotlin Timestamp.unscaled, Swift .unscaled, and moq-c moq_timestamp::unscaled. The new create_group doc ("declares nothing below it skipped") matches moq_net's skip semantics for append_group.

Pullfrog  | View workflow run | Using claude-opus-5-5 | 𝕏

@kixelated
kixelated added this pull request to the merge queue Nov 17, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Nov 17, 2026
@moq-bot

moq-bot Bot commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

quest land watch: the merge queue removed this pull request at a1b5a79 because it conflicts with main (merge_conflict).

The land fixer works on it in the background, and the watcher re-enqueues it at the back of the queue once it is fixed or judged flaky, or hands it back here with what it needs.

(written by quest land watch)

@moq-bot

moq-bot Bot commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

codex, gpt-6.1-sol, medium effort land fixer note:

The queue log identifies the conflict at lines 3–10:

Auto-merging doc/lib/c/index.md
CONFLICT (content): Merge conflict in doc/lib/c/index.md
Auto-merging rs/moq-c/README.md
Auto-merging rs/moq-c/src/api.rs
Auto-merging rs/moq-c/src/publish.rs
Auto-merging rs/moq-c/src/test.rs
Auto-merging rs/moq-ffi/src/test.rs
Automatic merge failed; fix conflicts and then commit the result.

Merged base into pr-9002 in dd6541d29bd2d669f786a0586738ec3ca98bc71c, preserving both the PR timestamp clarification and base catalog-timing paragraph. The resolution changes no behavior. No protected-path conflict required editing.

Ran nix develop --command just ci base. All executed modules passed except an initial moq-c test::server_accepts_a_session failure (session close returned -21 instead of 0). The focused rerun passed, and nix develop --command env CHECK_MODULES=rs just ci base passed all 541 selected Rust tests. Swift could not build or test: "swift check: no swift toolchain on PATH, skipping". Its check requires macOS. This validation gap requires a handback under the requested rules.

Ran nix develop --command just test interop --all: all checks passed. The harness skipped Chromium route-key proof because its bundled Chromium version lacks the exporter; Firefox route-key proof passed.

(written by codex, gpt-6.1-sol, medium effort)

@moq-bot

moq-bot Bot commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

codex, gpt-6.1-sol, medium effort land fixer: The documentation conflict is re·solved, but nix develop --command just ci base cannot validate Swift: "swift check: no swift toolchain on PATH, skipping".. Recommendation: Review merge commit dd6541d29bd2d669f786a0586738ec3ca98bc71c on pr-9002 and run nix develop --command just swift check on macOS before landing.

Auto-merging doc/lib/c/index.md
CONFLICT (content): Merge conflict in doc/lib/c/index.md
Auto-merging rs/moq-c/README.md
Auto-merging rs/moq-c/src/api.rs
Auto-merging rs/moq-c/src/publish.rs
Auto-merging rs/moq-c/src/test.rs
Auto-merging rs/moq-ffi/src/test.rs
Automatic merge failed; fix conflicts and then commit the result.

(written by codex, gpt-6.1-sol, medium effort)

…s client first

The merge with main brought a moq_publish_track_group call without the start this PR requires.
session_tokens_and_grants closed the accepted session first, so its client could end on its own
before moq_session_close reached it (123 of 400 parallel runs failed); it now closes the client
first, as the other session tests do.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

Since the last Pullfrog review (a1b5a79), this push merges main and fixes two moq-c tests.

  • Updated the merged abort test: the merge brought in track_abort_unwanted_keeps_a_late_reader, and it now calls moq_publish_track_group(track, &micros(0)), matching the frame it writes at micros(0).
  • Reordered the session_tokens_and_grants teardown: the test now closes the dialed client first, waits for the accepted session to end, and then asserts that moq_session_close(session) fails. This order is safe. The accepted-session task in rs/moq-c/src/session.rs removes its entry before it calls the terminal callback, so the handle is already gone when recv_terminal returns.

The doc/lib/c/index.md lines that differ in the range-diff are main's own edits brought in by the merge, not changes from this PR.

Pullfrog  | View workflow run | Using claude-opus-5-5 | 𝕏

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)
Reviewed commit: 5d0829f287e618f8c0eb0b70cd15f34c08296020

The new C tests look right: the inherited abort-unwanted test now supplies an explicit group start, and the token/grant test closes the client before its peer can end it, then checks that the peer handle was released. No new blocker in this test-only delta. The previous C ABI finding is unchanged: the legacy moq_publish_track_group link symbol still takes a new pointer argument. I did not run the C tests or bindings matrix.

@kixelated
kixelated added this pull request to the merge queue Nov 17, 2026
Merged via the queue into main with commit 86bf512 Nov 17, 2026
10 checks passed
@kixelated
kixelated deleted the quest/m2/time-addressing/bindings branch November 17, 2026 14:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant