Skip to content

Select replay gates from sampled event clocks - #577

Merged
jserv merged 4 commits into
mainfrom
reply
Nov 3, 2026
Merged

jserv merged 4 commits into
mainfrom
reply

Conversation

@jserv

@jserv jserv commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Adds the offline half of replay gate verification: given finalized, drained evidence, decide whether injected audio can be attributed to a tool or hint gate, and say unknown instead of aligned whenever the sampled clocks cannot order the events. A candidate acknowledgement is selected from sampled event clocks and then verified against the whole pinned process, room and session: a newer boundary, a replacement, a discarded socket, a terminal state or a stale age invalidates it, and an event that overlaps injection stays unknown.

To give the verifier intervals instead of instants, the room loop now records whether the interview is closing, logs a requested end before the farewell, and stamps each gate boundary with when it was handled as well as when it arrived, so an interruption queued behind a tool send cannot sort before the acknowledgement it followed. Observation lines are written once instead of field by field. The series also carries a fix for valid raw PCM whose first bytes only resemble a container signature (d2 c9 c6 c6 read as RIFF once the high bits were cleared), and the earlier fixture, evidence-retention and clock-correlation commits it builds on.

Verified with the full local gate (scripts/test.sh) on the rebased tip, the replay and floor-capture Node suites, the livekit Rust tests, and the Rust style check. Guard removals fail the new tests, including the queued-boundary and boundary-spanning-the-window cases. Not run: desktop checks that need WebKitGTK, and cargo-deny.

The helpers are not wired into a replay adapter yet, so no live interview or provider is exercised, and no endpointing, recovery, tool response or scoring policy changes. Left for follow-ups: emitting socket replacement and discard as gate boundaries from the producer instead of reading pressure summaries, deriving the JS tool and stage vocabulary from the Rust side, and a dedup key that tolerates two identical gate lines in one millisecond.


Summary by cubic

Adds the offline half of replay gate verification: given finalized diagnostic evidence, decide whether injected audio can be attributed to a tool or hint gate, and report "unknown" whenever sampled clocks cannot order the events. A candidate acknowledgement is selected from sampled event clocks and verified against the whole pinned process, room and session; a newer boundary, a replacement, a discarded socket, a terminal state, or a stale age invalidates it, and events overlapping injection stay unknown. This also adds named fixture validation, diagnostic stream retention, and clock correlation as the supporting pieces.

These helpers are not wired into a replay adapter yet, so no live interview, provider, endpointing, recovery, or scoring behavior changes.

Evidence and timing

  • The room loop now records whether the interview is closing, logs a requested end before the farewell, and stamps each gate boundary with when it was handled and when it arrived, so an interruption queued behind a tool send cannot sort before the acknowledgement it followed.
  • Floor records carry opening, first resolution, and candidate activity-end times on the interview clock; latency records carry sampled wall-clock and monotonic pairs with uncertainty.
  • Complete floor and latency streams are retained through shutdown with byte-bounded lines and drain-before-close.
  • Observation lines are written once instead of field by field.

Bug Fixes

  • Fixes valid raw PCM whose leading bytes resemble a container signature being refused: d2 c9 c6 c6 previously read as a RIFF header once the high bits were cleared.
  • Malformed floor record tests now assert the exact refusal reason instead of accepting any of six alternatives.

Written for commit a47bc32. Summary will update on new commits.

View guided diff Turn on auto-fix

@jserv
jserv requested a review from ColtenOuO as a code owner November 2, 2026 22:24
cubic-dev-ai[bot]

This comment was marked as resolved.

@jserv jserv added the explained A maintainer has seen the contributor explain this large change label Nov 2, 2026
jserv added 4 commits November 3, 2026 07:20
Turn-taking replay needs a reproducible input identity before a browser
or socket run can distinguish provider behavior from a malformed clip.
The existing interruption smoke and fixed speech probes do not provide
that shared contract. Define one named synthetic clip with exact-byte
digests, transport-specific audio, fresh scenario gates and a bounded
post-playback observation window.

Reject ambiguous JSON, incompatible audio, container headers, unsafe
file references and inputs beyond the byte or duration limits before
transport access. Resumed-pause fixtures must contain an annotated quiet
interval with signal on both sides. Document the existing entry points
and the clock correlation, durable evidence and budget requirements the
adapters must satisfy, without changing the current barge smoke or any
Gemini endpointing, interruption, recovery or scoring behavior.

Fourteen offline tests exercise the loader, CLI, file races, audio and
pause boundaries. Twelve guard-removal mutants fail in an isolated copy.
The full gate and final portable/browser lanes pass, including 2683
browser tests with no skips. Native desktop checks and cargo-deny remain
unverified because required libraries or tools are unavailable. The
fixtures prove input mechanics; browser/socket replay, provider outcome
judgment and live stability acceptance remain separate work.

Change-Id: Ia773182cf969d40955be4953b4acbf10eb07cfcd
A replay cannot attribute a floor outcome to injected audio from a
rolling agent tail or the order in which records arrive. Pauses can
settle long before teardown emits them, and tool context can expire
without an active latency trace.

Retain complete floor and latency diagnostic streams through owned
server and agent shutdown. Bound UTF-8 lines by bytes, mark incomplete
capture and source discontinuities, and drain each pipe before closing.
Attempt every cleanup without replacing a primary budget or soak error.
External dispatch logs explicitly report unavailable evidence.

Record floor opening, first resolution and candidate activity end on the
interview clock. Add sampled wall and monotonic clock uncertainty,
process identity, unconditional accepted-handler boundary observations,
and allowlisted accepted tool names with actual hint-counter changes.
These observations do not change endpointing, interruption, recovery,
tool responses, scoring or what is considered audible.

Exercise real child streams, source rotation, truncation, short writes,
UTF-8 limits and cleanup failure precedence. Cover first-resolution and
activity-loss timing, tool acceptance and boundary emission without a
trace. The full local gate passes with 2,705 browser tests and no
browser skips; desktop checks need WebKitGTK and cargo-deny is not
installed. Fifteen targeted guard-removal mutants fail their regression
suites.

This supplies replay evidence producers and retention. Named browser and
socket adapters, event-clock joins and common trial judgment remain
separate work; no live provider or physical audio acceptance is implied.

Change-Id: Id22928886806469d60ca10394f0a3eb545b7e436
Floor records can resolve inside an observation window and appear only
at teardown. Correlate their opening times instead of treating emission
order or the highest sequence as proof that injected audio caused them.
Keep process, room, session and socket identity explicit and return all
candidate, preexisting, ambiguous and unusable evidence without choosing
an outcome that happens to match the fixture.

Join trial-local producer and harness clock samples with distinct clock
identities, sampled-drift checks and conservative uncertainty hulls.
Include millisecond quantization and reserve uncertainty margins inside
both sample brackets. Missing anchors, excessive uncertainty and events
outside the supported brackets cannot support attribution.

Read complete floor, reply, boundary, replacement and pressure records
with physical UTF-8 byte limits and explicit finalization requirements.
Reject incomplete framing, duplicate evidence and malformed timestamps;
retain legacy unknown clocks as unusable. Share bounded regular-file and
duplicate-key JSON validation with the existing fixture loader,
including change detection across the read and BOM refusal.

Cover delayed preexisting records, teardown resolution, closed interval
edges, replaced sockets, real producer pressure summaries and input
corruption. Forty-five focused tests pass, and thirteen targeted guard
removals fail those tests in isolated scratch copies. The full local
gate passes with 2,736 browser tests and no browser skips; desktop
checks need WebKitGTK and cargo-deny is not installed.

This supplies offline correlation for the remaining replay adapters.
Live gate verification, named browser and socket injection, common trial
judgment and provider acceptance remain separate work. No endpointing,
recovery, tool response or scoring policy changes are included.

Change-Id: Ibebfcb66e2320da6d1f6c65b586bc4461c6314c2
Injected audio can only be attributed to a tool or hint gate when the
acknowledgement that opened it is still the newest context. Select a
candidate from sampled event clocks, then verify it against finalized
evidence: a newer boundary, a replacement, a discarded socket or a
terminal state invalidates the gate, as does a stale age. Anything the
clocks cannot order stays unknown rather than aligned.

Record the closing state and the end of each event so these checks have
the interval they need. A boundary also ends when it is handled, not
when it arrived, so an interruption queued behind a tool send cannot
sort before the acknowledgement it followed. Replies and gate boundaries
carry whether the interview is closing, and a requested end is logged
before the farewell. Keep read failures typed so an unexpected error
still surfaces instead of reading as unknown evidence.

Each observation line is written once, so a line cannot be split or
cost a syscall per field. The helpers are not wired into a replay
adapter yet.

Change-Id: Ia87d1522947d41d793eeb100ff2d5d3313949e12
@jserv
jserv merged commit 8f5ad5c into main Nov 3, 2026
17 checks passed
@jserv
jserv deleted the reply branch November 3, 2026 00:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

explained A maintainer has seen the contributor explain this large change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant