fix(voice): place recorded audio where it actually played - #6921
Open
longcw wants to merge 1 commit into
Open
Conversation
RecorderIO built the agent channel by re-composition: it collected the frames it forwarded, then placed them from the wall-clock moment playback_finished arrived and padded the difference with silence. Nothing measured where the audio went, so a sink that ran dry mid-segment, a late flush, audio discarded on pause, or a microphone that stopped delivering all moved the audio around instead of recording the gap. Sinks that own their playback device now report it. AudioOutput gains playback_progressed alongside playback_started and playback_finished, and _ParticipantAudioOutput reports one run at a time off its own queue: where the run sits in the segment, and when it began to play. Reports are past tense, so they are never revised, and an offset jump describes audio that was discarded rather than played. A sink that reports nothing is described by its segment endpoints instead, which is all a remote avatar worker can know. RecorderIO places both channels on one absolute timeline and leaves unwritten time silent. That drops the pause reconstruction, the end-anchored padding and the playback_position truncation.
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
RecorderIO rebuilds the agent channel by inference. It collects the frames it forwarded, places them from the wall-clock moment
playback_finishedarrives, and pads the difference with silence. Nothing measures where the audio really went.Four conditions break that inference: a sink that runs dry inside a segment, a late flush, audio discarded on
pause(), and a microphone that stops delivering. Each one moves the audio, and none of them records the gap. A slow TTS or a muted participant may hit it.Fix
A sink that owns its playback device now reports where its audio went.
AudioOutputgainsplayback_progressednext toplayback_startedandplayback_finished, and_ParticipantAudioOutputreports one run at a time off its own queue. Each report is past tense, so it is never revised, and a jump in its offset describes audio the sink discarded rather than played.RecorderIO places both channels on one absolute timeline and leaves unwritten time silent. When a sink reports nothing, the recorder uses its segment endpoints instead, which is all a remote avatar worker can know. That removes the pause reconstruction, the end-anchored padding, and the
playback_positiontruncation.