feat(audio): negotiate stereo opus on the subscriber answer - #1007
Open
kirill-jjj wants to merge 3 commits into
Open
feat(audio): negotiate stereo opus on the subscriber answer#1007kirill-jjj wants to merge 3 commits into
kirill-jjj wants to merge 3 commits into
Conversation
The server advertises a stereo publisher track via sprop-stereo=1 in its offer, but libwebrtc's Opus decoder only decodes incoming packets as stereo when the local answer's fmtp line also carries stereo=1. Without it, stereo tracks published by other participants are received as mono. - StereoSdpMunging: extract sprop-stereo mids from the server offer and add stereo=1 to the matching fmtp lines of the answer (mirrors client-sdk-js remoteStereoMids handling), applied in onServerOffer - LocalAudioTrackOptions.stereo + TF_STEREO track feature so publishers can declare a stereo capture - AudioTrackPublishOptions.stereo sent as AddTrackRequest.stereo, mirroring client-sdk-js forceStereo Includes unit tests built from real device SDP captures and a changeset.
kirill-jjj
requested review from
MaxHeimbrock,
davidliu and
xianshijing-lk
as code owners
August 23, 2026 13:55
🦋 Changeset detectedLatest commit: 0212c81 The changes in this PR will be included in the next version bump. Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
|
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
…ment stereo option Addresses Devin review feedback on livekit#1007: - if the munged answer is rejected by libwebrtc, retry with the original answer and send that instead of leaving the subscriber without a local description (mirrors PeerConnectionTransport.setMungedSdp) - add KDoc to LocalParticipant.AudioTrackPublishOptions.stereo
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When a remote participant publishes a stereo audio track, Android subscribers receive it as mono. libwebrtc's native Opus decoder downmixes incoming stereo packets to mono unless the local answer negotiates
stereo=1in the fmtp line — even when the server offer correctly announces the track withsprop-stereo=1.Observed on real hardware (Android 10, Huawei): stereo capture works (
setUseStereoInput(true), confirmed via AudioRecord), the track is published withstereo=1in AddTrackRequest, the server relayssprop-stereo=1in the subscriber offer — yet the decoded audio plays back mono untilstereo=1is manually added to the answer.Related issues
setUseStereoInput(true), but that only covers the publisher side. The subscriber side remained broken.extractStereoAndNackAudioFromOffercollectsremoteStereoMidsfrom the offer, andensureAudioNackAndStereorewrites the answer's fmtp lines.Changes
Receive path (the main fix):
StereoSdpMunging.kt: extracts mids carryingsprop-stereo=1from the server offer and addsstereo=1to the matching fmtp lines of the answer. Invoked inRTCEngine.onServerOfferbeforesetLocalDescription/sendAnswer. Fully fail-safe on parse failures.Publish path (API parity with client-sdk-js):
LocalAudioTrackOptions.stereo— capture-side flag, surfaced as theTF_STEREOtrack featureAudioTrackPublishOptions.stereo— sent asAddTrackRequest.stereo, mirroring JS'sforceStereooptionTesting
livekit-android-test, using SDP fixtures captured from a real device (the subscriber offer from a live LiveKit session)./gradlew test,spotlessCheck,detektReleasepassAssisted by AI; verified end-to-end on real hardware.