fix: upgrade OpenAI Realtime API to GA and resolve audio feedback loop#21
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Title: fix: Upgrade OpenAI Realtime API to GA and resolve audio feedback loop
Description
This PR implements critical hotfixes to stabilize the voice streaming infrastructure. It migrates the backend from the deprecated OpenAI Beta API to the General Availability (GA) release and resolves acoustic feedback loops on the client side that were causing infinite VAD interruption cycles.
Changelog
OpenAI-Betaheader. Updated thesession.updatepayload to match the GA schema, explicitly defininginputandoutputaudio formats.response.output_audio.deltaandresponse.output_transcript.done).echoCancellation,noiseSuppression,autoGainControl) intonavigator.mediaDevices.getUserMediato suppress acoustic feedback and prevent false-positive VAD triggers./apiand/wstraffic through the Nginx reverse proxy (8080) instead of the isolated FastAPI container port (8000).frontend/node_modules/from source control.Acceptance Criteria Checklist
[x] OpenAI WebSocket connects using the
gpt-realtime-1.5model without 3000/4000 closure codes.[x] Client microphone input is filtered for echo, preventing infinite
response_cancelledloops.[x] Frontend Vite proxy successfully forwards WebSocket upgrade requests to the backend.