fix(sarvam): request word timestamps on the REST recognize path - #6935
fix(sarvam): request word timestamps on the REST recognize path#6935devbm7 wants to merge 2 commits into
Conversation
REST recognize() never sent Sarvam's with_timestamps field, so the timestamps block the response parser already handles never came back and start_time/end_time silently fell back to a buffer-duration estimate. Now requested whenever the model isn't on the legacy translate endpoint. Sarvam removed server-side support for input_audio_codec in January 2026 (the API now negotiates it automatically); the plugin now warns when it's passed and no longer sends or acts on it, while keeping the parameter in both signatures for backward compatibility.
|
|
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 944b3b4e42
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| self._audio_encoding = self._opts.input_audio_codec or "audio/wav" | ||
| # input_audio_codec is deprecated and never forwarded (see STT.__init__/stream()); | ||
| # always report the actual wire format we send. | ||
| self._audio_encoding = "audio/wav" |
There was a problem hiding this comment.
When a caller uses streaming with input_audio_codec="pcm_s16le"/pcm_l16/pcm_raw, _process_audio still sends raw int16 frame bytes, but this change now forces every audio message to claim audio/wav and also stops putting the selected codec in the WebSocket URL. That regresses the only path that let raw PCM streams be described correctly; those bytes are not WAV containers, so Sarvam can receive mislabeled/unlabeled audio for callers that previously configured the codec.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
the streaming api has already been deprecated by sarvam but yes we should introduce this once it is shelved
The legacy streaming audio message schema documents `encoding` as a required per-chunk field, and Sarvam's PCM formats (which is what _process_audio always sends — raw int16 samples, never a real WAV container) still require explicit codec declaration even after the January 2026 change that made codec auto-detection the default for self-describing formats (wav, mp3, etc). Deprecating input_audio_codec was based on an incomplete reading of that changelog entry and would have made every stream mislabel its audio as audio/wav regardless of what the caller actually sends.
devbm7
changed the title
2 participants
Summary
_recognize_implnever sent Sarvam'swith_timestampsfield, so thetimestampsblock the response parser already handles never came back from the API —start_time/end_timesilently fell back to a buffer-duration estimate instead of real word timing. Now requested whenever the model isn't on the legacy/speech-to-text-translateendpoint (gated via a new_model_supports_timestampshelper, mirroring the existing_model_supports_mode/_model_supports_vad_paramspattern).(An earlier revision of this PR also deprecated
input_audio_codec, based on an incomplete reading of Sarvam's changelog. That's been reverted: Sarvam's docs confirm PCM formats — which is what this plugin's streaming path always sends — still require explicit codec declaration even after codec auto-detection became the default for self-describing formats like wav/mp3. Soinput_audio_codecis left untouched.)Test plan
ruff format --check/ruff checkcleanmypystrict cleanuv run pytest livekit-plugins/livekit-plugins-sarvam/tests/— newtest_timestamps_model_gating.pypasses; existingtest_speech_timing.pyunaffected. (Note:test_language_probability.pyhas 16 pre-existing failures onmainunrelated to this change — anAttributeErroron_pending_eosin theSpeechStream.__new__test-bypass pattern — not touched by this PR.)