feat(elevenlabs): stream eleven_v3/eleven_v3_conversational via text-to-dialogue by ameyakhare · Pull Request #6838 · livekit/agents

ameyakhare · 2026-08-13T03:47:28Z

Summary

Adds streaming support for eleven_v3 / eleven_v3_conversational by routing them through ElevenLabs' text-to-dialogue API — the regular text-to-speech streaming websocket (multi-stream-input) rejects these models outright today, so SynthesizeStream currently fails for them.

ChunkedStream now posts to /v1/text-to-dialogue/stream for these models.
New _DialogueConnection mirrors the existing _Connection (same shared-connection, multi-context design for barge-in) but speaks text-to-dialogue's multi-stream-input protocol instead.
Added eleven_v3_conversational to TTSModels (eleven_v3 was already listed but non-functional for streaming).

Quickstart guide here
Websocket documentation here

Testing

Lint/format/mypy all pass
16 new unit tests (26/26 total). Verified against the live API for both models — ChunkedStream and SynthesizeStream, including concurrent streams to simulate barge-in.

CLAassistant · 2026-08-13T03:47:37Z

All committers have signed the CLA.

…to-dialogue

…ellation in aclose

…e doesn't block

…upted

…_DialogueConnection

NikoLandgraf · 2026-08-20T10:20:40Z

Keen on getting this reviewed & merged!

…s-v3-dialogue-support Carries the new inactive-context handling from livekit#6936 into _DialogueConnection._recv_loop (snake_case is_final, cleanup via _cleanup_context so the turn-drain bookkeeping is released too) and tags the dialogue loop's content-bearing log extras with lk.pii.data (livekit#6356).

… of waiting for turn drain The text-to-dialogue server now drains queued frames before close_context finalizes (previously it could discard frames queued behind the close, dropping the last turn's audio), so the client-side flush/turn-boundary bookkeeping and deferred close tasks are no longer needed.

… instead of exc_info

chenghao-mou

Thanks for the PR! Left some comments.

- read the documented alignment field instead of normalized_alignment (reserved/unused on text-to-dialogue) - send only the supported voice_settings field (stability) on the dialogue init packet and HTTP settings body, and warn about the dropped fields - send a per-context keep_alive when the outgoing queue is idle so open contexts survive the server's 20s inactivity finalize; keep-alives stop once close_context is sent for a context

The server's 20s inactivity timer is per context, so an idle context must get keep-alives even while other contexts on the connection have traffic. Track the last send time per context and emit due keep-alives from the send loop on both queue timeout and after each message.

ameyakhare · 2026-08-25T17:40:14Z

Thanks for the first pass! Addressed comments

chenghao-mou

lgtm, left one more nit

…ue synthesize body

tinalenguyen · 2026-08-26T03:35:42Z

@ameyakhare thank you for the pr!!

feat(elevenlabs): stream eleven_v3/eleven_v3_conversational via text-…

fa13eaa

…to-dialogue

ameyakhare force-pushed the ameya/elevenlabs-v3-dialogue-support branch from 6d0b6e6 to fa13eaa Compare August 13, 2026 03:58

ameyakhare marked this pull request as ready for review August 13, 2026 04:02

ameyakhare requested a review from a team as a code owner August 13, 2026 04:02