Skip to content

feat(maya): add Maya Research TTS plugin - #6899

Open
saicherry93479 wants to merge 3 commits into
livekit:mainfrom
MayaResearch:maya-tts
Open

feat(maya): add Maya Research TTS plugin#6899
saicherry93479 wants to merge 3 commits into
livekit:mainfrom
MayaResearch:maya-tts

Conversation

@saicherry93479

@saicherry93479 saicherry93479 commented Aug 19, 2026

Copy link
Copy Markdown

Summary

  • Adds livekit-plugins-maya (livekit-plugins/livekit-plugins-maya/), a WebSocket TTS provider covering ten Indian languages plus Indian English, with every voice speaking all eleven. Connections come from utils.ConnectionPool, so one socket carries a whole conversation and the handshake is paid once rather than per utterance.
  • Maya's v2 protocol needs a handshake per connection, so _connect_ws sends {"type": "start", "v2": true, ...} and waits for the metadata reply before returning the socket — turns sent before that are rejected rather than served in the older frame shape, so the pool must never hand out a socket that hasn't completed it. A rejected handshake raises APIError instead of yielding one that would fail every turn.
  • Each agent turn maps to one Maya context: SynthesizeStream mints a context_id, sends each tokenized sentence under it with continue: true, and closes with an empty continue: false frame, yielding exactly one end. Sentences are not gated on the previous one's audio, so a multi-sentence reply streams continuously.
  • A turn that ends without its terminator — the caller interrupted, or the task was cancelled — is dropped with a targeted cancel before the socket returns to the pool. Without it the server keeps generating and the next borrower of that connection receives audio from a dead turn.
  • Voice, language and model are passed through as given rather than checked against a fixed list; Maya validates them and answers with an error frame, so values it adds later work without a release here.
  • ChunkedStream runs over the same websocket rather than Maya's HTTP endpoint, so the plugin has one transport and one auth path.
  • Registers the workspace member, the maya optional dependency, and an entry in the cross-provider SYNTHESIZE_TTS suite. version.py tracks the livekit-agents version (1.6.10), matching the sibling plugins so the livekit-plugins-maya>=1.6.10 pin resolves.

Example usage

from livekit.plugins import maya

tts = maya.TTS(voice="Ananya", language="hi")

The API key is read from MAYA_API_KEY.

Test plan

  • uv run pytest tests/test_plugin_maya_tts.py — 25 tests against a local server speaking the v2 protocol, covering the handshake contents, the turn model, connection reuse across turns, cancel-on-interrupt, no cancel for a turn that finished, a rejected handshake, and error frames
  • uv run ruff check && uv run ruff format --check
  • uv run mypy livekit-plugins/livekit-plugins-maya/livekit/plugins/maya/
  • Live against the production endpoint: one-shot synthesis, multi-sentence turns sharing one context, consecutive turns reusing the socket, and repeated barge-in followed by a clean turn with no leftover audio

Happy to adjust anything to match house style.

@saicherry93479
saicherry93479 requested a review from a team as a code owner August 19, 2026 08:18
@CLAassistant

CLAassistant commented Aug 19, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

devin-ai-integration[bot]

This comment was marked as resolved.

Streaming speech for ten Indian languages and Indian English, every voice
speaking all eleven.

One websocket carries a whole conversation, and each sentence is sent as the
LLM writes it, so the agent starts speaking before the reply is finished.
Interrupting it stops generation at the server rather than only muting
playback, which keeps a reused connection from carrying a turn nobody is
listening to into the next one.

Voice, language and model are passed through as given rather than checked
against a fixed list, so values Maya adds later work without a release here.
Leaving the language unset lets Maya detect it per sentence, which is what
code-switched Hinglish needs.
Start the receive timeout when a sentence has actually gone out, rather than
when the turn opens: until then Maya has nothing to answer, so the timeout was
measuring how long the LLM took to write its first sentence and a slow one
failed the whole reply.

Separate sentences with a space. The tokenizer strips its tokens, so the last
word of one sentence reached the model glued to the first word of the next.

Skip the closing frame for a turn that produced no text, on both the streaming
and one-shot paths. A context only exists once it has been given text, so
closing one that was never opened is rejected rather than answered.

Cancel an abandoned one-shot turn and ignore frames belonging to another
context. Pooled connections outlive a turn, so a turn dropped before its
terminator could otherwise leave the server generating and its audio would be
read as part of whatever turn borrowed the connection next.
devin-ai-integration[bot]

This comment was marked as resolved.

The connection pool only takes ownership once the handshake returns, so a
socket abandoned there had nothing left to close it. Only a timeout and the
two validation branches closed it; a dropped connection mid-handshake, a
malformed reply, or cancellation left it open.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants