feat(tts): add aivis_tts_python (Aivis Cloud, Japanese realtime HTTP streaming) - #2280
Open
Therealratoshen wants to merge 4 commits into
Open
feat(tts): add aivis_tts_python (Aivis Cloud, Japanese realtime HTTP streaming)#2280Therealratoshen wants to merge 4 commits into
Therealratoshen wants to merge 4 commits into
Conversation
- Add aivis_tts_python under ten_packages/extension/ (AsyncTTS2HttpExtension subclass that streams Aivis Cloud TTS WAV and yields PCM to TEN). - Append voice_assistant_aivis graph (Deepgram ja STT + OpenAI LLM + Aivis TTS + main_control + message_collector). - Document AIVIS_API_KEY / AIVIS_MODEL_UUID / AIVIS_BASE_URL in .env.example alongside the other TTS providers.
Aivis Cloud API supports `mono` and `stereo` for `output_audio_channels`
and `config.AivisTTSConfig.update_params` already defaults / forwards
this key in the request body. The manifest's JSON schema didn't reflect
it, so a TEN runtime property validator would silently accept the key
without documenting it. Add the string enum so the contract matches the
behaviour.
Verified `AsyncTTS2HttpExtension` is the right base class for Aivis (vs
`AsyncTTS2BaseExtension` used by Gradium / Tencent / Rime): Aivis's API
is a single-shot `POST /v1/tts/synthesize` returning a streamed WAV
blob, which is exactly the AsyncTTS2HttpClient.get() iterator shape
that Groq (also HTTP) implements. The HTTP class name is the historical
one; the underlying model is "single request yields streamed chunks".
No code change needed for the base class choice.
Also verified that the `@patch("aivis_tts_python.extension.AivisTTSClient")`
target used in `tests/test_basic.py` resolves cleanly via the existing
`from .aivis_tts import AivisTTSClient` re-binding in `extension.py`,
so no extension.py change is required for the mocked-extension tests.
- docs/IMPLEMENTATION_NOTES.md — short reviewer-facing rationale for the non-obvious decisions: base-class choice (HTTP vs Base), custom WavStreamParser + first-chunk emission for TTFB, forced output_format, get_extra_metadata, mock-target resolution. - manifest.json: package.include now globs **.md so the new doc ships with the built package.
Therealratoshen
force-pushed
the
feat/aivis-tts
branch
from
August 29, 2026 06:21
7799ba2 to
7505740
Compare
- aivis_tts.py: replace the private-attribute reach-in for the first PCM chunk with `await parser.__anext__()`. The previous code yielded the parser's `_first_pcm_chunk` once, then iterated the parser which yielded the same chunk again — the first audio frame was delivered twice. - aivis_tts.py: drop the unreachable `except HTTPStatusError` branch (`httpx.AsyncClient.stream()` only raises it when raise_for_status() is called, which we don't do — status codes are checked inline instead). - aivis_tts.py: replace `try/finally: pass` around `client.aclose()` with an `except` that logs a warning, matching Groq's intent. - extension.py: guard `synthesize_audio_sample_rate()` against the case where the base class calls it before `create_config()` populates `self.config`. - manifest.json: extend `params.properties` schema to declare every key `AivisTTSConfig.update_params` defaults/forwards, including the previously missing `output_format`, `use_ssml`, `leading_silence_seconds`, `trailing_silence_seconds` (see IMPLEMENTATION_NOTES.md for the rationale). - README.md: remove the non-existent `style_name` parameter (the Aivis API only supports `style_id`), document `leading_silence_seconds` and `trailing_silence_seconds` in the parameter table, and rephrase the Supertone note in past tense (the API shut down on 2026-08-31). - tests/test_state_machine.py: assign `mock_instance.get` via `side_effect` so the base class actually receives an async iterator (previously the coroutine function was assigned directly, so the test silently never exercised the streamed path). - tests/test_state_machine.py: drop the unused `MagicMock` import, the `__main__` boilerplate, the magic-number `1` for the audio-end reason (now accepts the string or integer forms the base class may emit), and standardize `get_property_to_json(None)` to match the rest of the suite. - tests/test_params.py: also mock `cancel()` so the client construction contract is complete even if the test were extended to send tts text. - tests/test_robustness.py: drop the unreachable `elif name == "tts_audio_end"` block that duplicated the earlier `if` branch. - tests/test_robustness.py: add `test_empty_text_does_not_call_vendor` and `test_whitespace_text_does_not_call_vendor`, asserting that the short-circuit in `aivis_tts.py:64-70` actually fires (matches Gradium's equivalent coverage).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a new TTS extension
aivis_tts_pythonbacked by Aivis Cloud API and wires it into the voice-assistant example asvoice_assistant_aivis.Aivis is a Japanese-focused realtime TTS provider with public HTTP API and WAV streaming — fills the regional TTS gap (Supertone sunsets 2026-08-31, signup already closed).
Changes
ai_agents/agents/ten_packages/extension/aivis_tts_python/— new extension modeled on the existing Gradium/Groq TTS extensions:AsyncTTS2HttpExtensionsubclass streamingPOST /v1/tts/synthesizeviahttpx.AsyncClient.WavStreamParserstrips the WAV header so the first PCM chunk reaches TEN immediately, giving the base class a correct TTFB measurement.INVALID_KEY_ERROR; cancellation →FLUSH; other HTTP errors →ERROR.output_formatis forced towavbecause TEN needs raw PCM.docs/IMPLEMENTATION_NOTES.md— short reviewer-facing rationale for the non-obvious decisions (class choice, TTFB handling, manifest schema).ai_agents/agents/examples/voice-assistant/tenapp/property.json— appendsvoice_assistant_aivis(Deepgram ja STT + OpenAI LLM + Aivis TTS + main_control + message_collector).ai_agents/.env.example— documentsAIVIS_API_KEY,AIVIS_MODEL_UUID,AIVIS_BASE_URLalongside the other TTS providers.Test plan
tests/test_config.py— 4 unit tests (validation, default param normalization, client-only key stripping, URL construction) pass without TEN runtime.tests/bin/startend-to-end in a TEN checkout where the runtime has been built (task install && task buildinai_agents/) — this requires Docker on macOS and so could not be done in this PR's environment. Same prerequisite as every other TTS extension's tests (verified against Gradium).AIVIS_API_KEY:python -c "from aivis_tts_python.aivis_tts import AivisTTSClient; ..."— happy to add the smoke transcript as a comment once an AIVIS key is available.Notes
AIVIS_MODEL_UUIDpoints at Aivis's public default model (a59cb814-0083-4369-8542-f51a29e72af7).claude-reviewCI check failing on this PR is unrelated to the diff — it fails on every external TEN PR because the bot lacks fork write permissions.Made with Cursor