Chatterbox voice-cloning: combining diacritics silently dropped on standard (NFC) input

[LukaszStormnet]

Description

When generating speech with the chatterbox family (--task clon) using text containing precomposed diacritic characters — as virtually all typed, copy-pasted, or web-sourced text is encoded (Unicode NFC) — the diacritics are silently dropped, with no error or warning. Polish ą is pronounced as plain a, ż as plain z, and the same is very likely true for other diacritic letters in Chatterbox's other supported languages (the family's documented language list includes da, de, el, es, fi, fr, hi, it, ko, ms, nl, no, pl, pt, sv, sw, tr — most of which use combining diacritics).

This is not a pronunciation-accuracy issue with the underlying model — it's a text-handling bug specific to this C++ integration. The same reference voice and same underlying Chatterbox weights produce correct diacritics when the input text is pre-converted to Unicode NFD (decomposed) form before being sent to audiocpp_cli/audiocpp_server.

Steps to Reproduce

1. Build with Vulkan (bug is not backend-specific, but this is what was tested):

   scripts/build_linux.sh --backend vulkan --model-set custom --models chatterbox --target audiocpp_cli
   

2. Install the Chatterbox GGUF package (chatterbox_q8_0 or chatterbox_f16 — bug reproduces on both).
3. Run with standard NFC-encoded Polish text (this is how the text appears if typed directly, copy-pasted from any normal source, or received from any typical client):

   build/linux-vulkan-release/bin/audiocpp_cli \
     --task clon --family chatterbox \
     --model models/Chatterbox-GGUF/chatterbox-q8_0.gguf \
     --backend vulkan --device 0 --language pl \
     --text "Nie dlatego, że tak trzeba. Dlatego, że Hanka nie mogła zasnąć, kiedy było cicho. A nocą, przy brzęku bębna i zapachu proszku, cicho było najgorsze." \
     --voice-ref assets/resources/sample.wav \
     --out out_nfc.wav
   

   Result: ą is pronounced as a, ż as z. The generated speech is comprehensible but every diacritic letter is flattened to its plain ASCII base form.
4. Now run the identical text, but normalized to Unicode NFD first:

   import unicodedata
   text = "Nie dlatego, że tak trzeba. Dlatego, że Hanka nie mogła zasnąć, kiedy było cicho. A nocą, przy brzęku bębna i zapachu proszku, cicho było najgorsze."
   nfd_text = unicodedata.normalize("NFD", text)

   Feed nfd_text (via --batch-text-file to avoid shell re-normalization) through the identical command. Result: all diacritics are pronounced correctly.

Byte length of the NFC input vs. NFD input differs (NFD is longer, since every accented letter becomes two codepoints: base letter + combining mark) — this is an easy way to confirm the two inputs are genuinely different at the byte level, not a shell/encoding artifact.

Root Cause (hypothesis, not confirmed against source)

The reference Chatterbox Multilingual PyTorch pipeline performs text.lower() followed by unicodedata.normalize("NFKD", text) before tokenizing (visible in community ONNX export fixes for this same model family, e.g. Folx/chatterbox-ONNX-polish). NFKD/NFD decomposition splits a precomposed character like ą into a base letter (a) plus a standalone combining mark (U+0328, combining ogonek (tail)).

Based on the observed behavior, it appears audio.cpp's Chatterbox text-processing path does not perform this decomposition step internally, and its tokenizer vocabulary only contains tokens for the decomposed form (base letter + combining mark) rather than the precomposed NFC character. When NFC text is passed in, the tokenizer either drops the unrecognized precomposed codepoint or silently falls back to something that produces the bare base-letter sound — either way, the combining-mark information never reaches the model.

I have not inspected the tokenizer source directly, so this is offered as a working hypothesis backed by consistent, reproducible black-box behavior (A/B tested against the same text, same voice reference, same seed range), not a confirmed code-level diagnosis.

Suggested Fix

Either:

1. Perform NFKD/NFD normalization internally, matching the reference PyTorch pipeline, before tokenizing — this would fix the bug for all callers without requiring any client-side workaround, and would match upstream model behavior exactly.
2. If (1) is intentionally avoided for some reason (e.g. to keep the C++ text path minimal), at minimum document that callers must pre-normalize non-English/diacritic text to NFD before calling the Chatterbox family — currently this isn't mentioned anywhere in docs/tts.md or the CLI --help output for --language, so callers have no way to discover this without independently diagnosing it (as was done here).

Current Workaround

Confirmed effective: normalize all non-English input text to NFD (unicodedata.normalize("NFD", text) in Python, or equivalent) before passing it to audiocpp_cli/audiocpp_server. English/ASCII-only text is unaffected either way (NFD is a no-op on plain ASCII).

Environment

• audio.cpp built from main, commit a8fccb478fa1fda648c75f1a50dcff2d13744bd4
• Backend: Vulkan (RADV), AMD RX 9070 XT (RDNA4, gfx1201)
• Model: chatterbox_q8_0 GGUF, installed via tools/model_manager_v2.py
• OS: Ubuntu 26.04

Generated output samples atached.

nfc_line_1.wav

nfd_line_1.wav

[0xShug0]

@LukaszStormnet Please test this PR and see if it solves your issue: #468
Chatterbox originally used unicode_cpts_normalize_nfd from the llama tokenizer path. The PR switches Chatterbox multilingual preprocessing to the framework normalize_nfkd_codepoints helper.

P.S: I only used ASR to validate the results, so it would be better to have you confirm them as well.