BreezeTTS sub-chunk streaming (draft PR ready; stacked on sm70 fallback #423)

[Th-Underscore]

What

BreezeTTS sub-chunk streaming — multiple speech.audio.delta per text chunk (intra-chunk), via a resumable AR stepper + prefix-decode lookahead margin.

Full implementation, validation, and artifacts: Th-Underscore#1 (draft).

Blocked by

Stacked on the sm70 attention fallback — #423. Must land first.

• Breeze on main hardcodes the flash path, no eager fallback: generator.cpp:104-105,151-152 (FlashGroupedViewKV, prefill+static), speech_decoder.cpp:720, speech_encoder.cpp:209 (Lowering::Flash). sm70 (Volta/Turing) → MMA kernel absent → every decode crashes. The streaming work is built and validated on that fallback.
• main-based rebase strips the fallback from the same hot loop → untestable on sm70, which is the only hardware available to the author.

Cherry-pick of the two streaming commits onto origin/main: 2 conflicts, both sm70-context leakage (attention spec entry / allow_flash_attention_ ctor arg) — trivially resolvable, no real entanglement. Just not runnable on sm70.

After #423 merges

Th-Underscore/audio.cpp#1 rebases onto dev (drop the now-upstream attention hunk from its diff) → retarget main/dev, or re-open.

[0xShug0]

@Th-Underscore Please update the PR decription of #423 to explain your changes, and @ me when the PR is ready for review. Otherwise, I won’t know whether the PR is actually ready for review or if you’re still working on it...

Also, in your case, I don’t see a strong reason why the streaming improvement needs to be stacked on #423. They look like two independent changes to me. The streaming logic should be backend-agnostic and based on main, while #423 only affects whether the code can run on your backend.