Skip to content

FE-1657: Separate Brunch-authored Voice speech from on-screen responses - #9622

Closed
kostandinang wants to merge 5 commits into
kostandin/fe-1630-improved-voice-relayfrom
voice/separate-brunch-speech
Closed

FE-1657: Separate Brunch-authored Voice speech from on-screen responses#9622
kostandinang wants to merge 5 commits into
kostandin/fe-1630-improved-voice-relayfrom
voice/separate-brunch-speech

Conversation

@kostandinang

@kostandinang kostandinang commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Negative control — do not merge. After the Voice instructions explicitly ordered speech authoring, a restarted Haiku-backed live retry still made zero brunch_set_voice_response calls and wrote no speech data. The result is preserved here because it rules out this prompt-only variant; the PR is closed unmerged.

🌟 What is the purpose of this PR?

Test whether Brunch can author brief spoken content separately from a complete on-screen response while retaining domain authority.

Deterministic tests proved the routing, persistence, completion gate, and replay design. Live diagnosis did not prove model adherence: the speech tool and Voice overlay were present, but Haiku omitted the tool before and after the explicit-order wording experiment.

🔗 Related links

🚫 Blocked by

  • The negative live outcome is recorded.
  • The bounded wording experiment and its tests are preserved in separate commits.
  • No merge: this variant did not produce substantive speech.

🔍 What does this change?

  • Adds a Brunch speech-authoring tool and response-associated speech data.
  • Waits for the whole correlated reply before playback.
  • Keeps complete canonical prose on screen and Realtime delivery-only.
  • Explicitly orders evidence gathering → speech authoring → optional question marking → visible delivery.
  • Falls back to a fixed reading notice when no valid authored speech exists.
🏗️ Agent notes

The implementation uses existing structured data and conversation history; it adds no store or Realtime domain authority. A later substantive tool invalidates earlier speech unless Brunch replaces it.

Live store inspection confirmed Voice context and speech-tool registration, but zero speech-tool calls and zero speech data. This locates the observed failure at model speech authoring, not transport recovery or UI selection. It does not establish general adherence rates.

The app-only wording experiment was committed after a separate authority commit. Its negative result is recorded separately. No additional prompt iteration, repair mechanism, or provider campaign is authorized by this PR.

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

  • The branch contains the previously prepared Petrinaut changeset.

📜 Does this require a change to the docs?

  • Internal implementation and verification documentation is included.

🕸️ Does this require a change to the Turbo Graph?

  • No execution-graph change is required.

⚠️ Known issues

  • The live model did not author speech, so playback used the fixed notice.
  • Whole-reply gating tests content separation, not best-possible relay latency.
  • Acoustic quality, human acceptance, deployed behavior, and broad model adherence were not established.

🐾 Next steps

Continue the matched short/long/tool/queue/interruption evaluation in #9638. Do not add another prompt tweak or a repair mechanism until that comparison is complete.

🛡 What tests cover this?

  • Brunch app: 28 files / 206 tests passed.
  • App build and TypeScript checks passed.
  • Lint passed with 14 existing warnings.
  • Changed TypeScript formatting and git diff --check passed.

❓ How to test this?

The exact deterministic commands, live submission IDs, limits, and retry observations are in the verification record. No further provider run is required for this closed negative control.

📹 Demo

No successful audible demonstration: the live retry produced no authored speech.

ampagent and others added 2 commits September 9, 2026 10:54
Co-authored-by: Kostandin Angjellari <ka@hash.ai>
Co-authored-by: Kostandin Angjellari <ka@hash.ai>
@kostandinang
kostandinang added this pull request to stack #9623 September 9, 2026 11:27
@vercel

vercel Bot commented Sep 9, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
petrinaut Ready Ready Preview Sep 10, 2026 7:27pm UTC
3 Skipped Deployments
Project Deployment Actions Updated
hash Ignored Ignored Preview Sep 10, 2026 7:27pm UTC
hashdotdesign-tokens Ignored Ignored Preview Sep 10, 2026 7:27pm UTC
petrinaut-docs Ignored Ignored Preview Sep 10, 2026 7:27pm UTC

Request Review

@github-actions github-actions Bot added area/infra Relates to version control, CI, CD or IaC (area) area/libs Relates to first-party libraries/crates/packages (area) type/eng > frontend Owned by the @frontend team area/tests New or updated tests area/apps labels Sep 9, 2026
@CLAassistant

CLAassistant commented Sep 9, 2026

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ kostandinang
❌ ampagent
You have signed the CLA already but the status is still pending? Let us recheck it.

@kostandinang kostandinang changed the title voice/separate brunch speech Separate Brunch-authored Voice speech from on-screen responses Sep 9, 2026
@kostandinang kostandinang self-assigned this Sep 9, 2026
@kostandinang kostandinang changed the title Separate Brunch-authored Voice speech from on-screen responses FE-1657: Separate Brunch-authored Voice speech from on-screen responses Sep 10, 2026
kostandinang and others added 3 commits September 10, 2026 21:20
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@kostandinang

Copy link
Copy Markdown
Contributor Author

Closing unmerged: after the Voice overlay explicitly ordered speech authoring, the restarted Haiku-backed retry still made zero speech-tool calls and wrote no speech data. The branch and evidence are retained as a negative control; the next matched comparison is #9638.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/apps area/infra Relates to version control, CI, CD or IaC (area) area/libs Relates to first-party libraries/crates/packages (area) area/tests New or updated tests type/eng > frontend Owned by the @frontend team

Development

Successfully merging this pull request may close these issues.

3 participants