Skip to content

FE-1630: Optimize and measure the Brunch Voice relay - #9585

Draft
kostandinang wants to merge 57 commits into
mainfrom
kostandin/fe-1630-improved-voice-relay
Draft

FE-1630: Optimize and measure the Brunch Voice relay#9585
kostandinang wants to merge 57 commits into
mainfrom
kostandin/fe-1630-improved-voice-relay

Conversation

@kostandinang

@kostandinang kostandinang commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

⏸️ Paused: This experiment and its active Voice follow-up are paused while FE-1661 evaluates the GPT-Live-1 migration. We'll reassess them after that spike.

Baseline outcome: the relay preserved complete canonical responses, but did not make short Voice follow-ups natural enough. This PR remains the shared baseline for the follow-up experiments; it does not approve a new ownership architecture.

🌟 What is the purpose of this PR?

Determine whether Voice-specific Brunch prompting and bounded Realtime delivery make the existing relay sufficiently natural before reconsidering split ownership.

The experiment produced a negative naturalness result: the recorded short clarification remained 151 words and reached only a delivery notice 7.4 seconds after transcription. Long reports stayed complete on screen and were read only on request.

This includes an explicitly approved, maintained local Flue 2.0.3 delivery-context patch. It is not an upstream-supported API. Local before/after evidence and an audible demonstration are included; deployed verification and human acceptance remain outstanding.

🔗 Related links

🚫 Blocked by

  • FE-1580: Reconcile Voice turn behavior on the shared Brunch conversation #9564 merged and the baseline was reconciled with current main.
  • Current main was merged again to resolve PR conflicts while preserving both newer interruption tests and this experiment's delivery tests.
  • Owner review of the negative result and demonstration.
  • Full preview verification of the patched Brunch backend; local evidence is not deployment evidence.

🔍 What does this change?

  • Carries optional per-delivery Voice context through the pinned Flue runtime/SDK using existing submission and recovery records.
  • Adds Voice-only Brunch instructions while keeping typed prompts unchanged.
  • Keeps Realtime tool-free and non-autonomous.
  • Preserves full canonical reports on screen, using only a fixed bounded offer for long-response playback.
  • Retains admission correlation, interruption versus durable Stop, explicit full-response reading, and reload without autoplay.
🏗️ Agent notes

Throughline: Voice input → shared AI SDK transport → one Flue admission with context.responseMode: "voice" → Brunch canonical response → bounded Realtime delivery.

Observed result: the same-input clarification changed from 192 to 151 words but remained too long and repetitive. The long report remained complete and was read only after explicit selection. The fixed offers succeeded after increasing the provider-dependent cap from 128 to 256 tokens.

Authority boundary: Brunch retains domain meaning, questions, conclusions, workpiece state, and tools. Realtime may deliver only application-selected text or non-substantive event-backed notices. No delegation, client-tool handback, new store, or workpiece redesign is introduced.

Evidence limits: deterministic tests establish context isolation, recovery, idempotency, and delivery lifecycle. They do not establish naturalness, first-audible latency, domain correctness, or deployment behavior. The local Flue patch remains a maintained exception with a documented removal path.

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

  • No publishable package release is required.

📜 Does this require a change to the docs?

  • Internal experiment, patch, and verification documentation is included.

🕸️ Does this require a change to the Turbo Graph?

  • No execution-graph change is required.

⚠️ Known issues

  • The short clarification still fails the naturalness goal.
  • Waiting for Brunch preserves authority but retains round-trip delay; the long fixed offer arrived 34.7 seconds after transcription.
  • Preview Voice configuration returned HTTP 500 on both this branch and its unchanged parent path; remote patched-backend behavior is unverified.
  • One browser-tool continuation stall was not reproduced and is not claimed fixed.
  • Human conversational acceptance, uninterrupted report audio, and domain correctness were not established.

🐾 Next steps

Use this PR as the common baseline. Preserve #9622 as the negative Brunch-authored-speech control and continue the bounded Realtime rephrasing comparison in #9638. Judge usefulness, fidelity, queueing, interruption, and actual audible timing separately before selecting an architecture.

🛡 What tests cover this?

  • Brunch runtime and architecture: 33 tests.
  • AI SDK transport: 49 tests.
  • Website Voice and policy: 176 tests.
  • Affected builds and TypeScript checks passed; lint had no errors.
  • Changed-file formatting and git diff --check passed.

The current conflict-resolution merge preserves main's replacement of obsolete architecture tests and combines both sides of the bridge-test coverage. CI is the verification oracle for the merged current-main result because local dependency linking is presently blocked by a full disk.

❓ How to test this?

Run the targeted commands recorded in the evidence file. For the live comparison, open /?brunch-fixture=crew-reservation-v1, ask the two recorded inputs, inspect complete canonical text, explicitly read the long response, test interruption and Stop separately, then reload and confirm no autoplay or duplicate admission.

📹 Demo

demo-2026-09-08.mp4

lunelson and others added 30 commits September 8, 2026 15:55
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
…nsport

Prepare one honestly labelled crew-reservation fixture whose starting
workpiece enters canonical Flue history exactly once as a tagged system
dispatch signal with idempotent retry, scope the read/`addArc` client tools
to that fixture, settle a runtime manifest only after conversation,
workpiece, and document state are observable, and surface a visible refusal
when the mounted Brunch route is absent.

The fixture rides Mission 5's landed browser `ChatTransport`: the panel
transport keeps the conversation tracker and accepts fixture-scoped client
tool names as an option, the history hook keeps the live observation and
additionally exposes the observed snapshot with its offset, and AI SDK user
sends carry a deterministic idempotency key.

The real two-tab witness remains blocked on the configured Anthropic
credential; the blocker and handoff are recorded in the FE-1575 evidence
note. No persona testing was run.
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Add dispositions G1 to G22 to the decision log, revise the mini spec to its
third state with the settled-revision protocol, typed basis, verifiable
transition record, document reconciliation, readiness ownership, and probe
decision tables, and retain the follow-up review as design evidence.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ared basis

Rewrite the Mission 7 draft at cut-level detail as the consolidated
construct-and-explain mission with a two-step authority, probe decision
tables, and a conversion map; recut Mission 9 as repeatable projection
breadth over the Mission 7 seam; re-express Missions 10 and 11 over settled
revisions, declared basis, and transition records with three-gate readiness
for the handoff; and record the provenance relations and tool admission
locks plus a lossless planning-content migration matrix in MISSION.next.md.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Point the Deferred section at the 2026-09-04 provenance replanning, carry the
fixture-authorship and fenced-block admissions into the close record, and note
in Status that no other section of the live authority changed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Record dispositions H0 to H12 in the decision log, fix revision identity to
the tool call id, make the transition record and elicited evidence relation
precise, replace every oracle gap in the Step A leaves with an exact
prospective test, artifact, or adjudication, classify probe outcomes as
eligible, rework, or terminal stop under the owner's no-narrowing rule, give
the two-step authority a lawful document shape, and add the pre-cut owner
checklist and the ask/sweep keep-remove-archive inventory.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…revision numbering

The index route now owns the brunch-fixture search key beside the shared
example contract, so a selection write no longer drops the prepared
fixture mid-session. The fixture selector and fixture URL fall back to
the ordinary demo while Brunch is unconfigured, the fixture's browser
tool catalog keeps the always-mounted docs reader answerable, workpiece
revisions are numbered by the history rather than by manifest count, and
an unchanged-document mutation result no longer claims the requested
state already existed.
kostandinang and others added 2 commits September 9, 2026 10:02
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@codecov

codecov Bot commented Sep 9, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 65.90%. Comparing base (fb728e4) to head (038e747).
⚠️ Report is 6 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #9585   +/-   ##
=======================================
  Coverage   65.90%   65.90%           
=======================================
  Files        1887     1887           
  Lines      198443   198443           
  Branches     8248     8248           
=======================================
  Hits       130774   130774           
  Misses      66139    66139           
  Partials     1530     1530           
Flag Coverage Δ
rust.hash-graph-atlas 80.36% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit fd47c96. Configure here.

…roved-voice-relay

Co-authored-by: Cursor <cursoragent@cursor.com>

# Conflicts:
#	apps/brunch-agent/test/architecture/boundaries.integration.ts
#	apps/petrinaut-website/src/main/app/voice-interview/realtime-brunch-bridge.test.ts
Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/apps area/deps Relates to third-party dependencies (area) area/infra Relates to version control, CI, CD or IaC (area) area/libs Relates to first-party libraries/crates/packages (area) area/tests New or updated tests type/eng > frontend Owned by the @frontend team

Development

Successfully merging this pull request may close these issues.

2 participants