feat: LiveKit audio recordings and traces

[David Elner (delner)]

Adds automatic voice tracing for LiveKit Agents on Node, producing the conversation structure and audio references for an improved experience in Braintrust.

• Caller and assistant turns with transcripts, model calls, tool executions, and usage metrics.
• Cascade (STT-TTS) and realtime (voice-to-voice) conversations, including tool continuations and interrupted playback.
• Optional call recordings and turn-level playback selections through the separate @braintrust/audio package.

Setting up

import { initLogger } from "braintrust";

initLogger({ projectName: "voice-agent" });
// Start LiveKit normally.

# Traces and transcripts:
node --import braintrust/hook.mjs agent.mjs

# Optional recordings: install the audio extension and enable capture:
npm install @braintrust/audio
BRAINTRUST_CAPTURE_USER_AUDIO_ATTACHMENTS=true \
BRAINTRUST_CAPTURE_AGENT_AUDIO_ATTACHMENTS=true \
node --import braintrust/hook.mjs agent.mjs

Supports @livekit/agents >=1.9.1 <1.10.0.

Additional configuration

import { configureInstrumentation, initLogger } from "braintrust";

// Requires @braintrust/audio to be installed for either recording format.

configureInstrumentation({
  integrations: {
    livekit: {
      captureAudio: true, // Alternative to the environment flag; does not install the extension.
      audioFormat: "wav", // Optional override; default: compressed Ogg.
      recording: { segmentDurationSeconds: 60 }, // Forwarded to @braintrust/audio.
    },
  },
});
initLogger({ projectName: "voice-agent" });
// Start LiveKit normally.

Option	Default	Purpose
captureAudio	undefined	Enable/disable capture of caller & agent audio. Set true or false. Default disabled.
captureUserAudio, captureAgentAudio	undefined	Override capture of caller or agent audio. Set true or false.
captureContent	true	Include transcripts and tool content, respecting LiveKit’s content policy.
audioFormat	"ogg"	Audio recording format: either ogg or wav.

recording forwards limits to the shared audio recorder. Both formats require the extension; tracing still works without it.

Environment variables

Variable	Default	Effect on LiveKit
BRAINTRUST_CAPTURE_USER_AUDIO_ATTACHMENTS	unset	Enable caller recording with true; defaults to false.
BRAINTRUST_CAPTURE_AGENT_AUDIO_ATTACHMENTS	unset	Enable agent recording with true; defaults to false.

Capture flags accept true or 1 to enable, false or 0 to disable. Options take precedence over environment variables.

What traces look like

livekit.agent_session                 call audio segments
├─ user_turn                         transcript + playback selections
│  └─ livekit.user_speaking
└─ assistant_turn                    transcript + playback selections
   ├─ llm_response                   model, usage, tool request
   ├─ lookup_order                   tool arguments and result
   ├─ llm_response                   answer after tool result
   ├─ tts                            cascade only
   └─ livekit.agent_speaking          confirmed local playback

Design

flowchart LR
  A[LiveKit hooks and native spans] --> B[LiveKit trace adapter]
  B --> C[Braintrust spans]
  B -->|Opt-in PCM and timing| D[Optional @braintrust/audio]
  B -->|Turn and playback associations| D
  D --> C

Loading

• Maps native operations to common voice spans, preserving useful contrib.livekit.* metadata.
• Associates transcripts and confirmed playback with conversation turns, including interruptions.
• Suppresses duplicate provider instrumentation within LiveKit inference work.
• Skips audio capture and extension loading when recording is disabled.

Performance

28.12-second replay, two trials per mode. Baseline is LiveKit without a Braintrust processor; other columns show deltas versus baseline.

Metric	Baseline (actual)	Metadata only Δ	WAV Δ	Ogg/Opus Δ (default)
Capture callback overhead, p99	0.005 ms	+0.000 ms	+0.097 ms	+0.103 ms
CPU time during call	473.41 ms	+1.10 ms	+222.25 ms	+604.52 ms
Sampled RSS growth	5.21 MiB	-0.34 MiB	+14.44 MiB	+30.41 MiB
Local HTTP latency, p99	0.250 ms	-0.010 ms	+0.096 ms	+0.008 ms
Trace updates, JSON	2 B	+6.1 KiB	+30.7 KiB	+30.7 KiB
Audio files	0.0 KiB	0	+2637.7 KiB	+65.4 KiB

Test segments: 5 seconds (rotation stress test); production default: 60 seconds
Latency: capture callback overhead, not caller-heard response time
Upload: simulated; memory logger; no production network/serialization costs