feat: Common audio recorder for integrations

[David Elner (delner)]

Adds @braintrust/audio, an optional recording library for voice integrations.

• Records audio with configurable segmentation for progressively flushing long-duration audio.
• Compressed Ogg/Opus by default, with optional WAV output.
• Tracks playback selections and recording states (to signal audio completeness).

It has no runtime dependency on braintrust or a voice framework.

Setting up

npm install @braintrust/audio

Integrations use createRecording() to supply timestamped PCM, span associations, and an attachment uploader. Installing the package does not enable recording; capture is controlled by the integration.

import { Attachment } from "braintrust";
import { createRecording } from "@braintrust/audio";

// The integration supplies the session span, timing, and turn selections.
const recording = createRecording({
  // Optional configuration, shown with defaults.
  audioFormat: "ogg", // Use "wav" for uncompressed audio.
  options: {
    segmentDurationSeconds: 60,
    maxDurationSeconds: 1800,
    maxBufferBytes: 33554432,
    flushFraction: 0.5,
  },
  span: sessionSpan,
  flush: () => logger.flush(),
  createAttachment: (options) => new Attachment(options),
  snapshot: () => ({
    origin: sessionStartUnixMs,
    basis: "local_capture",
    closed: sessionClosed,
    source: (channel_index) => ({ channel_index }),
  }),
  targets: () => turnSelections, // Entries: { span, intervals }.
});

// Call record() for each incoming audio frame to copy its samples into the recording.
recording.recorder.record({
  pcm, // Int16Array; copied by the recorder.
  rate: 24000,
  channels: 1,
  channel: 0, // Caller; channel 1 is the agent.
  at: frameOffsetMs, // Relative to sessionStartUnixMs.
});

// No more audio will arrive before this offset; segments may now be finalized.
recording.recorder.advance(confirmedThroughMs);

// At session end, drain encoding and uploads.
sessionClosed = true;
await recording.recorder.finish();
recording.publishManifest();

Option	Default
audioFormat	"ogg"
options.segmentDurationSeconds	60
options.maxDurationSeconds	1800
options.maxBufferBytes	33554432
options.flushFraction	0.5
turnGrouping.maxPauseMs	1500

Design

PCM → Recorder → AudioSegment → Segment exporter → Trace publisher
                                encode + upload    states + selections

• Recorder: owns capture, limits, segment boundaries, and scheduling.
• Segment exporter: encodes in a worker, uploads ordinary attachments, and releases payloads.
• Trace publisher: publishes recording references and playback selections; logging failures preserve uploaded audio.

Node 20+. Ogg uses bundled WebAssembly; no FFmpeg or native compilation. Audio dependencies stay outside the default braintrust install.

[Abhijeet Prasad (AbhiPrasad)]

the public api here is kinda weird. Why isn't this all just wrapped up in one class that gets exposed via createRecording?

[Abhijeet Prasad (AbhiPrasad)]

this class, and logic like this, is very hard to read and understand IMO.

[Abhijeet Prasad (AbhiPrasad)]

do we really have to serialize here? I don't really understand the manifest concept overall.

[Abhijeet Prasad (AbhiPrasad)]

we should fix this with linting/prettier, but I prefer if we used curly braces for conditionals

Suggested change

	if (this.retained + bytes > this.maximum)
	throw new Error("capture_byte_limit");
	if (this.retained + bytes > this.maximum) {
	throw new Error("capture_byte_limit");
	}

[Abhijeet Prasad (AbhiPrasad)]

why do we have a byte budget here? I think attachments tracks budget under the hood anyway, that should be enough.

[David Elner (delner)]

My original intent was to bound buffers so we didn't overflow memory on long calls. I agree its not necessary for the attachment once the audio is encoded and ready to upload. I'll examine this more carefully.

[Abhijeet Prasad (AbhiPrasad)]

this needs to be a state machine. so we need a singular entry point around state transitions.

Like

`segment.dispatch({ type: "ready", ...options });