From b2fe85f0ad68f921a8955b150909296a8c8f2996 Mon Sep 17 00:00:00 2001 From: tomaszs Date: Fri, 14 Aug 2026 10:01:51 +0100 Subject: [PATCH 1/2] Document session parking for the Realtime API Session parking lets a Realtime session release its compute during long stretches of near-silent audio, and resume automatically or via new PauseRecognition/ResumeRecognition messages, without ending the session or losing the connection. - spec/realtime.yaml: add PauseRecognition/ResumeRecognition and their RecognitionPaused/RecognitionResumed acks, plus the new Warning (not_allowed), Error (park_resume_failed) and Info (session_parked, session_resumed) values these introduce. - New session-parking.mdx page covering automatic parking, manual pause/resume, the preserve_timeline option, and session limits. Add the page to the Realtime sidebar. --- .../realtime/session-parking.mdx | 73 +++++++++++++++++++ docs/speech-to-text/realtime/sidebar.ts | 4 + spec/realtime.yaml | 72 ++++++++++++++++++ 3 files changed, 149 insertions(+) create mode 100644 docs/speech-to-text/realtime/session-parking.mdx diff --git a/docs/speech-to-text/realtime/session-parking.mdx b/docs/speech-to-text/realtime/session-parking.mdx new file mode 100644 index 00000000..6c506d31 --- /dev/null +++ b/docs/speech-to-text/realtime/session-parking.mdx @@ -0,0 +1,73 @@ +--- +description: 'Keep a Realtime session open without holding compute capacity during long stretches of near-silent audio' +keywords: + [ + speechmatics, + realtime, + session parking, + pause, + resume, + transcription, + speech recognition, + asr, + ] +--- + +import SchemaNode from '@theme/Schema'; +import realtimeSchema from "!asyncapi-schema-loader!@site/spec/realtime.yaml"; + +# Session parking + +Some integrations open a Realtime session and send only a trickle of audio — a small chunk every few seconds — for hours, just to keep the connection alive. Session parking frees the compute capacity allocated to a session like this without ending it: the WebSocket stays open and keeps accepting audio, and transcription resumes as soon as real audio arrives again. + +:::note +Session parking must be enabled for your account. Contact your account manager or [support](https://support.speechmatics.com) to turn it on. +::: + +## How it works + +Once enabled, parking is automatic and transparent: Speechmatics detects when a session's audio rate has stayed below real time for a while and parks it, with no message sent to the client. Audio sent while parked is buffered and acknowledged as normal, so client-side flow control keeps working. When audio returns to real time, the session resumes on its own. + +A parked session's transcript timeline (timestamps and speaker labels) either continues across the pause or restarts, depending on `preserve_timeline` — see [Manual parking](#manual-parking) below. Automatically detected parks default to preserving the timeline. + +If a resume fails, the session is closed with an `Error` of type `park_resume_failed`. + +## Manual parking + +Clients can also park and resume a session explicitly with `PauseRecognition` and `ResumeRecognition`. This is the cheapest fix for an integration that only trickles audio: pause instead of holding the connection open at a low rate. + +```json +{"message": "PauseRecognition"} -> {"message": "RecognitionPaused"} +{"message": "PauseRecognition", "preserve_timeline": true} -> {"message": "RecognitionPaused"} +{"message": "ResumeRecognition"} -> {"message": "RecognitionResumed"} +``` + + +
+ +
+ +`RecognitionPaused` is sent once the transcript for any audio already sent has arrived and the session's compute has been released. `RecognitionResumed` is sent once the session is transcribing again. Audio sent right after `ResumeRecognition` doesn't need to wait for it — it's acknowledged in order once the resume completes. + +:::tip +Send audio immediately after `ResumeRecognition`, without waiting for `RecognitionResumed`. Waiting for the acknowledgement adds a full round trip to time-to-first-final — typically 200-500ms, depending on network conditions — for no benefit, since the resumed session accepts and queues that audio correctly either way. +::: + +`preserve_timeline` is optional and applies to that pause only: + +- Omitted, or `false` — the session resumes as a new one. Timestamps restart from zero and speaker labels are not carried over. This is the default for a manual pause. +- `true` — the transcript timeline continues: timestamps and speaker labels carry across the pause, and audio sent while parked is transcribed on resume. + +A session that hasn't sent any audio yet, or that can't preserve its timeline at all, always resumes as a new session regardless of `preserve_timeline`. + +If `PauseRecognition` is sent on a session that can't be parked, the server replies with a `Warning` of type `not_allowed` and the session continues normally — pausing was simply refused, not the whole connection. + +## Limits + +A parked session holds no compute, so it has no idle deadline of its own — the account's usual [session limits](/speech-to-text/realtime/limits#session-limits) (idle timeout, max duration) still apply and can end it while parked. A session parked by its own `PauseRecognition` is exempt from the idle timeout for as long as it stays parked; the max-duration limit still applies. + +Audio buffered while parked is capped. If the cap is reached, the oldest buffered audio is dropped to make room for new audio — this can only happen with `preserve_timeline: true`, since audio isn't buffered for replay otherwise. + +## Deployment + +Session parking is available on SaaS on Cloud only. diff --git a/docs/speech-to-text/realtime/sidebar.ts b/docs/speech-to-text/realtime/sidebar.ts index a791a8eb..1c8ab349 100644 --- a/docs/speech-to-text/realtime/sidebar.ts +++ b/docs/speech-to-text/realtime/sidebar.ts @@ -30,6 +30,10 @@ export default { type: "doc", id: "speech-to-text/realtime/turn-detection", }, + { + type: "doc", + id: "speech-to-text/realtime/session-parking", + }, { type: "category", label: "Guides", diff --git a/spec/realtime.yaml b/spec/realtime.yaml index 53bd0887..865961b1 100644 --- a/spec/realtime.yaml +++ b/spec/realtime.yaml @@ -40,6 +40,10 @@ channels: $ref: "#/components/messages/SetRecognitionConfig" GetSpeakers: $ref: "#/components/messages/GetSpeakers" + PauseRecognition: + $ref: "#/components/messages/PauseRecognition" + ResumeRecognition: + $ref: "#/components/messages/ResumeRecognition" subscribe: address: / messages: @@ -73,6 +77,10 @@ channels: $ref: "#/components/messages/Error" SpeakersResult: $ref: "#/components/messages/SpeakersResult" + RecognitionPaused: + $ref: "#/components/messages/RecognitionPaused" + RecognitionResumed: + $ref: "#/components/messages/RecognitionResumed" operations: publish: action: send @@ -87,6 +95,8 @@ operations: - $ref: "#/channels/publish/messages/ForceEndOfUtterance" - $ref: "#/channels/publish/messages/SetRecognitionConfig" - $ref: "#/channels/publish/messages/GetSpeakers" + - $ref: "#/channels/publish/messages/PauseRecognition" + - $ref: "#/channels/publish/messages/ResumeRecognition" subscribe: action: receive channel: @@ -107,6 +117,8 @@ operations: - $ref: "#/channels/subscribe/messages/Warning" - $ref: "#/channels/subscribe/messages/Error" - $ref: "#/channels/subscribe/messages/SpeakersResult" + - $ref: "#/channels/subscribe/messages/RecognitionPaused" + - $ref: "#/channels/subscribe/messages/RecognitionResumed" components: messages: GetSpeakers: @@ -142,6 +154,16 @@ components: summary: Requests a finalized transcript. payload: $ref: "#/components/schemas/ForceEndOfUtterance" + PauseRecognition: + summary: | + Requests that the session be parked: the transcriber is released while the WebSocket + connection stays open and continues to accept audio. See [Session parking](https://docs.speechmatics.com/speech-to-text/realtime/session-parking). + payload: + $ref: "#/components/schemas/PauseRecognition" + ResumeRecognition: + summary: Requests that a parked session resume transcription. + payload: + $ref: "#/components/schemas/ResumeRecognition" SetRecognitionConfig: summary: | Allows the client to re-configure the recognition session. @@ -224,6 +246,14 @@ components: summary: Additional information sent from the server to the client. payload: $ref: "#/components/schemas/Info" + RecognitionPaused: + summary: Server response to `PauseRecognition`, confirming the session has parked and the transcriber has been released. + payload: + $ref: "#/components/schemas/RecognitionPaused" + RecognitionResumed: + summary: Server response to `ResumeRecognition`, confirming the session is running on a worker again. + payload: + $ref: "#/components/schemas/RecognitionResumed" Warning: summary: Warning messages sent from the server to the client. payload: @@ -400,6 +430,26 @@ components: description: "Timestamp of the audio data that corresponds to the force end of utterance request. It's the number of seconds since the beginning of the audio." required: - message + PauseRecognition: + type: object + properties: + message: + const: PauseRecognition + preserve_timeline: + type: boolean + description: >- + Keep the transcript timeline (timestamps, speaker labels) continuous across this + pause. Optional; without it, the session resumes as a new one when it comes back: + timestamps restart from zero and speaker labels are not carried over. + required: + - message + ResumeRecognition: + type: object + properties: + message: + const: ResumeRecognition + required: + - message SetRecognitionConfig: type: object properties: @@ -546,6 +596,20 @@ components: type: number description: The time (in seconds) that the end of utterance was detected. format: float + RecognitionPaused: + type: object + properties: + message: + const: RecognitionPaused + required: + - message + RecognitionResumed: + type: object + properties: + message: + const: RecognitionResumed + required: + - message AddPartialTranslation: type: object properties: @@ -1254,9 +1318,13 @@ components: | --- | --- | | `recognition_quality` | Informs the client what particular quality-based model is used to handle the recognition. Sent to the client immediately after the WebSocket handshake is completed.| | `concurrent_session_usage` | Informs the client of their quota for concurrent sessions and how much of it they are using. Sent to the client immediately after the WebSocket handshake is completed.| + | `session_parked` | The session has been parked. See [Session parking](https://docs.speechmatics.com/speech-to-text/realtime/session-parking). Sent only if your account is configured to notify on parking events; parking is otherwise silent. | + | `session_resumed` | A parked session has resumed. Sent only if your account is configured to notify on parking events. | enum: - recognition_quality - concurrent_session_usage + - session_parked + - session_resumed WarningTypeEnum: type: string description: | @@ -1271,6 +1339,7 @@ components: | `empty_translation_target_list` | No supported translation target languages specified. Translation will not run. | `add_audio_after_eos` | Protocol specification doesn't allow adding audio after `EndOfStream` has been received. Any `AddAudio messages after this, will be ignored. | `speaker_id` | Informs the client about any speaker ID related issues. | + | `not_allowed` | A `PauseRecognition` was requested on a session that cannot be parked. The session continues normally; this is distinct from the `Error` type of the same name, which closes the connection. | enum: - duration_limit_exceeded - unsupported_translation_pair @@ -1279,6 +1348,7 @@ components: - empty_translation_target_list - add_audio_after_eos - speaker_id + - not_allowed ErrorTypeEnum: type: string # TODO if OpenAPI/AsyncAPI ever adds enum descriptors, we can move this description there @@ -1303,6 +1373,7 @@ components: | `timelimit_exceeded` | Usage quota for the contract has been reached | | `idle_timeout` | Idle duration limit was reached (no audio data sent within the last hour), a closing handshake with code 1008 follows this in-band error. | | `session_timeout` | Max session duration was reached (maximum session duration of 48 hours), a closing handshake with code 1008 follows this in-band error. | + | `park_resume_failed` | A parked session (see [Session parking](https://docs.speechmatics.com/speech-to-text/realtime/session-parking)) failed to resume. The session is closed. | | `unknown_error` | An error that did not fit any of the types above. | :::info @@ -1325,6 +1396,7 @@ components: - timelimit_exceeded - idle_timeout - session_timeout + - park_resume_failed - unknown_error AudioEventStartData: type: object From 11062d830b4b51e864d11007eb8218b944282a4c Mon Sep 17 00:00:00 2001 From: tomaszs Date: Fri, 14 Aug 2026 11:09:34 +0100 Subject: [PATCH 2/2] Various fixes in readability --- .../realtime/session-parking.mdx | 58 ++++++++++--------- 1 file changed, 32 insertions(+), 26 deletions(-) diff --git a/docs/speech-to-text/realtime/session-parking.mdx b/docs/speech-to-text/realtime/session-parking.mdx index 6c506d31..e93ca887 100644 --- a/docs/speech-to-text/realtime/session-parking.mdx +++ b/docs/speech-to-text/realtime/session-parking.mdx @@ -1,5 +1,6 @@ --- -description: 'Keep a Realtime session open without holding compute capacity during long stretches of near-silent audio' +title: Session parking +description: 'Keep a Realtime session open without holding compute capacity during long stretches of silence' keywords: [ speechmatics, @@ -18,56 +19,61 @@ import realtimeSchema from "!asyncapi-schema-loader!@site/spec/realtime.yaml"; # Session parking -Some integrations open a Realtime session and send only a trickle of audio — a small chunk every few seconds — for hours, just to keep the connection alive. Session parking frees the compute capacity allocated to a session like this without ending it: the WebSocket stays open and keeps accepting audio, and transcription resumes as soon as real audio arrives again. +## How session parking works -:::note -Session parking must be enabled for your account. Contact your account manager or [support](https://support.speechmatics.com) to turn it on. -::: +Speechmatics detects when a session's audio rate has stayed significantly below real time, eg less than 50%, for a while and parks the session automatically. + +Speechmatics acknowledges audio sent while parked, so client-side flow control keeps working. When the audio stream returns to close to real time, the session resumes on its own. -## How it works +A parked session's transcript timeline (timestamps and speaker labels) either continues across the pause or restarts, depending on `preserve_timeline`, covered in [Pausing and resuming explicitly](#pausing-and-resuming-explicitly). -Once enabled, parking is automatic and transparent: Speechmatics detects when a session's audio rate has stayed below real time for a while and parks it, with no message sent to the client. Audio sent while parked is buffered and acknowledged as normal, so client-side flow control keeps working. When audio returns to real time, the session resumes on its own. +## Pausing and resuming explicitly -A parked session's transcript timeline (timestamps and speaker labels) either continues across the pause or restarts, depending on `preserve_timeline` — see [Manual parking](#manual-parking) below. Automatically detected parks default to preserving the timeline. +You can also pause and resume a session explicitly with `PauseRecognition` and `ResumeRecognition`. This is the best fix for an integration that needs connections prewarmed and ready: pause instead of holding the connection open at a low rate. -If a resume fails, the session is closed with an `Error` of type `park_resume_failed`. +Resuming a paused session is faster than closing the connection and opening a new one. A new session pays for a fresh WebSocket connection — DNS lookup, TCP handshake, TLS handshake — followed by a `StartRecognition` / `RecognitionStarted` round trip; a large custom dictionary in `StartRecognition` can add hundreds of milliseconds to that round trip on its own. -## Manual parking +`ResumeRecognition` skips all of that: the connection is already open, so resuming is a single message on it. Depending on network distance and throughput, that can save on the order of 1.5 seconds compared to opening a new session. -Clients can also park and resume a session explicitly with `PauseRecognition` and `ResumeRecognition`. This is the cheapest fix for an integration that only trickles audio: pause instead of holding the connection open at a low rate. +The following table shows messages involved in explict session parking: -```json -{"message": "PauseRecognition"} -> {"message": "RecognitionPaused"} -{"message": "PauseRecognition", "preserve_timeline": true} -> {"message": "RecognitionPaused"} -{"message": "ResumeRecognition"} -> {"message": "RecognitionResumed"} -``` +| Client sends | Server replies | +|---|---| +| `{"message": "PauseRecognition"}` | `{"message": "RecognitionPaused"}` | +| `{"message": "PauseRecognition", "preserve_timeline": true}` | `{"message": "RecognitionPaused"}` | +| `{"message": "ResumeRecognition"}` | `{"message": "RecognitionResumed"}` |

-`RecognitionPaused` is sent once the transcript for any audio already sent has arrived and the session's compute has been released. `RecognitionResumed` is sent once the session is transcribing again. Audio sent right after `ResumeRecognition` doesn't need to wait for it — it's acknowledged in order once the resume completes. +`RecognitionPaused` is sent once the transcript for any audio already sent has arrived and the session's compute has been released. `RecognitionResumed` is sent once the session is transcribing again. :::tip -Send audio immediately after `ResumeRecognition`, without waiting for `RecognitionResumed`. Waiting for the acknowledgement adds a full round trip to time-to-first-final — typically 200-500ms, depending on network conditions — for no benefit, since the resumed session accepts and queues that audio correctly either way. +Send audio immediately after `ResumeRecognition`, without waiting for `RecognitionResumed`. Waiting for the acknowledgement adds a full round trip to time-to-first-final, depending on network conditions even 200-500ms, resumed session accepts and queues that audio correctly. ::: `preserve_timeline` is optional and applies to that pause only: -- Omitted, or `false` — the session resumes as a new one. Timestamps restart from zero and speaker labels are not carried over. This is the default for a manual pause. -- `true` — the transcript timeline continues: timestamps and speaker labels carry across the pause, and audio sent while parked is transcribed on resume. +- Omitted, or `false` — the session resumes as a new one. Timestamps restart from zero and speaker labels are not carried over. +- `true` — the transcript timeline continues: timestamps and speaker labels carry across the pause. A session that hasn't sent any audio yet, or that can't preserve its timeline at all, always resumes as a new session regardless of `preserve_timeline`. -If `PauseRecognition` is sent on a session that can't be parked, the server replies with a `Warning` of type `not_allowed` and the session continues normally — pausing was simply refused, not the whole connection. +If you send `PauseRecognition` on a session that can't be parked, the server replies with a `Warning` of type `not_allowed` and the session continues normally: pausing was refused, not the connection. + +## Session limits while parked + +A parked session still counts toward your account's [concurrent session limit](/speech-to-text/realtime/limits) — parking releases compute, not the connection. -## Limits +The account's [session limits](/speech-to-text/realtime/limits#session-limits) continue to apply, but which ones depends on how the session was parked: -A parked session holds no compute, so it has no idle deadline of its own — the account's usual [session limits](/speech-to-text/realtime/limits#session-limits) (idle timeout, max duration) still apply and can end it while parked. A session parked by its own `PauseRecognition` is exempt from the idle timeout for as long as it stays parked; the max-duration limit still applies. +- **Parked automatically**, because of extremely low audio rate — both the idle timeout and the max duration limit can end the session. +- **Parked explicitly** with `PauseRecognition` — exempt from the idle timeout for as long as it stays parked. The max duration limit still applies. -Audio buffered while parked is capped. If the cap is reached, the oldest buffered audio is dropped to make room for new audio — this can only happen with `preserve_timeline: true`, since audio isn't buffered for replay otherwise. +Speechmatics caps how much audio it buffers while a session is parked. If the cap is reached, Speechmatics drops the oldest buffered audio to make room for new audio — this can only happen with `preserve_timeline: true`, since audio isn't buffered for replay otherwise. -## Deployment +## Where session parking is available -Session parking is available on SaaS on Cloud only. +Session parking is available on SaaS on Cloud and when RT SaaS helm chart is used for on Prem.