diff --git a/docs/deployments/container/accessing-images.mdx b/docs/deployments/container/accessing-images.mdx index 7dc5c2aa..fda15cae 100644 --- a/docs/deployments/container/accessing-images.mdx +++ b/docs/deployments/container/accessing-images.mdx @@ -64,6 +64,12 @@ See [how to run the Core Speech CPU container here.](/deployments/container/cpu- The Transcription GPU images are required to use the most accurate models. +See [how to run the Transcription GPU container here.](/deployments/container/gpu-speech-to-text) + +:::info +To access additional language configurations for containers, please [speak to our Support Team](https://support.speechmatics.com). +::: + ### Standard model There is a single image available that supports all languages for the Standard model. There are language specific images available that support the Enhanced and Standard models. @@ -112,11 +118,16 @@ Depending on which Enhanced model languages are required, you can pull specific -See [how to run the Transcription GPU container here.](/deployments/container/gpu-speech-to-text) +### Standard and Enhanced model -:::info -To access additional language configurations for containers, please [speak to our Support Team](https://support.speechmatics.com). -::: +A single image supports all languages for both the Standard and Enhanced models, so you do not need to pull individual language pack images. The `sm-gpu-inference-server-standard-all` image above covers all languages for the Standard model only. + + + {`# pulling the Transcription GPU inference server supporting all languages for both Enhanced and Standard models with the ${smVariables.latestContainerVersion} tag: +docker pull speechmaticspublic.azurecr.io/sm-gpu-inference-server-all-lang:${smVariables.latestContainerVersion}`} + + +This image contains both models. To load only one of them, see [Running only one model](/deployments/container/gpu-speech-to-text#running-only-one-model). To load only some of the image's languages, see [Loading only selected languages](/deployments/container/gpu-speech-to-text#loading-only-selected-languages). ### Melia 1 model diff --git a/docs/deployments/container/gpu-speech-to-text.mdx b/docs/deployments/container/gpu-speech-to-text.mdx index fc82ca0a..ccef0308 100644 --- a/docs/deployments/container/gpu-speech-to-text.mdx +++ b/docs/deployments/container/gpu-speech-to-text.mdx @@ -117,6 +117,21 @@ To save GPU memory for throughput, you can run the server with only one model lo When running the all language standard model GPU inference server you must set the `SM_MODEL` environment variable to `standard` ::: +### Loading only selected languages + +Images that contain more than one language model load every language at startup. To save GPU memory for throughput, pass the `SM_LANGUAGES` environment variable to the container and set it to a comma-separated list of language codes. Only those languages are loaded: + +```bash +# load only English, German, and French +-e SM_LANGUAGES=en,de,fr +``` + +Codes must be languages the image contains. Any code the image does not contain causes startup to fail with an error. + +When `SM_LANGUAGES` is unset, all of the image's languages are loaded. `SM_LANGUAGES` is available from container version 15.18.0. + +`SM_LANGUAGES` and `SM_MODEL` can be used together. For example, `SM_MODEL=standard` with `SM_LANGUAGES=en,de` loads the Standard model for English and German only. + ### Monitoring the server The inference server is based on [Nvidia's Triton architecture](https://developer.nvidia.com/nvidia-triton-inference-server) and as such diff --git a/docs/deployments/container/performance-and-cost.mdx b/docs/deployments/container/performance-and-cost.mdx index b158c220..addd0525 100644 --- a/docs/deployments/container/performance-and-cost.mdx +++ b/docs/deployments/container/performance-and-cost.mdx @@ -27,6 +27,7 @@ The benchmark uses the following configuration: | Benchmark details | | |-------------------|----------------------------------------------| | Version | 15.7.0 (Standard, Enhanced), 1.3.0 (Melia-1) | +| Language | English only | | CPU | D16ds_v5 | | GPU Standard | Standard_NC16as_T4_v3 | | GPU Enhanced | Standard_NC8as_T4_v3 | @@ -54,6 +55,7 @@ This benchmark uses the following configuration[^4]: | Benchmark details | Value | | ------------------| ----------------------------------- | | Version | 13.4.0 | +| Language | English only | | CPU | D16ds_v5 | | GPU Standard | Standard_NC16as_T4_v3 | | GPU Enhanced | Standard_NC8as_T4_v3 | diff --git a/sm-variables.ts b/sm-variables.ts index b6af3db1..1e520837 100644 --- a/sm-variables.ts +++ b/sm-variables.ts @@ -1,7 +1,7 @@ export const smVariables = { jsonOutputVersion: "2.9", latestApplianceVersion: "6.3.0", - latestContainerVersion: "15.14.0", + latestContainerVersion: "15.18.0", latestMelia1ContainerVersion: "1.3.0", usageContainerVersion: "0.3.0", helmChartVersion: "1.4.0",