Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 15 additions & 4 deletions docs/deployments/container/accessing-images.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,12 @@ See [how to run the Core Speech CPU container here.](/deployments/container/cpu-

The Transcription GPU images are required to use the most accurate models.

See [how to run the Transcription GPU container here.](/deployments/container/gpu-speech-to-text)

:::info
To access additional language configurations for containers, please [speak to our Support Team](https://support.speechmatics.com).
:::

### Standard model

There is a single image available that supports all languages for the Standard model. There are language specific images available that support the Enhanced and Standard models.
Expand Down Expand Up @@ -112,11 +118,16 @@ Depending on which Enhanced model languages are required, you can pull specific
</CodeBlock>
</details>

See [how to run the Transcription GPU container here.](/deployments/container/gpu-speech-to-text)
### Standard and Enhanced model

:::info
To access additional language configurations for containers, please [speak to our Support Team](https://support.speechmatics.com).
:::
A single image supports all languages for both the Standard and Enhanced models, so you do not need to pull individual language pack images. The `sm-gpu-inference-server-standard-all` image above covers all languages for the Standard model only.

<CodeBlock language="bash">
{`# pulling the Transcription GPU inference server supporting all languages for both Enhanced and Standard models with the ${smVariables.latestContainerVersion} tag:
docker pull speechmaticspublic.azurecr.io/sm-gpu-inference-server-all-lang:${smVariables.latestContainerVersion}`}
</CodeBlock>

This image contains both models. To load only one of them, see [Running only one model](/deployments/container/gpu-speech-to-text#running-only-one-model). To load only some of the image's languages, see [Loading only selected languages](/deployments/container/gpu-speech-to-text#loading-only-selected-languages).

### Melia 1 model

Expand Down
15 changes: 15 additions & 0 deletions docs/deployments/container/gpu-speech-to-text.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,21 @@ To save GPU memory for throughput, you can run the server with only one model lo
When running the all language standard model GPU inference server you must set the `SM_MODEL` environment variable to `standard`
:::

### Loading only selected languages

Images that contain more than one language model load every language at startup. To save GPU memory for throughput, pass the `SM_LANGUAGES` environment variable to the container and set it to a comma-separated list of language codes. Only those languages are loaded:

```bash
Comment thread
smvenkateshc marked this conversation as resolved.
# load only English, German, and French
-e SM_LANGUAGES=en,de,fr
```

Codes must be languages the image contains. Any code the image does not contain causes startup to fail with an error.

When `SM_LANGUAGES` is unset, all of the image's languages are loaded. `SM_LANGUAGES` is available from container version 15.18.0.

`SM_LANGUAGES` and `SM_MODEL` can be used together. For example, `SM_MODEL=standard` with `SM_LANGUAGES=en,de` loads the Standard model for English and German only.

### Monitoring the server

The inference server is based on [Nvidia's Triton architecture](https://developer.nvidia.com/nvidia-triton-inference-server) and as such
Expand Down
2 changes: 2 additions & 0 deletions docs/deployments/container/performance-and-cost.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ The benchmark uses the following configuration:
| Benchmark details | |
|-------------------|----------------------------------------------|
| Version | 15.7.0 (Standard, Enhanced), 1.3.0 (Melia-1) |
| Language | English only |
Comment thread
smvenkateshc marked this conversation as resolved.
| CPU | D16ds_v5 |
| GPU Standard | Standard_NC16as_T4_v3 |
| GPU Enhanced | Standard_NC8as_T4_v3 |
Expand Down Expand Up @@ -54,6 +55,7 @@ This benchmark uses the following configuration[^4]:
| Benchmark details | Value |
| ------------------| ----------------------------------- |
| Version | 13.4.0 |
| Language | English only |
| CPU | D16ds_v5 |
| GPU Standard | Standard_NC16as_T4_v3 |
| GPU Enhanced | Standard_NC8as_T4_v3 |
Expand Down
2 changes: 1 addition & 1 deletion sm-variables.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
export const smVariables = {
jsonOutputVersion: "2.9",
latestApplianceVersion: "6.3.0",
latestContainerVersion: "15.14.0",
latestContainerVersion: "15.18.0",
latestMelia1ContainerVersion: "1.3.0",
usageContainerVersion: "0.3.0",
helmChartVersion: "1.4.0",
Expand Down