feat(whisper): timing log, default prompt and hotwords; chart affinity and spread

.github/workflows/ci.yml

@@ -34,3 +34,19 @@ jobs:
 
       - name: Build
         run: make build
+
+  # ── Python: whisper server tests (stubbed model, no download) ─────────────
+  whisper:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v7
+
+      - uses: actions/setup-python@v6
+        with:
+          python-version: "3.12"
+
+      - name: Install test dependencies
+        run: pip install fastapi httpx python-multipart pytest
+
+      - name: Test
+        run: pytest -q deploy/whisper

.gitignore

@@ -8,3 +8,4 @@ vendor/
 .env.*
 deploy/helm/charts/
 /mockbackend
+__pycache__/

AGENTS.md

@@ -72,7 +72,7 @@ Key details:
 
 ### Whisper (Standalone)
 
-Model is baked directly into the image at build time. No init container, no PVC needed. The image runs `deploy/whisper/server.py`, which speaks the OpenAI audio API itself (the router forwards paths unchanged). `/health` does not queue behind inference. Pin `av` (see Containerfile) — newer PyAV majors break faster-whisper.
+Model is baked directly into the image at build time. No init container, no PVC needed. The image runs `deploy/whisper/server.py`, which speaks the OpenAI audio API itself (the router forwards paths unchanged). `/health` does not queue behind inference. Pin `av` (see Containerfile) — newer PyAV majors break faster-whisper. Each request logs one JSON line (`whisper_request`, timings, RTF; never prompt/transcript text). `WHISPER_INITIAL_PROMPT` / `WHISPER_HOTWORDS` are server-side defaults (see README). Tests: `pytest deploy/whisper` (stubbed model).
 
 ### Router (Go Proxy)
 

README.md

@@ -192,9 +192,69 @@ Measured on CPU: ~2.0 GiB RSS peak; roughly real-time speed on 2 threads (a
 - `response_format`: `json` (default), `text`, `verbose_json`, `srt`, `vtt`.
 - Set `whisper.language` (e.g. `de`) to skip language auto-detection; a
   request's own `language` field wins.
+- Vocabulary hints, two mechanisms (both optional, both off by default):
+  - `prompt` (form field, OpenAI-compatible) becomes faster-whisper's
+    `initial_prompt`. Set a server-side default with `whisper.initialPrompt`
+    (`WHISPER_INITIAL_PROMPT`) for clients that send no prompt (e.g. an STT
+    integration that only sends `model` and `language`). A request prompt is
+    **appended** to the default (`<default> <request>`): Whisper weighs the end
+    of the prompt most, and an over-long prompt is cut from the front, so the
+    request-specific text survives. Whisper reads at most 223 tokens
+    (`max_length // 2 - 1`); faster-whisper keeps the last 223 and drops the
+    rest. The server additionally caps the combined text at 2000 characters
+    (tail kept).
+  - `hotwords` (form field, extension; free text such as
+    `Kubernetes Grafana Postgres`) is passed to faster-whisper's `hotwords`.
+    Server default: `whisper.hotwords` (`WHISPER_HOTWORDS`). A request's
+    `hotwords` **replaces** the default (no merging). faster-whisper puts the
+    hotwords in front of the previous-text prompt; it ignores them only when
+    `prefix` is set, which this server never sets, so `prompt` and `hotwords`
+    work together (checked in `faster_whisper/transcribe.py`, `get_prompt`,
+    version 1.2.1). Both count against the same 223-token window of their own.
+  - Neutral example for `whisper.initialPrompt`:
+    `Release notes for Kubernetes, PostgreSQL and Grafana.`
+  - Hints raise the odds of a spelling; they do not guarantee it. Keep them
+    short and spell terms the way they should appear.
+- Per-request log line (JSON after the prefix `whisper_request`), e.g.
+  `whisper_request {"status":"ok","audio_s":9.0,"prep_s":0.4,"infer_s":7.6,"total_s":8.1,"rtf":0.889,...}`:
+  audio duration (`audio_s`, `audio_after_vad_s`), upload, queue wait, decode +
+  VAD (`prep_s`) and inference (`infer_s`) time, real-time factor
+  (`rtf = (prep_s + infer_s) / audio_s`), model, language, `beam_size`, `vad`,
+  file size, and `prompt_chars` / `hotwords_chars` / `prompt_source`
+  (`none|default|request|both`). Prompt, hotwords and transcript text are
+  never logged.
 - `/v1/audio/translations` (to English) is exposed, but large-v3-turbo is not
   trained for translation; use a non-turbo model for that.
 
+### Scheduling (`affinity`, `topologySpreadConstraints`)
+
+`embedding` and `whisper` accept optional `affinity` and
+`topologySpreadConstraints` (empty by default, so the rendered manifests do not
+change). With `embedding.replicaCount > 1` the replicas can land on one node,
+and then the PodDisruptionBudget does not protect against a node failure.
+Example, soft spreading that never blocks scheduling:
+
+```yaml
+embedding:
+  replicaCount: 2
+  affinity:
+    podAntiAffinity:
+      preferredDuringSchedulingIgnoredDuringExecution:
+        - weight: 100
+          podAffinityTerm:
+            topologyKey: kubernetes.io/hostname
+            labelSelector:
+              matchLabels:
+                app.kubernetes.io/component: embedding
+  topologySpreadConstraints:
+    - maxSkew: 1
+      topologyKey: kubernetes.io/hostname
+      whenUnsatisfiable: ScheduleAnyway
+      labelSelector:
+        matchLabels:
+          app.kubernetes.io/component: embedding
+```
+
 ## Environment Variables
 
 | Variable               | Default | Description                   |

deploy/helm/templates/embedding/deployment.yaml

@@ -46,6 +46,14 @@ spec:
       tolerations:
         {{- toYaml . | nindent 8 }}
       {{- end }}
+      {{- with .Values.embedding.affinity }}
+      affinity:
+        {{- toYaml . | nindent 8 }}
+      {{- end }}
+      {{- with .Values.embedding.topologySpreadConstraints }}
+      topologySpreadConstraints:
+        {{- toYaml . | nindent 8 }}
+      {{- end }}
       initContainers:
       - name: model-init
         image: {{ include "inference-stack.image" (dict "registry" .Values.global.imageRegistry "repo" .Values.embedding.modelInit.image.repository "tag" .Values.embedding.modelInit.image.tag "appVersion" .Chart.AppVersion) }}

deploy/helm/templates/whisper/deployment.yaml

@@ -33,6 +33,14 @@ spec:
       tolerations:
         {{- toYaml . | nindent 8 }}
       {{- end }}
+      {{- with .Values.whisper.affinity }}
+      affinity:
+        {{- toYaml . | nindent 8 }}
+      {{- end }}
+      {{- with .Values.whisper.topologySpreadConstraints }}
+      topologySpreadConstraints:
+        {{- toYaml . | nindent 8 }}
+      {{- end }}
       containers:
       - name: whisper
         image: {{ include "inference-stack.image" (dict "registry" .Values.global.imageRegistry "repo" .Values.whisper.image.repository "tag" .Values.whisper.image.tag "appVersion" .Chart.AppVersion) }}
@@ -50,6 +58,10 @@ spec:
           value: {{ .Values.whisper.maxConcurrent | quote }}
         - name: WHISPER_LANGUAGE
           value: {{ .Values.whisper.language | quote }}
+        - name: WHISPER_INITIAL_PROMPT
+          value: {{ .Values.whisper.initialPrompt | quote }}
+        - name: WHISPER_HOTWORDS
+          value: {{ .Values.whisper.hotwords | quote }}
         ports:
         - name: http
           containerPort: {{ .Values.whisper.service.port }}

deploy/helm/values.yaml

@@ -193,6 +193,27 @@ embedding:
   imagePullSecrets: []
   nodeSelector: {}
   tolerations: []
+  # Optional scheduling rules; empty = scheduler default. Example: prefer
+  # spreading replicas over nodes with a soft anti-affinity
+  #   affinity:
+  #     podAntiAffinity:
+  #       preferredDuringSchedulingIgnoredDuringExecution:
+  #       - weight: 100
+  #         podAffinityTerm:
+  #           topologyKey: kubernetes.io/hostname
+  #           labelSelector:
+  #             matchLabels:
+  #               app.kubernetes.io/component: embedding
+  affinity: {}
+  # Example: spread replicas over nodes, but never block scheduling
+  #   topologySpreadConstraints:
+  #   - maxSkew: 1
+  #     topologyKey: kubernetes.io/hostname
+  #     whenUnsatisfiable: ScheduleAnyway
+  #     labelSelector:
+  #       matchLabels:
+  #         app.kubernetes.io/component: embedding
+  topologySpreadConstraints: []
 
 # ── Reranker — BAAI/bge-reranker-v2-m3 ───────────────────────────────────────
 # Uses TEI (Rust/ONNX via ORT backend) — same runtime image as embedding.
@@ -315,6 +336,18 @@ whisper:
   # per request. A request's own `language` field always wins.
   language: ""
 
+  # Server-side default prompt (WHISPER_INITIAL_PROMPT), empty = off. Use it for
+  # vocabulary and spelling that clients cannot send themselves (e.g. an STT
+  # integration that only sends model + language). A request's `prompt` is
+  # appended after it. Whisper uses at most ~223 tokens; longer text is cut
+  # from the front. Example: "Release notes for Kubernetes, PostgreSQL and
+  # Grafana."
+  initialPrompt: ""
+
+  # Server-side default hotwords (WHISPER_HOTWORDS), empty = off. A request's
+  # `hotwords` field replaces it. Free text, e.g. "Kubernetes Grafana Postgres".
+  hotwords: ""
+
   # CTranslate2 threads. Must match the CPU limit: CTranslate2 sizes itself by
   # the node's cores, not by the cgroup limit.
   cpuThreads: 4
@@ -368,6 +401,27 @@ whisper:
   imagePullSecrets: []
   nodeSelector: {}
   tolerations: []
+  # Optional scheduling rules; empty = scheduler default. Example: prefer
+  # spreading replicas over nodes with a soft anti-affinity
+  #   affinity:
+  #     podAntiAffinity:
+  #       preferredDuringSchedulingIgnoredDuringExecution:
+  #       - weight: 100
+  #         podAffinityTerm:
+  #           topologyKey: kubernetes.io/hostname
+  #           labelSelector:
+  #             matchLabels:
+  #               app.kubernetes.io/component: whisper
+  affinity: {}
+  # Example: spread replicas over nodes, but never block scheduling
+  #   topologySpreadConstraints:
+  #   - maxSkew: 1
+  #     topologyKey: kubernetes.io/hostname
+  #     whenUnsatisfiable: ScheduleAnyway
+  #     labelSelector:
+  #       matchLabels:
+  #         app.kubernetes.io/component: whisper
+  topologySpreadConstraints: []
 
 # ── ServiceMonitor (Prometheus Operator) ──────────────────────────────────────
 serviceMonitor: