diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 97551bf..8bc673d 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -34,3 +34,19 @@ jobs: - name: Build run: make build + + # ── Python: whisper server tests (stubbed model, no download) ───────────── + whisper: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v7 + + - uses: actions/setup-python@v6 + with: + python-version: "3.12" + + - name: Install test dependencies + run: pip install fastapi httpx python-multipart pytest + + - name: Test + run: pytest -q deploy/whisper diff --git a/.gitignore b/.gitignore index 1b29ec2..4938b74 100644 --- a/.gitignore +++ b/.gitignore @@ -8,3 +8,4 @@ vendor/ .env.* deploy/helm/charts/ /mockbackend +__pycache__/ diff --git a/AGENTS.md b/AGENTS.md index 9e8c875..1f3a948 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -72,7 +72,7 @@ Key details: ### Whisper (Standalone) -Model is baked directly into the image at build time. No init container, no PVC needed. The image runs `deploy/whisper/server.py`, which speaks the OpenAI audio API itself (the router forwards paths unchanged). `/health` does not queue behind inference. Pin `av` (see Containerfile) — newer PyAV majors break faster-whisper. +Model is baked directly into the image at build time. No init container, no PVC needed. The image runs `deploy/whisper/server.py`, which speaks the OpenAI audio API itself (the router forwards paths unchanged). `/health` does not queue behind inference. Pin `av` (see Containerfile) — newer PyAV majors break faster-whisper. Each request logs one JSON line (`whisper_request`, timings, RTF; never prompt/transcript text). `WHISPER_INITIAL_PROMPT` / `WHISPER_HOTWORDS` are server-side defaults (see README). Tests: `pytest deploy/whisper` (stubbed model). ### Router (Go Proxy) diff --git a/README.md b/README.md index 7efc01f..3c92272 100644 --- a/README.md +++ b/README.md @@ -192,9 +192,69 @@ Measured on CPU: ~2.0 GiB RSS peak; roughly real-time speed on 2 threads (a - `response_format`: `json` (default), `text`, `verbose_json`, `srt`, `vtt`. - Set `whisper.language` (e.g. `de`) to skip language auto-detection; a request's own `language` field wins. +- Vocabulary hints, two mechanisms (both optional, both off by default): + - `prompt` (form field, OpenAI-compatible) becomes faster-whisper's + `initial_prompt`. Set a server-side default with `whisper.initialPrompt` + (`WHISPER_INITIAL_PROMPT`) for clients that send no prompt (e.g. an STT + integration that only sends `model` and `language`). A request prompt is + **appended** to the default (` `): Whisper weighs the end + of the prompt most, and an over-long prompt is cut from the front, so the + request-specific text survives. Whisper reads at most 223 tokens + (`max_length // 2 - 1`); faster-whisper keeps the last 223 and drops the + rest. The server additionally caps the combined text at 2000 characters + (tail kept). + - `hotwords` (form field, extension; free text such as + `Kubernetes Grafana Postgres`) is passed to faster-whisper's `hotwords`. + Server default: `whisper.hotwords` (`WHISPER_HOTWORDS`). A request's + `hotwords` **replaces** the default (no merging). faster-whisper puts the + hotwords in front of the previous-text prompt; it ignores them only when + `prefix` is set, which this server never sets, so `prompt` and `hotwords` + work together (checked in `faster_whisper/transcribe.py`, `get_prompt`, + version 1.2.1). Both count against the same 223-token window of their own. + - Neutral example for `whisper.initialPrompt`: + `Release notes for Kubernetes, PostgreSQL and Grafana.` + - Hints raise the odds of a spelling; they do not guarantee it. Keep them + short and spell terms the way they should appear. +- Per-request log line (JSON after the prefix `whisper_request`), e.g. + `whisper_request {"status":"ok","audio_s":9.0,"prep_s":0.4,"infer_s":7.6,"total_s":8.1,"rtf":0.889,...}`: + audio duration (`audio_s`, `audio_after_vad_s`), upload, queue wait, decode + + VAD (`prep_s`) and inference (`infer_s`) time, real-time factor + (`rtf = (prep_s + infer_s) / audio_s`), model, language, `beam_size`, `vad`, + file size, and `prompt_chars` / `hotwords_chars` / `prompt_source` + (`none|default|request|both`). Prompt, hotwords and transcript text are + never logged. - `/v1/audio/translations` (to English) is exposed, but large-v3-turbo is not trained for translation; use a non-turbo model for that. +### Scheduling (`affinity`, `topologySpreadConstraints`) + +`embedding` and `whisper` accept optional `affinity` and +`topologySpreadConstraints` (empty by default, so the rendered manifests do not +change). With `embedding.replicaCount > 1` the replicas can land on one node, +and then the PodDisruptionBudget does not protect against a node failure. +Example, soft spreading that never blocks scheduling: + +```yaml +embedding: + replicaCount: 2 + affinity: + podAntiAffinity: + preferredDuringSchedulingIgnoredDuringExecution: + - weight: 100 + podAffinityTerm: + topologyKey: kubernetes.io/hostname + labelSelector: + matchLabels: + app.kubernetes.io/component: embedding + topologySpreadConstraints: + - maxSkew: 1 + topologyKey: kubernetes.io/hostname + whenUnsatisfiable: ScheduleAnyway + labelSelector: + matchLabels: + app.kubernetes.io/component: embedding +``` + ## Environment Variables | Variable | Default | Description | diff --git a/deploy/helm/templates/embedding/deployment.yaml b/deploy/helm/templates/embedding/deployment.yaml index 6416039..d535180 100644 --- a/deploy/helm/templates/embedding/deployment.yaml +++ b/deploy/helm/templates/embedding/deployment.yaml @@ -46,6 +46,14 @@ spec: tolerations: {{- toYaml . | nindent 8 }} {{- end }} + {{- with .Values.embedding.affinity }} + affinity: + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.embedding.topologySpreadConstraints }} + topologySpreadConstraints: + {{- toYaml . | nindent 8 }} + {{- end }} initContainers: - name: model-init image: {{ include "inference-stack.image" (dict "registry" .Values.global.imageRegistry "repo" .Values.embedding.modelInit.image.repository "tag" .Values.embedding.modelInit.image.tag "appVersion" .Chart.AppVersion) }} diff --git a/deploy/helm/templates/whisper/deployment.yaml b/deploy/helm/templates/whisper/deployment.yaml index b651954..a296ef5 100644 --- a/deploy/helm/templates/whisper/deployment.yaml +++ b/deploy/helm/templates/whisper/deployment.yaml @@ -33,6 +33,14 @@ spec: tolerations: {{- toYaml . | nindent 8 }} {{- end }} + {{- with .Values.whisper.affinity }} + affinity: + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.whisper.topologySpreadConstraints }} + topologySpreadConstraints: + {{- toYaml . | nindent 8 }} + {{- end }} containers: - name: whisper image: {{ include "inference-stack.image" (dict "registry" .Values.global.imageRegistry "repo" .Values.whisper.image.repository "tag" .Values.whisper.image.tag "appVersion" .Chart.AppVersion) }} @@ -50,6 +58,10 @@ spec: value: {{ .Values.whisper.maxConcurrent | quote }} - name: WHISPER_LANGUAGE value: {{ .Values.whisper.language | quote }} + - name: WHISPER_INITIAL_PROMPT + value: {{ .Values.whisper.initialPrompt | quote }} + - name: WHISPER_HOTWORDS + value: {{ .Values.whisper.hotwords | quote }} ports: - name: http containerPort: {{ .Values.whisper.service.port }} diff --git a/deploy/helm/values.yaml b/deploy/helm/values.yaml index 7960f34..c5b2779 100644 --- a/deploy/helm/values.yaml +++ b/deploy/helm/values.yaml @@ -193,6 +193,27 @@ embedding: imagePullSecrets: [] nodeSelector: {} tolerations: [] + # Optional scheduling rules; empty = scheduler default. Example: prefer + # spreading replicas over nodes with a soft anti-affinity + # affinity: + # podAntiAffinity: + # preferredDuringSchedulingIgnoredDuringExecution: + # - weight: 100 + # podAffinityTerm: + # topologyKey: kubernetes.io/hostname + # labelSelector: + # matchLabels: + # app.kubernetes.io/component: embedding + affinity: {} + # Example: spread replicas over nodes, but never block scheduling + # topologySpreadConstraints: + # - maxSkew: 1 + # topologyKey: kubernetes.io/hostname + # whenUnsatisfiable: ScheduleAnyway + # labelSelector: + # matchLabels: + # app.kubernetes.io/component: embedding + topologySpreadConstraints: [] # ── Reranker — BAAI/bge-reranker-v2-m3 ─────────────────────────────────────── # Uses TEI (Rust/ONNX via ORT backend) — same runtime image as embedding. @@ -315,6 +336,18 @@ whisper: # per request. A request's own `language` field always wins. language: "" + # Server-side default prompt (WHISPER_INITIAL_PROMPT), empty = off. Use it for + # vocabulary and spelling that clients cannot send themselves (e.g. an STT + # integration that only sends model + language). A request's `prompt` is + # appended after it. Whisper uses at most ~223 tokens; longer text is cut + # from the front. Example: "Release notes for Kubernetes, PostgreSQL and + # Grafana." + initialPrompt: "" + + # Server-side default hotwords (WHISPER_HOTWORDS), empty = off. A request's + # `hotwords` field replaces it. Free text, e.g. "Kubernetes Grafana Postgres". + hotwords: "" + # CTranslate2 threads. Must match the CPU limit: CTranslate2 sizes itself by # the node's cores, not by the cgroup limit. cpuThreads: 4 @@ -368,6 +401,27 @@ whisper: imagePullSecrets: [] nodeSelector: {} tolerations: [] + # Optional scheduling rules; empty = scheduler default. Example: prefer + # spreading replicas over nodes with a soft anti-affinity + # affinity: + # podAntiAffinity: + # preferredDuringSchedulingIgnoredDuringExecution: + # - weight: 100 + # podAffinityTerm: + # topologyKey: kubernetes.io/hostname + # labelSelector: + # matchLabels: + # app.kubernetes.io/component: whisper + affinity: {} + # Example: spread replicas over nodes, but never block scheduling + # topologySpreadConstraints: + # - maxSkew: 1 + # topologyKey: kubernetes.io/hostname + # whenUnsatisfiable: ScheduleAnyway + # labelSelector: + # matchLabels: + # app.kubernetes.io/component: whisper + topologySpreadConstraints: [] # ── ServiceMonitor (Prometheus Operator) ────────────────────────────────────── serviceMonitor: diff --git a/deploy/whisper/server.py b/deploy/whisper/server.py index 021e840..768f70a 100644 --- a/deploy/whisper/server.py +++ b/deploy/whisper/server.py @@ -10,15 +10,23 @@ file audio file (mp3, wav, m4a, ogg, flac, webm, mp4, ...) model accepted for compatibility, ignored (one model is loaded) language ISO-639-1 code; empty = auto-detect (default: WHISPER_LANGUAGE) - prompt optional initial prompt + prompt optional initial prompt (domain vocabulary, spelling hints); + appended to WHISPER_INITIAL_PROMPT, see combine_prompt() + hotwords optional hint phrases; replaces WHISPER_HOTWORDS response_format json (default) | text | verbose_json | srt | vtt temperature float, default 0 Inference runs in a worker thread and is serialised by a semaphore, so the event loop (and with it /health) stays responsive while a transcription runs. + +Every request writes one JSON log line ("whisper_request {...}") with timings, +audio duration, real-time factor and parameters. Prompt, hotwords and +transcript are never logged, only whether and how long the prompt/hotwords +were. """ import asyncio +import json import os import tempfile import time @@ -37,6 +45,13 @@ BEAM_SIZE = int(os.environ.get("BEAM_SIZE", "1")) VAD_FILTER = os.environ.get("VAD_FILTER", "true").lower() == "true" PORT = int(os.environ.get("PORT", "9000")) +# Server-side defaults; many STT clients send neither prompt nor hotwords. +DEFAULT_PROMPT = os.environ.get("WHISPER_INITIAL_PROMPT", "").strip() +DEFAULT_HOTWORDS = os.environ.get("WHISPER_HOTWORDS", "").strip() +# Upper bound on input size per field. faster-whisper itself keeps only the last +# 223 tokens (max_length 448 // 2 - 1) of prompt and hotwords; this cap only +# protects against absurd inputs and also keeps the tail. +MAX_PROMPT_CHARS = 2000 FORMATS = {"json", "text", "verbose_json", "srt", "vtt"} @@ -88,27 +103,64 @@ def _ts(seconds: float, sep: str) -> str: return f"{h:02d}:{m:02d}:{s:02d}{sep}{ms:03d}" -def _transcribe(path: str, task: str, language: str | None, prompt: str | None, temperature: float): +def combine_prompt(default: str, request: str | None) -> tuple[str | None, str]: + """Merge the configured default prompt with a request prompt. + + The request prompt is appended after the default. Whisper conditions most + strongly on the end of the prompt, and faster-whisper cuts an over-long + prompt from the front (it keeps the last 223 tokens), so the more specific + request text survives and the generic default is what gets cut first. + Returns (prompt or None, source) with source in none|default|request|both. + """ + request = (request or "").strip() + parts = [p for p in (default, request) if p] + if not parts: + return None, "none" + source = "both" if default and request else ("default" if default else "request") + return " ".join(parts)[-MAX_PROMPT_CHARS:], source + + +def resolve_hotwords(default: str, request: str | None) -> str | None: + """A request's hotwords replace the default (no merging).""" + return ((request or "").strip() or default)[:MAX_PROMPT_CHARS] or None + + +def _transcribe(path: str, task: str, language: str | None, prompt: str | None, hotwords: str | None, + temperature: float): + t0 = time.monotonic() + # transcribe() decodes the whole file (PyAV/FFmpeg, resampling to 16 kHz), + # runs VAD and builds the features before it returns the lazy generator. segments, info = _model.transcribe( path, task=task, language=language, - initial_prompt=prompt or None, + initial_prompt=prompt, + hotwords=hotwords, temperature=temperature, beam_size=BEAM_SIZE, vad_filter=VAD_FILTER, ) - # transcribe() is lazy: decoding happens while iterating. - return list(segments), info + t1 = time.monotonic() + # The actual decoding happens while iterating. + segments = list(segments) + t2 = time.monotonic() + return segments, info, {"prep_s": t1 - t0, "infer_s": t2 - t1} + + +def _log_request(**fields): + print("whisper_request " + json.dumps(fields, separators=(",", ":")), flush=True) async def _handle(file: UploadFile, task: str, language: str | None, prompt: str | None, - response_format: str, temperature: float): + hotwords: str | None, response_format: str, temperature: float): if _model is None: raise HTTPException(status_code=503, detail="model not loaded yet") if response_format not in FORMATS: raise HTTPException(status_code=400, detail=f"response_format must be one of {sorted(FORMATS)}") language = (language or DEFAULT_LANGUAGE) or None + prompt, prompt_source = combine_prompt(DEFAULT_PROMPT, prompt) + hotwords = resolve_hotwords(DEFAULT_HOTWORDS, hotwords) + t_start = time.monotonic() limit = MAX_UPLOAD_MB * 1024 * 1024 size = 0 @@ -122,12 +174,46 @@ async def _handle(file: UploadFile, task: str, language: str | None, prompt: str if size == 0: raise HTTPException(status_code=400, detail="empty file") + t_queued = time.monotonic() async with _sem: + t_run = time.monotonic() try: - segments, info = await asyncio.to_thread(_transcribe, tmp.name, task, language, prompt, temperature) + segments, info, timing = await asyncio.to_thread( + _transcribe, tmp.name, task, language, prompt, hotwords, temperature) except Exception as exc: # undecodable audio etc. + _log_request(status="error", error=type(exc).__name__, task=task, file_bytes=size, + total_s=round(time.monotonic() - t_start, 3)) raise HTTPException(status_code=400, detail=f"could not process audio: {exc}") from exc + total_s = time.monotonic() - t_start + work_s = timing["prep_s"] + timing["infer_s"] + audio_s = info.duration or 0.0 + _log_request( + status="ok", + task=task, + model=MODEL_NAME, + language=info.language, + language_forced=language is not None, + compute_type=COMPUTE_TYPE, + cpu_threads=CPU_THREADS, + beam_size=BEAM_SIZE, + vad=VAD_FILTER, + temperature=temperature, + prompt_chars=len(prompt or ""), + prompt_source=prompt_source, + hotwords_chars=len(hotwords or ""), + response_format=response_format, + file_bytes=size, + audio_s=round(audio_s, 2), + audio_after_vad_s=round(getattr(info, "duration_after_vad", audio_s) or 0.0, 2), + upload_s=round(t_queued - t_start, 3), + queue_s=round(t_run - t_queued, 3), + prep_s=round(timing["prep_s"], 3), + infer_s=round(timing["infer_s"], 3), + total_s=round(total_s, 3), + rtf=round(work_s / audio_s, 3) if audio_s else None, + ) + text = "".join(s.text for s in segments).strip() if response_format == "json": return {"text": text} @@ -158,10 +244,11 @@ async def transcriptions( model: str | None = Form(None), language: str | None = Form(None), prompt: str | None = Form(None), + hotwords: str | None = Form(None), response_format: str = Form("json"), temperature: float = Form(0.0), ): - return await _handle(file, "transcribe", language, prompt, response_format, temperature) + return await _handle(file, "transcribe", language, prompt, hotwords, response_format, temperature) @app.post("/v1/audio/translations") @@ -169,10 +256,11 @@ async def translations( file: UploadFile = File(...), model: str | None = Form(None), prompt: str | None = Form(None), + hotwords: str | None = Form(None), response_format: str = Form("json"), temperature: float = Form(0.0), ): - return await _handle(file, "translate", None, prompt, response_format, temperature) + return await _handle(file, "translate", None, prompt, hotwords, response_format, temperature) if __name__ == "__main__": diff --git a/deploy/whisper/tests/test_server.py b/deploy/whisper/tests/test_server.py new file mode 100644 index 0000000..a18c122 --- /dev/null +++ b/deploy/whisper/tests/test_server.py @@ -0,0 +1,142 @@ +"""Tests for deploy/whisper/server.py with a stubbed model (no model download). + +Run: pip install fastapi httpx python-multipart pytest && pytest deploy/whisper +""" + +import io +import json +import os +import sys +from types import SimpleNamespace + +import pytest +from fastapi.testclient import TestClient + +sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..")) +import server # noqa: E402 + +SECRET_PROMPT = "geheimes-fachwort-im-prompt" +SECRET_HOTWORD = "geheimes-hotword" +SECRET_TEXT = "geheimer transkriptinhalt" + + +class FakeModel: + def __init__(self): + self.calls = [] + + def transcribe(self, path, **kwargs): + self.calls.append(kwargs) + seg = SimpleNamespace(text=f" {SECRET_TEXT}", start=0.0, end=2.0, avg_logprob=-0.1, no_speech_prob=0.0) + info = SimpleNamespace(language="de", duration=10.0, duration_after_vad=8.0) + return iter([seg]), info + + +@pytest.fixture +def client(monkeypatch): + model = FakeModel() + monkeypatch.setattr(server, "_model", model) + monkeypatch.setattr(server, "DEFAULT_PROMPT", "") + monkeypatch.setattr(server, "DEFAULT_HOTWORDS", "") + monkeypatch.setattr(server, "DEFAULT_LANGUAGE", None) + import asyncio + + monkeypatch.setattr(server, "_sem", asyncio.Semaphore(1)) + c = TestClient(server.app) + c.model = model + return c + + +def post(client, **data): + return client.post( + "/v1/audio/transcriptions", + files={"file": ("a.wav", io.BytesIO(b"RIFFxxxx"), "audio/wav")}, + data=data, + ) + + +def test_combine_prompt(): + assert server.combine_prompt("", None) == (None, "none") + assert server.combine_prompt("", " ") == (None, "none") + assert server.combine_prompt("A", None) == ("A", "default") + assert server.combine_prompt("", "B") == ("B", "request") + assert server.combine_prompt("A", "B") == ("A B", "both") + + +def test_combine_prompt_keeps_tail(): + prompt, _ = server.combine_prompt("d" * 3000, "REQUEST") + assert len(prompt) == server.MAX_PROMPT_CHARS + assert prompt.endswith("REQUEST") + + +def test_resolve_hotwords(): + assert server.resolve_hotwords("", None) is None + assert server.resolve_hotwords("default", None) == "default" + assert server.resolve_hotwords("default", "req") == "req" + assert server.resolve_hotwords("default", " ") == "default" + + +def test_no_prompt_by_default(client): + assert post(client).json() == {"text": SECRET_TEXT} + kw = client.model.calls[0] + assert kw["initial_prompt"] is None and kw["hotwords"] is None + assert kw["beam_size"] == server.BEAM_SIZE and kw["vad_filter"] == server.VAD_FILTER + + +def test_request_fields_are_forwarded(client): + post(client, prompt="Alpha", hotwords="Beta Gamma", language="de", temperature="0.2") + kw = client.model.calls[0] + assert kw["initial_prompt"] == "Alpha" + assert kw["hotwords"] == "Beta Gamma" + assert kw["language"] == "de" + assert kw["temperature"] == 0.2 + + +def test_defaults_combined_with_request(client, monkeypatch): + monkeypatch.setattr(server, "DEFAULT_PROMPT", "Standard.") + monkeypatch.setattr(server, "DEFAULT_HOTWORDS", "StdWort") + post(client) + kw = client.model.calls[-1] + assert kw["initial_prompt"] == "Standard." and kw["hotwords"] == "StdWort" + post(client, prompt="Anfrage.", hotwords="Eigenes") + kw = client.model.calls[-1] + assert kw["initial_prompt"] == "Standard. Anfrage." + assert kw["hotwords"] == "Eigenes" + + +def test_translations_forward_hotwords(client): + client.post( + "/v1/audio/translations", + files={"file": ("a.wav", io.BytesIO(b"RIFFxxxx"), "audio/wav")}, + data={"hotwords": "X"}, + ) + assert client.model.calls[0]["hotwords"] == "X" + assert client.model.calls[0]["task"] == "translate" + + +def test_log_line_has_timings_and_no_content(client, capsys, monkeypatch): + monkeypatch.setattr(server, "DEFAULT_PROMPT", "Standard.") + post(client, prompt=SECRET_PROMPT, hotwords=SECRET_HOTWORD) + out = capsys.readouterr().out + lines = [l for l in out.splitlines() if l.startswith("whisper_request ")] + assert len(lines) == 1 + rec = json.loads(lines[0].split(" ", 1)[1]) + assert rec["status"] == "ok" and rec["audio_s"] == 10.0 and rec["audio_after_vad_s"] == 8.0 + assert rec["prompt_source"] == "both" + assert rec["prompt_chars"] == len("Standard. " + SECRET_PROMPT) + assert rec["hotwords_chars"] == len(SECRET_HOTWORD) + assert rec["file_bytes"] == 8 and rec["language"] == "de" + for key in ("prep_s", "infer_s", "queue_s", "total_s", "rtf", "beam_size", "vad", "model"): + assert key in rec + for secret in (SECRET_PROMPT, SECRET_HOTWORD, SECRET_TEXT, "Standard."): + assert secret not in out + + +def test_error_log_without_content(client, capsys): + def boom(path, **kw): + raise RuntimeError("decode failed") + + client.model.transcribe = boom + r = post(client, prompt=SECRET_PROMPT) + assert r.status_code == 400 + out = capsys.readouterr().out + assert '"status":"error"' in out and SECRET_PROMPT not in out diff --git a/internal/proxy/proxy_test.go b/internal/proxy/proxy_test.go index 35dbb3a..64554f9 100644 --- a/internal/proxy/proxy_test.go +++ b/internal/proxy/proxy_test.go @@ -210,6 +210,45 @@ func TestRouting_Whisper(t *testing.T) { } } +// The router must hand multipart bodies to the ASR backend byte for byte, +// including the vocabulary fields prompt, hotwords and language. +func TestRouting_WhisperMultipartPassthrough(t *testing.T) { + body := "--b\r\nContent-Disposition: form-data; name=\"file\"; filename=\"a.wav\"\r\n\r\nRIFFdata\r\n" + + "--b\r\nContent-Disposition: form-data; name=\"language\"\r\n\r\nde\r\n" + + "--b\r\nContent-Disposition: form-data; name=\"prompt\"\r\n\r\nAlpha, Beta\r\n" + + "--b\r\nContent-Disposition: form-data; name=\"hotwords\"\r\n\r\nGamma Delta\r\n--b--\r\n" + + var gotBody, gotCT string + backend := mockBackend(t, map[string]http.HandlerFunc{ + "/v1/audio/transcriptions": func(w http.ResponseWriter, r *http.Request) { + b, _ := io.ReadAll(r.Body) + gotBody, gotCT = string(b), r.Header.Get("Content-Type") + jsonOK(`{"text":"ok"}`)(w, r) + }, + }) + cfg := buildConfig( + config.Backend{Name: "embedding"}, + config.Backend{Name: "reranker"}, + config.Backend{Name: "whisper", BaseURL: backend.URL, Enabled: true, Timeout: 5 * time.Second}, + ) + router := buildRouter(t, cfg) + + req := httptest.NewRequest(http.MethodPost, "/v1/audio/transcriptions", strings.NewReader(body)) + req.Header.Set("Content-Type", "multipart/form-data; boundary=b") + w := httptest.NewRecorder() + router.ServeHTTP(w, req) + + if w.Code != http.StatusOK { + t.Fatalf("status = %d, want 200", w.Code) + } + if gotBody != body { + t.Errorf("backend body differs from client body:\n got: %q\nwant: %q", gotBody, body) + } + if gotCT != "multipart/form-data; boundary=b" { + t.Errorf("Content-Type = %q, want unchanged", gotCT) + } +} + func TestModelsAggregation(t *testing.T) { embedBackend := mockBackend(t, map[string]http.HandlerFunc{ "/v1/models": jsonOK(modelsJSON("BAAI/bge-m3")),