[eksrha]

Closes #22

What

• tei-reranker-model-init now takes the model's own onnx/model.onnx when the repository ships one (copying only the files TEI needs) and runs the Optimum export only otherwise. MODEL_REVISION can pin the model revision. While testing I found that the repeated --exclude flags in the old hf download call replaced each other, so every excluded file was downloaded anyway; the patterns are now passed to a single --exclude. For the new default the resulting init image holds about 0.5 GB instead of about 1.5 GB.
• Chart default switches to cross-encoder/mmarco-mMiniLMv2-L12-H384-v1 (tag mmarco-mminilmv2-l12-h384-v1-2026.10). New value reranker.modelName is shown in NOTES.txt. BAAI/bge-reranker-v2-m3 stays selectable by setting the init image tag and modelName.
• release-models.yml builds both init images (matrix); only the default also gets :latest.
• Reranker chart defaults are sized from measurements for the small model: memory request 1536Mi / limit 2Gi (measured working set about 1.4 GiB, also under 4 concurrent requests), CPU request 2 / limit 4, --max-client-batch-size 32 added next to --max-batch-tokens 2048. The BGE model needs 4Gi/8Gi, noted in the values.
• README: p95 latency by passage count and length on a 4-vCPU pod (e.g. 0.9 s for 15 passages of 500 characters, 2.1 s for 40; about 3 to 4x higher with four concurrent requests since inference is CPU-bound), truncation behaviour at 512 tokens, licence of both models (Apache-2.0), quantised ONNX variants noted as not used.
• make docker/tei-reranker-model-init RERANKER_MODEL_ID=... selects the model locally.
• README: new "Reranker model choice" section with the measured latency and quality tables, caveats, and how to switch back. The stale FlagEmbedding rows were corrected.
• ADR-001: appended an addendum that corrects the latency statement (0.27 to 0.47 s was for 5 short pairs); the original text is untouched.

Verified

• helm lint and helm template render with the default and with the bge-reranker-v2-m3 tag/modelName override.
• Built the init image from the Containerfile with the pinned mmarco revision; it contains only onnx/model.onnx plus config and tokenizer files. Ran it against an empty volume, started TEI cpu-1.9.4 (4 CPUs, --max-batch-tokens 2048, offline) on that volume and POSTed /rerank with three German/English texts: HTTP 200, relevant text ranked first (0.997), irrelevant last (0.0002), about 0.07 to 0.15 s.
• go test -race ./... and golangci-lint run ./... pass (no Go code changed).

Not verified / notes for the reviewer

• The ONNX-export branch of the Containerfile (bge-reranker-v2-m3) was not rebuilt here; its code path is unchanged apart from the download flags and the final existence check.
• The new default tag does not exist in the registry until release-models.yml has run on this change (the tag carries the build month, as before), so merge, run the workflow, and check the tag before relying on a chart release.
• Quality claims come from a small, saturated relevance set and cannot separate the two models (see the issue). Latency ratios come from an older AVX2 CPU.
• Behaviour change: tei-reranker-model-init:latest now points at the small model.
• The init step copies with --ignore-existing, so switching models on an existing release needs the old reranker model PVC deleted (documented in the README).

[eksrha]

Follow-up measurement in a Kubernetes pod (4 CPU limit, same TEI cpu-1.9.4 image and arguments, 800-character documents, through a port-forward, 3 runs each for the small model, 1 run for the current default):

Documents	Small model	Current default
5	0.34 to 0.37 s	3.6 s
10	0.57 to 0.59 s	5.4 s
20	0.98 to 1.07 s	11.7 s
40 (2 x 20)	2.1 to 2.2 s	23.6 s

That is about 0.05 s per pair against about 0.55 s per pair, so roughly 11x. A sanity probe with 3 German and 3 English queries (one relevant document among 20) ranked the relevant document first every time. German scores were noticeably lower than English, so consumers should rank by order and not rely on absolute score thresholds. 20 documents fit a 1.5 s budget only narrowly; 10 to 15 documents leave comfortable headroom.