[tanishqbaweja]

Summary

Fixes #3837

ContextualParaformer.__call__ (inherited by SeacoParaformer) computes the hotword embedding once per call, as a [num_hotwords, 512] array, but on main the batch loop overwrites it with its batched copy:

bias_embed = np.expand_dims(bias_embed, axis=0)
bias_embed = np.repeat(bias_embed, feats.shape[0], axis=0)

The first batch gets [batch, num_hotwords, 512], and each later batch one more dimension. So when a call has more files than batch_size:

• funasr_onnx: ONNX Runtime rejects the second batch (Invalid rank for input: bias_embed Got: 4 Expected: 3), and the except ONNXRuntimeError branch logs input wav is silence or noise and adds no result. With the default batch_size=1, model([a, b, c], hotwords) returns only a's result; with batch_size=2, a's and b's. In a plain script the warning is not even shown, because the package calls logging.basicConfig(level=logging.ERROR).
• funasr_torch: the same loop (torch.unsqueeze(bias_embed, 0).repeat(...)) raises RuntimeError: Number of dimensions of repeat dims can not be smaller than number of dimensions of tensor on the second batch.

The README documents str, np.ndarray, List[str] input, and both funasr_onnx demos (demo_contextual_paraformer.py, demo_seaco_paraformer.py) pass their wav as a one-item list; adding a second file to either demo hits this bug.

This builds the batched copy in a new variable, batch_bias_embed, in both packages (also in the .cuda() branch of funasr_torch), so every batch gets [batch, num_hotwords, 512].

tests/test_runtime_eval_native.py gets a second module fixture: a tiny embedder (hotword -> hw_embed) and backbone (speech, speech_lengths, bias_embed -> logits, token_num) with the input names and layouts of the hotword exports, exported to ONNX and traced to TorchScript, plus model files and three wavs (0.25, 0.5 and 0.75 s, which decode to 4, 11 and 17 tokens). test_hotword_wrappers_decode_every_batch runs ContextualParaformer and SeacoParaformer from both packages with batch_size 1 and 2 through the real frontend, ONNX Runtime and TorchScript, and checks that one call with all three files returns the results of one call per file. test-runtime-eval.yml already runs this file for changes under both runtime packages.

Type of change

• Bug fix
• Documentation
• Example or demo
• Runtime or deployment
• Benchmark or evaluation
• Model/training change

Validation

• python -m compileall funasr examples tests
• Docs or links checked
• Runtime/deployment command tested

Windows 11, CPU, in two environments: Python 3.11.17 with torch 2.10.0+cpu and numpy 2.4.6, and Python 3.12.15 with the workflow's pins (torch 2.8.0+cpu, numpy 1.26.4); both with onnx 1.19.0, onnxruntime 1.23.2 and kaldi-native-fbank 1.22.5.

• python -m pytest -q tests/test_runtime_eval_native.py -k hotword against main (1125860): 8 failed in both environments. The 4 Torch cases raise the RuntimeError above, and the 4 ONNX cases get 1 (batch_size=1) or 2 (batch_size=2) of the 3 results.
• The workflow's command, python -m pytest -q tests/test_runtime_eval_cli.py tests/test_runtime_eval_native.py, on this branch: 52 passed in both environments.
• The funasr-onnx wheel built from this branch, installed into a fresh Python 3.12 venv without torch (onnxruntime 1.31.1): twine check and the five test scripts the workflow runs against the installed wheel (test_funasr_onnx_installed.py, ..._paraformer_online_overlap.py, ..._paraformer_online_empty_final.py, ..._vad_online_short_chunks.py, ..._vad_multiple_inputs.py) pass, and its ContextualParaformer and SeacoParaformer return 3 results for 3 files with batch_size 1 and 2.
• black --line-length=100 (24.4.0, the pre-commit pin) leaves the test unchanged and reports nothing in the changed lines of the two wrappers. git diff --check is clean.

User impact

Users of the hotword models through funasr_onnx who pass more files than batch_size in one call, such as a list of paths with the default batch_size=1, get one result per file instead of silently losing every batch after the first. funasr_torch users get results instead of a RuntimeError. Calls that fit in one batch are unchanged.

Notes for reviewers

• Synthetic graphs with fixed weights, not pretrained models: the test checks batching through the real wrappers, not recognition quality. The fixture backbone returns logits and token_num only, so the timestamp outputs of the SeACo export are not exercised.
• The new test runs on the CPU and does not read FUNASR_TEST_DEVICE: on CUDA, the Torch hotword wrapper moves its inputs to the GPU but loads the TorchScript modules without map_location, so the CPU-traced fixture would get CUDA inputs.
• Not changed: a batch that does fail in ONNX Runtime still adds no entry (the except ONNXRuntimeError branch).

[LauraGPT]

Reviewed all three changed files at fe89f75d7113c2eb8e29f623adfe7764023274d7. Keeping the unbatched hotword embedding separate prevents rank growth across batches in both runtime wrappers, including the CUDA branch; the inherited Seaco call follows the same path.

Independent validation against current main and the exact potential merge tree:

• Main reproduces all eight submitted multi-batch failures; candidate passes all 52 selected runtime/CLI tests with no skips.
• Real L20Z FP32 checks pass 48 cases across ONNX/TorchScript, both wrapper classes, batch sizes 1/2/3, normal/reversed/repeated file lists and empty lists. Actual execution records 1850 Torch CUDA events and 590 ONNX CUDA nodes, with zero ONNX CPU nodes. These are retained analytic fixed-weight fixtures with the real frontend, not pretrained recognition-quality or SeACo timestamp-output validation; TorchScript fixture buffers were serialized on CUDA for the existing loader.
• Exact-head hosted ONNX-package, Torch-package and runtime-eval CI passed; downloaded wheel/sdist module bytes match the reviewed source, and both runtime CI artifacts contain 52 passing tests including all eight new regressions.

Negative validation history is retained: the first local interpreter lacked ONNX Runtime (eight skips, not acceptance), then the existing runtime environment was used. A baseline GPU profiler filename collision corrupted its evidence and crashed cleanup; that run is not counted as passed. After unique per-session paths were added to the private validator, the previously unrun candidate GPU checks passed. CPU tests were not replayed.

Scope: source/runtime batching fix only. Existing official funasr-onnx 0.4.6 and funasr-torch 0.1.4 do not acquire this change by merging. No Windows/NPU/full-suite claim, new model download, protection bypass or production change.