[tanishqbaweja]

Summary

Fixes #3830

Fsmn_vad (runtime/python/onnxruntime/funasr_onnx/vad_bin.py) takes str, np.ndarray, List[str] input (README), but for a list of files it put the segments of all files into the first result list, and batch_size above 1 failed. __call__ started from segments = [[]] * self.batch_size and added the segments of each batch to segments[batch_num] for every batch_num below batch_size. With the default batch_size=1, each file is its own batch, so the segments of every file were added to segments[0].

With the files of the new test (a: 3 s, noise at 0.5-1.5 s; b: 4.5 s, noise at 1-2 s and 3-3.5 s; silence: 2 s):

Call	main	This PR
model(a)	[[[220, 1660]]]	unchanged
model(b)	[[[720, 2160], [2720, 3660]]]	unchanged
model([a, b])	[[[220, 1660], [720, 2160], [2720, 3660]]]	[[[220, 1660]], [[720, 2160], [2720, 3660]]]
model([silence, b])	[[[720, 2160], [2720, 3660]]]	[[], [[720, 2160], [2720, 3660]]]

On main, b's times start again from 0 after a's last segment, and in the last call b's segments are reported for the silent file while b gets no entry. Code that pairs the inputs with the results attributes every segment to the first file. Paraformer and SenseVoiceSmall in the same package return one result per input.

batch_size above 1 failed on main in two ways (numpy 1.26.4):

• One file: E2EVadModel returns one segment list per waveform, and only when it found segments, but the loop read segments_part[batch_num] for every batch_num below batch_size. With batch_size=2, model(a) raised IndexError: list index out of range, and model(silence) returned two aliased empty lists, [[], []].
• Two or more files in a batch: the call failed before the scorer. np.array(waveform) raised ValueError (inhomogeneous shape) for files of different lengths, such as [a, b], and int(feats_len) raised TypeError: only length-1 arrays can be converted to Python scalars for files of equal length, such as [a, a]. Past those lines, E2EVadModel takes one waveform per call (assert len(self.scores) == 1, "only support batch_size = 1 now"), and prepare_cache builds the FSMN cache with a batch dimension of 1.

This PR scores the inputs one at a time (for beg_idx in range(0, waveform_nums) with end_idx = beg_idx + 1), creates one list per input (segments = [[] for _ in range(waveform_nums)]) and adds each input's segments to its own list (segments[beg_idx] += segments_part[0]). Each batch already started from a new E2EVadModel and the initial FSMN cache, so now each input does, and a file in a list gets the same segments as on its own. A single input still returns [segments], and with the default batch_size=1 the loop runs the same batches as on main.

Tests

tests/test_funasr_onnx_vad_multiple_inputs.py builds Fsmn_vad with __new__ (no model files): a real WavFrontend with the frontend_conf of funasr/models/fsmn_vad_streaming/template.yaml and no CMVN, the real E2EVadModel, and infer replaced by a stub that scores loud frames as speech, with the same scoring as the stub of tests/test_funasr_onnx_vad_online_short_chunks.py. It writes the three files above to a temporary directory.

• test_single_input: a path or an array of each file returns [segments]. This passes on main too.
• test_list_input: [], [a, b], [b, a], [silence, b] and [a, silence, b] return one segment list per file, in input order.
• test_batch_size_above_one: with batch_size 2 and 3, each file on its own and the lists [a, b], [a, a], [silence, b] and [a, silence, b] return the batch_size=1 results.

I added the file to test-funasr-onnx-package.yml (both path filters, and a step after the streaming VAD check), so CI runs it against the built wheel.

Type of change

• Bug fix
• Documentation
• Example or demo
• Runtime or deployment
• Benchmark or evaluation
• Model/training change

Validation

• python -m compileall funasr examples tests
• Docs or links checked
• Runtime/deployment command tested

Windows, Python 3.11.17 and 3.12.15, numpy 1.26.4, onnxruntime 1.31.1, kaldi-native-fbank 1.22.5, librosa 0.11.0, soundfile 0.14.0. I built the wheel from runtime/python/onnxruntime at main (f122f6c) and the wheel and the sdist on this branch, installed them into Python 3.11 and 3.12 venvs and ran the test steps of the workflow:

• python tests/test_funasr_onnx_vad_multiple_inputs.py:
  • main: FAILED (failures=7, errors=12) on 3.11 and 3.12. test_single_input passes. The five list cases fail as in the table ([] gives [[]]). With batch_size 2 and 3, a and b raise the IndexError, silence returns [[], []] and [[], [], []], and the four lists raise the ValueError or, for [a, a], the TypeError.
  • This branch: Ran 3 tests ... OK on 3.11 and 3.12, from the wheel and from the sdist.
• The other test steps pass on this branch: tests/test_funasr_onnx_installed.py (6 tests, from the wheel and from the sdist), tests/test_funasr_onnx_paraformer_online_overlap.py (3), tests/test_funasr_onnx_paraformer_online_empty_final.py (12), tests/test_funasr_onnx_vad_online_short_chunks.py (6), the two rich transcription test files (36 passed), twine check, pip check and the workflow's clean-install check, which asserts version 0.4.5. tests/test_funasr_onnx_release.py (8 tests) passes with PYTHONUTF8=1. Without it, test_readme_names_the_standalone_fix_release cannot decode README.md here (read_text() uses the Windows locale encoding, cp1252); this PR changes neither that test nor README.md.
• black --line-length=100 (24.4.0, the pre-commit pin) leaves the new test unchanged. On vad_bin.py it reports the same existing formatting as on main, none of it in the changed lines. git diff --check is clean.

I did not run a real FSMN-VAD ONNX model. The stub replaces only the session call.

User impact

Users of funasr_onnx.Fsmn_vad who pass a list of files, as the README documents, get one segment list per file, in input order, instead of the segments of every file in the first list. With batch_size above 1, a call returns the same segments as with batch_size=1 instead of raising IndexError, ValueError or TypeError, or returning batch_size empty lists for a file without speech. Results for a single input with the default batch_size=1 are unchanged.

Notes for reviewers

• Fsmn_vad no longer uses batch_size: it scores one input at a time, which is all that E2EVadModel and prepare_cache support. On main, no call with batch_size above 1 returned a segment. The argument is still accepted, and I left the README's batch_size line for Fsmn_vad as it is.
• Blame points at 861147c, which re-indented the file. The aggregation came with 435a590 (April 2023), which added the loop over the inputs. Fsmn_vad支持多线程并发调用 #2519 moved the scorer into __call__ for concurrent calls and kept the aggregation.
• model([]) now returns [] instead of [[]], as Paraformer and SenseVoiceSmall do for an empty list.
• The package version stays at 0.4.5, which the workflow checks.

[LauraGPT]

Reviewed the three changed files at 76b5189e0afb62e83131f3a0fe1c2c23530a4217. Per-input result allocation and indexing fix the attribution bug; sequential scoring matches the existing single-waveform scorer/cache contract. I also checked ordering, silent entries, duplicate inputs and repeated calls on the same instance.

Independent validation:

• The submitted regression reproduced on the current main: two methods failed and the single-input control passed.
• All 74 selected CPU/package regressions now pass. The first audit export omitted three unchanged support files, yielding 49 passes and 25 missing-file failures/errors. After recovering those exact Git blobs, only those 25 cases were rerun and passed. This was an audit-directory error, not a product fix or a hidden test exclusion.
• 42 real L20Z/CUDA contract cases: main failed 36; this head passed all 42. FP32, requested batch sizes 1/2/3, effective model batch 1. Actual ONNX Runtime profiles show 396 CUDA and zero CPU node events. The graph is an explicitly synthetic analytic scoring/cache fixture reused from prior validation, with actual wrapper/frontend/state machine; no pretrained weights or downloads. This does not establish pretrained VAD/ASR quality, Windows/NPU compatibility or a full-suite pass.
• Exact-head CI succeeded; actual Python 3.11/3.12 wheel/sdist source bytes and test logs, plus Python 3.10/3.12 runtime-evaluation artifacts, were inspected.

This is a source fix only. Existing official funasr-onnx 0.4.5 bytes are not overwritten and do not gain this fix merely because the PR merges. The next package release must use a new version. No protection or CI bypass.