Skip to content

fix(runtime): repair evaluation CLIs and honor GPU device selection - #3813

Merged
LauraGPT merged 1 commit into
mainfrom
fix/runtime-eval-devices-20261101
Nov 1, 2026
Merged

LauraGPT merged 1 commit into
mainfrom
fix/runtime-eval-devices-20261101

Conversation

@LauraGPT

@LauraGPT LauraGPT commented Nov 1, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Keep test_cer.py, test_rtf.py and test_rtf_gpu.py import-safe, with lazy
    funasr_onnx / funasr_torch imports instead of the nonexistent
    funasr.runtime namespace. Existing script paths and boolean aliases remain.
  • Pass the requested device to inference. The batched GPU tool defaults to
    CUDA0 and refuses ONNX CPU fallback. TorchScript model buffers and input
    tensors now use the same explicit device.
  • Use actual audio duration for warmup/total RTF, synchronize Torch CUDA timing,
    preserve spaces in SCP paths, reject malformed/empty input and missing
    inference rows, and write space-delimited transcript token files.
  • Add offline CLI/native regression tests, Python3.10/3.12 CPU CI and usage notes.

Actual Validation

  • CLI boundary RED28fail ->28pass; compatibility extension RED2fail/4controls.
    Final focused run: 44passed, zero skips (38CLI +6real native CPU cases).
  • Native CPU baseline:2controls passed. Real L20Z GPU RED: Torch buffers were
    CPU despite device0; ONNX control passed. After repair: 6native GPU cases
    passed
    , covering real frontend/decoder and transcript/serial RTF subprocesses.
  • Additional real batched GPU CLI, batch2 with one warmup, uses fixed synthetic
    TorchScript/ONNX weights and generated 1.0s+0.5s audio. Torch profiler records
    6CUDA kernels; ORT records 10CUDA nodes /0CPU nodes, including MatMul.
    Raw transcripts, source/fixture hashes, device/dependencies and profiles are
    retained on shared storage. Float32 tensors use existing TF32 settings.
  • Base bare offline pytest: one collection error and two skips remain.
    tests/test_funasr_onnx_paraformer_online_empty_final.py cannot import the
    separately installed funasr_onnx. No test exclusion or skip was added to
    hide this; the three runtime-tool collection failures are removed.

Synthetic graphs prove runtime/device/CLI mechanics, not pretrained ASR
accuracy, leaderboard results, BF16 behavior, all timestamp variants, multiple
physical GPUs or performance improvements. No new weights were downloaded.
No Nano work or release is included. The old libtorch setup.py package-selection
issue is separate; README documents using checkout modules explicitly.

Reviewed in a separate author self-review, not independent GitHub/human review.
Prepared with OpenAI Codex assistance. Normal repository protections still apply.

Keep runtime evaluation import-safe and use the requested inference device.

Signed-off-by: LauraGPT <zhifu.gzf@alibaba-inc.com>
@LauraGPT
LauraGPT merged commit 91510db into main Nov 1, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant