Skip to content

whisper: align no_speech fallback logic with upstream#1430

Open
AdityaBiranje wants to merge 1 commit into
ml-explore:mainfrom
AdityaBiranje:fix/whisper-no-speech-fallback
Open

whisper: align no_speech fallback logic with upstream#1430
AdityaBiranje wants to merge 1 commit into
ml-explore:mainfrom
AdityaBiranje:fix/whisper-no-speech-fallback

Conversation

@AdityaBiranje

Copy link
Copy Markdown

Summary

Fixes #1427

decode_with_fallback in mlx_whisper/transcribe.py suppressed fallback
decoding whenever no_speech_prob exceeded no_speech_threshold, regardless
of avg_logprob. Upstream openai/whisper only suppresses fallback when
the segment looks like silence by both measures — high no_speech_prob
and low avg_logprob.

Because the MLX condition was broader, a segment that triggered fallback for
an unrelated reason (e.g. high compression ratio / repetitive output) could
still get marked "silence" and have fallback wrongly cancelled, letting
low-quality decodes through undetected.

Change

whisper/mlx_whisper/transcribe.py: added the missing logprob_threshold
/ avg_logprob check to the "silence" condition, matching upstream:

if (
    no_speech_threshold is not None
    and decode_result.no_speech_prob > no_speech_threshold
    and logprob_threshold is not None
    and decode_result.avg_logprob < logprob_threshold
):
    needs_fallback = False  # silence

I extracted this into a small _needs_fallback helper so the logic could be
unit tested directly, rather than only through the decode_with_fallback
closure.

Testing

  • Added a unit test covering the case that previously broke: high
    no_speech_prob with low compression ratio/logprob threshold still
    triggering fallback correctly.
  • Ran the existing test suite in whisper/tests/ — all passing.
  • Manually verified on a short local audio sample that decode behavior
    matches upstream openai/whisper on the same clip.

Notes

No public API changes. This only affects the internal fallback-suppression
condition and should make behavior more consistent with upstream.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

whisper: align no_speech_threshold fallback condition with openai/whisper

1 participant