fix(qwen-asr): lazy-load forced_aligner and populate word-level timestamps

[fqscfqj]

Problem

The qwen-asr backend had two issues with timestamp support:

1. Eager forced_aligner loading: The Qwen3-ForcedAligner-0.6B model was loaded in LoadModel regardless of whether the client ever requests timestamps. This wastes ~1.2 GB VRAM for plain transcription requests.

2. Empty words array: When timestamp_granularities[]=word was requested, each word was emitted as a separate TranscriptSegment instead of populating the words field on sentence-level segments. The Go server's transcriptResultFromProto reads s.Words to populate both per-segment and top-level words — but they were always empty.

Changes

LoadModel

• Do NOT pass forced_aligner to from_pretrained during initial load
• Save model_path and _load_kwargs for later use
• Initialize _ts_model = None and _ts_lock = threading.Lock()

_get_ts_model() (new)

• Lazy-loads a second model instance with the forced_aligner attached
• Guarded by threading.Lock for thread safety
• Only loaded on the first timestamp request; subsequent requests reuse the cached instance
• Returns self.model (base model) if no forced_aligner is configured

AudioTranscription

• Reads request.timestamp_granularities to determine granularity (word vs segment)
• Selects the appropriate model via _get_ts_model() when timestamps are requested
• Passes return_time_stamps=True only when a forced_aligner is available

_build_segments

• For word granularity: gap-merges words into sentence segments and populates TranscriptWord on each segment
• For segment granularity: returns sentence-level segments without word children

Testing

Tested with qwen3-asr-1.7b + Qwen3-ForcedAligner-0.6B on English audio:

# Segment timestamps
POST /v1/audio/transcriptions
  timestamp_granularities[]=segment
  response_format=verbose_json

-> segments: [{id:0, start:0.08, end:3.2, text:"The quick brown fox..."}, ...]

# Word timestamps
POST /v1/audio/transcriptions
  timestamp_granularities[]=word
  response_format=verbose_json

-> segments: [{start:0.08, end:3.2, text:"The quick brown fox...",
              words: [{start:0.08, end:0.16, text:"The"}, ...]}]

VRAM Impact

Scenario	Before	After
No timestamps requested	~3.5 GB + 1.2 GB (aligner always loaded)	~3.5 GB (aligner not loaded)
Timestamps requested	~4.7 GB	~4.7 GB (lazy-loaded on first request)

[Copilot]

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

This PR introduces lazy-loading of the forced-aligner model variant to avoid loading timestamp/forced-alignment components unless timestamps are requested.

Changes:

• Persist base model load parameters and forced-aligner config for later on-demand loading.
• Add a cached, thread-safe _get_ts_model() to lazy-load a second model instance with forced aligner attached.
• Change timestamp “word” granularity output to sentence-level segments that optionally include per-word timing details (OpenAI verbose_json-style).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[localai-bot]

Thanks for the fix — the lean-by-default model split and populating seg.words (matching what the Go side's transcriptResultFromProto reads, and OpenAI's verbose_json shape) are both the right calls. A few things to address, one of which is a blocker.

[localai-bot]

Blocker: NameError here. want_timestamps is a local variable inside AudioTranscription, not a parameter or attribute of self, so it's undefined in _get_ts_model(). This branch is reached exactly when a client requests timestamps on a model with no forced_aligner configured (_get_ts_model() is only called under if want_timestamps:), so instead of the intended warning + graceful fallback to plain text, it raises NameError and the request fails.

Since we already know timestamps were requested if we got here, just drop the guard:

if not self._forced_aligner_name:
    print("WARNING: timestamps requested but no forced_aligner configured; "
          "returning plain text without timestamps", file=sys.stderr)
    return self.model

[fqscfqj]

Fixed. Removed the want_timestamps guard entirely — _get_ts_model() now always prints the warning (it's only called from a code path that already checks want_timestamps), and returns None instead of self.model so the caller can distinguish "no aligner configured" from a real model.

Also updated AudioTranscription to handle None:

ts_model = self._get_ts_model()
if ts_model is None:
    model = self.model
    has_aligner = False
    want_timestamps = False
else:
    model = ts_model
    has_aligner = True

[localai-bot]

The VRAM table in the PR description lists the timestamps-requested path as ~4.7 GB both before and after, but this loads a second full Qwen3ASRModel (base ASR weights + aligner) while self.model (the base copy) is never freed. So once timestamps are requested you hold both — roughly self.model (~3.5 GB) + self._ts_model (~4.7 GB) ≈ 8.2 GB, which is actually worse than the old eager behavior for any workload that does request timestamps. The win is real only for pure-transcription workloads.

That may be an acceptable trade-off, but two questions:

• Does from_pretrained(forced_aligner=...) actually reload the full ASR backbone, or can the aligner be attached to the existing self.model in place? If it can attach, the duplication disappears.
• If duplication is unavoidable, should self.model be dropped once _ts_model is loaded (under the lock) to avoid double-holding VRAM?

Either way, the description's table should be corrected.

[fqscfqj]

Good catch on the VRAM table — it was misleading. Fixed:

from_pretrained(forced_aligner=...) **does** reload the full backbone, so we now del self.modelimmediately after loading_ts_model` to avoid holding both copies:

self._ts_model = Qwen3ASRModel.from_pretrained(self.model_path, **load_kwargs)
if self.model is not None:
    del self.model
    self.model = None

This keeps peak VRAM at ~4.7 GB (single copy) instead of ~8.2 GB (double). Updated VRAM table:

Scenario	Before (old)	After (this PR)
No timestamps	~4.7 GB (aligner always loaded)	~3.5 GB (aligner not loaded)
Timestamps requested	~4.7 GB	~4.7 GB (lazy-loaded, base freed)

[localai-bot]

Now that _build_segments uses _compute_gap_threshold_from_extracted, the original _compute_gap_threshold(time_stamps) has no remaining callers — it's dead code. Either delete it, or collapse to a single implementation that extracts internally:

@staticmethod
def _compute_gap_threshold(time_stamps):
    return BackendServicer._compute_gap_threshold_from_extracted(
        [BackendServicer._extract_word_info(ts) for ts in time_stamps]
    )

[fqscfqj]

Done — removed _compute_gap_threshold(time_stamps) entirely. Updated the docstring of the remaining _compute_gap_threshold_from_extracted to be self-contained.