[ANDROIDHASSAN]

Problem

sentence_postprocess collects a BPE word in word_item until a token without @@ closes it. When the token list ends on a @@ piece, nothing closes it and the piece is dropped:

>>> sentence_postprocess(["hello", "wor@@"])
('hello', ['hello'])
>>> sentence_postprocess(["你", "好", "wor@@"])
('你好', ['你', '好'])

Streaming Paraformer hits this every time a word spans two chunks: paraformer_streaming/model.py and funasr_onnx/paraformer_online_bin.py postprocess each chunk on its own, so the first part of that word disappears from the transcript.

Fix

After the loop, a non-empty word_item is added as a word, with the [begin, end] already tracked for it when timestamps are given. The same 6 lines go into all three copies of postprocess_utils.py (funasr, funasr_onnx, funasr_torch). Token lists that end on a complete word are unchanged.

Tests

Added to tests/test_sentence_postprocess_timestamps.py, over all three copies: ["hello", "wor@@"], ["你", "好", "wor@@"], and the timestamped ["hello", "wor@@", "l@@"] → ["hello", "worl"], [[0, 100], [100, 300]].

• Without the fix: 9 failed subtests (3 cases × 3 copies), each one on its own.
• With the fix, the text-postprocessing CI set (test_postprocess_hotwords.py, test_rich_transcription_postprocess.py, test_sentence_postprocess_timestamps.py): 26 passed, 6 skipped (the fuzzy hotword tests, pypinyin/rapidfuzz not installed here; same 6 skipped on main).

Scope

A @@ piece followed by a Chinese character in mixed text (["hel@@", "你"]) is still mishandled the way it is today; that is a different case and not changed here.