Skip to content

fix(paraformer): trim timestamp predictor padding per sample - #3554

Merged
LauraGPT merged 1 commit into
modelscope:mainfrom
John6666cat:fix/paraformer-timestamp-padding
Aug 30, 2026
Merged

fix(paraformer): trim timestamp predictor padding per sample#3554
LauraGPT merged 1 commit into
modelscope:mainfrom
John6666cat:fix/paraformer-timestamp-padding

Conversation

@John6666cat

Copy link
Copy Markdown
Contributor

Summary

Fix per-sample timestamp predictor padding handling in Paraformer dynamic batches.

When the predictor has tail_mask=True, timestamp inputs are now trimmed per sample using:

  • encoder_out_lens[i]
  • one additional tail frame when tail_threshold > 0
  • the available widths of both timestamp tensors as an upper bound

Behavior is unchanged when tail_mask=False, when the predictor has no tail_mask attribute, or when the tail threshold is zero.

A CPU-only, network-free regression test is included.

Type of change

  • Bug fix
  • Documentation
  • Example or demo
  • Runtime or deployment
  • Benchmark or evaluation
  • Model/training change

Validation

Focused regression test:

python -m pytest -q tests/test_paraformer_timestamp_padding.py

On the unmodified baseline:

3 failed, 2 passed

With this change:

5 passed

The tests cover:

  • per-sample trimming with tail_mask=True

  • unchanged behavior with tail_mask=False

  • unchanged behavior when tail_mask is absent

  • zero tail_threshold

  • clamping to both timestamp tensor widths

  • python -m compileall funasr examples tests

  • Docs or links checked

  • Runtime/deployment command tested

Also checked:

git diff --check

User impact

In dynamic batches, shorter samples can otherwise pass padded predictor frames into timestamp post-processing.

This change limits timestamp processing to each sample's valid encoder extent when tail masking is enabled, preventing padded tail frames from affecting generated timestamps while preserving the existing behavior for other predictor configurations.

Notes for reviewers

The scope is intentionally narrow and limited to Paraformer timestamp padding.

This is separate from the pred_timestamp / speaker-output contract fix handled in PR A.

No EParaformer, English/BPE, or broader timestamp flag normalization changes are included.

The regression test is CPU-only and requires no network access or model download.

@LauraGPT LauraGPT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reproduced the focused result (5 passed), but the two timestamp PRs are not currently green together. Applying #3553 then #3554 and running python -m pytest -q tests/test_paraformer_timestamp_contract.py tests/test_paraformer_timestamp_padding.py tests/test_punc_model_none.py gives 20 passed, 1 failed: #3553 constructs a lightweight Paraformer without predictor, while the new #3554 branch reads self.predictor directly and raises AttributeError. Please preserve the intended production behavior while making the access compatible, e.g. bind predictor = getattr(self, "predictor", None) before checking getattr(predictor, "tail_mask", None). Please add a combined regression (or extend the fixture) so both PRs remain green together. DCO is also missing on 947d7116eaffea2dde962a49d5251e02eb55eebb; amend with --signoff before the force-with-lease push.

@LauraGPT

Copy link
Copy Markdown
Collaborator

Fresh current-main revalidation confirms the existing requested change without adding scope. Against main@4300a6b2a158e126ff5d97ed55816b43ae88522a:

Environment: Python 3.12, NumPy 1.26.4, SciPy 1.12.0, Torch 2.11.0+cu130. The server-wide NumPy 2.x/SciPy ABI mismatch was isolated and is not attributed to this PR.

Please apply the minimal compatibility change on #3554 and amend both PR commits with your own Signed-off-by trailers. I will re-run the exact replacement heads; I will not add DCO on the contributor’s behalf.

@John6666cat
John6666cat force-pushed the fix/paraformer-timestamp-padding branch from 947d711 to 6042ace Compare August 30, 2026 08:26
@John6666cat

John6666cat commented Aug 30, 2026

Copy link
Copy Markdown
Contributor Author

Thanks. I applied the requested minimal compatibility change by binding predictor = getattr(self, "predictor", None) and using that value for the two checks. I also added coverage for the missing predictor attribute case, amended the commit with my DCO sign off, and force pushed the replacement head. New SHA: 6042ace.

@LauraGPT LauraGPT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The requested compatibility fix and missing-predictor regression are correct. I applied exact heads #3553 449414665d36953f3bd83e514c31125a228c0fe4 and #3554 6042ace22d617a69c47735503cb67be04c7825fd together on current main@55b662ccf9ea77237ba9253b3bddd953d4184f84; all 24 focused timestamp, padding, punctuation-none, and timestamp-tool tests pass. py_compile and git diff --check also pass.

The only remaining blocker is the trailer syntax. The commit currently contains:

Signed-off-by: John6666 186692226+John6666cat@users.noreply.github.com

DCO requires angle brackets:

Signed-off-by: John6666 <186692226+John6666cat@users.noreply.github.com>

Please ensure user.name and user.email are configured, then run git commit --amend --no-edit --signoff and git push --force-with-lease. Please verify the resulting message with git log -1 --format=%B. No code changes are requested.

@John6666cat
John6666cat force-pushed the fix/paraformer-timestamp-padding branch 2 times, most recently from f7d8545 to aa4375b Compare August 30, 2026 11:50
Signed-off-by: John6666 <186692226+John6666cat@users.noreply.github.com>
@John6666cat
John6666cat force-pushed the fix/paraformer-timestamp-padding branch from aa4375b to b2fb22d Compare August 30, 2026 11:50

@LauraGPT LauraGPT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Revalidated the exact cleaned replacement head b2fb22d1addcfa0284b3ca0c63bf2e02d02a6ad4 after #3553 on current main@917ca2f4a673b4ee8b0aea4b64fbe2d6c2034a67.

  • the accidental C# obj/Debug artifacts are absent; only the two intended files remain
  • the source tree is identical to the previously reviewed implementation
  • the corrected Signed-off-by trailer satisfies DCO
  • the combined #3553 + #3554 focused suite passes 24/24
  • py_compile and git diff --check pass

Approved for merge after #3553.

@LauraGPT
LauraGPT merged commit e5a1f4b into modelscope:main Aug 30, 2026
@John6666cat

Copy link
Copy Markdown
Contributor Author

Thanks for revalidating the cleaned replacement head and the combined suite. I appreciate the thorough review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants