Skip to content

fix(diarization): honor configurable speaker caps for long recordings - #3794

Closed
phoenixray2000 wants to merge 2 commits into
modelscope:mainfrom
phoenixray2000:codex-configurable-speaker-cap
Closed

phoenixray2000 wants to merge 2 commits into
modelscope:mainfrom
phoenixray2000:codex-configurable-speaker-cap

Conversation

@phoenixray2000

@phoenixray2000 phoenixray2000 commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Automatic speaker counting switches from spectral clustering to UMAP/HDBSCAN at 2048 speaker embeddings. The density-based branch ignores the spectral speaker limit, so a configured maximum of 5 can still produce 8 speaker groups.

Expose a construction-time top-level AutoModel(..., max_num_spks=5) argument, defaulting to 15. Forward that value explicitly to ClusterBackend so both automatic clustering paths obey the same cap. max_num_spks inside spk_kwargs or spk_kwargs["cb_kwargs"] raises TypeError, including when a top-level value is also supplied. This leaves one AutoModel configuration entry point without alias or precedence rules. Other backend options, such as merge_thr, continue to use the existing component configuration.

Excess UMAP/HDBSCAN clusters are greedily merged by centroid cosine similarity. Noise labels are preserved and excluded from the count; pair selection excludes the diagonal and handles nonpositive similarities and zero centroids. An explicit preset_spk_num/oracle_num takes precedence on the known-count spectral and KMeans paths.

The default limit of 15 now also applies to automatic UMAP/HDBSCAN results. Callers can increase it for larger meetings. This is an upper bound, not an exact count: setting it below the actual number of speakers can merge different people. The English and Chinese SDK guides document the top-level construction argument, defaults, validation and examples.

Validation:

  • The 2048-embedding limit regression fails against the original implementation for caps 1, 3 and 5 with NumPy and Torch inputs. Top-level routing and nested-location rejection regressions also fail before the API change; all pass afterwards.
  • Core clustering and SDK documentation checks: 66 passed, 22 subtests passed. Coverage includes top-level caps 1/3/16, the default of 15, invalid values, both nested locations alone or alongside equal/conflicting top-level values, preservation of other backend options and caller dictionaries, known-count precedence, the 2047/2048 boundary, noise, closest-centroid selection and zero/nonpositive similarities. The existing real UMAP/HDBSCAN wrapper test also passes.
  • NumPy workflow selection plus SDK, component-device and timestamp regressions: 284 passed, 42 subtests passed, 1 deselected locally. The deselected MP3 decoder case reopens a still-open NamedTemporaryFile on Windows; it remains selected in Linux CI.
  • Black 24.4.0 with the repository's 100-character line length and git diff --check pass.
  • tests/test_cluster_backend.py is included in the path filters and test selection of the NumPy 1.26.4 / 2.4.0 CI matrix.

The existing product-site browser environment has a separate incompatibility: its ubuntu-latest runner uses Ubuntu 26.04 with Playwright 1.55.0. The previous browser run fails at Install Chromium, before browser tests run, with Error: ERROR: Playwright does not support chromium on ubuntu26.04-x64.

Latest head 7bec62b9763bee22c513a156ddfcc35c55d91774: NumPy 1.26.4, NumPy 2.4.0, KWS output, model contracts, and legacy document links pass (NumPy, KWS, model contracts). The current product-site build/validation is still in progress; its browser stage is pending.

LauraGPT added a commit that referenced this pull request Oct 31, 2026
Integrate the reviewed #3794 speaker-cap changes on current main, preserving the original fork and contributor credit.

Co-authored-by: phoenixray2000 <phoenixray2000@gmail.com>

Signed-off-by: LauraGPT <zhifu.gzf@alibaba-inc.com>
LauraGPT added a commit that referenced this pull request Oct 31, 2026
Integrate phoenixray2000's #3794 with contributor credit. Reviewed and validated by 66 focused CPU tests, synthetic CAM++ CUDA embeddings with actual native clustering, and new-head CI/artifacts. Not pretrained diarization accuracy, BF16, the full suite, or a release.

Signed-off-by: LauraGPT <zhifu.gzf@alibaba-inc.com>
@LauraGPT

Copy link
Copy Markdown
Collaborator

Thanks @phoenixray2000. Your seven reviewed speaker-cap changes are now integrated through #3809 on current main, with contributor credit in the signed+DCO commit. The original fork/head is unchanged. This PR is being closed as superseded, not described as a merged original commit.

Validation: the previous source reproduces 41 expected failures, while the candidate passes 66 focused CPU tests with no skips. Real L20Z execution used random-initialized reduced CAMPPlus parameters, FP32, batch 32, and 2048 unique synthetic feature rows. Actual CPU UMAP/HDBSCAN and spectral clustering exercised the cap; the CUDA profiler recorded 972 kernel events in its first batch. This is not pretrained diarization accuracy, BF16 or a full-suite claim.

The old browser failure was not bypassed or rerun. The integrated head and final main each passed their new CI, including browser/build validation and both NumPy versions. Actual evidence artifacts and logs were downloaded and checked against each exact commit, including all 50 cluster tests and six existing empty-batch regressions per NumPy matrix. Completed local experiments were reused only after full source/receipt hash verification. No new package release is claimed.

@LauraGPT LauraGPT closed this Oct 31, 2026
@LauraGPT LauraGPT mentioned this pull request Nov 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants