Skip to content

[test] Benchmark README: batch functions take their size and random numbers from their arguments - #6568

Closed
JazyJiang wants to merge 1 commit into
mainfrom
docs/benchmark-batch-generator
Closed

JazyJiang wants to merge 1 commit into
mainfrom
docs/benchmark-batch-generator

Conversation

@JazyJiang

Copy link
Copy Markdown
Collaborator

Related issue

Follow-up to #6565; documentation only.

Background

A model's batch_fn is called by the maximum search with changing sizes and on every rank of a multi-device measurement with a differently seeded generator. The README did not say so, so a batch function written with a fixed size breaks the search, and one using the global random state gives every rank the same batch, which makes a MoE router route identically on every rank.

Changes

  • tests/kit/benchmark/README.md, "Adding a model": a paragraph that batch_fn builds the batch at the size it is given and takes its random numbers from generator, why, and that these functions go into the same pull request as the entry (or into config/data.py when several models can use one).

Not in this PR: code changes; the existing batch functions (causal_lm_batch, PackedText, LatentVideo) already follow this.

User and compatibility impact

None.

Test environment / Test commands and results

Documentation only; no tests run.

Known issues

None.

Risks and rollback

None; revert the commit.

Documentation and release impact

README only.

@JazyJiang

Copy link
Copy Markdown
Collaborator Author

Folded into #6566 as its second commit (unchanged), so the README note lands with the benchmark worker change. Closing this one.

@JazyJiang JazyJiang closed this Nov 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant