Skip to content

fix(dataloader): track distributed progress by global batch - #204

Open
chen2021673 wants to merge 1 commit into
masterfrom
fix/dataloader-global-batches
Open

fix(dataloader): track distributed progress by global batch#204
chen2021673 wants to merge 1 commit into
masterfrom
fix/dataloader-global-batches

Conversation

@chen2021673

Copy link
Copy Markdown
Contributor

概述

修复分布式训练中 DataLoader 使用局部 batch 进行数据划分和索引,可能导致 batch 访问越界的问题。

主要修改

  • 统一使用 global batch 作为分布式数据划分和读取单位。
  • 按 data parallel rank 计算各进程应读取的数据范围。
  • 增加 DataLoader 单元测试,覆盖多 rank 下的 batch 划分和边界行为。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant