Skip to content

Prevent ranks from pinning to one CPU - #112

Open
michaelmckinsey1 wants to merge 2 commits into
LBANN:mainfrom
michaelmckinsey1:michaelmckinsey1-patch-17
Open

michaelmckinsey1 wants to merge 2 commits into
LBANN:mainfrom
michaelmckinsey1:michaelmckinsey1-patch-17

Conversation

@michaelmckinsey1

Copy link
Copy Markdown
Collaborator

The current run command in develop results in CPU processes running on one APU

image

Typically, this would be handled by mpibind on our systems (for flux must run with flux run --exclusive), but torchrun-hpc sets -ompibind=off, so I think that is why this occurs.

This setting improves performance for rocm7.2.1 by 11% and rocm7.14 by 47%

@michaelmckinsey1 michaelmckinsey1 self-assigned this Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant