Repository navigation
[optimizer] add OrScale and OrScale-LM - #6571
Open
yuxuan-lou wants to merge 1 commit into
Open
yuxuan-lou wants to merge 1 commit into
yuxuan-lou wants to merge 1 commit into
Conversation
Add colossalai.nn.optimizer.OrScale: Muon's orthogonalized update scaled by a layer-wise trust ratio, with the weight decay coupled into the scaled step (arXiv:2605.07815). It subclasses Muon, so it has the same built-in AdamW branch for non-matrix parameters and runs under TorchDDP and LowLevelZeRO. The arithmetic follows the official implementation (NUS-HPC-AI-Lab/OrScale @ a45fc2b) statement by statement: accumulated float32 momentum, bf16 Polar Express orthogonalization, float32 norms. Run eagerly, the official code and this one give the same bits on CPU and CUDA. The official code compiles its orthogonalization with torch.compile on CUDA, which alone moves the result by 1-3%, as much as bf16 moves it from a float64 iteration. - orscale.py: OrScale with variant "orscale" or "orscale_lm", and polar_express. OrScale-LM's per-layer constant is calibrated at the first step and kept in the state as a float, which LowLevelZeRO's checkpoints carry. Under ZeRO each rank sums the squares of its slice and one all-reduce per bucket gives the two norms of the trust ratio, so a sharded step equals the unsharded one up to float32 rounding, not bit for bit. The ablation variants and the flattening of convolution weights of the official code are left out. - muon.py: the orthogonalization and the application of the update are methods a subclass overrides; no change in behavior. - gemini_plugin.py, torch_fsdp_plugin.py: the error names the optimizer class. - orscale_reference.py: excerpt of the official code (MIT) as the reference of the tests. - test_orscale.py: Polar Express and 3 steps of both variants against the reference under check_optimizer, with clipped and unclipped trust ratios and a zero-initialized layer; one optimizer against the official pair (OrScale + AdamW); wrong hyperparameters, decoupled weight decay and Muon's averaged momentum must fail. Grouping, state_dict round trip, error cases, 2-GPU TorchDDP. - test_dist_muon.py: OrScale on hand-made shards for 2 and 3 ranks and under LowLevelZeRO (2 ranks against 1 after one step within the float32 bound, 4 ranks within the bf16 bound, resume bit for bit with sharded and unsharded optimizer checkpoints). A trust ratio taken from the local slice must fail. - run_cpu_unit_tests.sh: run test_orscale.py in the CPU job. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
4 of 7 tasks
4 of 7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
1. Related issues
No issue yet. This is the third step of the Muon line: OrScale and OrScale-LM.
It is stacked on #6521: the base branch is
feat/muon-optimizer, so the diff shows this step only. The step it builds on, Muon under LowLevelZeRO (#6570), has been merged into that branch.The PR gates (
ci_layer_1_pr.yml,ci_layer_2_gpu_pr.yml,fork_pr_checks.yml) run for pull requests tomainonly, so they do not run here until this PR is retargeted tomainafter #6521 merges. Section 6 has the local runs on a merge withmain.2. Background
OrScale (arXiv:2605.07815, official code NUS-HPC-AI-Lab/OrScale, MIT) scales Muon's orthogonalized update by a layer-wise trust ratio and couples the weight decay into the scaled step. For a matrix W with Nesterov momentum M, Q = PolarExpress(M) and a shape scale s:
The official implementation uses two optimizers (OrScale for the matrices, AdamW for the rest) and its own ZeRO-1 wrapper. A Booster takes one optimizer, and LowLevelZeRO shards every parameter into flat slices.
3. Changes
colossalai/nn/optimizer/orscale.py:OrScalesubclassesMuon, so it has the same built-in AdamW branch, takesmuon_param_groups(model), and runs underTorchDDPPluginandLowLevelZeroPlugin.variantis"orscale"or"orscale_lm".polar_express), float32 norms.c_denomis calibrated at each layer's first step and kept in the optimizer state as a Python float. A 0-dim tensor would not do: LowLevelZeRO's checkpoints treat every tensor in the state as sharded.colossalai/nn/optimizer/muon.py: the orthogonalization and the application of the update become methods a subclass overrides. No change in behavior.colossalai/booster/plugin/gemini_plugin.py,torch_fsdp_plugin.py: the error atboostnames the optimizer class, so it reads correctly for OrScale too.tests/test_optimizer/orscale_reference.py: an excerpt of the official code, cut down to the two variants of the paper, with its MIT license notice. It is the reference of the tests.tests/test_optimizer/test_orscale.py, usingtests/kit/numerics:polar_expressagainst the official orthogonalization, with the official iteration in float64 as the truth; Muon's Newton–Schulz must fail the same check. Independently of the official code, the result must be within 10% of U Vᵀ from an SVD.check_optimizer, on CPU and CUDA: with and without weight decay, calibrated and givenc_denom. The weights are scaled so that the trust ratio is clipped from below, unclipped, and clipped from above, plus one zero-initialized layer.r_min0.6,r_max1.4,eps1e-3, 4 iterations, the other variant, ac_denomoff by 2%, weight decay decoupled from the trust ratio, and Muon's moving-average momentum.state_dictround trip (the calibrated constants included), invalid arguments, and 2-GPUTorchDDPPluginagainst one device.tests/test_optimizer/test_dist_muon.py:c_denomwithin the float32 relative bound.LowLevelZeroPluginwith 2 ranks against 1 after one step, within the float32 bound on the optimizer state (stage 1 fp32, stage 2 bf16, stage 2 bf16 with CPU offload); 4 ranks over 3 steps within the bf16 bound..github/scripts/ci/run_cpu_unit_tests.sh: runstest_orscale.pyin the CPU job.Not included: the ablation variants of the official code, its flattening of convolution weights (matrix groups take 2D parameters only),
torch.compileof the orthogonalization, Gemini / FSDP / tensor parallelism, convergence experiments, docs.4. User and compatibility impact
colossalai.nn.optimizer.OrScale. Nothing existing changes behavior; the Gemini / FSDP error message for Muon now starts with the class name.lr=0.02,weight_decay=0,momentum=0.95, trust ratio clipped to [0.5, 1.5], as in the official code. The defaultvariantis"orscale", while the official class defaults to"orscale_lm". The AdamW groups sharelrunless a group sets its own; the official default is a tenth of it.momentum_buffer(float32, accumulated, so it has the official scale and not that of Muon's buffer) and, for OrScale-LM, a floatc_denomper matrix..item()once per matrix and step, as in the official code. No throughput numbers in this PR.test_orscale.pyexcept the DDP test runs on torch 2.5.1 in the CPU job.tests/test_optimizer.5. Test environment
torch2.13-cu130-tf5.17.0nvidia-smiCUDA Version 13.2)torch.version.cuda13.06. Test commands and results
torch.optim.Muon, 23 need GPUs), exit code 0.Details:
ORSCALE_DISABLE_TORCH_COMPILE=1): the orthogonalization for 7 shapes up to 3072×768, and 5 steps of the optimizer (weights, momentum,c_denom) for both variants, weight decay 0 and 0.1, calibrated and givenc_denom, both clipping ranges, parameters in float32, float64 and bfloat16. The official code, the excerpt and this implementation give the same bits everywhere, on CPU and CUDA. This check is a one-off script, not part of the test suite.torch.compilewhen CUDA is available. Compiled, it differs from this implementation by 1.3–3.2% (relative Frobenius error of Q), which is the size of the bf16 error itself: against the float64 iteration the compiled official function is off by 0.9–2.1% and this one by 1.1–2.6%.c_denomwithin 9.9e-8 (bound 1.26e-6). With the trust ratio taken from the local slice the weights are 34–1011× outside the bound.c_denomwithin 7.1e-8. With the local trust ratio the two matrices are 31× and 4.6× outside the bound. 4 ranks over 3 steps: updates within 5.6e-7 (bf16 bound 8.29e-2).xfailor--ignore; the only skips are the torch-version and GPU-count conditions above.7. Known issues
main(section 1).tests/test_optimizer, not in the PR gate.8. Risk and rollback
Only code that builds
OrScaleis affected. The change tomuon.pymoves code into overridable methods;test_muon.pyand the Muon cases oftest_dist_muon.pypass unchanged. Rollback: revert this commit.9. Docs and release
No README, docs, changelog or
version.txtchanges. The plugin compatibility table ofdistributed_optimizers.mddoes not list Muon or OrScale yet.Checklist
🤖 Generated with Claude Code