Repository navigation
[AffineIR] Check affine holders and source regions without pairwise comparisons - #9011
Conversation
|
#9010 and #9011 carry the same code as the fork branches I read on Nov 1; One addition to the remaining-work list: for rank, held in sorted(by_rank.items()):
if boxes_disjoint([frame.local for frame in held]) and sum(frame.local.volume
for frame in held) == held[0].shard_numel:
continue
# ... existing budget check and pairwise walk ...I tested 8 ranks with interleaved rows, re-sharded to 8 row shards with 64 columns. Planning drops from 1920 ms to 190 ms at 4096 pieces, and from 148 ms to 29 ms at 1024. The affine tests still pass (581). Maps that pass also no longer hit the per-rank |
21324f4 to
bea2e09
Compare
|
Thanks for the review and for the Both #9010 and this PR are rebased onto current master. While looking for other pairwise checks, I found one in |
|
Thanks @jinyouzhi. I checked #9011 the same way as #9039, running
One observation, not blocking: LGTM otherwise. |
|
Thanks @Achyuthan-S, good catch. 12663c3 gives |
12663c3 to
82b30c7
Compare
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c91a7ba441
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
_validate_holders compared every piece with every other piece, and _sources compared every source region with every other region, so a map with 1024 pieces already exceeded the default max_pair_checks and planning time grew quadratically. When no rank repeats a box and distinct boxes do not overlap, a rank covers a box exactly when it holds that very box. The holders of each box can then be read from a table. Add boxes_disjoint to affine_geometry: it sweeps along the most varied axis and compares a box only with the boxes still open where it starts. Maps that fail the fast checks take the original pairwise walk, which names the offending piece, so error messages are unchanged. A randomized test checks that plans and errors match the pairwise walk, including replicas split at different points, overlapping pieces and wrong holder lists. Signed-off-by: Jin, Youzhi <youzhi.jin@intel.com>
A rank whose pieces sweep as disjoint in its shard and fill it passes _shards_tile with boxes_disjoint; any other rank takes the pairwise walk, so error messages are unchanged. Maps that pass no longer hit the per-rank len(held)**2 budget. Suggested-by: 0z5a Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Jin, Youzhi <youzhi.jin@intel.com>
A disjoint layout can keep every box open for the whole sweep, so boxes_disjoint was still quadratic without a budget of its own. Give it an optional max_checks: once exceeded it reports the boxes as not shown disjoint, and since the sweep compares at most n*(n-1)/2 pairs, the caller's pairwise fallback under the same budget refuses at once. The planner passes max_pair_checks at every call. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Jin, Youzhi <youzhi.jin@intel.com>
|
Thanks @jinyouzhi, 82b30c7 does it: On the Codex note: I'd keep the pairwise walk as the oracle, since it is what master checks, but compare the plans and the exception types rather than the message text, and check that every accepted plan reproduces |
|
@delock ready to review, Thanks! |
…pairwise walk Signed-off-by: Jin, Youzhi <youzhi.jin@intel.com>
1befaa5
Brings in deepspeedai#8997, deepspeedai#9011 and deepspeedai#9068; no overlap with this branch's files. Co-Authored-By: Claude Code <noreply@anthropic.com>
deepspeedai#9027) ## Why A replica policy (deepspeedai#8936) or cost model (deepspeedai#8252) needs to know what a batch will cost before running it: messages per peer, rounds, and staging memory. deepspeedai#8967 builds that report today by importing the executor's private `_packed_rounds` and `build_operations`. It then re-derives the in-place decision and the arena sizing, so its numbers would drift silently whenever the executor's staging changes. It asked for a public API instead. ## What `schedule_transfers(plans, endpoints, budget, element_size, process=None)` in `affine_transfer_executor` returns, for one process or for every process the endpoints name, the `ProcessSchedule` that `execute_transfers` would run. It uses no buffers and no communication: - the rounds per peer and direction (`ScheduledRound`: elements, chunks, and whether the round is sent or received in place); - the staging arena each peer direction allocates; - the local copies; - the round count that `TransferStats.rounds` reports. `execute_transfers` now builds its rounds and arenas from the same private computation, so the report and the run cannot disagree. The in-place decision is made from geometry rather than from the buffer. `_region` is an `as_strided` view of a packed shard, so it is contiguous exactly when `(chunk_shape, strides)` is row-major packed, ignoring extent-1 axes. Unchanged: - `plan_digest`, the batch fingerprint and `TransferStats`, so mixed versions still agree on the schedule; - `build_operations` and `_packed_rounds`, which keep their names and signatures until deepspeedai#8967 switches. ## Tests - `test_schedule_is_what_the_executor_runs` (2 and 4 ranks; mixed packed and strided regions with 16-element messages) checks: - rounds, sent, received and local-copy bytes against the executor's `TransferStats`; - that both sides of every message agree; - the arena sizes; - that the batch exercises both the in-place and the staged path. - `test_packed_regions_are_exactly_the_contiguous_views` compares the geometric rule with `Tensor.is_contiguous` over 2000 random shapes and strides, including extent-1 axes. - A single-process case covers local-only batches and the argument checks. - Forcing every round to be staged, or dropping the extent-1 exception, makes these tests fail. On macOS arm64 CPU (Python 3.12, torch 2.15.0): - The executor, external-group, planner and shard-map suites pass (263). - All other affine checkpoint tests pass, except the four `TestAffineUniversalCheckpointResume` cases, which fail the same way on master in this environment (uninstalled package version). - With deepspeedai#8967 merged on top, its analysis tests pass unchanged (59 together with the executor suite). - pre-commit is clean. This merges cleanly with deepspeedai#8936, deepspeedai#9010 and deepspeedai#9011. With deepspeedai#8966 and deepspeedai#8967 the only conflict is that both append a section to `affine-transfer.rst`. --------- Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de> Co-authored-by: Ma, Guokai <guokai.ma@gmail.com>
…ons (deepspeedai#9039) **Rebased on master** (PRs deepspeedai#9010 and deepspeedai#9011 have merged; this branch is rebased on master at `5dc44e8`, so the diff contains only this PR's changes). `ParamAffineMap._validate_destinations` (added in deepspeedai#8922) checks the box-shaped pieces of a column split by comparing every box of a rank with every other one. `validate()` runs whenever a ZeRO, AutoEP or block-gather map is built and before every conversion, and this pass has no budget, so validation grows quadratically with the number of strided pieces per rank. This PR sweeps the boxes with `boxes_disjoint` from deepspeedai#9011 first. Only a rank that fails the sweep takes the pairwise walk, which names the shared shard address, so error messages are unchanged. The second commit moves the remaining hand-written geometry onto `affine_geometry`: `_dest_box` returns a `LogicalBox` built with `coordinate()`, the shared address comes from `intersection()` and `dot()`, and `address_range()` / `overlaps_itself()` move there from `affine.py`. The transfer executor reuses `address_range()` for its buffer-reach check and `dot()` for chunk offsets. No behavior or message changes. `validate()` of 8 ranks with interleaved columns (64 rows; ms, single CPU process): | pieces per rank | before | after | |---|---|---| | 128 | 29 | 4 | | 512 | 441 | 17 | | 2048 | 6654 | 68 | Tests: - Randomized check over 200 seeds: column blocks of row bands tiling a shard, half with one block shifted onto another. `validate()` refuses exactly the maps whose pieces write some shard address twice (enumerated independently in the test), and every refusal names an address that is written twice. The test checks this contract only, so it does not depend on which helper does the sweep. - Mutation checks: making the sweep always report "disjoint" fails 48 of the 200 cases. Forcing every rank through the pairwise walk still passes all 200. - All `tests/unit/checkpoint/test_*affine*` files: 839 passed, 1 skipped on 4× NVIDIA L20 at f6e1f6c. At ac3a329 on CPU (`DS_ACCELERATOR=cpu`, 4 ranks): 842 passed, 5 skipped. ## AI assistance Implementation, tests and this description were drafted with GitHub Copilot CLI and reviewed/validated by the author. --------- Signed-off-by: Jin, Youzhi <youzhi.jin@intel.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Ma, Guokai <guokai.ma@gmail.com>
…epspeedai#9070) Follow-up to the column case in deepspeedai#9026. deepspeedai#9011, which this was stacked on, has merged; the branch is now on master. A ZeRO-3 partition cut at a tensor-parallel column edge holds boxes narrower than the parameter, and it stores them at the parameter's own row-major strides (`(cols, 1)`), not as dense runs. `plan_transfer` refuses those today (`dest axis 0 strides …`). The only workaround is one piece per row, which runs into `max_pieces_per_map` for real weights. **Change.** In `_frame_of`, a flat shard may now hold a piece at the parameter's own strides, alongside the dense run it already accepts. Each such piece records its shift: its shard offset minus the row-major index of its origin. The tiling check for these pieces is a separate branch, `_in_order_shard_tiles`: - All pieces of the rank must share one shift. Then logical index `i` sits at address `i + shift`, which is injective. - With one shift, disjoint logical boxes mean disjoint addresses. Together with the shard bounds and the volume sum, the shard is tiled exactly. - Anything else is refused as before: two shifts in one shard, a mix with dense runs of another width, overlapping boxes, a box past the shard end, or strides that match neither form. - The disjointness sweep counts against `max_pair_checks`, the same as in deepspeedai#9011. Shards whose pieces are all dense runs take the existing path unchanged. **Docs.** A second commit updates `affine_ir_spec.md` §10.2–10.3 to match: the two forms a flat shard may take, the tiling argument for the second, and that `nest_affine_maps` still emits dense boxes only, since the parameter's own strides fit only a shard that is a run of the parameter's flattening. **Tests** (`tests/unit/checkpoint/test_affine_transfer_plan.py`, all single-process): - ZeRO partitions cut at a column edge, including cuts in the middle of a row. Planned both from and into a column split, each checked element by element. - The same storage described one piece per row. The plan moves the same elements. - Five refusals, each matched to its reason. Without the planner change, the 5 acceptance cases are refused. The 5 refusal cases refuse either way; the tests pin the new reasons. On CPU (gloo, up to 4 processes), `test_affine_*.py`, `test_autoep_affine*.py`, `test_zero_affine.py` and `v1/moe/test_autoep_checkpoint.py` pass: 765 passed and 4 skipped, against 755 and 4 on master. **Outside the unit tests** (CPU, against this branch). These use the two-layout map from deepspeedai#9026: ZeRO-3 partitions plus the surviving TP shards, cut at each other's edges. - 300 random cases with uneven partitions, row or column TP splits, and one or both TP shards surviving. All are exact, whether the cut map is the source or the target. - 100 of those plans run through `execute_transfers` on 4 gloo processes, in both directions, at staging budgets of 8 MiB and 256 B. Nothing is wrong: 0 of 200 target shards reading from the cut map, and 0 of 483 writing into it. - 4096×4096 weight, 8 ZeRO-3 partitions plus both shards of a TP2 column split, planned into TP2 along columns: - with this change: 32 source pieces plan in about 1 ms under the default budget, giving 16 segments and a 1.3 KB plan; - one piece per row: 16,384 pieces, refused under the default budget; with an unbounded budget, 5.1 s, 8,192 segments and a 642 KB plan. --------- Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de> Co-authored-by: Ma, Guokai <guokai.ma@gmail.com>
Stacked on #9010 (only the last two commits are new here; I will rebase after #9010 merges).
_validate_holderscompared every piece with every other piece,_sourcescompared every source region with every other region, and_shards_tilecompared each rank's pieces pairwise by shard address. Planning time therefore grew quadratically with the number of pieces, and a map with 1024 pieces already exceeded the defaultmax_pair_checks.When no rank repeats a box and distinct boxes do not overlap, a rank covers a box exactly when it holds that very box, so the holders of each box can be read from a table. This PR adds
boxes_disjointtoaffine_geometry. It sweeps along the axis whose box origins vary most and compares each box only with the boxes still open where it starts. The same sweep proves that a rank's pieces tile its shard when they are disjoint by address and fill it (suggested by 0z5a in review). Maps that fail the fast checks fall back to the original pairwise walk, which names the offending piece, so error messages are unchanged, and maps that pass no longer hit the per-ranklen(held)**2budget.The sweep counts its comparisons against
max_pair_checkstoo (suggested by Achyuthan-S in review). A disjoint layout can keep every box open for the whole sweep: for example, h full-height one-column boxes plus 2h one-element boxes beside them. When the sweep runs out of budget it reports "not shown disjoint". It compares at most n·(n−1)/2 pairs, so running out implies n² >max_pair_checks, and the pairwise fallback refuses at once. With 4,095 such pieces, planning went from 19.2 s to a budget refusal in 0.6 s.Planning time (8 ranks with interleaved rows, re-sharded to 8 row shards, 64 columns; ms, single CPU process):
Tests:
boxes_disjointis checked against an all-pairs reference over 100 seeds.max_pair_checks(refused before this change).max_pair_checksjust below its sweep cost. Without the sweep budget, that test plans instead of refusing.tests/unit/checkpoint -k affinepasses on 8× NVIDIA L20 (645 passed). At 12663c3 on CPU (DS_ACCELERATOR=cpu, 4 ranks): 642 passed, 5 skipped.The remaining near-quadratic steps, the source × target join and
_covers_union, overlap with #8936 and will follow after it lands; segment merging in_coalescehas its ownmax_merge_checksbudget and will follow separately.