Deprecate and remove graph harvesting - #8579
hwchen2017 wants to merge 2 commits into
Conversation
Signed-off-by: Hongwei Chen <hongweichen@microsoft.com>
Signed-off-by: Hongwei Chen <hongweichen@microsoft.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e3b0f811c8
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| "sparse_attention": | ||
| "DeepSpeed Sparse Attention has been removed; the 'sparse_attention' configuration block is no longer " | ||
| f"supported. See {_REMOVED_FEATURES_ISSUE}.", | ||
| "graph_harvesting": |
There was a problem hiding this comment.
This is a single-parent, non-merge commit, and git log --format='%b' f1f03e8aaf9ac03006ae9a49ae52a1375fe1f992 confirms that its message has no Signed-off-by trailer. Add the author sign-off so the commit satisfies the repository's commit and DCO requirements.
AGENTS.md reference: AGENTS.md:L8-L9
Useful? React with 👍 / 👎.
1 participant
• Remove the graph_harvesting config option; supplying it (any value, including false / null ) now raises DeepSpeedConfigError , consistent with other removed features.
• Remove graph_process / graph_cache and the use_graph argument from get_global_norm_of_tensors / clip_tensors_by_global_norm , collapsing them to the direct (non-graph) path.
• Optimize global-norm computation: previously each tensor's p‑th power was accumulated in-place into a reused, cached scratch buffer ( graph_cache['norm_tensors_compute_buffer'] ) so memory addresses stayed fixed for graph replay. It now computes each tensor's norm into a fresh list, stacks them, raises to the p‑th power, and sums after the model/expert-parallel all-reduce).
• Drop the graph_harvesting plumbing in BF16_Optimizer , DeepSpeedEngine , and constants.py ; simplify update_hp_grads (the graph-only CPU flag-fixup loop was redundant).
• Add a config-rejection test.