Add DualPipeV

[phoeenniixx]

Implements DualPipeV from DeepSeek team
Part of #7100.

This adds DualPipeV, the V-shaped schedule from DualPipe, as an opt-in alternative to 1F1B. Existing pipeline users are not affected. A model opts in by building a DualPipeVModule instead of a PipelineModule.

from deepspeed.pipe import DualPipeVModule

net = DualPipeVModule(layers=layers, num_stages=4, loss_fn=loss_fn)
engine, _, _, _ = deepspeed.initialize(config=config, model=net, model_parameters=net.parameters())
loss = engine.train_batch()

Changes

• Adds DualPipeVModule, which cuts the model into 2P stages and builds stages r and 2P-1-r on rank r.
• Adds DualPipeVEngine, a child of PipelineEngine. It replaces only the send and receive methods with batched isend/irecv.
• Adds DualPipeVSchedule, which follows the eight steps of the reference dualpipev.py.
• Adds P2POp and batch_isend_irecv to deepspeed.comm.
• PipelineEngine gets three small hook methods for the schedules and the loss rank, and PipelineModule reads self.num_stages when partitioning. Neither changes 1F1B behaviour.
• Tests for the schedule protocol, and an end-to-end test against an unpartitioned nn.Sequential.

I also tried it using a small experiment on Kaggle's 2 T4 using this scipt. See the results here

schedule	step time (s)	samples/s	peak mem rank 0 (MB)	peak mem rank 1 (MB)	final loss
1F1B (PipelineModule)	0.465	68.9	621	525	1.000726
DualPipeV (DualPipeVModule)	0.439	72.9	761	909	1.000726

Limitations

• gradient_accumulation_steps must be at least twice the number of pipeline ranks.
• dynamic_shape, pipe_partitioned, grad_partitioned and boolean attention masks are not supported.
• No forward/backward overlap and no deferred weight gradients yet, so the speedup is smaller than in the DualPipe README.

Follow-ups

If this approach is accepted, I plan to raise follow-up PRs for:

• The bidirectional DualPipe schedule. It can reuse parts of DualPipeVEngine (maybe move the common parts to a common base class in future?) and needs a second steps() generator plus data loading and loss on both end ranks.
• Forward/backward overlap and the zero-bubble weight-gradient split.
• Tutorial and API reference docs.

[phoeenniixx]

FYI @delock @ehartford

[phoeenniixx]

Hi @delock, can you please review this PR whenever you have some time.
Thanks