Fix ZeRO++ secondary shard copy for small params

[ZyEng-Art]

Summary

This PR fixes a ZeRO++ edge case where _partition_param_sec() can call torch.narrow() with an out-of-range start index when a very small parameter does not overlap with the computed secondary shard.

When secondary_start > param.ds_numel, sec_numel becomes 0, but the old code still attempted to execute:

one_dim_param.narrow(0, secondary_start, sec_numel)

PyTorch raises IndexError for this case even when sec_numel == 0.

The fix is to skip the secondary copy when sec_numel == 0. This preserves the existing behavior for overlapping shards and avoids the invalid empty narrow() call for non-overlapping shards.

This PR also adds a focused regression test covering the small-parameter, no-overlap secondary shard case.

Testing

python3 -m pytest -q tests/unit/runtime/zero/test_zeropp.py -k 'test_zero_hpz_partition_size_config or test_zero_hpz_small_param_secondary_shard_without_overlap'

Observed locally:

• 2 passed

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fb6daa6efc

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Zero-fill skipped secondary shards

In HPZ runs with zero_quantized_weights enabled, partitioned_param_coordinator.py:580 passes quantize=True and _all_gather_coalesced quantizes a torch.cat of every ds_secondary_tensor (partition_parameters.py:1406-1409). When sec_numel == 0, this branch skips the only write into the newly allocated torch.empty secondary tensor, so a small no-overlap parameter can feed arbitrary padding values into the shared quantization group for neighboring parameters instead of deterministic zeros. Please initialize the skipped shard before it can be gathered or quantized.

Useful? React with 👍 / 👎.

[goransh-buh]

This looks like a solid, focused change.