Remove no-op operation in sharded tensor pool (meta-pytorch#3551)

Summary:
Pull Request resolved: meta-pytorch#3551

During implementation for item usharding added a no-op operation by mistake. Since item usharding already updates the device map to correctly move all the parameters of corresponding LocalShardPool to cpu / cuda (depending upon where the shard is located), setting device for shard here is not required. Removing this line to avoid confusion

Reviewed By: cp2923

Differential Revision: D87244659

fbshipit-source-id: ea2a1d3fcaaba830b1cf6456c3326be2a66554fe

Author: faran928

torchrec/distributed/tensor_pool.py

@@ -305,7 +305,6 @@ def __init__(
     @torch.jit.export
     def set_device(self, device_str: str) -> None:
         self.current_device = torch.device(device_str)
-        self._shard.to(self.current_device)
 
     def forward(self, rank_ids: torch.Tensor) -> torch.Tensor:
         """