unshard_dtensor¶

paddle.distributed. unshard_dtensor ( dist_tensor ) [source]

Converts a distributed tensor to a dense tensor. unshard_dtensor first make the dist_tensor be Replicate state on all processes and then converts it to a dense paddle.Tensor. It can be treated as a reverse operation of shard_tensor.

Parameters: dist_tensor (paddle.Tensor) – The distributed tensor which is constructed from a dense tensor with shard_tensor.
Returns: The original dense tensor of the input dist_tensor.
Return type: paddle.Tensor

Examples

           >>> import paddle
>>> import paddle.distributed as dist
>>> from paddle.distributed import Replicate, Shard

>>> 
>>> mesh = dist.ProcessMesh([0, 1], dim_names=["x"])
>>> original_tensor = paddle.rand([4, 1024, 512])
>>> dist_tensor = dist.shard_tensor(original_tensor, mesh, [Shard(0)])
>>> # dense_tensor's shape is the same as original_tensor
>>> dense_tensor = dist.unshard_dtensor(dist_tensor)

          