gather¶

paddle.distributed. gather ( tensor, gather_list=None, dst=0, group=None, sync_op=True ) [source]

Gather tensors from all participators.

Parameters

• tensor (Tensor) – The input Tensor. Its data type should be float16, float32, float64, int32, int64, int8, uint8, bool or bfloat16.

• gather_list (list) – A list of Tensors to hold the gathered tensors. Every element in the list must be a Tensor whose data type should be float16, float32, float64, int32, int64, int8, uint8, bool or bfloat16. Default value is None.

• dst (int) – The dst rank id. Default value is 0.

• group (Group, optional) – The group instance return by new_group or None for global default group.

• sync_op (bool, optional) – Whether this op is a sync op. The default value is True.

Returns

Async work handle,which can be wait on, if async_op is set to True. None, if not async_op

Examples

>>> 
>>> import paddle
>>> import paddle.distributed as dist

>>> dist.init_parallel_env()
>>> gather_list = []
>>> if dist.get_rank() == 0:
...     data = paddle.to_tensor([1, 2, 3])
...     dist.gather(data, gather_list, dst=0)
>>> else:
...     data = paddle.to_tensor([4, 5, 6])
...     dist.gather(data1, gather_list, dst=0)
>>> print(gather_list)
>>> # [[1, 2, 3], [4, 5, 6]] (2 GPUs, out for rank 0)
>>> # [] (2 GPUs, out for rank 1)