ProcessMesh

class paddle.distributed. ProcessMesh ( mesh: npt.NDArray[Any] | NestedNumericSequence | None = None, dim_names: list[str] | None = None, shape: _NumpyShapeLike | None = None, process_ids: Iterable[Any] | None = None ) [source]

The ProcessMesh object describes the Cartesian topology of the used processes.

Parameters

• mesh (list|numpy.array) – an n-dimensional array describes the topology of the processes.

• dim_names (list, optional) – the i-th element of this list gives the name of the i-th dimension of the mesh.

Examples

>>> import paddle
>>> import paddle.distributed as dist

>>> mesh = dist.ProcessMesh([[2, 4, 5], [0, 1, 3]], dim_names=["x", "y"])
>>> assert mesh.shape == [2, 3]
>>> assert mesh.process_ids == [2, 4, 5, 0, 1, 3]

property mesh : npt.NDArray[Any]

Get the underlying mesh of ProcessMesh.

property dim_names : list[str]

Get the underlying dimension names of ProcessMesh.

property unique_id : int

Get the unique id of ProcessMesh. NOTE Unique id only take process_ids and shape into account. Different ProcessMesh with same process_ids and shape have same unique id.

get_submesh_with_dim ( dim_name: str ) → ProcessMesh

get_submesh_with_dim¶

Slice the current ProcessMesh based on the dim_name given to create a submesh with single dimension remained.

Parameters

dim_name (str) – the name of the mesh dimension of the ProcessMesh to create the submesh for.

Returns

A ProcessMesh object

Examples

>>> import paddle
>>> import paddle.distributed as dist

>>> dist.init_parallel_env()
>>> mesh_2d = dist.ProcessMesh([[0, 1, 2, 3], [4, 5, 6, 7]], dim_names=["dp", "tp"])

>>> dp_mesh = mesh_2d.get_submesh_with_dim("dp")
>>> # ProcessMesh:([0, 4]) on rank 0, 4
>>> # ProcessMesh:([1, 5]) on rank 1, 5
>>> # ProcessMesh:([2, 6]) on rank 2, 6
>>> # ProcessMesh:([3, 7]) on rank 3, 7

>>> tp_mesh = mesh_2d.get_submesh_with_dim("tp")
>>> # ProcessMesh:([0, 1, 2, 3]) on rank 0, 1, 2, 3
>>> # ProcessMesh:([4, 5, 6, 7]) on rank 4, 5, 6, 7

>>> mesh_3d = dist.ProcessMesh([[[0, 1],[2, 3]], [[4, 5], [6, 7]]], dim_names=["pp","dp","tp"])

>>> pp_mesh = mesh_3d.get_submesh_with_dim("pp")
>>> # ProcessMesh:([0, 4]) on rank 0, 4
>>> # ProcessMesh:([1, 5]) on rank 1, 5
>>> # ProcessMesh:([2, 6]) on rank 2, 6
>>> # ProcessMesh:([3, 7]) on rank 3, 7

>>> dp_mesh = mesh_3d.get_submesh_with_dim("dp")
>>> # ProcessMesh:([0, 2]) on rank 0, 2
>>> # ProcessMesh:([1, 3]) on rank 1, 3
>>> # ProcessMesh:([4, 6]) on rank 4, 6
>>> # ProcessMesh:([5, 7]) on rank 5, 7

>>> tp_mesh = mesh_3d.get_submesh_with_dim("tp")
>>> # ProcessMesh:([0, 1]) on rank 0, 1
>>> # ProcessMesh:([2, 3]) on rank 2, 3
>>> # ProcessMesh:([4, 5]) on rank 4, 5
>>> # ProcessMesh:([6, 7]) on rank 6, 7

get_group ( dim_name: Optional[str] = None ) → Group [source]

get_group¶

Convert single dimension ProcessMesh to the corresponding Group.

Parameters

dim_name (str, optional) – it can be the name of the mesh dimension. Default is None.

Returns

A Group object.

contains ( self: paddle.base.libpaddle.ProcessMesh, arg0: int ) → bool

contains¶

dim_size ( *args, **kwargs )

dim_size¶

Overloaded function.

1. dim_size(self: paddle.base.libpaddle.ProcessMesh, arg0: int) -> int

2. dim_size(self: paddle.base.libpaddle.ProcessMesh, arg0: str) -> int

empty ( self: paddle.base.libpaddle.ProcessMesh ) → bool

empty¶