ConcatDataset¶

class paddle.io. ConcatDataset ( datasets: Iterable[Dataset[Any]] ) [source]

Dataset as a concatenation of multiple datasets.

This class is useful to assemble different existing datasets.

Parameters

datasets (sequence) – List of datasets to be concatenated

Returns

A Dataset which concatenated by multiple datasets.

Return type

Dataset

Examples

>>> import numpy as np
>>> import paddle
>>> from paddle.io import Dataset, ConcatDataset

>>> # define a random dataset
>>> class RandomDataset(Dataset):  # type: ignore[type-arg]
...     def __init__(self, num_samples):
...         self.num_samples = num_samples
...
...     def __getitem__(self, idx):
...         image = np.random.random([32]).astype('float32')
...         label = np.random.randint(0, 9, (1, )).astype('int64')
...         return image, label
...
...     def __len__(self):
...         return self.num_samples
...
>>> dataset = ConcatDataset([RandomDataset(10), RandomDataset(10)])  # type: ignore[var-annotated]
>>> for i in range(len(dataset)):
...     image, label = dataset[i]
...     # do something