label_smooth

paddle.fluid.layers.nn. label_smooth ( label, prior_dist=None, epsilon=0.1, dtype='float32', name=None ) [source]
Alias_main

paddle.nn.functional.label_smooth :alias: paddle.nn.functional.label_smooth,paddle.nn.functional.common.label_smooth :old_api: paddle.fluid.layers.label_smooth

Label smoothing is a mechanism to regularize the classifier layer and is called label-smoothing regularization (LSR).

Label smoothing is proposed to encourage the model to be less confident, since optimizing the log-likelihood of the correct label directly may cause overfitting and reduce the ability of the model to adapt. Label smoothing replaces the ground-truth label \(y\) with the weighted sum of itself and some fixed distribution \(\mu\). For class \(k\), i.e.

\[\begin{split}\\tilde{y_k} = (1 - \epsilon) * y_k + \epsilon * \mu_k,\end{split}\]

where \(1 - \epsilon\) and \(\epsilon\) are the weights respectively, and \(\\tilde{y}_k\) is the smoothed label. Usually uniform distribution is used for \(\mu\).

See more details about label smoothing in https://arxiv.org/abs/1512.00567.

Parameters

  • label (Variable) – The input variable containing the label data. The label data should use one-hot representation. It’s a multidimensional tensor with a shape of \([N_1, ..., Depth]\), where Depth is class number. The dtype can be “float32” and “float64”.

  • prior_dist (Variable, optional) – The prior distribution to be used to smooth labels. If not provided, an uniform distribution is used. It’s a multidimensional tensor with a shape of \([1, class\_num]\) . The default value is None.

  • epsilon (float, optional) – The weight used to mix up the original ground-truth distribution and the fixed distribution. The default value is 0.1.

  • dtype (np.dtype|core.VarDesc.VarType|str, optional) – The data type can be set as ‘float32’, ‘float64’. The default value is ‘float32’.

  • name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to Name.

Returns

The tensor variable containing the smoothed labels.

Return type

Variable

Examples

import paddle.fluid as fluid
import paddle.fluid.layers as layers

label = layers.data(name="label", shape=[1], dtype="int32")
one_hot_label = layers.one_hot(input=label, depth=10)
smooth_label = layers.label_smooth(
    label=one_hot_label, epsilon=0.1, dtype="float32")