KaimingNormal¶

class paddle.nn.initializer. KaimingNormal ( fan_in=None, negative_slope=0.0, nonlinearity='relu' ) [source]

Implements the Kaiming Normal initializer

This class implements the weight initialization from the paper Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification by Kaiming He, Xiangyu Zhang, Shaoqing Ren and Jian Sun. This is a robust initialization method that particularly considers the rectifier nonlinearities.

In case of Normal distribution, the mean is 0 and the standard deviation is

\[\frac{gain}{\sqrt{{fan\_in}}}\]

Parameters

• fan_in (float32|None, optional) – fan_in (in_features) of trainable Tensor, If None, it will be infered automatically. If you don’t want to use in_features of the Tensor, you can set the value of ‘fan_in’ smartly by yourself. Default is None.

• negative_slope (float, optional) – negative_slope (only used with leaky_relu). Default is 0.0.

• nonlinearity (str, optional) – the non-linear function. Default is relu.

Note

It is recommended to set fan_in to None for most cases.

Examples

>>> import paddle
>>> import paddle.nn as nn

>>> linear = nn.Linear(2, 4, weight_attr=nn.initializer.KaimingNormal())
>>> data = paddle.rand([30, 10, 2], dtype='float32')
>>> res = linear(data)