MSRAInitializer¶

class paddle.fluid.initializer. MSRAInitializer ( uniform=True, fan_in=None, seed=0 ) [source]

Implements the MSRA initializer a.k.a. Kaiming Initializer

This class implements the weight initialization from the paper Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification by Kaiming He, Xiangyu Zhang, Shaoqing Ren and Jian Sun. This is a robust initialization method that particularly considers the rectifier nonlinearities. In case of Uniform distribution, the range is [-x, x], where

$$\begin{split}x = \sqrt{\\frac{6.0}{fan\_in}}\end{split}$$

In case of Normal distribution, the mean is 0 and the standard deviation is

$$\begin{split}\sqrt{\\frac{2.0}{fan\_in}}\end{split}$$

Parameters

• uniform (bool) – whether to use uniform or normal distribution

• fan_in (float32|None) – fan_in for MSRAInitializer. If None, it is

• None. (inferred from the variable. default is) –

• seed (int32) – random seed

Note

It is recommended to set fan_in to None for most cases.

Examples

import paddle
import paddle.fluid as fluid
paddle.enable_static()
x = fluid.data(name="data", shape=[8, 32, 32], dtype="float32")
fc = fluid.layers.fc(input=x, size=10,
    param_attr=fluid.initializer.MSRA(uniform=False))