XavierUniform¶

class paddle.nn.initializer. XavierUniform ( fan_in=None, fan_out=None, name=None ) [source]

This class implements the Xavier weight initializer from the paper Understanding the difficulty of training deep feedforward neural networks by Xavier Glorot and Yoshua Bengio.

This initializer is designed to keep the scale of the gradients approximately same in all the layers. In case of Uniform distribution, the range is [-x, x], where

\[x = \sqrt{\frac{6.0}{fan\_in + fan\_out}}\]

Parameters

• fan_in (float, optional) – fan_in for Xavier initialization, it is inferred from the tensor. The default value is None.

• fan_out (float, optional) – fan_out for Xavier initialization, it is inferred from the tensor. The default value is None.

• name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to Name.

Returns

A parameter initialized by Xavier weight, using a uniform distribution.

Examples

import paddle

data = paddle.ones(shape=[3, 1, 2], dtype='float32')
weight_attr = paddle.framework.ParamAttr(
    name="linear_weight",
    initializer=paddle.nn.initializer.XavierUniform())
bias_attr = paddle.framework.ParamAttr(
    name="linear_bias",
    initializer=paddle.nn.initializer.XavierUniform())
linear = paddle.nn.Linear(2, 2, weight_attr=weight_attr, bias_attr=bias_attr)
# linear.weight:  [[-0.04229349 -1.1248565 ]
#                  [-0.10789523 -0.5938053 ]]
# linear.bias:  [ 1.1983747  -0.40201235]

res = linear(data)
# res:  [[[ 1.0481861 -2.1206741]]
#        [[ 1.0481861 -2.1206741]]
#        [[ 1.0481861 -2.1206741]]]