Linear

class paddle.nn. Linear ( in_features, out_features, weight_attr=None, bias_attr=None, name=None ) [source]

Fully-connected linear transformation layer. For each input \(X\) , the equation is:

\[Out = XW + b\]

where \(W\) is the weight and \(b\) is the bias.

Linear layer takes only one multi-dimensional tensor as input with the shape \([batch\_size, *, in\_features]\) , where \(*\) means any number of additional dimensions. It multiplies input tensor with the weight (a 2-D tensor of shape \([in\_features, out\_features]\) ) and produces an output tensor of shape \([batch\_size, *, out\_features]\) . If \(bias\_attr\) is not False, the bias (a 1-D tensor of shape \([out\_features]\) ) will be created and added to the output.

Parameters

  • in_features (int) – The number of input units.

  • out_features (int) – The number of output units.

  • weight_attr (ParamAttr, optional) – The attribute for the learnable weight of this layer. The default value is None. If the Initializer of the param_attr is not set, the parameter is initialized with Xavier. For detailed information, please refer to paddle.ParamAttr.

  • bias_attr (ParamAttr|bool, optional) – The attribute for the learnable bias of this layer. If it is set to False, no bias will be added to the output. If it is set to None or one kind of ParamAttr, a bias parameter will be created according to ParamAttr. For detailed information, please refer to paddle.ParamAttr. The default value is None and the bias will be initialized to zero.

  • name (str, optional) – Normally there is no need for user to set this parameter. For detailed information, please refer to Name .

Attribute:

weight (Parameter): the learnable weight of this layer.

bias (Parameter): the learnable bias of this layer.

Shape:

  • input: Multi-dimensional tensor with shape \([batch\_size, *, in\_features]\) . Its data types are float16, float32, float64 ,The default is float32 .

  • output: Multi-dimensional tensor with shape \([batch\_size, *, out\_features]\) . The data type is the same as the input .

Examples

>>> import paddle
>>> paddle.seed(100)

>>> # Define the linear layer.
>>> weight_attr = paddle.ParamAttr(
...     name="weight",
...     initializer=paddle.nn.initializer.Constant(value=0.5))
>>> bias_attr = paddle.ParamAttr(
...     name="bias",
...     initializer=paddle.nn.initializer.Constant(value=1.0))
>>> linear = paddle.nn.Linear(2, 4, weight_attr=weight_attr, bias_attr=bias_attr)
>>> print(linear.weight)
Parameter containing:
Tensor(shape=[2, 4], dtype=float32, place=Place(cpu), stop_gradient=False,
[[0.50000000, 0.50000000, 0.50000000, 0.50000000],
 [0.50000000, 0.50000000, 0.50000000, 0.50000000]])

>>> print(linear.bias)
Parameter containing:
Tensor(shape=[4], dtype=float32, place=Place(cpu), stop_gradient=False,
[1., 1., 1., 1.])

>>> x = paddle.randn((3, 2), dtype="float32")
>>> y = linear(x)
>>> print(y)
Tensor(shape=[3, 4], dtype=float32, place=Place(cpu), stop_gradient=False,
[[ 0.42121571,  0.42121571,  0.42121571,  0.42121571],
 [ 0.85327661,  0.85327661,  0.85327661,  0.85327661],
 [-0.05398512, -0.05398512, -0.05398512, -0.05398512]])
forward ( input )

forward

Defines the computation performed at every call. Should be overridden by all subclasses.

Parameters

  • *inputs (tuple) – unpacked tuple arguments

  • **kwargs (dict) – unpacked dict arguments

extra_repr ( )

extra_repr

Extra representation of this layer, you can have custom implementation of your own layer.

Used in the guide/tutorials