glu¶

paddle.nn.functional. glu ( x, axis=- 1, name=None ) [source]

The gated linear unit. The input is evenly splited into 2 parts along a given axis. The first part is used as the content, and the second part is passed through a sigmoid function then used as the gate. The output is a elementwise multiplication of the content and the gate.

\[\mathrm{GLU}(a, b) = a \otimes \sigma(b)\]

Parameters

• x (Tensor) – The input Tensor with data type float32, float64.

• axis (int, optional) – The axis along which split the input tensor. It should be in range [-D, D), where D is the dimensions of x . If axis < 0, it works the same way as \(axis + D\) . Default is -1.

• name (str, optional) – Name for the operation (optional, default is None). For more information, please refer to Name.

Returns

A Tensor with the same data type as x. The size of the given aixs is halved.

Examples

import paddle
from paddle.nn import functional as F

x = paddle.to_tensor(
    [[-0.22014759, -1.76358426,  0.80566144,  0.04241343],
     [-1.94900405, -1.89956081,  0.17134808, -1.11280477]]
)
print(F.glu(x).numpy())
# array([[-0.15216254, -0.9004892 ],
#        [-1.0577879 , -0.46985325]], dtype=float32)