glu

paddle.nn.functional. glu ( x, axis=- 1, name=None ) [source]

The gated linear unit. The input is evenly splited into 2 parts along a given axis. The first part is used as the content, and the second part is passed through a sigmoid function then used as the gate. The output is a elementwise multiplication of the content and the gate.

\[\mathrm{GLU}(a, b) = a \otimes \sigma(b)\]
Parameters

  • x (Tensor) – The input Tensor with data type float32, float64.

  • axis (int, optional) – The axis along which split the input tensor. It should be in range [-D, D), where D is the dimensions of x . If axis < 0, it works the same way as \(axis + D\) . Default is -1.

  • name (str, optional) – For details, please refer to Name. Generally, no setting is required. Default: None.

Returns

A Tensor with the same data type as x. The size of the given axis is halved.

Examples

>>> import paddle
>>> from paddle.nn import functional as F
>>> x = paddle.to_tensor(
...     [[-0.22014759, -1.76358426,  0.80566144,  0.04241343],
...         [-1.94900405, -1.89956081,  0.17134808, -1.11280477]]
... )
>>> print(F.glu(x))
Tensor(shape=[2, 2], dtype=float32, place=Place(cpu), stop_gradient=True,
[[-0.15216254, -0.90048921],
[-1.05778778, -0.46985325]])