fused_dropout_add¶

paddle.incubate.nn.functional. fused_dropout_add ( x, y, p=0.5, training=True, mode='upscale_in_train', name=None ) [source]

Fused Dropout and Add.

Parameters

• x (Tensor) – The input tensor. The data type is bfloat16, float16, float32 or float64.

• y (Tensor) – The input tensor. The data type is bfloat16, float16, float32 or float64.

• p (float|int, optional) – Probability of setting units to zero. Default: 0.5.

• training (bool, optional) – A flag indicating whether it is in train phrase or not. Default: True.

• mode (str, optional) –

  [‘upscale_in_train’(default) | ‘downscale_in_infer’].

  1. upscale_in_train (default), upscale the output at training time

     • train: \(out = x \times \frac{mask}{(1.0 - dropout\_prob)} + y\)

     • inference: \(out = x + y\)

  2. downscale_in_infer, downscale the output at inference

     • train: \(out = input \times mask + y\)

     • inference: \(out = input \times (1.0 - dropout\_prob) + y\)

• name (str, optional) – Name for the operation, Default: None. For more information, please refer to Name.

Returns

A Tensor representing the fused dropout and add, has same shape and data type as x .

Examples

# required: gpu
import paddle
from paddle.incubate.nn.functional import fused_dropout_add

x = paddle.randn([4, 10], dtype='float16')
y = paddle.randn([4, 10], dtype='float16')
out = fused_dropout_add(x, y, p=0.5)