FusedDropoutAdd¶

class paddle.incubate.nn. FusedDropoutAdd ( p=0.5, mode='upscale_in_train', name=None ) [source]

Fused Dropout and Add.

Parameters

• p (float|int, optional) – Probability of setting units to zero. Default: 0.5

• mode (str, optional) –

  [‘upscale_in_train’(default) | ‘downscale_in_infer’]

  1. upscale_in_train (default), upscale the output at training time

     • train: \(out = x \times \frac{mask}{(1.0 - p)} + y\)

     • inference: \(out = x + y\)

  2. downscale_in_infer, downscale the output at inference

     • train: \(out = x \times mask + y\)

     • inference: \(out = x \times (1.0 - p) + y\)

• name (str, optional) – Name for the operation, Default: None. For more information, please refer to Name.

Shape:

• x: N-D tensor.

• y: N-D tensor.

• output: N-D tensor, the same shape as x.

Examples

>>> 
>>> import paddle
>>> paddle.device.set_device('gpu')
>>> from paddle.incubate.nn.layer.fused_dropout_add import FusedDropoutAdd

>>> x = paddle.to_tensor([[1,2,3], [4,5,6]], dtype="float32")
>>> y = paddle.to_tensor([[1,2,3], [4,5,6]], dtype="float32")

>>> m = FusedDropoutAdd(p=0.5)

>>> out = m(x, y)

forward ( x, y )

forward¶

Defines the computation performed at every call. Should be overridden by all subclasses.

Parameters

• *inputs (tuple) – unpacked tuple arguments

• **kwargs (dict) – unpacked dict arguments

extra_repr ( )

extra_repr¶

Extra representation of this layer, you can have custom implementation of your own layer.