SimpleRNNCell

class paddle.nn. SimpleRNNCell ( input_size, hidden_size, activation='tanh', weight_ih_attr=None, weight_hh_attr=None, bias_ih_attr=None, bias_hh_attr=None, name=None ) [source]

Elman RNN (SimpleRNN) cell. Given the inputs and previous states, it computes the outputs and updates states.

The formula used is as follows:

\[ \begin{align}\begin{aligned}h_{t} & = act(W_{ih}x_{t} + b_{ih} + W_{hh}h_{t-1} + b_{hh})\\y_{t} & = h_{t}\end{aligned}\end{align} \]

where \(act\) is for activation.

Please refer to Finding Structure in Time for more details.

Parameters

  • input_size (int) – The input size.

  • hidden_size (int) – The hidden size.

  • activation (str, optional) – The activation in the SimpleRNN cell. It can be tanh or relu. Defaults to tanh.

  • weight_ih_attr (ParamAttr, optional) – The parameter attribute for \(weight_ih\). Default: None.

  • weight_hh_attr (ParamAttr, optional) – The parameter attribute for \(weight_hh\). Default: None.

  • bias_ih_attr (ParamAttr, optional) – The parameter attribute for the \(bias_ih\). Default: None.

  • bias_hh_attr (ParamAttr, optional) – The parameter attribute for the \(bias_hh\). Default: None.

  • name (str, optional) – Name for the operation (optional, default is None). For more information, please refer to Name.

Variables:

  • weight_ih (Parameter): shape (hidden_size, input_size), input to hidden weight, corresponding to \(W_{ih}\) in the formula.

  • weight_hh (Parameter): shape (hidden_size, hidden_size), hidden to hidden weight, corresponding to \(W_{hh}\) in the formula.

  • bias_ih (Parameter): shape (hidden_size, ), input to hidden bias, corresponding to \(b_{ih}\) in the formula.

  • bias_hh (Parameter): shape (hidden_size, ), hidden to hidden bias, corresponding to \(b_{hh}\) in the formula.

Inputs:

  • inputs (Tensor): shape [batch_size, input_size], the input, corresponding to \(x_{t}\) in the formula.

  • states (Tensor, optional): shape [batch_size, hidden_size], the previous hidden state, corresponding to \(h_{t-1}\) in the formula. When states is None, zero state is used. Defaults to None.

Returns

shape [batch_size, hidden_size], the output, corresponding to \(h_{t}\) in the formula. - states (Tensor): shape [batch_size, hidden_size], the new hidden state, corresponding to \(h_{t}\) in the formula.

Return type

  • outputs (Tensor)

Notes

All the weights and bias are initialized with Uniform(-std, std) by default. Where std = \(\frac{1}{\sqrt{hidden\_size}}\). For more information about parameter initialization, please refer to ParamAttr.

Examples

>>> import paddle

>>> x = paddle.randn((4, 16))
>>> prev_h = paddle.randn((4, 32))

>>> cell = paddle.nn.SimpleRNNCell(16, 32)
>>> y, h = cell(x, prev_h)
>>> print(y.shape)
[4, 32]
forward ( inputs, states=None )

forward

Defines the computation performed at every call. Should be overridden by all subclasses.

Parameters

  • *inputs (tuple) – unpacked tuple arguments

  • **kwargs (dict) – unpacked dict arguments

property state_shape

Abstract method (property). Used to initialize states. A (possibly nested structure of) shape[s], where a shape is a list/tuple of integers (-1 for batch size would be automatically inserted into a shape if shape is not started with it). Not necessary to be implemented if states are not initialized by get_initial_states or the shape argument is provided when using get_initial_states.

extra_repr ( )

extra_repr

Extra representation of this layer, you can have custom implementation of your own layer.