DynamicRNN¶

class paddle.fluid.layers.control_flow. DynamicRNN ( name=None ) [source]

Api_attr

Static Graph

Note: the input of this class should be LoDTensor which holds the information of variable-length sequences. If the input is fixed-length Tensor, please use StaticRNN (fluid.layers. api_fluid_layers_StaticRNN ) for better performance.

DynamicRNN can process a minibatch of variable-length sequences. The length of each sample can be different and is recorded in LoD. In DynamicRNN, an input sequence will be unfolded into time steps and users can define how to process each time step in block() . The total number of time steps is determined by the longest sequence. DynamicRNN will not pad all sequences to the same length, instead it will sort the sequences internally by the sequence length in descending order. The input sequences will be shrank because only sequences of which the length is larger than the time step will participate the remaining calculation.

If defined drnn = DynamicRNN(), then users can call drnn() to obtain the result sequences. It is a LoDTensor gained by merging all time steps’s output. When RNN’s input sequence x meets x.lod_level == 1, the output LoDTensor will have the same LoD with x. The result of drnn() includes RNN’s outputs of all time steps, users can call api_fluid_layers_sequence_last_step to extract the data of the last time step.

Warning

Currently it is not supported to set is_sparse = True of any layers defined within DynamicRNN’s block function.

Parameters

name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to Name .

Examples

import paddle.fluid as fluid

sentence = fluid.data(name='sentence', shape=[None, 32], dtype='float32', lod_level=1)
encoder_proj = fluid.data(name='encoder_proj', shape=[None, 32], dtype='float32', lod_level=1)
decoder_boot = fluid.data(name='boot', shape=[None, 10], dtype='float32')

drnn = fluid.layers.DynamicRNN()
with drnn.block():
    # Set sentence as RNN's input, each time step processes a word from the sentence
    current_word = drnn.step_input(sentence)
    # Set encode_proj as RNN's static input
    encoder_word = drnn.static_input(encoder_proj)
    # Initialize memory with boot_memory, which need reorder according to RNN's input sequences
    memory = drnn.memory(init=decoder_boot, need_reorder=True)
    fc_1 = fluid.layers.fc(input=encoder_word, size=30)
    fc_2 = fluid.layers.fc(input=current_word, size=30)
    decoder_inputs = fc_1 + fc_2
    hidden, _, _ = fluid.layers.gru_unit(input=decoder_inputs, hidden=memory, size=30)
    # Update memory with hidden
    drnn.update_memory(ex_mem=memory, new_mem=hidden)
    out = fluid.layers.fc(input=hidden, size=10, bias_attr=True, act='softmax')
    # Set hidden and out as RNN's outputs
    drnn.output(hidden, out)

# Get RNN's result
hidden, out = drnn()
# Get RNN's result of the last time step
last = fluid.layers.sequence_last_step(out)

step_input ( x, level=0 )

step_input¶

This function is used to set sequence x as DynamicRNN’s input. The maximum sequence length in x determines the number of time steps the RNN unit will be executed. DynamicRNN can take multiple inputs. When all inputs’ lod_level are 1, all inputs should hold the same LoD. When x.lod_level >= 2 , the input sequence will be unfold along specified level, and the slice of each time step is a LoDTensor whose lod_level is x.lod_level - level - 1 . In this case, the specified LoD level of multiple inputs should be the same.

• Case 1:

# input, where Si is slice data of shape [1, N]
level = 0
x.lod = [[2, 1, 3]]
x.shape = [6, N]
x.data = [[S0],
          [S0],
          [S1],
          [S2],
          [S2],
          [S2]]

# output
# step 0, time step data of 3 sequences
out.lod = [[]]
out.shape = [3, N]
out.data = [[S2],
            [S0],
            [S1]]

# step 1, time step data of 2 sequences
out.lod = [[]]
out.shape = [2, N]
out.data = [[S2],
            [S0]]

# step 2, time step data of 1 sequences
out.lod = [[]]
out.shape = [1, N]
out.data = [[S2]]

Parameters

• x (Variable) – The input LoDTensor which holds information of a minibatch of variable-length sequences and should meet x.lod_level >= 1 . When RNN has multiple inputs, the first dimension should match across all inputs, but other shape components may differ. Optional data types are: bool, float16, float32, float64, int8, int16, int32, int64, uint8.

• level (int, optional) – The level of lod used to split steps. It should be in range \([0, x.lod\_level)\) . The default value is 0.

Returns

The current time step in the input sequence. If there are num_sequences

sequences in x whose length is larger than step_idx , the returned Variable will only hold the step_idx -th time step of those num_sequences sequences. The data type is the same as input. If x.lod_level == 1 , the return value is a Tensor of shape \(\{num\_sequences, x.shape[1], ...\}\) , or it will be a variable-length LoDTensor.

Return type

Variable

Raises

• ValueError – When step_input() is called outside block() .

• TypeError – When x is not a Variable.

Examples

import paddle.fluid as fluid

sentence = fluid.data(name='sentence', shape=[None, 1], dtype='int64', lod_level=1)
embedding = fluid.layers.embedding(input=sentence, size=[65536, 32], is_sparse=True)

drnn = fluid.layers.DynamicRNN()
with drnn.block():
    # Set embedding as RNN's input, each time step processes a word from the sentence
    word = drnn.step_input(embedding)
    # Initialize memory to a Tensor whose value is 0, shape=[batch_size, 200],
    # where batch_size is the number of sequences in embedding.
    memory = drnn.memory(shape=[200])
    hidden = fluid.layers.fc(input=[word, memory], size=200, act='relu')
    # Update memory to hidden
    drnn.update_memory(ex_mem=memory, new_mem=hidden)
    # Set hidden as RNN's output
    drnn.output(hidden)

# Get RNN's result
rnn_output = drnn()

static_input ( x )

static_input¶

This function is used to set x as DynamicRNN’s static input. It is optional.

• Case 1, set static input with LoD

# RNN's input is the same as the case listed in step_input
# static input, where Si is slice data of shape [1, M]
x.lod = [[3, 1, 2]]
x.shape = [6, M]
x.data = [[S0],
          [S0],
          [S0],
          [S1],
          [S2],
          [S2]]

# step 0, batch data corresponding to the 3 input sequences
out.lod = [[2, 3, 1]]
out.shape = [6, M]
out.data = [[S2],
            [S2],
            [S0],
            [S0],
            [S0],
            [S1]]

# step 1, batch data corresponding to the 2 input sequences
out.lod = [[2, 3]]
out.shape = [5, M]
out.data = [[S2],
            [S2],
            [S0],
            [S0],
            [S0]]

# step 2, batch data corresponding to the 1 input sequences
out.lod = [[2]]
out.shape = [2, M]
out.data = [[S2],
            [S2]]

• Case 2, set static input without LoD

# RNN's input is the same as the case listed in step_input
# static input, where Si is slice data of shape [1, M]
x.lod = [[]]
x.shape = [3, M]
x.data = [[S0],
          [S1],
          [S2]]

# step 0, batch data corresponding to the 3 input sequences
out.lod = [[]]
out.shape = [3, M]
out.data = [[S2],
            [S0],
            [S1]]

# step 1, batch data corresponding to the 2 input sequences
out.lod = [[]]
out.shape = [2, M]
out.data = [[S2],
            [S0]]

# step 2, batch data corresponding to the 1 input sequences
out.lod = [[]]
out.shape = [1, M]
out.data = [[S2]]

Parameters

x (Variable) – The static input LoDTensor which should hold the same number of sequences as RNN’s input (the input LoDTensor set by step_input() ). If the LoD is None, the input x will be treated as a minibatch with x.shape[0] sequences of length 1. Optional data types are: bool, float16, float32, float64, int8, int16, int32, int64, uint8.

Returns

The input LoDTensor after sorted and shrank. If there are num_sequences

sequences in RNN’s input LoDTensor whose length is larger than step_idx , the static input Tensor will be sorted to the same order as RNN’s input and will only retain data corresponding to those num_sequences sequences. The data type is the same as input. If x.lod == None , the return value is a Tensor of shape \(\{num\_sequences, x.shape[1], ...\}\) , or it will be a variable-length LoDTensor.

Return type

Variable

Raises

• ValueError – When static_input() is called outside block() .

• TypeError – When x is not a Variable.

• RuntimeError – When static_input() is called before step_input() .

Examples

import paddle.fluid as fluid

sentence = fluid.data(name='sentence', shape=[None, 32], dtype='float32', lod_level=1)
encoder_proj = fluid.data(name='encoder_proj', shape=[None, 32], dtype='float32', lod_level=1)
decoder_boot = fluid.data(name='boot', shape=[None, 10], dtype='float32')

drnn = fluid.layers.DynamicRNN()
with drnn.block():
    # Set sentence as RNN's input, each time step processes a word from the sentence
    current_word = drnn.step_input(sentence)
    # Set encode_proj as RNN's static input
    encoder_word = drnn.static_input(encoder_proj)
    # Initialize memory with boot_memory, which need reorder according to RNN's input sequences
    memory = drnn.memory(init=decoder_boot, need_reorder=True)
    fc_1 = fluid.layers.fc(input=encoder_word, size=30)
    fc_2 = fluid.layers.fc(input=current_word, size=30)
    decoder_inputs = fc_1 + fc_2
    hidden, _, _ = fluid.layers.gru_unit(input=decoder_inputs, hidden=memory, size=30)
    # Update memory with hidden
    drnn.update_memory(ex_mem=memory, new_mem=hidden)
    out = fluid.layers.fc(input=hidden, size=10, bias_attr=True, act='softmax')
    # Set out as RNN's output
    drnn.output(out)

# Get RNN's result
rnn_output = drnn()

block ( )

block¶

The function is used to list the operations executed during each time step in RNN. The operation list will be executed max_sequence_len times (where max_sequence_len is the maximum length of RNN’s input sequences).

Raises

ValueError – When block() is called multi-times.

memory ( init=None, shape=None, value=0.0, need_reorder=False, dtype='float32' )

memory¶

Create a memory Variable for DynamicRNN to deliver data cross time steps. It can be initialized by an existing Tensor or a constant Tensor of given dtype and shape.

Parameters

• init (Variable, optional) – LoDTensor used to initialize the memory. If init is not None, it should hold the same number of sequences as RNN’s input (the input LoDTensor set by step_input() ) and the memory will be initialized to it. If init’s LoD is None, it will be treated as a minibatch with init.shape[0] sequences of length 1. The default value is None.

• shape (list|tuple, optional) – When init is None, it is used to specify the memory’s shape. Note that the shape does not include the batch_size. If setting shape to \(\{D_1, D_2, ...\}\) , the shape of memory Tensor will be \(\{batch\_size, D_1, D_2, ...\}\) , where batch_size is determined by RNN’s input sequences. The default value is None.

• value (float, optional) – When init is None, it is used as initialized value of memory. The default value is 0.0.

• need_reorder (bool, optional) – When init is not None, it determines whether the memory needs to reorder like the RNN’s input sequences. It should be set to True when the initialized memory depends on the order of input samples. The default value is False.

• dtype (str|numpy.dtype, optional) – When init is None, it is used to set the data type of memory. The default value is “float32”. Optional data types are: “float32”, “float64”, “int32”, “int64”.

Returns

The memory LoDTensor after shrank. If there are num_sequences

sequences in RNN’s input LoDTensor whose length is larger than step_idx , the memory Tensor also need to be shrank and will only retain data corresponding to those num_sequences sequences.

Return type

Variable

Raises

• ValueError – When memory() is called outside block() .

• TypeError – When init is set and is not a Variable.

• ValueError – When memory() is called before step_input() .

Examples

import paddle.fluid as fluid

sentence = fluid.data(name='sentence', shape=[None, 32], dtype='float32', lod_level=1)
boot_memory = fluid.data(name='boot', shape=[None, 10], dtype='float32')

drnn = fluid.layers.DynamicRNN()
with drnn.block():
    # Set sentence as RNN's input, each time step processes a word from the sentence
    word = drnn.step_input(sentence)
    # Initialize memory with boot_memory, which need reorder according to RNN's input sequences
    memory = drnn.memory(init=boot_memory, need_reorder=True)
    hidden = fluid.layers.fc(input=[word, memory], size=10, act='tanh')
    # Update memory with hidden
    drnn.update_memory(ex_mem=memory, new_mem=hidden)
    # Set hidden as RNN's output
    drnn.output(hidden)

# Get RNN's result
rnn_output = drnn()

Examples

import paddle.fluid as fluid

sentence = fluid.data(name='sentence', shape=[None, 32], dtype='float32', lod_level=1)

drnn = fluid.layers.DynamicRNN()
with drnn.block():
    # Set sentence as RNN's input, each time step processes a word from the sentence
    word = drnn.step_input(sentence)
    # Initialize memory to a Tensor whose value is 0, shape=[batch_size, 10],
    # where batch_size is the number of sequences in sentence.
    memory = drnn.memory(shape=[10], dtype='float32', value=0)
    hidden = fluid.layers.fc(input=[word, memory], size=10, act='tanh')
    # Update memory with hidden
    drnn.update_memory(ex_mem=memory, new_mem=hidden)
    # Set hidden as RNN's output
    drnn.output(hidden)

# Get RNN's result
rnn_output = drnn()

update_memory ( ex_mem, new_mem )

update_memory¶

Update the memory which need to be delivered across time steps.

Parameters

• ex_mem (Variable) – The memory data of previous time step.

• new_mem (Variable) – The new memory data produced in current time step. The shape and data type of ex_mem and new_mem should be the same.

Returns

None

Raises

• ValueError – When update_memory() is called outside block() .

• TypeError – When ex_mem or new_mem is not a Variable.

• ValueError – When ex_mem is defined by memory() .

• ValueError – When update_memory() is called before step_input() .

output ( *outputs )

output¶

This function is used to set outputs as RNN’s output.

Parameters

*outputs (Variable ...) – The output Tensor. DynamicRNN can mark multiple Variables as its output.

Returns

None

Raises

ValueError – When output() is called outside block() .