linear_chain_crf

paddle.fluid.layers.nn. linear_chain_crf ( input, label, param_attr=None, length=None ) [source]
Api_attr

Static Graph

Linear Chain CRF.

Conditional Random Field defines an undirected probabilistic graph with nodes denoting random variables and edges denoting dependencies between these variables. CRF learns the conditional probability $P(Y|X)$, where $X = (x_1, x_2, … , x_n)$ are structured inputs and $Y = (y_1, y_2, … , y_n)$ are labels for the inputs.

Linear chain CRF is a special case of CRF that is useful for sequence labeling task. Sequence labeling tasks do not assume a lot of conditional independences among inputs. The only constraint they impose is that the input and output must be linear sequences. Thus, the graph of such a CRF is a simple chain or a line, which results in the linear chain CRF.

This operator implements the Forward-Backward algorithm for the linear chain CRF. Please refer to http://www.cs.columbia.edu/~mcollins/fb.pdf and http://cseweb.ucsd.edu/~elkan/250Bwinter2012/loglinearCRFs.pdf for details.

Equation:

  1. Denote Input(Emission) to this operator as $x$ here. 2. The first D values of Input(Transition) to this operator are for starting weights, denoted as $a$ here. 3. The next D values of Input(Transition) of this operator are for ending weights, denoted as $b$ here. 4. The remaning values of Input(Transition) are for transition weights, denoted as $w$ here. 5. Denote Input(Label) as $s$ here.

The probability of a sequence $s$ of length $L$ is defined as: $$P(s) = (1/Z) exp(a_{s_1} + b_{s_L} + sum_{l=1}^L x_{s_l} + sum_{l=2}^L w_{s_{l-1},s_l})$$

where $Z$ is a normalization value so that the sum of $P(s)$ over all possible sequences is 1, and $x$ is the emission feature weight to the linear chain CRF.

Finally, the linear chain CRF operator outputs the logarithm of the conditional likelihood of each training sample in a mini-batch.

NOTE:

  1. The feature function for a CRF is made up of the emission features and the transition features. The emission feature weights are NOT computed in this operator. They MUST be computed first before this operator is called.

  2. Because this operator performs global normalization over all possible sequences internally, it expects UNSCALED emission feature weights. Please do not call this op with the emission feature being output of any nonlinear activation.

  3. The 2nd dimension of Input(Emission) MUST be equal to the tag number.

Parameters

  • input (Variable) – (LoDTensor/Tensor<float>). When a LoDTensor input,A 2-D LoDTensor with shape [N x D], where N is the size of the mini-batch and D is the total tag number. The unscaled emission weight matrix for the linear chain CRF. When a Tensor input,A Tensor with shape [N x S x D], where N is batch number,S is max length of sequences, D is the total tag number.A LoDTensor or Tensor with type float32, float64

  • label (Variable) – (LoDTensor/Tensor<int64_t>), when a LoDTensor input, [N x 1], where N is the total element number in a mini-batch. when a Tensor input, [N x S], where N is batch number. S is max length of sequences. The ground truth.A LoDTensor or Tensor with int64

  • Length (Variable) – (Tensor, default Tensor<int64_t>) A Tensor with shape [M x 1], where M is the sequence number in a mini-batch.A Tensor with type int64

  • param_attr (ParamAttr) – The attribute of the learnable parameter for transition parameter.

Returns

(Tensor, default Tensor<float>), the same shape with Emission. The exponentials of Input(Emission). This is an intermediate computational result in forward computation, and will be reused in backward computation.A LoDTensor or Tensor with type float32, float64

output(Variable): (Tensor, default Tensor<float>) A 2-D Tensor with shape [(D + 2) x D]. The exponentials of Input(Transition). This is an intermediate computational result in forward computation, and will be reused in backward computation.A LoDTensor or Tensor with type float32, float64

output(Variable): (Tensor, default Tensor<float>) The logarithm of the conditional likelihood of each training sample in a mini-batch. This is a 2-D tensor with shape [S x 1], where S is the sequence number in a mini-batch. Note: S is equal to the sequence number in a mini-batch. A Tensor with type float32, float64

Return type

output(Variable)

Examples

import paddle.fluid as fluid
import numpy as np
import paddle
paddle.enable_static()

#define net structure, using LodTensor
train_program = fluid.Program()
startup_program = fluid.Program()
with fluid.program_guard(train_program, startup_program):
    input_data = fluid.data(name='input_data', shape=[-1,10], dtype='float32')
    label = fluid.data(name='label', shape=[-1,1], dtype='int')
    emission= fluid.layers.fc(input=input_data, size=10, act="tanh")
    crf_cost = fluid.layers.linear_chain_crf(
        input=emission,
        label=label,
        param_attr=fluid.ParamAttr(
        name='crfw',
        learning_rate=0.01))
use_cuda = False
place = fluid.CUDAPlace(0) if use_cuda else fluid.CPUPlace()
exe = fluid.Executor(place)
exe.run(startup_program)
#define data, using LoDTensor
a = fluid.create_lod_tensor(np.random.rand(12,10).astype('float32'), [[3,3,4,2]], place)
b = fluid.create_lod_tensor(np.array([[1],[1],[2],[3],[1],[1],[1],[3],[1],[1],[1],[1]]),[[3,3,4,2]] , place)
feed1 = {'input_data':a,'label':b}
loss= exe.run(train_program,feed=feed1, fetch_list=[crf_cost])
print(loss)

#define net structure, using padding
train_program = fluid.Program()
startup_program = fluid.Program()
with fluid.program_guard(train_program, startup_program):
    input_data2 = fluid.data(name='input_data2', shape=[-1,10,10], dtype='float32')
    label2 = fluid.data(name='label2', shape=[-1,10,1], dtype='int')
    label_length = fluid.data(name='length', shape=[-1,1], dtype='int')
    emission2= fluid.layers.fc(input=input_data2, size=10, act="tanh", num_flatten_dims=2)
    crf_cost2 = fluid.layers.linear_chain_crf(
        input=emission2,
        label=label2,
        length=label_length,
        param_attr=fluid.ParamAttr(
         name='crfw',
         learning_rate=0.01))

use_cuda = False
place = fluid.CUDAPlace(0) if use_cuda else fluid.CPUPlace()
exe = fluid.Executor(place)
exe.run(startup_program)

#define data, using padding
cc=np.random.rand(4,10,10).astype('float32')
dd=np.random.rand(4,10,1).astype('int64')
ll=np.array([[3],[3],[4],[2]])
feed2 = {'input_data2':cc,'label2':dd,'length':ll}
loss2= exe.run(train_program,feed=feed2, fetch_list=[crf_cost2])
print(loss2)
#[array([[ 7.8902354],
#        [ 7.3602567],
#        [ 10.004011],
#        [ 5.86721  ]], dtype=float32)]

#you can use find_var to get transition parameter.
transition=np.array(fluid.global_scope().find_var('crfw').get_tensor())
print(transition)