ExponentialDecay¶

class paddle.fluid.dygraph.learning_rate_scheduler. ExponentialDecay ( learning_rate, decay_steps, decay_rate, staircase=False, begin=0, step=1, dtype='float32' ) [source]

Api_attr

imperative

Applies exponential decay to the learning rate.

The algorithm can be described as following.

\[decayed\_learning\_rate = learning\_rate * decay\_rate ^ y\]

If staircase is set to False, then:

\[\begin{split}y = \\frac{global\_step}{decay\_steps}\end{split}\]

If staircase is set to True, then:

\[\begin{split}y = math.floor(\\frac{global\_step}{decay\_steps})\end{split}\]

Parameters

• learning_rate (Variable|float) – The initial learning rate. If the type is Variable, it’s a tensor with shape [1], the data type can be float32 or float64. It also can be set to python int number.

• decay_steps (int) – The decay step size. It determines the decay cycle.

• decay_rate (float) – The decay rate.

• staircase (bool, optional) – If set to True, decay the learning rate at discrete intervals. The default value is False.

• begin (int, optional) – The begin step. The initial value of global_step described above. The default value is 0.

• step (int, optional) – The step size used to calculate the new global_step in the description above. The default value is 1.

• dtype (str, optional) – The data type used to create the learning rate variable. The data type can be set as ‘float32’, ‘float64’. The default value is ‘float32’.

Returns

None.

Examples

import paddle.fluid as fluid
base_lr = 0.1
with fluid.dygraph.guard():
    sgd_optimizer = fluid.optimizer.SGD(
          learning_rate=fluid.dygraph.ExponentialDecay(
              learning_rate=base_lr,
              decay_steps=10000,
              decay_rate=0.5,
              staircase=True))

create_lr_var ( lr )

create_lr_var¶

convert lr from float to variable

Parameters

lr – learning rate

Returns

learning rate variable

set_dict ( state_dict )

set_dict¶

Loads the schedulers state.

set_state_dict ( state_dict )

set_state_dict¶

Loads the schedulers state.

state_dict ( )

state_dict¶

Returns the state of the scheduler as a dict.

It is a subset of self.__dict__ .