spectral_norm¶

paddle.nn.utils. spectral_norm ( layer, name='weight', n_power_iterations=1, eps=1e-12, dim=None ) [source]

Applies spectral normalization to a parameter according to the following Calculation:

Step 1: Generate vector U in shape of [H], and V in shape of [W]. While H is the dim th dimension of the input weights, and W is the product result of remaining dimensions.

Step 2: n_power_iterations should be a positive integer, do following calculations with U and V for power_iters rounds.

\[ \begin{align}\begin{aligned}\mathbf{v} := \frac{\mathbf{W}^{T} \mathbf{u}}{\|\mathbf{W}^{T} \mathbf{u}\|_2}\\\mathbf{u} := \frac{\mathbf{W} \mathbf{v}}{\|\mathbf{W} \mathbf{v}\|_2}\end{aligned}\end{align} \]

Step 3: Calculate \(\sigma(\mathbf{W})\) and normalize weight values.

\[ \begin{align}\begin{aligned}\sigma(\mathbf{W}) = \mathbf{u}^{T} \mathbf{W} \mathbf{v}\\\mathbf{W} = \frac{\mathbf{W}}{\sigma(\mathbf{W})}\end{aligned}\end{align} \]

Refer to Spectral Normalization .

Parameters

layer (Layer) – Layer of paddle, which has weight.
name (str, optional) – Name of the weight parameter. Default: ‘weight’.
n_power_iterations (int, optional) – The number of power iterations to calculate spectral norm. Default: 1.
eps (float, optional) – The epsilon for numerical stability in calculating norms. Default: 1e-12.
dim (int, optional) – The index of dimension which should be permuted to the first before reshaping Input(Weight) to matrix, it should be set as 0 if Input(Weight) is the weight of fc layer, and should be set as 1 if Input(Weight) is the weight of conv layer. Default: None.

Returns

Layer, the original layer with the spectral norm hook.

Examples

           >>> from paddle.nn import Conv2D
>>> from paddle.nn.utils import spectral_norm
>>> paddle.seed(2023)
>>> conv = Conv2D(3, 1, 3)
>>> sn_conv = spectral_norm(conv)
>>> print(sn_conv)
Conv2D(3, 1, kernel_size=[3, 3], data_format=NCHW)
>>> # Conv2D(3, 1, kernel_size=[3, 3], data_format=NCHW)
>>> print(sn_conv.weight)
Tensor(shape=[1, 3, 3, 3], dtype=float32, place=Place(cpu), stop_gradient=False,
[[[[ 0.01668976,  0.30305523,  0.11405435],
   [-0.06765547, -0.50396705, -0.40925547],
   [ 0.47344422,  0.03628403,  0.45277366]],
  [[-0.15177251, -0.16305730, -0.15723954],
   [-0.28081197, -0.09183260, -0.08081978],
   [-0.40895155,  0.18298769, -0.29325116]],
  [[ 0.21819633, -0.01822380, -0.50351536],
   [-0.06262003,  0.17713565,  0.20517939],
   [ 0.16659889, -0.14333329,  0.05228264]]]])

          