BatchNormalizationLayer

BatchNormalizationLayer[]

represents a trainable net layer that normalizes its input data by learning the data mean and variance.

Details and Options

  • The following optional parameters can be included:
  • "Epsilon"0.001`stability parameter
    "Momentum"0.9momentum used during training
  • The following learnable arrays can be included:
  • "Biases"Automaticlearnable bias array
    "MovingMean"Automaticmoving estimate of the mean
    "MovingVariance"Automaticmoving estimate of the variance
    "Scaling"Automaticlearnable scaling array
  • With Automatic settings, the biases, scaling, moving mean, and moving variance arrays are initialized automatically when NetInitialize or NetTrain is used.
  • If biases, scaling, moving variance, and moving mean have been added, BatchNormalizationLayer[][input] explicitly computes the output from applying the layer.
  • BatchNormalizationLayer[][{input1,input2,}] explicitly computes outputs for each of the inputi.
  • When given a NumericArray as input, the output will be a NumericArray.
  • NetExtract can be used to extract biases, scaling, moving variance, and moving mean arrays from a BatchNormalizationLayer object.
  • BatchNormalizationLayer is typically used inside NetChain, NetGraph, etc. to regularize and speed up network training.
  • BatchNormalizationLayer exposes the following ports for use in NetGraph etc.:
  • "Input"a vector, matrix or higher-rank array
    "Output"a vector, matrix or higher-rank array
  • When it cannot be inferred from other layers in a larger net, the option "Input"->{n1,n2,} can be used to fix the input dimensions of BatchNormalizationLayer.
  • BatchNormalizationLayer updates the values of "MovingVariance" and "MovingMean" during training with NetTrain.

Examples

open all close all

Basic Examples  (2)

Create a BatchNormalizationLayer:

In[1]:=
Click for copyable input
Out[1]=

Create an initialized BatchNormalizationLayer that takes a vector and returns a vector:

In[1]:=
Click for copyable input
Out[1]=

Apply the layer to an input vector:

In[2]:=
Click for copyable input
Out[2]=

Scope  (2)

Options  (3)

Applications  (1)

Properties & Relations  (1)

Possible Issues  (3)

Introduced in 2016
(11.0)
|
Updated in 2019
(12.0)