Wolfram Language & System 11.0 (2016)|Legacy Documentation

This is documentation for an earlier version of the Wolfram Language.View current documentation (Version 11.2)

BatchNormalizationLayer

BatchNormalizationLayer[]
represents a trainable net layer that normalizes its input data by learning the data mean and variance.

Details and OptionsDetails and Options

  • The following optional parameters can be included:
  • "Gamma"Automaticlearnable scaling parameters
    "Beta"Automaticlearnable bias parameters
    "MovingVariance"Automaticmoving estimate of the variance
    "MovingMean"Automaticmoving estimate of the mean
    "Momentum"0.9momentum used during training
    "Epsilon"0.001`stability parameter
  • With Automatic settings, gamma, beta, moving variance, and moving mean are added automatically when NetInitialize or NetTrain is used.
  • If gamma, beta, moving variance, and moving mean have been added, BatchNormalizationLayer[][input] explicitly computes the output from applying the layer.
  • BatchNormalizationLayer[][{input1,input2,}] explicitly computes outputs for each of the inputi.
  • NetExtract can be used to extract gamma, beta, moving variance, and moving mean from a BatchNormalizationLayer object.
  • BatchNormalizationLayer is typically used inside NetChain, NetGraph, etc.
  • BatchNormalizationLayer exposes the following ports for use in NetGraph etc.:
  • "Input"a rank-1 or rank-3 numerical tensor
    "Output"a rank-1 or rank-3 numerical tensor
  • When it cannot be inferred from other layers in a larger net, the option "Input"->{n1,n2,} can be used to fix the input dimensions of BatchNormalizationLayer.

ExamplesExamplesopen allclose all

Basic Examples  (2)Basic Examples  (2)

Create a batch normalization layer:

In[1]:=
Click for copyable input
Out[1]=

Create an uninitialized layer with the input dimensions specified:

In[1]:=
Click for copyable input
Out[1]=

Initialize the layer with random weights:

In[2]:=
Click for copyable input
Out[2]=

Apply the layer to an input vector to produce an output vector:

In[3]:=
Click for copyable input
Out[3]=
Introduced in 2016
(11.0)