gives a net in which all uninitialized learnable parameters in net have been given initial values.


gives a net in which all learnable parameters have been given initial values.

Details and Options

  • NetInitialize[net,All] overwrites any existing training or preset learnable parameters in net.
  • NetInitialize typically assigns random values to parameters representing weights and zero to parameters representing biases.
  • The following optional parameters can be included:
  • MethodAutomaticwhich initialization method to use
    RandomSeedingInheritedseeding of pseudorandom number generator
  • Possible settings for Method include:
  • "Xavier"choose weights to preserve variance of random arrays when propagated through affine layers
    "Orthogonal"choose weights to be orthogonal matrices
    "Random"choose weights from a given univariate distribution
    "Identity"choose weights so as to preserve components of arrays when propogated through affine layers
  • Suboptions for specific methods can be specified using Method{"method",opt1val1,}.
  • For the method "Xavier", the following suboptions are supported:
  • "FactorType""Mean"one of "In", "Out", or "Mean"
    "Distribution""Normal"either "Normal" or "Uniform"
  • For the method "Random", the following suboptions are supported:
  • "Weights"NormalDistribution[0,1]random distribution to use to initialize weight matrices
    "Biases"Nonerandom distribution to use to initialize bias vectors
  • For the method "Identity", the following suboption is supported:
  • "Distribution"NormalDistribution[0,0.01]random distribution used to add noise to the initial identity matrices in order to break symmetries
  • For any suboption that expects a distribution, a numeric value stddev can be specified and is taken to mean NormalDistribution[0,stddev].
  • By default, all methods initialize bias vectors to zero.
  • Possible settings for RandomSeeding include:
  • Automaticautomatically reseed every time the function is called
    Inheriteduse externally seeded random numbers
    seeduse an explicit integer or strings as a seed


open all close all

Basic Examples  (1)

Create an uninitialized layer:

Click for copyable input

Initialize the layer with random weights:

Click for copyable input

Extract the new initialized weights:

Click for copyable input

Scope  (1)

Options  (1)

Properties & Relations  (2)

Possible Issues  (2)

Neat Examples  (1)

Introduced in 2016
Updated in 2019