Wolfram Language & System 11.0 (2016)|Legacy Documentation

This is documentation for an earlier version of the Wolfram Language.View current documentation (Version 11.2)

NetInitialize

NetInitialize[net]
gives a net in which all uninitialized learnable parameters in net have been given initial values.

NetInitialize[net,All]
gives a net in which all learnable parameters have been given initial values.

Details and OptionsDetails and Options

  • NetInitialize[net,All] overwrites any existing training or preset learnable parameters in net.
  • NetInitialize typically assigns random values to parameters representing weights and zero to parameters representing biases.
  • The following optional parameters can be included:
  • MethodAutomaticwhich initialization method to use
  • Possible settings for Method include:
  • "Xavier"choose weights to preserve variance of random tensors when propagated through affine layers
    "Orthogonal"choose weights to be orthogonal matrices
    "Random"choose weights from a given univariate distribution
    "Identity"choose weights so as to preserve components of tensors when propogated through affine layers
  • Suboptions for specific methods can be specified using Method{"method",opt1val1,}.
  • For the method "Xavier", the following suboptions are supported:
  • "FactorType""Mean"one of "In", "Out", or "Mean"
    "Distribution""Normal"either "Normal" or "Uniform"
  • For the method "Random", the following suboptions are supported:
  • "Weights"NormalDistribution[0,1]random distribution to use to initialize weight matrices
    "Biases"Nonerandom distribution to use to initialize bias vectors
  • For the method "Identity", the following suboption is supported:
  • "Distribution"NormalDistribution[0,0.01]random distribution used to add noise to the initial identity matrices in order to break symmetries
  • For any suboption that expects a distribution, a numeric value stddev can be specified and is taken to mean NormalDistribution[0,stddev].
  • By default, all methods initialize bias vectors to zero.
Introduced in 2016
(11.0)