LearningRate

LearningRate

is an option for NetTrain that specifies the rate at which to adjust neural net weights in order to minimize the training loss.

Details

  • LearningRate->Automatic specifies that NetTrain should choose a learning rate automatically.
  • LearningRate->r specifies that the learning rate should be the number r.
  • The base learning rate for all learned weights is established using LearningRate. Use LearningRateMultipliers to modify this rate for particular parts of the net.
  • Typical learning rates are numbers much smaller than 1; values of between 0.001 and 0.01 are common. Choosing a learning rate that is too high can cause the net to fail to converge to a good solution. Choosing a learning rate that is too low will cause the training process to take longer than necessary.
  • Learning rates are usually not comparable across different optimizers specified via the Method option of NetTrain.

Examples

Basic Examples  (1)

Train a network with a learning rate of 0.01:

In[1]:=
Click for copyable input
Out[1]=
Introduced in 2019
(12.0)