- LearningRate->Automatic specifies that NetTrain should choose a learning rate automatically.
- LearningRate->r specifies that the learning rate should be the number r.
- The base learning rate for all learned weights is established using LearningRate. Use LearningRateMultipliers to modify this rate for particular parts of the net.
- Typical learning rates are numbers much smaller than 1; values of between 0.001 and 0.01 are common. Choosing a learning rate that is too high can cause the net to fail to converge to a good solution. Choosing a learning rate that is too low will cause the training process to take longer than necessary.
- Learning rates are usually not comparable across different optimizers specified via the Method option of NetTrain.
Introduced in 2019