LearningRate
is an option for NetTrain that specifies the rate at which to adjust neural net weights in order to minimize the training loss.
Details
- LearningRate->Automatic specifies that NetTrain should choose a learning rate automatically.
- LearningRate->r specifies that the learning rate should be the number r.
- The base learning rate for all learned weights is established using LearningRate. Use LearningRateMultipliers to modify this rate for particular parts of the net.
- Typical learning rates are numbers much smaller than 1; values of between 0.001 and 0.01 are common. Choosing a learning rate that is too high can cause the net to fail to converge to a good solution. Choosing a learning rate that is too low will cause the training process to take longer than necessary.
- Learning rates are usually not comparable across different optimizers specified via the Method option of NetTrain.
Examples
Wolfram Research (2019), LearningRate, Wolfram Language function, https://reference.wolfram.com/language/ref/LearningRate.html.
Text
Wolfram Research (2019), LearningRate, Wolfram Language function, https://reference.wolfram.com/language/ref/LearningRate.html.
CMS
Wolfram Language. 2019. "LearningRate." Wolfram Language & System Documentation Center. Wolfram Research. https://reference.wolfram.com/language/ref/LearningRate.html.
APA
Wolfram Language. (2019). LearningRate. Wolfram Language & System Documentation Center. Retrieved from https://reference.wolfram.com/language/ref/LearningRate.html