LearningRate

LearningRate

is an option for NetTrain that specifies the rate at which to adjust neural net weights in order to minimize the training loss.

Details

  • LearningRate->Automatic specifies that NetTrain should choose a learning rate automatically.
  • LearningRate->r specifies that the learning rate should be the number r.
  • The base learning rate for all learned weights is established using LearningRate. Use LearningRateMultipliers to modify this rate for particular parts of the net.
  • Typical learning rates are numbers much smaller than 1; values of between 0.001 and 0.01 are common. Choosing a learning rate that is too high can cause the net to fail to converge to a good solution. Choosing a learning rate that is too low will cause the training process to take longer than necessary.
  • Learning rates are usually not comparable across different optimizers specified via the Method option of NetTrain.

Examples

Basic Examples  (1)

Train a network with a learning rate of 0.01:

Wolfram Research (2019), LearningRate, Wolfram Language function, https://reference.wolfram.com/language/ref/LearningRate.html.

Text

Wolfram Research (2019), LearningRate, Wolfram Language function, https://reference.wolfram.com/language/ref/LearningRate.html.

CMS

Wolfram Language. 2019. "LearningRate." Wolfram Language & System Documentation Center. Wolfram Research. https://reference.wolfram.com/language/ref/LearningRate.html.

APA

Wolfram Language. (2019). LearningRate. Wolfram Language & System Documentation Center. Retrieved from https://reference.wolfram.com/language/ref/LearningRate.html

BibTeX

@misc{reference.wolfram_2022_learningrate, author="Wolfram Research", title="{LearningRate}", year="2019", howpublished="\url{https://reference.wolfram.com/language/ref/LearningRate.html}", note=[Accessed: 19-August-2022 ]}

BibLaTeX

@online{reference.wolfram_2022_learningrate, organization={Wolfram Research}, title={LearningRate}, year={2019}, url={https://reference.wolfram.com/language/ref/LearningRate.html}, note=[Accessed: 19-August-2022 ]}