- With the default of TrainingStoppingCriterionNone, no early stopping is performed.
- Setting TrainingStoppingCriterion"measurement" specifies that training should be stopped if measurement stops improving. Possible values for measurement include:
"Loss" network training loss key a TrainingProgressMeasurements key
- The values available for key depend on the specification of TrainingProgressMeasurements. Often, the key is the same as the measurement. For example, with TrainingProgressMeasurements"ErrorRate", key can be "ErrorRate". However, this is not always the case.
- TrainingStoppingCriterionAutomatic is equivalent to TrainingStoppingCriterion"Loss".
- By default, TrainingStoppingCriterion is based on the metrics and loss from the validation set. If the ValidationSet option of NetTrain is None, then a warning will be issued and the training set will be used. If the validation set is present, the stopping criterion is checked whenever the validation loss and metrics are calculated (once per round by default); otherwise, the stopping criterion is checked once per round.
- TrainingStoppingCriterion has a number of suboptions that can be specified using the <|"Criterion""measurement",opt1val1,opt2val2,…|> syntax.
- Setting TrainingStoppingCriterion<|"Criterion""measurement","Patience"n|> specifies that training should be stopped if an improvement in measurement is not seen for n rounds in a row. The default value for n is 0.
- Setting TrainingStoppingCriterion<|"Criterion""measurement","InitialPatience"n |> specifies that the stopping criterion is only checked for the first time after n rounds. The default value for n is 0.
- Setting TrainingStoppingCriterion<|"Criterion""measurement","improvement"v|> specifies the minimum change in measurement that is considered an improvement. Possible values for improvement are:
"AbsoluteChange" stop training if measurement improves by less than v "RelativeChange" stop training if measurement improves by less than a factor v of the current best value
- If an improvement specification is not given, then "AbsoluteChange"0 is used.
- The form "RelativeChange"Quantity[q,"Percent"] can also be given.
- It is also possible to specify a function as the stopping criterion using TrainingStoppingCriterion<|"Criterion"func,…|>, where func should return True if training should be stopped.
- The function func is provided an association with the following keys:
RoundLoss total loss for the training set ValidationLoss total loss for the validation set RoundMeasurements association of requested measurements for the training set ValidationMeasurements association of requested measurements for the validation set
- The validation properties are only available if there is a validation set.
- The "RelativeChange" and "AbsoluteChange" options cannot be used with a function criterion.
- If a measurement is specified for TrainingStoppingCriterion, the value of the measurement will be used to select the optimal trained net to return from NetTrain. Note that the choices of "Patience", "InitialPatience", "AbsoluteChange" and "RelativeChange" have no effect on this selection. If a function is specified for TrainingStoppingCriterion, the default behavior will be used when selecting the optimal trained net.
Examplesopen all close all
Basic Examples (1)
Use TrainingStoppingCriterion to stop training when the validation loss stops improving:
Properties & Relations (2)
Introduced in 2019