TrainingProgressFunction

TrainingProgressFunction

is an option for NetTrain that specifies a function to run periodically during training.

Details

  • With the default value of TrainingProgressFunction->None, no function is run.
  • Setting TrainingProgressFunction->f specifies that f[assoc] is evaluated after every training round, where assoc is an association with the following keys:
  • "Net"current, partially trained network
    "Weights"association between weight position within net and current value
    "Gradients"association between weight position within net and current gradient
    "RoundLoss"average loss of most recent round
    "BatchLoss"average loss of most recent batch
    "Round"current round number
    "Batch"current batch number within this round
    "AbsoluteBatch"total number of batches processed so far
    "MaxRounds"maximum number of rounds that will be performed
    "MaxBatches"maximum number of batches that will be performed
    "BatchSize"the number of inputs contained in a batch
    "BatchesPerRound"the number of batches contained in a single round
    "BatchesPerSecond"the current training rate in batches per second
    "InputsPerSecond"the training rate in inputs per second
    "ProgressFraction"progress represented as a number between 0 and 1
    "TimeRemaining"estimated time remaining, in seconds
    "TimeElapsed"time elapsed since training began, in seconds
    "ValidationLoss"most recent validation loss
    "LowestValidationLoss"lowest validation loss encountered so far
  • The keys "ValidationLoss" and "LowestValidationLoss" are applicable only in the case that the option ValidationSet was specified to NetTrain.
  • Setting TrainingProgressFunction->{f,"Interval"->Quantity[n,"unit"]} specifies the interval at which to apply f. Possible forms of "unit" include:
  • "Rounds"net training rounds
    "Batches"training data batches
    "Seconds","Minutes","Hours"absolute time
  • The suboption {f,,"MinimumInterval"n} specifies that f should not be applied more frequently than once every n seconds. If unspecified, there is no limit on how frequently f is applied.
  • Setting TrainingProgressFunction->{spec1,spec2,} specifies multiple functions to evaluate, which can have different intervals.

Examples

Basic Examples  (2)

Use TrainingProgressFunction to append information about the state of training to a file. Create a log file:

In[1]:=
Click for copyable input
Out[1]=

Define functions to append the batch number and loss to the log file:

In[2]:=
Click for copyable input

Define the training data and perform training:

In[3]:=
Click for copyable input
In[4]:=
Click for copyable input

Read the log file:

In[5]:=
Click for copyable input
Out[5]=

Put the saved data into a Dataset:

In[6]:=
Click for copyable input
Out[6]=

Plot the loss over value over time:

In[7]:=
Click for copyable input
Out[7]=

Use TrainingProgressFunction to plot the intermediate curves fitted to a least-squares problem. First generate training data:

In[1]:=
Click for copyable input
In[2]:=
Click for copyable input
In[3]:=
Click for copyable input
Out[3]=

Create a network to fit the data:

In[4]:=
Click for copyable input

Every 100 training rounds, append a visualization of the current solution to a list:

In[5]:=
Click for copyable input

Animate the list to show the solution converging over time:

In[6]:=
Click for copyable input
Out[6]=

See Also

NetTrain  TrainingProgressReporting  TrainingProgressCheckpointing  ClassifierMeasurements

Introduced in 2017
(11.1)