is an option for NetTrain that specifies which arrays of the network can be updated at each step of the optimization process.


TrainingUpdateSchedule
is an option for NetTrain that specifies which arrays of the network can be updated at each step of the optimization process.
Details

- With the default value of TrainingUpdateScheduleAutomatic, all arrays are updated at any step of the optimization process.
- TrainingUpdateSchedule{group1,group2,…,groupn} specifies which arrays will be updated for each respective optimization step, this schedule being repeated until the end of training.
- In TrainingUpdateSchedule{group1,group2,…,groupn}, each of the groupi can be of the following forms:
-
"layer" all arrays of a named layer or subnetwork n all arrays of the nth layer m;;n all arrays of layers m through n {layer,"array"} a particular array in a layer or subnetwork {part1,part2,…} - arrays of a nested layer or subnetwork
spec1|spec2|… any of the specified arrays _ all arrays in the network specs repeat the same specification for s consecutive steps - The hierarchical specification {part1,part2,…} used by TrainingUpdateSchedule to refer to a subpart of a net is equivalent to that used by NetExtract and NetReplacePart.
- Specifications of a subnet (e.g. a nested NetChain or NetGraph) apply to all layers and arrays within that subnet.
- Any group of parameters not specified in TrainingUpdateSchedule is held constant during training.
Examples
open all close allBasic Examples (1)
Train a NetGANOperator by alternating updates of the discriminator and updates of the generator:
Scope (2)
Create and initialize a net with three layers:
Train this net by updating alternately the first and third layers, and collect the net arrays after each optimization iteration on a batch:
Show the evolution of the first array value through iterations and check that the arrays are alternately updated:
Create and initialize a NetGraph with named layers:
Train the net by updating a subpart of the NetGraph 10 times more than another:
Check how the arrays are updated:
Train the same NetGraph by updating arrays separately, one by one:
Possible Issues (1)
When a shared array occurs at several places in the network, only a unique training update schedule will be applied to all the occurrences of the shared array.
Create a network with shared arrays:
Train with a TrainingUpdateSchedule that specifies that only the first layer should be updated and collects the value of the weights of the third layer after each update:
The shared weights are updated at each epoch:
The results are the same as training without any update schedule:
Tech Notes
History
Text
Wolfram Research (2020), TrainingUpdateSchedule, Wolfram Language function, https://reference.wolfram.com/language/ref/TrainingUpdateSchedule.html.
CMS
Wolfram Language. 2020. "TrainingUpdateSchedule." Wolfram Language & System Documentation Center. Wolfram Research. https://reference.wolfram.com/language/ref/TrainingUpdateSchedule.html.
APA
Wolfram Language. (2020). TrainingUpdateSchedule. Wolfram Language & System Documentation Center. Retrieved from https://reference.wolfram.com/language/ref/TrainingUpdateSchedule.html
BibTeX
@misc{reference.wolfram_2025_trainingupdateschedule, author="Wolfram Research", title="{TrainingUpdateSchedule}", year="2020", howpublished="\url{https://reference.wolfram.com/language/ref/TrainingUpdateSchedule.html}", note=[Accessed: 15-August-2025]}
BibLaTeX
@online{reference.wolfram_2025_trainingupdateschedule, organization={Wolfram Research}, title={TrainingUpdateSchedule}, year={2020}, url={https://reference.wolfram.com/language/ref/TrainingUpdateSchedule.html}, note=[Accessed: 15-August-2025]}