NetInsertSharedArrays

NetInsertSharedArrays[net]

converts all ordinary arrays in net into NetSharedArray objects.

NetInsertSharedArrays[net,"prefix"]

uses a prefix for the names of all newly shared arrays.

Details

  • Each array in the net will be shared using a name that is derived from its position within the net.
  • By sharing all arrays in a net, multiple instances of that net within a larger net will behave identically during training and evaluation.

Examples

open allclose all

Basic Examples  (3)

Convert arrays in a layer into shared arrays:

Convert arrays in a layer into shared arrays, using a specific prefix:

Convert arrays in layers within a net into shared arrays:

Properties & Relations  (1)

Insert shared arrays into a layer:

When used multiple times in a larger network, only one set of weights will be created and stored:

When extracted, the weights for layers 1 and 3 are identical:

The underlying shared array can also be extracted using NetExtract:

The amount of storage required for arrays in the entire net is the same as the storage required for a single layer:

Possible Issues  (1)

The "MovingMean" and "MovingVariance" arrays of BatchNormalizationLayer cannot be shared.

Create a BatchNormalizationLayer with shared arrays:

Train it on some data:

Extract the trained batch normalization layers:

The "Scaling" and "Biases" arrays were shared, but not "MovingMean" or "MovingVariance":

Introduced in 2018
 (11.3)
 |
Updated in 2019
 (12.0)