NetSharedArray

NetSharedArray["name"]

represents an array in a net that is shared by more than one layer.

Details

  • All instances of NetSharedArray["name"] in a net are treated as the same underlying array by NetInitialize, NetTrain, etc.
  • The value of a shared array can be extracted with NetExtract[net,NetSharedArray["name"]].
  • The value of a shared array can be updated with NetReplacePart[net,NetSharedArray["name"]->value].
  • Any ordinary array of a layer such as LinearLayer, ConvolutionLayer, etc. can be specified to be a shared array.
  • When one or more nets are combined into a larger net, the shared arrays of the larger net will be the union of the shared arrays of the original nets.
  • A list of the values of all shared arrays in a net can be retrieved with NetExtract[net,NetSharedArray[All]].
  • NetInsertSharedArrays[layer] will return a new layer in which all its arrays have been replaced with shared arrays.

Examples

open allclose all

Basic Examples  (4)

Create a layer containing a shared array:

Create a layer, both of whose arrays are shared:

Create a chain that contains a shared array, used in two layers:

Initialize all arrays and extract the value of the shared array:

Extracting the corresponding weights from the two linear layers returns the same value:

Create an LSTM layer whose arrays will be shared every time it is used in a larger net:

Use the LSTM layer on two separate inputs in a graph and combine the resulting states with a CatenateLayer:

Possible Issues  (1)

The "MovingMean" and "MovingVariance" arrays of BatchNormalizationLayer cannot be shared:

The other arrays of BatchNormalizationLayer can be shared:

Introduced in 2018
 (11.3)
 |
Updated in 2019
 (12.0)