NetFoldOperator

NetFoldOperator[net]

represents a net in which net is folded over a sequence of inputs, maintaining a recurrent state.

Details and Options

  • NetFoldOperator[net] represents a net that takes a sequence of tensors and outputs a sequence of the same length.
  • NetFoldOperator[net] accepts a single input sequence {x1,x2,,xn} and an initial state s0, and produces a sequence of states {s1,s2,,sn}, computed by applying the net repeatedly to xi and si-1 to compute si , effectively using net[<|"Input"->xi,"State"->si-1|>].
  • In NetFoldOperator[net], net should be a NetGraph[] with two input ports:
  • "Input"the port accepting sequence element xt
    "State"the port accepting previous state st-1
  • NetFoldOperator exposes the following ports for use in NetGraph etc.:
  • "Input"a sequence of inputs {x1,x2,,xn}
    "Output"the output sequence {s1,s2,,sn}
  • NetFoldOperator[net] also has a single state port, "State", which is a tensor that is the same shape as the "State" input to net.
  • Within a NetGraph, a connection of the form src->NetPort[layer,"State"] can be used to provide the initial value of the state for a NetFoldOperator, corresponding to s0 in the recurrence relation. The default initial value is a zero tensor.
  • Within a NetGraph, a connection of the form NetPort[layer,"State"]->dst can be used to obtain the final value of the state for a NetFoldOperator, corresponding to sn in the recurrence relation.
  • NetFoldOperator[net] can be seen as allowing a form of weight sharing between multiple copies of net, one for each sequence element that is being folded.
  • NetFoldOperator[net,"Input"->shape] allows the shape of the input to be specified. Possible forms for shape are:
  • NetEncoder[]encoder producing a sequence of vectors
    {len,p}sequence of len length-p vectors
    {len,Automatic}sequence of len vectors whose length is inferred
    {"Varying",p}varying number of vectors each of length p
    {"Varying",Automatic}varying number of vectors each of inferred length
    {lenspec,d1,d2,}sequence of numerical tensors of dimensions d1×d2×

Examples

open allclose all

Basic Examples  (1)

Create a "core" net that takes a state and an input and produces an output:

In[1]:=
Click for copyable input
Out[1]=

Evaluate the core on an input:

In[2]:=
Click for copyable input
Out[2]=

Create a NetFoldOperator that uses the core to operate on sequences of length-2 vectors:

In[3]:=
Click for copyable input
Out[3]=

Evaluate the net on an input sequence:

In[4]:=
Click for copyable input
Out[4]=

Scope  (3)

Applications  (1)

See Also

Fold  NetNestOperator  NetMapOperator  NetChain  NetGraph

Introduced in 2017
(11.1)