NetFoldOperator

NetFoldOperator[net]

represents a net in which net is folded over a sequence of inputs, maintaining a recurrent state.

NetFoldOperator[net,{"outi""in1",…}]

represents a net in which net is folded over its inputs, maintaining a recurrent state by feeding the outi of each step back to the ini of the next step.

NetFoldOperator[net,feedback,{"const1","const2",…}]

treats the inputs consti to net as being constant instead of mapped over.

NetFoldOperator[net,feedback,constants,{"out1",…}]

returns the sequence of values for the specific outputs outi.

Details and Options

  • NetFoldOperator[net] is equivalent to NetFoldOperator[net,{"Output"->"State"}] and specifies that the output called "Output" of the folded net will be fed back to the input called "State".
  • NetFoldOperator[…] represents a net that takes one or more sequences and outputs one or more sequences of the same length.
  • NetFoldOperator[net,{"out1""in1",…}] accepts the same inputs as net, except for those ini given by one of the feedback connections "outi"->"ini".
  • NetFoldOperator[net,feedback,constants] will map over any inputs that are not specified in feedback or constants. All mapped-over inputs should have the same length.
  • In the typical case in which net has two inputs named "Input" and "State" and one output named "Output", the single argument form NetFoldOperator[net] accepts a single input sequence {x1,x2,…,xn} named "Input" and produces a sequence of states {s1,s2,…,sn} named "Output", computed by applying the net repeatedly to xi and si-1 to compute si , effectively using net[<|"Input"->xi,"State"->si-1|>]. The initial state s0 is an array of zeros.
  • Within a NetGraph, a connection of the form src->NetPort[layer,"state"] can be used to provide the initial value of a recurrent state of a NetFoldOperator. Similarly, a connection of the form NetPort[layer,"state"]->dst can be used to obtain the final value of a recurrent state. If left unconnected, these states can be tracked by NetStateObject.
  • NetFoldOperator[net,{"out1""in1",…}] will have state ports named for all the recurrent ports ini whose values are fed back.
  • NetFoldOperator[net,…] can be seen as allowing a form of weight sharing between multiple copies of net, one for each sequence element that is being mapped over.
  • NetFoldOperator[…,"Input"->shape] allows the shape of the input to be specified. Possible forms for shape are:
  • NetEncoder[…]encoder producing an array
    na vector of length n
    {n1,n2,…}an array of dimensions n1×n2×…
    "Varying"a vector whose length is variable
    {"Varying",n2,n3,…}an array whose first dimension is variable and remaining dimensions are n2×n3×…
  • The following training parameter can be included:
  • LearningRateMultipliersAutomaticlearning rate multipliers for trainable arrays in the net
  • Options[NetFoldOperator] gives the list of default options to construct the operator. Options[NetFoldOperator[…]] gives the list of default options to evaluate the operator on some data.
  • Information[NetFoldOperator[…]] gives a report about the operator.
  • Information[NetFoldOperator[…],prop] gives the value of the property prop of NetFoldOperator[…]. Possible properties are the same as for NetGraph.

Examples

open allclose all

Basic Examples  (1)

Create a "core" net that takes a state and an input and produces an output:

Evaluate the core on an input:

Create a NetFoldOperator that uses the core to operate on sequences of arrays:

Evaluate the net on an input sequence of length-2 vectors:

Scope  (5)

Construct a net that takes a weighted average of the state and input:

Apply the net to an input and state:

Use NetFoldOperator to create a higher-order net that folds the average over an input sequence:

Fold the accumulator over a sequence of real numbers:

Use a NetMapOperator to map the folder over the color channels of an image, resulting in the rows of the image being accumulated:

Apply the operator to an image:

Create a net that can take a constant input that is used during the folding operation:

Apply the net to an input sequence, providing a value for the constant:

Create a net that has two independent states that are fed back. These states will additionally be used to provide the final output sequences:

Obtain the output sequences when given the input sequences:

Construct a net that combines elements from two sequences with a recurrent state using a simple nonlinear function:

Obtain the output sequence when given the input sequences:

Obtain the final state:

Create a NetGraph that allows the initial and final states of a NetFoldOperator to be set and obtained:

Apply the net to an input sequence, providing the initial state:

The final state is the last element of the output sequence:

If the initial and final states do not come from a larger network, it is also possible to use NetPort[All,"States"] to control these without using a NetGraph:

Applications  (1)

Create training data based on strings containing x's and y's and either Less, Greater or Equal by comparing the number of x's and y's. The training data consists of all possible sentences up to length 8:

Define a NetGraph representing the computation performed in a single time step by a basic recurrent layer with a state size of 5 and using a Ramp activation:

Allow the previous NetGraph to be applied to a sequence by embedding it in a NetFoldOperator:

Create a net with the NetFoldOperator defined previously to read an input string and predict one of Less, Greater or Equal:

Train the network:

Apply the trained network to a list of inputs:

Measure the accuracy on the entire training set:

Wolfram Research (2017), NetFoldOperator, Wolfram Language function, https://reference.wolfram.com/language/ref/NetFoldOperator.html (updated 2020).

Text

Wolfram Research (2017), NetFoldOperator, Wolfram Language function, https://reference.wolfram.com/language/ref/NetFoldOperator.html (updated 2020).

CMS

Wolfram Language. 2017. "NetFoldOperator." Wolfram Language & System Documentation Center. Wolfram Research. Last Modified 2020. https://reference.wolfram.com/language/ref/NetFoldOperator.html.

APA

Wolfram Language. (2017). NetFoldOperator. Wolfram Language & System Documentation Center. Retrieved from https://reference.wolfram.com/language/ref/NetFoldOperator.html

BibTeX

@misc{reference.wolfram_2024_netfoldoperator, author="Wolfram Research", title="{NetFoldOperator}", year="2020", howpublished="\url{https://reference.wolfram.com/language/ref/NetFoldOperator.html}", note=[Accessed: 19-April-2024 ]}

BibLaTeX

@online{reference.wolfram_2024_netfoldoperator, organization={Wolfram Research}, title={NetFoldOperator}, year={2020}, url={https://reference.wolfram.com/language/ref/NetFoldOperator.html}, note=[Accessed: 19-April-2024 ]}