NetBidirectionalOperator

NetBidirectionalOperator[net]

represents a net that applies net to a sequence and to its reverse, concatenating both results into one output sequence.

NetBidirectionalOperator[{fnet,bnet}]

uses fnet on the normal input and bnet on the reversed input.

NetBidirectionalOperator[nets,agg]

aggregates the two output sequences using the specified aggregation function.

Details and Options

  • In NetBidirectionalOperator[net], net should be a net that takes a sequence and produces a sequence. net is often one of the recurrent layers BasicRecurrentLayer, GatedRecurrentLayer, LongShortTermMemoryLayer or NetFoldOperator.
  • NetBidirectionalOperator[net] creates two copies of net that are trained independently, and is equivalent to NetBidirectionalOperator[{net,net}].
  • In NetBidirectionalOperator[…,agg], valid settings for agg include:
  • Catenatecatenate the forward and reversed outputs (default)
    Totaladd the forward and reversed outputs
    Meantake the mean of the forward and reversed outputs
  • The following training parameter can be included:
  • LearningRateMultipliersAutomaticlearning rate multipliers for trainable arrays in the net
  • NetBidirectionalOperator[net,Catenate] will concatenate each element of the forward and reversed outputs, rather than concatenating the sequences themselves.
  • If net produces a sequence of vectors of length d, NetBidirectionalOperator[net,Catenate] will produce a sequence of vectors of length 2×d. For the aggregation functions Total and Mean, the vectors will be of length d.
  • For the reversed application of net, the input is first reversed, then net is applied normally, then the output is again reversed if net produces a sequence of vectors, before combination with the forward application of net.
  • NetExtract allows access to the forward and reverse nets via "ForwardNet" and "BackwardNet".
  • The states of the forward and reverse nets are accessible in NetBidirectionalOperator[…] with the same names prefixed respectively by "Forward…" and "Backward…".
  • Options[NetBidirectionalOperator] gives the list of default options to construct the operator. Options[NetBidirectionalOperator[…]] gives the list of default options to evaluate the operator on some data.
  • Information[NetBidirectionalOperator[…]] gives a report about the operator.
  • Information[NetBidirectionalOperator[…],prop] gives the value of the property prop of NetBidirectionalOperator[…]. Possible properties are the same as for NetGraph.

Examples

open allclose all

Basic Examples  (4)

Create a bidirectional operator that contains two LSTM layers:

Apply the operator to an input:

Create a bidirectional operator using different nets for the forward and backward passes:

Apply the operator to an input:

Create a bidirectional operator with a specific aggregation function:

Apply the operator to an input:

Create an operator that yields the final elements of two sequences processed by a forward and a backward recurrent layer:

Apply the operator to an input:

Scope  (2)

Create a bidirectional recurrent network where initial states in both directions are exposed:

Apply the network:

Create a bidirectional recurrent network where initial states in both directions can be learned:

Apply the network:

Properties & Relations  (2)

Consider a network that takes and produces a sequence of vectors:

NetBidirectionalOperator[net,Catenate] is equivalent to the following NetGraph:

NetBidirectionalOperator[net,Total] is equivalent to the following NetGraph:

Consider a network that takes a sequence of vectors and produces a fixed-size vector:

NetBidirectionalOperator[net,Catenate] is equivalent to the following NetGraph:

NetBidirectionalOperator[net,Total] is equivalent to the following NetGraph:

Possible Issues  (1)

For aggregation functions Total and Mean, the forward and backward nets must produce sequences of arrays of the same size:

Wolfram Research (2018), NetBidirectionalOperator, Wolfram Language function, https://reference.wolfram.com/language/ref/NetBidirectionalOperator.html (updated 2020).

Text

Wolfram Research (2018), NetBidirectionalOperator, Wolfram Language function, https://reference.wolfram.com/language/ref/NetBidirectionalOperator.html (updated 2020).

CMS

Wolfram Language. 2018. "NetBidirectionalOperator." Wolfram Language & System Documentation Center. Wolfram Research. Last Modified 2020. https://reference.wolfram.com/language/ref/NetBidirectionalOperator.html.

APA

Wolfram Language. (2018). NetBidirectionalOperator. Wolfram Language & System Documentation Center. Retrieved from https://reference.wolfram.com/language/ref/NetBidirectionalOperator.html

BibTeX

@misc{reference.wolfram_2024_netbidirectionaloperator, author="Wolfram Research", title="{NetBidirectionalOperator}", year="2020", howpublished="\url{https://reference.wolfram.com/language/ref/NetBidirectionalOperator.html}", note=[Accessed: 21-November-2024 ]}

BibLaTeX

@online{reference.wolfram_2024_netbidirectionaloperator, organization={Wolfram Research}, title={NetBidirectionalOperator}, year={2020}, url={https://reference.wolfram.com/language/ref/NetBidirectionalOperator.html}, note=[Accessed: 21-November-2024 ]}