NetBidirectionalOperator

NetBidirectionalOperator[net]

represents a net that applies net to a sequence and to its reverse, concatenating both results into one output sequence.

NetBidirectionalOperator[{fnet,bnet}]

uses fnet on the normal input and bnet on the reversed input.

NetBidirectionalOperator[nets,agg]

aggregates the two output sequences using the specified aggregation function.

Details and Options

  • In NetBidirectionalOperator[net], net should be a net that takes a sequence and produces a sequence. net is often one of the recurrent layers BasicRecurrentLayer, GatedRecurrentLayer, LongShortTermMemoryLayer or NetFoldOperator.
  • NetBidirectionalOperator[net] creates two copies of net that are trained independently, and is equivalent to NetBidirectionalOperator[{net,net}].
  • In NetBidirectionalOperator[,agg], valid settings for agg include:
  • Catenatecatenate the forward and reversed outputs (default)
    Totaladd the forward and reversed outputs
    Meantake the mean of the forward and reversed outputs
  • The following training parameter can be included:
  • LearningRateMultipliersAutomaticlearning rate multipliers for trainable arrays in the net
  • NetBidirectionalOperator[net,Catenate] will concatenate each element of the forward and reversed outputs, rather than concatenating the sequences themselves.
  • If net produces a sequence of vectors of length d, NetBidirectionalOperator[net,Catenate] will produce a sequence of vectors of length 2×d. For the aggregation functions Total and Mean, the vectors will be of length d.
  • For the reversed application of net, the input is first reversed, then net is applied normally, then the output is again reversed if net produces a sequence of vectors, before combination with the forward application of net.
  • NetExtract allows access to the forward and reverse nets via "ForwardNet" and "BackwardNet".
  • The states of the forward and reverse nets are accessible in NetBidirectionalOperator[] with the same names prefixed respectively by "Forward" and "Backward".
  • Options[NetBidirectionalOperator] gives the list of default options to construct the operator. Options[NetBidirectionalOperator[]] gives the list of default options to evaluate the operator on some data.
  • Information[NetBidirectionalOperator[]] gives a report about the operator.
  • Information[NetBidirectionalOperator[],prop] gives the value of the property prop of NetBidirectionalOperator[]. Possible properties are the same as for NetGraph.

Examples

open allclose all

Basic Examples  (4)

Create a bidirectional operator that contains two LSTM layers:

Apply the operator to an input:

Create a bidirectional operator using different nets for the forward and backward passes:

Apply the operator to an input:

Create a bidirectional operator with a specific aggregation function:

Apply the operator to an input:

Create an operator that yields the final elements of two sequences processed by a forward and a backward recurrent layer:

Apply the operator to an input:

Scope  (2)

Create a bidirectional recurrent network where initial states in both directions are exposed:

Apply the network:

Create a bidirectional recurrent network where initial states in both directions can be learned:

Apply the network:

Properties & Relations  (2)

Consider a network that takes and produces a sequence of vectors:

NetBidirectionalOperator[net,Catenate] is equivalent to the following NetGraph:

NetBidirectionalOperator[net,Total] is equivalent to the following NetGraph:

Consider a network that takes a sequence of vectors and produces a fixed-size vector:

NetBidirectionalOperator[net,Catenate] is equivalent to the following NetGraph:

NetBidirectionalOperator[net,Total] is equivalent to the following NetGraph:

Possible Issues  (1)

For aggregation functions Total and Mean, the forward and backward nets must produce sequences of arrays of the same size:

Introduced in 2018
 (11.3)
 |
Updated in 2020
 (12.1)