CompiledLayer

CompiledLayer[func]

represents a net layer whose computation is defined by the compilable function func.

CompiledLayer[func,gradientfunc]

specifies a gradient propagating function allowing the layer to be used in NetTrain.

Details and Options

  • The functions func and gradientfunc must be valid Function objects suitable to be processed by FunctionCompile (e.g. their argument specification must be built using Typed and TypeSpecifier).
  • CompiledLayer currently only supports a single input and output port.
  • The function func represents the computation performed by the layer, which is also known as the forward pass.
  • The function func must take a single array as input (representing the layer input) and must produce a single array as output (representing the layer output).
  • The function gradientfunc is used to compute input gradients from output gradients, which is also known as the backward pass.
  • The function gradientfunc should take at least two arrays as input representing the layer input and the output gradients. Optionally, the previously computed output can be given as a third argument. The result of the function gradientfunc is a single array representing the layer input gradient.
  • The function func can be defined to take either a single example or a batch of examples. CompiledLayer[func,"Input"] can be used to specify the rank of the input to disambiguate between the two cases (by comparing with the rank defined in func). func and gradientfunc should either both take a single example or both take a batch of examples.
  • CompiledLayer exposes the following ports for use in NetGraph etc.:
  • "Input"an array
    "Output"an array
  • When given a NumericArray as input, the output will be a NumericArray.
  • Options[CompiledLayer] gives the list of default options to construct the layer. Options[CompiledLayer[]] gives the list of default options to evaluate the layer on some data.
  • Information[CompiledLayer[]] gives a report about the layer.
  • Information[CompiledLayer[],prop] gives the value of the property prop of CompiledLayer[]. Possible properties are the same as for NetGraph.

Examples

open allclose all

Basic Examples  (2)

Create a compiled layer from a function taking a rank-3 array of machine precision:

Create a CompiledLayer from a function taking a rank-2 array of 32-bit precision:

Apply the layer to an input:

Scope  (3)

Create a CompiledLayer with a gradient propagating function:

With the gradient propagating function, the layer can be used to compute gradients and in NetTrain:

Specify a third argument in the gradient propagating function in order to reuse the previously computed output when computing the gradient:

Compute the input gradient using the layer:

Create a CompiledLayer acting on a batch of elements. Define a function reversing an array of rank 2:

Specify an input of rank 1 in CompiledLayer. The function is now interpreted to act on the batch level:

When acting on a single vector of length 4, the vector is unchanged:

When acting on a batch, the order of the vectors is reversed:

When used in this way, CompiledLayer can make different elements in a batch interact with each other, which is normally not possible to do with ordinary layers.

Possible Issues  (4)

When specifying a gradient propagating function in the second argument, there is a number of strict compatibility constraints that must be satisfied:

When the layer function acts on batches of inputs, the batch dimension must be preserved:

The output shapes produced by the layer and gradient functions are expected to only depend on their input shapes, not on their values. Create a function whose output shape depends in the value of its input:

During shape inference, CompiledLayer runs its functions on test inputs to obtain an expected output shape:

Subsequent evaluations complete successfully if the output shape matches the expected shape:

If an unexpected shape is produced, an error is generated:

For speed purposes, CompiledLayer disables sanity checks that compiled functions normally enable. This can result in bad code causing unexpected behavior or even kernel crashes. Take, for example, a function that accesses an element outside the boundaries of its input array:

The CompiledCodeFunction generated by FunctionCompile throws an error:

CompiledLayer runs the function without checking the array boundaries, resulting in undefined behavior:

Wolfram Research (2020), CompiledLayer, Wolfram Language function, https://reference.wolfram.com/language/ref/CompiledLayer.html.

Text

Wolfram Research (2020), CompiledLayer, Wolfram Language function, https://reference.wolfram.com/language/ref/CompiledLayer.html.

BibTeX

@misc{reference.wolfram_2020_compiledlayer, author="Wolfram Research", title="{CompiledLayer}", year="2020", howpublished="\url{https://reference.wolfram.com/language/ref/CompiledLayer.html}", note=[Accessed: 15-January-2021 ]}

BibLaTeX

@online{reference.wolfram_2020_compiledlayer, organization={Wolfram Research}, title={CompiledLayer}, year={2020}, url={https://reference.wolfram.com/language/ref/CompiledLayer.html}, note=[Accessed: 15-January-2021 ]}

CMS

Wolfram Language. 2020. "CompiledLayer." Wolfram Language & System Documentation Center. Wolfram Research. https://reference.wolfram.com/language/ref/CompiledLayer.html.

APA

Wolfram Language. (2020). CompiledLayer. Wolfram Language & System Documentation Center. Retrieved from https://reference.wolfram.com/language/ref/CompiledLayer.html