represents a net layer whose computation is defined by the compilable function func.
Details and Options
- CompiledLayer is typically used to create a net layer that computes a custom function that cannot be implemented with other layers.
- CompiledLayer[func], such as CompiledLayer[func,Automatic], attempts to automatically differentiate func to produce a gradient propagating function. CompiledLayer[func,None] disables automatic differentiation.
- The functions func and gradientfunc must be valid Function objects suitable to be processed by FunctionCompile (e.g. their argument specification must be built using Typed and TypeSpecifier).
- CompiledLayer currently only supports a single input and output port.
- The function func represents the computation performed by the layer, which is also known as the forward pass.
- The function func must take a single array as input (representing the layer input) and must produce a single array as output (representing the layer output).
- The function gradientfunc is used to compute input gradients from output gradients, which is also known as the backward pass.
- The function gradientfunc should take at least two arrays as input representing the layer input and the output gradients. Optionally, the previously computed output can be given as a third argument. The result of the function gradientfunc is a single array representing the layer input gradient.
- The function func can be defined to take either a single example or a batch of examples. CompiledLayer[func,"Input"…] can be used to specify the rank of the input to disambiguate between the two cases (by comparing with the rank defined in func). func and gradientfunc should either both take a single example or both take a batch of examples.
- CompiledLayer exposes the following ports for use in NetGraph etc.:
"Input" an array "Output" an array
- When given a NumericArray as input, the output will be a NumericArray.
- Options[CompiledLayer] gives the list of default options to construct the layer. Options[CompiledLayer[…]] gives the list of default options to evaluate the layer on some data.
- Information[CompiledLayer[…]] gives a report about the layer.
- Information[CompiledLayer[…],prop] gives the value of the property prop of CompiledLayer[…]. Possible properties are the same as for NetGraph.
- CompiledLayer does not yet compile to GPU architectures. When setting TargetDevice"GPU", both training and inference will happen on the CPU.
Examplesopen allclose all
Basic Examples (2)
Create a CompiledLayer from a function taking a rank-2 array of 32-bit precision:
Create a CompiledLayer with a gradient propagating function:
With the gradient propagating function, the layer can be used to compute gradients and in NetTrain:
Create a CompiledLayer acting on a batch of elements. Define a function reversing an array of rank 2:
Specify an input of rank 1 in CompiledLayer. The function is now interpreted to act on the batch level:
When used in this way, CompiledLayer can make different elements in a batch interact with each other, which is normally not possible to do with ordinary layers.
Possible Issues (5)
The output shapes produced by the layer and gradient functions are expected to only depend on their input shapes, not on their values. Create a function whose output shape depends on the value of its input:
During shape inference, CompiledLayer runs its functions on test inputs to obtain an expected output shape:
For speed purposes, CompiledLayer disables sanity checks that compiled functions normally enable. This can result in bad code causing unexpected behavior or even kernel crashes. Take, for example, a function that accesses an element outside the boundaries of its input array:
CompiledLayer runs the function without checking the array boundaries, resulting in undefined behavior:
Wolfram Research (2020), CompiledLayer, Wolfram Language function, https://reference.wolfram.com/language/ref/CompiledLayer.html (updated 13).
Wolfram Language. 2020. "CompiledLayer." Wolfram Language & System Documentation Center. Wolfram Research. Last Modified 13. https://reference.wolfram.com/language/ref/CompiledLayer.html.
Wolfram Language. (2020). CompiledLayer. Wolfram Language & System Documentation Center. Retrieved from https://reference.wolfram.com/language/ref/CompiledLayer.html