DropoutLayer

[Experimental]

✖

DropoutLayer[]

represents a net layer that sets its input elements to zero with probability 0.5 during training.

✖

DropoutLayer[p]

sets its input elements to zero with probability p during training.

Details and Options

DropoutLayer is commonly used as a form of neural network regularization.
DropoutLayer is typically used inside NetChain, NetGraph, etc.
The following optional parameters can be included:
Method "Dropout" dropout method to use

"OutputPorts" "Output" output ports
Possible explicit settings for the Method option include:

	"AlphaDropout"	keeps the mean and variance of the input constant; designed to be used together with the ElementwiseLayer["SELU"] activation
	"Dropout"	sets the input elements to zero with probability p during training, multiplying the remainder by 1/(1-p)

Possible settings for the "OutputPorts" option include:
"BinaryMask" binary mask applied to input data

"Output" output of the dropout

{port₁,…} a list of valid ports
DropoutLayer exposes the following ports for use in NetGraph etc.:
"Input" an array or sequence of arrays of arbitrary rank

"Output" an array or sequence of arrays of arbitrary rank
DropoutLayer normally infers the dimensions of its input from its context in NetChain etc. To specify the dimensions explicitly as {n₁,n₂,…}, use DropoutLayer["Input"->{n₁,n₂,…}].
DropoutLayer[…][input] explicitly computes the output from applying the layer.
DropoutLayer[…][{input₁,input₂,…}] explicitly computes outputs for each of the input_i.
When given a NumericArray as input, the output will be a NumericArray.
DropoutLayer only randomly sets input elements to zero during training. During evaluation, DropoutLayer leaves the input unchanged, unless NetEvaluationMode->"Train" is specified when applying the layer.
Options[DropoutLayer] gives the list of default options to construct the layer. Options[DropoutLayer[…]] gives the list of default options to evaluate the layer on some data.
Information[DropoutLayer[…]] gives a report about the layer.
Information[DropoutLayer[…],prop] gives the value of the property prop of DropoutLayer[…]. Possible properties are the same as for NetGraph.

Examples

open allclose all

Basic Examples (1)Summary of the most common use cases

Create a DropoutLayer:

In[1]:=1

✖

https://wolfram.com/xid/08ux6othncg1yua-qv4tqb

Out[1]=1

Apply it to an input, which remains unchanged:

In[2]:=2

✖

https://wolfram.com/xid/08ux6othncg1yua-5ipqql

Out[2]=2

Use NetEvaluationMode to force training behavior of DropoutLayer:

In[3]:=3

✖

https://wolfram.com/xid/08ux6othncg1yua-u01m17

Out[3]=3

Scope (2)Survey of the scope of standard use cases

Create a DropoutLayer with a specific probability:

In[1]:=1

✖

https://wolfram.com/xid/08ux6othncg1yua-8f8zq2

Out[1]=1

Apply it to input data, which leaves the input unchanged:

In[2]:=2

✖

https://wolfram.com/xid/08ux6othncg1yua-bso35a

Out[2]=2

Apply it to input data, specifying that training behavior be used:

In[3]:=3

✖

https://wolfram.com/xid/08ux6othncg1yua-eji7wd

Out[3]=3

Create a DropoutLayer that takes an RGB image and returns an RGB image:

In[1]:=1

✖

https://wolfram.com/xid/08ux6othncg1yua-b4v9zw

Out[1]=1

The DropoutLayer acts on the image represented by a rank-3 array by randomly and independently zeroing the individual color components of each pixel:

In[2]:=2

✖

https://wolfram.com/xid/08ux6othncg1yua-s3jjxb

Out[2]=2

Options (2)Common values & functionality for each option

Method (1)

Create a DropoutLayer using "AlphaDropout" as the dropout method:

In[1]:=1

✖

https://wolfram.com/xid/08ux6othncg1yua-e9jjao

Out[1]=1

Apply it to input data, specifying that training behavior be used:

In[2]:=2

✖

https://wolfram.com/xid/08ux6othncg1yua-3jaubw

Out[2]=2

If the input data has a mean of 0 and a variance of 1, then the output will have the same mean and variance:

In[3]:=3

✖

https://wolfram.com/xid/08ux6othncg1yua-qow0nu

In[4]:=4

✖

https://wolfram.com/xid/08ux6othncg1yua-ya6a2s

In[5]:=5

✖

https://wolfram.com/xid/08ux6othncg1yua-onilwe

Out[5]=5

In[6]:=6

✖

https://wolfram.com/xid/08ux6othncg1yua-vqtb1o

Out[6]=6

This is not the case for the standard dropout method:

In[7]:=7

✖

https://wolfram.com/xid/08ux6othncg1yua-sdlv51

In[8]:=8

✖

https://wolfram.com/xid/08ux6othncg1yua-1nmoy8

Out[8]=8

In[9]:=9

✖

https://wolfram.com/xid/08ux6othncg1yua-osaray

Out[9]=9

"OutputPorts" (1)

Create a DropoutLayer that yields the binary mask besides the output:

In[1]:=1

✖

https://wolfram.com/xid/08ux6othncg1yua-ir4cxr

Out[1]=1

Apply it to input data:

In[2]:=2

✖

https://wolfram.com/xid/08ux6othncg1yua-vtqn2p

Out[2]=2

Apply it to input data, specifying that training behavior be used:

In[3]:=3

✖

https://wolfram.com/xid/08ux6othncg1yua-gcqlm4

Out[3]=3

Properties & Relations (1)Properties of the function, and connections to other functions

DropoutLayer can be used between recurrent layers in a NetChain to perform regularization. A typical network used to classify sentences might incorporate a DropoutLayer as follows:

In[1]:=1