---
title: "DropoutLayer"
language: "en"
type: "Symbol"
summary: "DropoutLayer[] represents a net layer that sets its input elements to zero with probability 0.5 during training. DropoutLayer[p] sets its input elements to zero with probability p during training."
keywords: 
- drop out
- dropout
- regularization
- regularizing neural nets
- drop connect
canonical_url: "https://reference.wolfram.com/language/ref/DropoutLayer.html"
source: "Wolfram Language Documentation"
related_guides: 
  - 
    title: "Neural Network Layers"
    link: "https://reference.wolfram.com/language/guide/NeuralNetworkLayers.en.md"
related_functions: 
  - 
    title: "BatchNormalizationLayer"
    link: "https://reference.wolfram.com/language/ref/BatchNormalizationLayer.en.md"
  - 
    title: "NetEvaluationMode"
    link: "https://reference.wolfram.com/language/ref/NetEvaluationMode.en.md"
  - 
    title: "NetChain"
    link: "https://reference.wolfram.com/language/ref/NetChain.en.md"
  - 
    title: "NetGraph"
    link: "https://reference.wolfram.com/language/ref/NetGraph.en.md"
  - 
    title: "NetTrain"
    link: "https://reference.wolfram.com/language/ref/NetTrain.en.md"
  - 
    title: "ElementwiseLayer"
    link: "https://reference.wolfram.com/language/ref/ElementwiseLayer.en.md"
related_tutorials: 
  - 
    title: "Neural Networks in the Wolfram Language"
    link: "https://reference.wolfram.com/language/tutorial/NeuralNetworksOverview.en.md"
---
[EXPERIMENTAL]

# DropoutLayer

DropoutLayer[] represents a net layer that sets its input elements to zero with probability 0.5 during training.

DropoutLayer[p] sets its input elements to zero with probability p during training.

## Details and Options

* ``DropoutLayer`` is commonly used as a form of neural network regularization.

* ``DropoutLayer`` is typically used inside ``NetChain``, ``NetGraph``, etc.

* The following optional parameters can be included:

|                |           |                       |
| -------------- | --------- | --------------------- |
| Method         | "Dropout" | dropout method to use |
| "OutputPorts"  | "Output"  | output ports          |

* Possible explicit settings for the ``Method`` option include:

|     |     |
| --- | --- |
| "AlphaDropout" | keeps the mean and variance of the input constant; designed to be used together with the ElementwiseLayer["SELU"] activation |
| "Dropout" | sets the input elements to zero with probability p during training, multiplying the remainder by 1 / (1 - p) |

* Possible settings for the ``"OutputPorts"`` option include:

|              |                                   |
| ------------ | --------------------------------- |
| "BinaryMask" | binary mask applied to input data |
| "Output"     | output of the dropout             |
| {port1, …}   | a list of valid ports             |

* ``DropoutLayer`` exposes the following ports for use in ``NetGraph`` etc.:

|          |                                                  |
| -------- | ------------------------------------------------ |
| "Input"  | an array or sequence of arrays of arbitrary rank |
| "Output" | an array or sequence of arrays of arbitrary rank |

* ``DropoutLayer`` normally infers the dimensions of its input from its context in ``NetChain`` etc. To specify the dimensions explicitly as ``{n1, n2, …}``, use ``DropoutLayer["Input" -> {n1, n2, …}]``.

* ``DropoutLayer[…][input]`` explicitly computes the output from applying the layer.

* ``DropoutLayer[…][{input1, input2, …}]`` explicitly computes outputs for each of the ``inputi``.

* When given a ``NumericArray`` as input, the output will be a ``NumericArray``.

* ``DropoutLayer`` only randomly sets input elements to zero during training. During evaluation, ``DropoutLayer`` leaves the input unchanged, unless ``NetEvaluationMode -> "Train"`` is specified when applying the layer.

* ``Options[DropoutLayer]`` gives the list of default options to construct the layer. ``Options[DropoutLayer[…]]`` gives the list of default options to evaluate the layer on some data.

* ``Information[DropoutLayer[…]]`` gives a report about the layer.

* ``Information[DropoutLayer[…], prop]`` gives the value of the property ``prop`` of ``DropoutLayer[…]``. [Possible properties](https://reference.wolfram.com/language/ref/NetGraph.en.md#495200340) are the same as for ``NetGraph``.

---

## Examples (7)

### Basic Examples (1)

Create a ``DropoutLayer``:

```wl
In[1]:= drop = DropoutLayer[]

Out[1]=
DropoutLayer[Association["Type" -> "Dropout", "Arrays" -> Association[], 
  "Parameters" -> Association["DropoutProbability" -> 0.5, "Method" -> "Dropout", 
    "OutputPorts" -> NeuralNetworks`ValidatedParameter[{"Output"}]], 
  "Inputs" -> Associa ... NeuralNetworks`AtomT]], 
  "Outputs" -> Association["Output" -> NeuralNetworks`TensorT[NeuralNetworks`ListT[
       NeuralNetworks`NaturalT, NeuralNetworks`SizeT], NeuralNetworks`AtomT]]], 
 Association["Version" -> "14.3.0", "Unstable" -> False]]
```

Apply it to an input, which remains unchanged:

```wl
In[2]:= drop[{1, 2, 3, 4, 5}]

Out[2]= {1., 2., 3., 4., 5.}
```

Use ``NetEvaluationMode`` to force training behavior of ``DropoutLayer`` :

```wl
In[3]:= drop[{1, 2, 3, 4, 5}, NetEvaluationMode -> "Train"]

Out[3]= {0., 0., 0., 0., 0.}
```

### Scope (2)

Create a ``DropoutLayer`` with a specific probability:

```wl
In[1]:= drop = DropoutLayer[0.8]

Out[1]=
DropoutLayer[Association["Type" -> "Dropout", "Arrays" -> Association[], 
  "Parameters" -> Association["DropoutProbability" -> 0.8, "Method" -> "Dropout", 
    "OutputPorts" -> NeuralNetworks`ValidatedParameter[{"Output"}]], 
  "Inputs" -> Associa ... NeuralNetworks`AtomT]], 
  "Outputs" -> Association["Output" -> NeuralNetworks`TensorT[NeuralNetworks`ListT[
       NeuralNetworks`NaturalT, NeuralNetworks`SizeT], NeuralNetworks`RealT]]], 
 Association["Version" -> "14.1.1", "Unstable" -> False]]
```

Apply it to input data, which leaves the input unchanged:

```wl
In[2]:= drop[{1, 2, 3, 4, 5, 6, 7, 8, 9, 10}]

Out[2]= {1., 2., 3., 4., 5., 6., 7., 8., 9., 10.}
```

Apply it to input data, specifying that training behavior be used:

```wl
In[3]:= drop[{1, 2, 3, 4, 5, 6, 7, 8, 9, 10}, NetEvaluationMode -> "Train"]

Out[3]= {5., 0., 0., 0., 0., 30., 0., 40., 0., 50.}
```

---

Create a ``DropoutLayer`` that takes an RGB image and returns an RGB image:

```wl
In[1]:= drop = DropoutLayer[0.95, "Input" -> NetEncoder["Image"], "Output" -> NetDecoder["Image"]]

Out[1]=
DropoutLayer[Association["Type" -> "Dropout", "Arrays" -> Association[], 
  "Parameters" -> Association["DropoutProbability" -> 0.95, "Method" -> "Dropout", 
    "OutputPorts" -> NeuralNetworks`ValidatedParameter[{"Output"}]], 
  "Inputs" -> Associ ... s`SizeT, NeuralNetworks`MatchT[
            _NeuralNetworks`LengthVar]}]], "$Channels" -> 3, "$Version" -> "14.1.1"], 
      NeuralNetworks`TensorT[{3, 128, 128}, NeuralNetworks`RealT]]]], 
 Association["Version" -> "14.1.1", "Unstable" -> False]]
```

The ``DropoutLayer`` acts on the image represented by a rank-3 array by randomly and independently zeroing the individual color components of each pixel:

```wl
In[2]:= drop[[image], NetEvaluationMode -> "Train"]

Out[2]= [image]
```

### Options (2)

#### Method (1)

Create a ``DropoutLayer`` using ``"AlphaDropout"`` as the dropout method:

```wl
In[1]:= drop = DropoutLayer[Method -> "AlphaDropout"]

Out[1]=
DropoutLayer[Association["Type" -> "Dropout", "Arrays" -> Association[], 
  "Parameters" -> Association["DropoutProbability" -> 0.5, "Method" -> "AlphaDropout", 
    "OutputPorts" -> NeuralNetworks`ValidatedParameter[{"Output"}]], 
  "Inputs" -> As ... NeuralNetworks`AtomT]], 
  "Outputs" -> Association["Output" -> NeuralNetworks`TensorT[NeuralNetworks`ListT[
       NeuralNetworks`NaturalT, NeuralNetworks`SizeT], NeuralNetworks`RealT]]], 
 Association["Version" -> "14.1.1", "Unstable" -> False]]
```

Apply it to input data, specifying that training behavior be used:

```wl
In[2]:= drop[{1, 2, 3, 4, 5, 6, 7}, NetEvaluationMode -> "Train"]

Out[2]= {1.6656, 2.552, -0.779194, 4.32481, -0.779194, -0.779194, -0.779194}
```

If the input data has a mean of 0 and a variance of 1, then the output will have the same mean and variance:

```wl
In[3]:= data = RandomVariate[NormalDistribution[0, 1], 20000];

In[4]:= out = drop[data, NetEvaluationMode -> "Train"];

In[5]:= Variance@out

Out[5]= 0.994984

In[6]:= Mean@out

Out[6]= 0.00731836
```

This is not the case for the standard dropout method:

```wl
In[7]:= out2 = DropoutLayer[][data, NetEvaluationMode -> "Train"];

In[8]:= Mean@out2

Out[8]= -0.0142753

In[9]:= Variance@out2

Out[9]= 2.00972
```

#### "OutputPorts" (1)

Create a ``DropoutLayer`` that yields the binary mask besides the output:

```wl
In[1]:= drop = DropoutLayer["OutputPorts" -> {"Output", "BinaryMask"}]

Out[1]=
DropoutLayer[Association["Type" -> "Dropout", "Arrays" -> Association[], 
  "Parameters" -> Association["DropoutProbability" -> 0.5, "Method" -> "Dropout", 
    "OutputPorts" -> NeuralNetworks`ValidatedParameter[{"Output", "BinaryMask"}]], 
  "Inpu ... zeT], NeuralNetworks`AtomT], 
    "BinaryMask" -> NeuralNetworks`TensorT[NeuralNetworks`ListT[NeuralNetworks`NaturalT, 
       NeuralNetworks`SizeT], NeuralNetworks`IndexIntegerT[0, 1]]]], 
 Association["Version" -> "14.1.1", "Unstable" -> False]]
```

Apply it to input data:

```wl
In[2]:= drop[{1, 2, 3, 4, 5, 6, 7, 8, 9, 10}]

Out[2]= <|"Output" -> {1., 2., 3., 4., 5., 6., 7., 8., 9., 10.}, "BinaryMask" -> {1, 1, 1, 1, 1, 1, 1, 1, 1, 1}|>
```

Apply it to input data, specifying that training behavior be used:

```wl
In[3]:= drop[{1, 2, 3, 4, 5, 6, 7, 8, 9, 10}, NetEvaluationMode -> "Train"]

Out[3]= <|"Output" -> {2., 4., 0., 8., 10., 0., 14., 16., 0., 20.}, "BinaryMask" -> {1, 1, 0, 1, 1, 0, 1, 1, 0, 1}|>
```

### Properties & Relations (1)

``DropoutLayer`` can be used between recurrent layers in a ``NetChain`` to perform regularization. A typical network used to classify sentences might incorporate a ``DropoutLayer`` as follows:

```wl
In[1]:= NetChain[{EmbeddingLayer[20], GatedRecurrentLayer[100], DropoutLayer[0.2], GatedRecurrentLayer[100], SequenceLastLayer[], LinearLayer[{}], LogisticSigmoid}, "Input" -> NetEncoder["Characters"], "Output" -> NetDecoder["Boolean"]]

Out[1]=
NetChain[Association["Type" -> "Chain", 
  "Nodes" -> Association["1" -> Association["Type" -> "Embedding", 
      "Arrays" -> Association["Weights" -> NeuralNetworks`TensorT[{97, 20}, NeuralNetworks`RealT]], 
      "Parameters" -> Association["Out ...   "InteriorStates" -> Association[{2, "State"} -> NeuralNetworks`NetPath["Nodes", "2", "States", 
      "State"], {4, "State"} -> NeuralNetworks`NetPath["Nodes", "4", "States", "State"]]], 
 Association["Version" -> "14.1.1", "Unstable" -> False]]
```

More sophisticated forms of dropout are possible by using the ``"Dropout"`` option of recurrent layers directly:

```wl
In[2]:= gru = GatedRecurrentLayer[100, "Dropout" -> {"VariationalInput" -> 0.2, "StateUpdate" -> 0.1}]

Out[2]=
GatedRecurrentLayer[Association["Type" -> "GatedRecurrent", 
  "Arrays" -> Association["InputGateInputWeights" -> NeuralNetworks`TensorT[
      {100, NeuralNetworks`SizeT}, NeuralNetworks`RealT], "InputGateStateWeights" -> 
     NeuralNetworks`Tens ... ralNetworks`TensorT[{NeuralNetworks`LengthVar[358005548], 
       100}, NeuralNetworks`RealT]], 
  "States" -> Association["State" -> NeuralNetworks`TensorT[{100}, NeuralNetworks`RealT]]], 
 Association["Version" -> "14.1.1", "Unstable" -> False]]

In[3]:= NetChain[{EmbeddingLayer[20], gru, gru, SequenceLastLayer[], LinearLayer[{}], LogisticSigmoid}, "Input" -> NetEncoder["Characters"], "Output" -> NetDecoder["Boolean"]]

Out[3]=
NetChain[Association["Type" -> "Chain", 
  "Nodes" -> Association["1" -> Association["Type" -> "Embedding", 
      "Arrays" -> Association["Weights" -> NeuralNetworks`TensorT[{97, 20}, NeuralNetworks`RealT]], 
      "Parameters" -> Association["Out ...   "InteriorStates" -> Association[{2, "State"} -> NeuralNetworks`NetPath["Nodes", "2", "States", 
      "State"], {3, "State"} -> NeuralNetworks`NetPath["Nodes", "3", "States", "State"]]], 
 Association["Version" -> "14.1.1", "Unstable" -> False]]
```

### Possible Issues (1)

By default, any randomness invoked by ``NetEvaluationMode -> "Train"`` is not affected by ``SeedRandom`` and ``BlockRandom`` :

```wl
In[1]:= drop = DropoutLayer[]

Out[1]=
DropoutLayer[Association["Type" -> "Dropout", "Arrays" -> Association[], 
  "Parameters" -> Association["DropoutProbability" -> 0.5, "Method" -> "Dropout", 
    "OutputPorts" -> NeuralNetworks`ValidatedParameter[{"Output"}]], 
  "Inputs" -> Associa ... NeuralNetworks`AtomT]], 
  "Outputs" -> Association["Output" -> NeuralNetworks`TensorT[NeuralNetworks`ListT[
       NeuralNetworks`NaturalT, NeuralNetworks`SizeT], NeuralNetworks`AtomT]]], 
 Association["Version" -> "14.1.1", "Unstable" -> False]]

In[2]:= Table[BlockRandom[drop[{1, 2, 3, 4}, NetEvaluationMode -> "Train"]], 6]

Out[2]= {{0., 0., 6., 8.}, {0., 0., 6., 0.}, {2., 0., 0., 0.}, {2., 4., 0., 0.}, {0., 4., 0., 0.}, {0., 4., 0., 8.}}
```

Use option ``RandomSeeding -> Inherited`` to change this behavior:

```wl
In[3]:=
Table[BlockRandom[drop[{1, 2, 3, 4}, NetEvaluationMode -> "Train", 
	RandomSeeding -> Inherited]], 6]

Out[3]= {{2., 4., 6., 0.}, {2., 4., 6., 0.}, {2., 4., 6., 0.}, {2., 4., 6., 0.}, {2., 4., 6., 0.}, {2., 4., 6., 0.}}
```

Use option ``RandomSeeding`` to control the randomness:

```wl
In[4]:= Table[drop[{1, 2, 3, 4}, NetEvaluationMode -> "Train", RandomSeeding -> 123], 6]

Out[4]= {{2., 4., 6., 0.}, {2., 4., 6., 0.}, {2., 4., 6., 0.}, {2., 4., 6., 0.}, {2., 4., 6., 0.}, {2., 4., 6., 0.}}
```

## See Also

* [`BatchNormalizationLayer`](https://reference.wolfram.com/language/ref/BatchNormalizationLayer.en.md)
* [`NetEvaluationMode`](https://reference.wolfram.com/language/ref/NetEvaluationMode.en.md)
* [`NetChain`](https://reference.wolfram.com/language/ref/NetChain.en.md)
* [`NetGraph`](https://reference.wolfram.com/language/ref/NetGraph.en.md)
* [`NetTrain`](https://reference.wolfram.com/language/ref/NetTrain.en.md)
* [`ElementwiseLayer`](https://reference.wolfram.com/language/ref/ElementwiseLayer.en.md)

## Tech Notes

* [Neural Networks in the Wolfram Language](https://reference.wolfram.com/language/tutorial/NeuralNetworksOverview.en.md)

## Related Guides

* [Neural Network Layers](https://reference.wolfram.com/language/guide/NeuralNetworkLayers.en.md)

## History

* [Introduced in 2016 (11.0)](https://reference.wolfram.com/language/guide/SummaryOfNewFeaturesIn110.en.md) \| [Updated in 2017 (11.1)](https://reference.wolfram.com/language/guide/SummaryOfNewFeaturesIn111.en.md) ▪ [2017 (11.2)](https://reference.wolfram.com/language/guide/SummaryOfNewFeaturesIn112.en.md) ▪ [2020 (12.1)](https://reference.wolfram.com/language/guide/SummaryOfNewFeaturesIn121.en.md)