CrossEntropyLossLayer

CrossEntropyLossLayer["Index"]

represents a net layer that computes the cross-entropy loss by comparing input class probability vectors with indices representing the target class.

CrossEntropyLossLayer["Probabilities"]

represents a net layer that computes the cross-entropy loss by comparing input class probability vectors with target class probability vectors.

CrossEntropyLossLayer["Binary"]

represents a net layer that computes the binary cross-entropy loss by comparing input probability scalars with target probability scalars, where each probability represents a binary choice.

Details and Options

  • CrossEntropyLossLayer exposes the following ports for use in NetGraph etc.:
  • "Input"real array of rank n
    "Target"real array of rank n or integer array of rank n-1
    "Loss"real number
  • When operating on multidimensional inputs, CrossEntropyLossLayer effectively threads over any extra array dimensions to produce an array of losses and returns the mean of these losses.
  • For CrossEntropyLossLayer["Binary"], the input and target should be scalar values between 0 and 1, or arrays of these.
  • For CrossEntropyLossLayer["Index"], the input should be a vector of probabilities {p1,,pc} that sums to 1, or an array of such vectors. The target should be an integer between 1 and c, or an array of such integers.
  • For CrossEntropyLossLayer["Probabilities"], the input and target should be a vector of probabilities that sums to 1, or an array of such vectors.
  • For the "Index" and "Probabilities" forms, where the input array has dimensions {d1,d2,,dn}, the final dimension dn is used to index the class. The output loss is taken to be the mean over the remaining dimensions {d1,,dn-1}.
  • CrossEntropyLossLayer[][<|"Input"->in,"Target"target|>] explicitly computes the output from applying the layer.
  • CrossEntropyLossLayer[][<|"Input"->{in1,in2,},"Target"->{target1,target2,}|>] explicitly computes outputs for each of the ini and targeti.
  • When given a NumericArray as input, the output will be a NumericArray.
  • CrossEntropyLossLayer is typically used inside NetGraph to construct a training network.
  • CrossEntropyLossLayer can operate on arrays that contain "Varying" dimensions.
  • A CrossEntropyLossLayer[] can be provided as the third argument to NetTrain when training a specific network.
  • When appropriate, CrossEntropyLossLayer is automatically used by NetTrain if an explicit loss specification is not provided. One of "Binary", "Probabilities", or "Index" will be chosen based on the final activation used for the output port and the form of any attached NetDecoder.
  • CrossEntropyLossLayer[form,"port"->shape] allows the shape of the input or target port to be specified. Possible forms for shape are:
  • "Real"a single real number
    "Integer"a single integer
    na vector of length n
    {n1,n2,}an array of dimensions n1×n2×
    "Varying"a vector whose length is variable
    {"Varying",n2,n3,}an array whose first dimension is variable and remaining dimensions are n2×n3×
    NetEncoder[]an encoder
    NetEncoder[{,"Dimensions"{n1,}}]an encoder mapped over an array of dimensions n1×

Examples

open all close all

Basic Examples  (3)

Create a CrossEntropyLossLayer object that takes a probability vector and an index:

In[1]:=
Click for copyable input
Out[1]=

Create a CrossEntropyLossLayer where the input is a probability vector and the target is an index:

In[1]:=
Click for copyable input
Out[1]=

Apply it to an input and a target:

In[2]:=
Click for copyable input
Out[2]=

Create a CrossEntropyLossLayer that operates on vectors generated from strings:

In[1]:=
Click for copyable input
In[2]:=
Click for copyable input
Out[2]=

Apply it to an input and a target:

In[3]:=
Click for copyable input
Out[3]=

Completely correct predictions produce a loss of 0:

In[4]:=
Click for copyable input
Out[4]=

Scope  (5)

Applications  (2)

Properties & Relations  (4)

Introduced in 2016
(11.0)
|
Updated in 2019
(12.0)