ClassifierMeasurements

ClassifierMeasurements[classifier,testset,prop]
gives measurements associated with property prop when classifier is evaluated on testset.

ClassifierMeasurements[classifier,testset]
yields a ClassifierMeasurements[] object that can be applied to any property. When repeated property lookups are required, this is typically more efficient.

Details and OptionsDetails and Options

  • The classifier is typically a ClassifierFunction object as generated by Classify.
  • Properties supported include:
  • {"Examples",i->j}all class i examples classified as class j
    "Accuracy"fraction of correctly classified examples
    "AccuracyRejectionPlot"plot of the accuracy as function of the rejection rate
    "ConfusionFunction"function giving confusion matrix values
    "ConfusionMatrix"counts of class i examples classified as class j
    "ConfusionMatrixPlot"plot of the confusion matrix
    "Error"fraction of incorrectly classified examples
    "FScore"F-score for each class
    "LogLikelihood"log-likelihood of the model given the test set
    "LogLikelihoodRate"average log-likelihood per test example
    "Precision"precision of classification for each class
    "Properties"list of measurement properties available
    "Recall"recall rate of classification for each class
    "RejectionRate"fraction of examples classified as Indeterminate
  • ClassifierMeasurements[,opts] specifies that the classifier should use the options opts when applied to the test set.
  • The following options can be given:
  • ClassPriorsAutomaticexplicit prior probabilities for classes
    IndeterminateThresholdAutomaticbelow what probability to return Indeterminate
    UtilityFunctionAutomaticutility expressed as a function of actual and predicted class
  • ClassifierMeasurements[][prop] can be used to give properties prop for a ClassifierMeasurements[] object.
  • ClassifierMeasurements[][prop,opts] specifies that the classifier should use the options opts when applied to the test set. It supersedes options given to ClassifierMeasurements.
  • Test examples classified as Indeterminate will be discarded when measuring properties , , , , and .

ExamplesExamplesopen allclose all

Basic Examples  (1)Basic Examples  (1)

Define a training set and a test set:

In[1]:=
Click for copyable input
In[2]:=
Click for copyable input

Create a classifier on the training set:

In[3]:=
Click for copyable input
Out[3]=

Measure the accuracy of the classifier on the test set:

In[4]:=
Click for copyable input
Out[4]=

Measure the F-score of each class:

In[5]:=
Click for copyable input
Out[5]=

Obtain a function giving the number of test examples for each pair of actual class and predicted class:

In[6]:=
Click for copyable input
Out[6]=

Use the function to obtain the number of examples of class predicted as :

In[7]:=
Click for copyable input
Out[7]=

Generate a ClassifierMeasurements[] object of the classifier with the test set:

In[8]:=
Click for copyable input
Out[8]=

Perform the previous measurements using the ClassifierMeasurements[] object:

In[9]:=
Click for copyable input
Out[9]//TableForm=
Introduced in 2014
(10.0)