represents an object generated by ClassifierMeasurements that can be applied to properties.

Details and Options

  • ClassifierMeasurementsObject[][prop] is used to look up property prop from the ClassifierMeasurementsObject.
  • Properties supported include:
  • "Accuracy"fraction of correctly classified examples
    "Accuracy"ntop-n accuracy
    "AccuracyBaseline"accuracy if predicting the commonest class
    "AccuracyRejectionPlot"plot of the accuracy as function of the rejection rate
    "AreaUnderROCCurve"area under the ROC curve for each class
    "BatchEvaluationTime"marginal time to predict one example when a batch is given
    "BestClassifiedExamples"examples having the highest actual-class probability
    "ClassifierFunction"ClassifierFunction[] being measured
    "ClassMeanCrossEntropy"mean cross entropy for each class
    "ClassRejectionRate"rejection rate for each class
    "CohenKappa"Cohen's kappa coefficient
    "ConfusionFunction"function giving confusion matrix values
    "ConfusionMatrix"counts cij of class i examples classified as class j
    "ConfusionMatrixPlot"plot of the confusion matrix
    "ConfusionMatrixPlot"{c1,c2,}confusion matrix plot restricted to classes c1, c2, etc.
    "ConfusionMatrixPlot"nconfusion matrix plot for the worst n-class subset
    "CorrectlyClassifiedExamples"examples correctly classified
    "DecisionUtilities"value of the utility function for each example
    "Error"fraction of incorrectly classified examples
    "EvaluationTime"time needed to classify one example
    "Examples"all test examples
    "Examples"{i,j}all class i examples classified as class j
    "F1Score"F1 score for each class
    "FalseDiscoveryRate"false discovery rate for each class
    "FalseNegativeExamples"false negative test examples for each class
    "FalseNegativeRate"false negative rate for each class
    "FalsePositiveExamples"false positive test examples for each class
    "FalsePositiveRate"false positive rate for each class
    "GeometricMeanProbability"geometric mean of the actual-class probabilities
    "IndeterminateExamples"test examples classified as Indeterminate
    "LeastCertainExamples"examples having the highest distribution entropy
    "LogLikelihood"log-likelihood of the model given the test set
    "MatthewsCorrelationCoefficient"Matthews correlation coefficient for each class
    "MeanCrossEntropy"mean cross entropy over test examples
    "MeanDecisionUtility"mean utility over test example
    "MisclassifiedExamples"examples misclassified
    "MostCertainExamples"examples having the lowest distribution entropy
    "NegativePredictedValue"negative predicted value for each class
    "Perplexity"exponential of the mean cross entropy
    "Precision"precision of classification for each class
    "Probabilities"actual-class classification probabilities
    "ProbabilityHistogram"histogram of actual-class probabilities
    "Properties"list of measurement properties available
    "Recall"recall rate of classification for each class
    "RejectionRate"fraction of examples classified as Indeterminate
    "Report"panel reporting main measurements
    "ROCCurve"Receiver Operating Characteristics curve
    "ScottPi"Scott's pi coefficient
    "Specificity"specificity for each class
    "TopConfusions"pairs of classes that are most confused
    "TopConfusions"nn most confused class pairs
    "TrueNegativeExamples"true negative test examples for each class
    "TruePositiveExamples"true positive test examples for each class
    "WorstClassifiedExamples"examples having the lowest actual-class probability
  • ClassifierMeasurementsObject[][prop,opts] specifies that the classifier should use the options opts when applied to the test set. It supersedes options given to ClassifierMeasurements.
  • Possible options are as given in ClassifierFunction.
  • Test examples classified as Indeterminate will be discarded when measuring properties "Accuracy", "Error", "Precision", "Recall" and "FScore".
  • Properties such as "Precision" or "Recall" give one measure for each possible "positive class". The "negative class" is the union of all the classes that are not the "positive class". ClassifierMeasurementsObject[][propclass] specifies that class is the "positive class".
  • In properties giving test examples such as "Examples" or "WorstClassifiedExamples", examples are in the form inputiclassi, where classi is the actual class.
  • Properties such as "WorstClassifiedExamples" or "MostCertainExamples" output up to 10 examples. ClassifierMeasurementsObject[][propn] can be used to output n examples.
  • ClassifierMeasurementsObject[] has the same options as ClassifierFunction[], with the following additions:
  • WeightsAutomaticweights to be associated with test set examples
    "Uncertainty"Falsewhether measures should be given with their statistical uncertainty
  • With the setting "Uncertainty"True, numerical measures will be given in the form measure±err where err represents the standard error (corresponding to a 68% confidence interval) associated with measure.
  • Possible settings for Weights include:
  • Automaticassociates weight 1 with all test examples
    {w1,w2,}associates weight wi with the i^(th) test examples
  • Changing the weight of a test example from 1 to 2 is equivalent to duplicating this example.
  • Weights affect measures as well as their uncertainties.


Basic Examples  (1)

Define a training set and a test set:

Click for copyable input
Click for copyable input

Create a classifier on the training set:

Click for copyable input

Generate a ClassifierMeasurementsObject of the classifier with the test set:

Click for copyable input

Measure accuracy and F-score and plot the confusion matrix from the ClassifierMeasurementsObject:

Click for copyable input

See Also

ClassifierMeasurements  ClassifierFunction  Classify  ClassifierInformation  PredictorMeasurementsObject

Introduced in 2015
| Updated in 2018