PredictorMeasurementsObject

PredictorMeasurementsObject[]

represents an object generated by PredictorMeasurements that can be applied to properties.

Details and Options

  • PredictorMeasurementsObject[][prop] is used to look up property prop from the PredictorMeasurementsObject.
  • PredictorMeasurementsObject[][{prop1,prop2,}] is used to look up many properties.
  • PredictorMeasurementsObject[][prop,opts] specifies that the predictor should use the options opts when applied to the test set. It supersedes options given to PredictorMeasurements.
  • Possible options are as given in PredictorFunction[], with the following addition:
  • ComputeUncertaintyFalsewhether measures should be given with their statistical uncertainty
  • When ComputeUncertaintyTrue, numerical measures will be returned as Around[result,err], where err represents the standard error (corresponding to a 68% confidence interval) associated with measure result.
  • Properties returning a single numeric value related to prediction abilities on the test set include:
  • "StandardDeviation"root mean square of the residuals
    "StandardDeviationBaseline"standard deviation of test set values
    "LogLikelihood"log-likelihood of the model given the test data
    "MeanCrossEntropy"mean cross entropy over test examples
    "MeanDeviation"mean absolute value of the residuals
    "MeanSquare"mean square of the residuals
    "Perplexity"exponential of the mean cross entropy
    "RejectionRate"fraction of examples predicted as Indeterminate
    "GeometricMeanProbabilityDensity"geometric mean of the actual-class probability densities
  • Test examples classified as Indeterminate will be discarded when measuring properties related to prediction abilities on the test set, such as "StandardDeviation" or "MeanCrossEntropy".
  • Properties returning graphics include:
  • "Report"panel reporting main measurements
    "ComparisonPlot"plot of predicted values versus test values
    "ProbabilityDensityHistogram"histogram of actual-class probability densities
    "ResidualHistogram"histogram of residuals
    "ResidualPlot"plot of the residuals
  • Timing-related properties include:
  • "EvaluationTime"time needed to predict one example of the test set
    "BatchEvaluationTime"marginal time to predict one example in a batch
  • Properties returning one value for each test-set example include:
  • "Residuals"list of differences between predicted and test values
    "ProbabilityDensities"actual-class prediction probability densities
  • Properties returning examples from the test set include:
  • "BestPredictedExamples"examples having the highest actual-class probability density
    "Examples"all test examples
    "Examples"{i1,i2}all examples in the interval i1 predicted in the interval i2
    "LeastCertainExamples"examples having the highest distribution entropy
    "MostCertainExamples"examples having the lowest distribution entropy
    "WorstPredictedExamples"examples having the lowest actual-class probability density
  • Examples are given in the form inputivaluei, where valuei is the actual value.
  • Properties such as "WorstPredictedExamples" or "MostCertainExamples" output up to 10 examples. PredictorMeasurementsObject[][propn] can be used to output n examples.
  • Other properties include:
  • "PredictorFunction"PredictorFunction[] being measured
    "Properties"list of measurement properties available

Examples

Basic Examples  (1)

Define a training set and a test set:

In[1]:=
Click for copyable input
In[2]:=
Click for copyable input

Create a predictor with the training set:

In[3]:=
Click for copyable input
Out[3]=

Generate a PredictorMeasurementsObject of the predictor with the test set:

In[4]:=
Click for copyable input
Out[4]=

Measure the standard deviation and plot the residuals from the PredictorMeasurementsObject:

In[5]:=
Click for copyable input
Out[5]=
Introduced in 2015
(10.1)
|
Updated in 2019
(12.0)