PredictorMeasurements
PredictorMeasurements[predictor,testset,prop]
gives measurements associated with the property prop when predictor is evaluated on testset.
PredictorMeasurements[…,{prop1,prop2,…}]
gives properties prop1, prop2, etc.
PredictorMeasurements[predictor,testset]
yields a PredictorMeasurementsObject[…] that can be applied to any property.
Details and Options




- The predictor is typically a PredictorFunction object as generated by Predict.
- PredictorMeasurements[…,opts] specifies that the predictor should use the options opts when applied to the test set. Possible options are as given in PredictorFunction.
- PredictorMeasurementsObject[…][prop] can be used to look up prop from a PredictorMeasurementsObject. When repeated property lookups are required, this is typically more efficient.
- PredictorMeasurementsObject[…][prop,opts] specifies that the predictor should use the options opts when applied to the test set. It supersedes options given to PredictorMeasurements.
- PredictorMeasurements has the same options as PredictorFunction[…], with the following additions:
-
Weights Automatic weights to be associated with test set examples ComputeUncertainty False whether measures should be given with their statistical uncertainty - When ComputeUncertaintyTrue, numerical measures will be returned as Around[result,err], where err represents the standard error (corresponding to a 68% confidence interval) associated with measure result.
- Possible settings for Weights include:
-
Automatic associates weight 1 with all test examples {w1,w2,…} associates weight wi with the i test examples
- Changing the weight of a test example from 1 to 2 is equivalent to duplicating the example.
- Weights affect measures as well as their uncertainties.
- Properties returning a single numeric value related to prediction abilities on the test set include:
-
"StandardDeviation" root mean square of the residuals "StandardDeviationBaseline" standard deviation of test set values "LogLikelihood" log-likelihood of the model given the test data "MeanCrossEntropy" mean cross entropy over test examples "MeanDeviation" mean absolute value of the residuals "MeanSquare" mean square of the residuals "Perplexity" exponential of the mean cross entropy "RejectionRate" fraction of examples predicted as Indeterminate "GeometricMeanProbabilityDensity" geometric mean of the actual-class probability densities - Test examples classified as Indeterminate will be discarded when measuring properties related to prediction abilities on the test set, such as "StandardDeviation" or "MeanCrossEntropy".
- Properties returning graphics include:
-
"Report" panel reporting main measurements "ComparisonPlot" plot of predicted values versus test values "ProbabilityDensityHistogram" histogram of actual-class probability densities "ResidualHistogram" histogram of residuals "ResidualPlot" plot of the residuals - Timing-related properties include:
-
"EvaluationTime" time needed to predict one example of the test set "BatchEvaluationTime" marginal time to predict one example in a batch - Properties returning one value for each test-set example include:
-
"Residuals" list of differences between predicted and test values "ProbabilityDensities" actual-class prediction probability densities - Properties returning examples from the test set include:
-
"BestPredictedExamples" examples having the highest actual-class probability density "Examples" all test examples "Examples"{i1,i2} all examples in the interval i1 predicted in the interval i2 "LeastCertainExamples" examples having the highest distribution entropy "MostCertainExamples" examples having the lowest distribution entropy "WorstPredictedExamples" examples having the lowest actual-class probability density - Examples are given in the form inputivaluei, where valuei is the actual value.
- Properties such as "WorstPredictedExamples" or "MostCertainExamples" output up to 10 examples. PredictorMeasurementsObject[…][propn] can be used to output n examples.
- Other properties include:
-
"PredictorFunction" PredictorFunction[…] being measured "Properties" list of measurement properties available
Examples
open all close allBasic Examples (1)
Define a training set and a test set:
In[1]:= |
In[2]:= |
Create a predictor with the training set:
Visualize the residual values:
Compute the root mean square of the residuals:
Generate a PredictorMeasurementsObject of the predictor with the test set:
Perform the previous measurements using the PredictorMeasurementsObject:
Scope (1)
Options (5)
Applications (2)
Introduced in 2014
Updated in 2019
(10.0)
|
(12.0)