--- title: "ActivePrediction" language: "en" type: "Symbol" summary: "ActivePrediction[f, {conf1, conf2, ...}] gives an object representing the result of active prediction obtained by using the function f to determine values for the example configurations confi. ActivePrediction[f, reg] generates configurations within the region specified by reg. ActivePrediction[f, sampler] generates configurations by applying the function sampler. ActivePrediction[f, {conf1, conf2, ...} -> nsampler] applies the function nsampler to successively generate configurations starting from one of the confi." keywords: - active predict - active predicting - active learning - active data selection - dynamic data selection - dynamic predicting - cost effective selection - efficient sampling canonical_url: "https://reference.wolfram.com/language/ref/ActivePrediction.html" source: "Wolfram Language Documentation" related_guides: - title: "Supervised Machine Learning" link: "https://reference.wolfram.com/language/guide/SupervisedMachineLearning.en.md" - title: "Machine Learning" link: "https://reference.wolfram.com/language/guide/MachineLearning.en.md" related_functions: - title: "Predict" link: "https://reference.wolfram.com/language/ref/Predict.en.md" - title: "ActivePredictionObject" link: "https://reference.wolfram.com/language/ref/ActivePredictionObject.en.md" - title: "BayesianMinimization" link: "https://reference.wolfram.com/language/ref/BayesianMinimization.en.md" - title: "BayesianMaximization" link: "https://reference.wolfram.com/language/ref/BayesianMaximization.en.md" --- [EXPERIMENTAL] # ActivePrediction ActivePrediction[f, {conf1, conf2, …}] gives an object representing the result of active prediction obtained by using the function f to determine values for the example configurations confi. ActivePrediction[f, reg] generates configurations within the region specified by reg. ActivePrediction[f, sampler] generates configurations by applying the function sampler. ActivePrediction[f, {conf1, conf2, …} -> nsampler] applies the function nsampler to successively generate configurations starting from one of the confi. ## Details and Options * ``ActivePrediction[…]`` returns an ``ActivePredictionObject[…]`` whose properties can be obtained using ``ActivePredictionObject[…]["prop"]``. * Possible properties include: | | | | ----------------------------- | -------------------------------------------------------- | | "EvaluationHistory" | configurations explored and values corresponding to them | | "Method" | method used for active prediction | | "PredictorFunction" | best PredictorFunction[…] obtained | | "PredictorMeasurementsObject" | latest PredictorMeasurementsObject[…] obtained | | "OracleFunction" | original function f used to determine values | | "LearningCurve" | plot of mean cross-entropy evolution | | "Properties" | list of all available properties | * Configurations can be of any form accepted by ``Predict`` (single data element, list of data elements, association of data elements, etc.) and of any type accepted by ``Predict`` (numerical, textual, sounds, images, etc.). * When applied to a configuration ``conf``, the output of the function ``f`` must be a real-number value. * In ``ActivePrediction[f, spec]``, ``spec`` defines the domain of the function ``f``. A domain can be defined by a list of configurations, a geometric region or a configuration generator function. * In ``ActivePrediction[f, sampler]``, ``sampler[]`` must output a configuration suitable for ``f`` to be applied to it. * In ``ActivePrediction[f, {conf1, conf2, …} -> nsampler]``, ``nsampler[conf]`` must output a configuration. * ``ActivePrediction`` has the same options as ``Predict``, with the following additions and changes: [] | | | | | ------------------------- | --------- | ------------------------------------------------------------------------------------ | | InitialEvaluationHistory | None | initial set of configurations and values | | MaxIterations | 2000 | maximum number of iterations | | Method | Automatic | method used to determine configurations to query and the prediction algorithm to use | | RandomSeeding | 1234 | what seeding of pseudorandom generators should be done internally | * Possible settings for ``Method`` include: | | | | ------------ | --------------------------------------------------------------------- | | Automatic | automatically choose method | | "Randomized" | choose random configurations from the domain | | "MaxEntropy" | choose configurations for which the predictor has maximum uncertainty | | assoc | association specifying the evaluation strategy and prediction method | * In the form ``Method -> assoc``, the association can have elements: | | | | -------------------- | ---------------------------------------------------- | | "EvaluationStrategy" | method for determining which configurations to query | | "PredictionMethod" | method to use for prediction | * Possible settings for ``RandomSeeding`` include: | | | | --------- | ------------------------------------------------------ | | Automatic | automatically reseed every time the function is called | | Inherited | use externally seeded random numbers | | seed | use an explicit integer or strings as a seed | ### List of all options | | | | | ------------------------- | --------- | ------------------------------------------------------------------------------------ | | AcceptanceThreshold | Automatic | rarer probability threshold for anomaly detector | | AnomalyDetector | None | anomaly detector used by the predictor | | FeatureExtractor | Identity | how to extract features from which to learn | | FeatureNames | Automatic | feature names to assign for input data | | FeatureTypes | Automatic | feature types to assume for input data | | IndeterminateThreshold | 0 | below what probability density to return Indeterminate | | InitialEvaluationHistory | None | initial set of configurations and values | | MaxIterations | 2000 | maximum number of iterations | | Method | Automatic | method used to determine configurations to query and the prediction algorithm to use | | MissingValueSynthesis | Automatic | how to synthesize missing values | | PerformanceGoal | Automatic | aspects of performance to try to optimize | | RandomSeeding | 1234 | what seeding of pseudorandom generators should be done internally | | RecalibrationFunction | Automatic | how to post-process predicted value | | TargetDevice | "CPU" | the target device on which to perform training | | TimeGoal | Automatic | how long to spend training the classifier | | TrainingProgressReporting | Automatic | how to report progress during training | | UtilityFunction | Automatic | utility as function of actual and predicted value | | ValidationSet | Automatic | data on which to validate the model generated | --- ## Examples (11) ### Basic Examples (3) Train an ``ActivePredictionObject[…]`` to find the predictor for a function, given a set of configurations: ```wl In[1]:= apo = ActivePrediction[# + 2&, {2.0, 4.3, 9.1, 13.4, 20.5}] Out[1]= ActivePredictionObject[«1»] ``` Extract the resulting predictor: ```wl In[2]:= p = apo["PredictorFunction"] Out[2]= PredictorFunction[Association["ExampleNumber" -> 5, "Input" -> Association["Preprocessor" -> MachineLearning`MLProcessor["ToMLDataset", Association["Input" -> Association["f1" -> Association["Type" -> "Numerical"]], "Output" -> As ... e" -> DateObject[{2025, 5, 22, 15, 10, 7.591531`7.632904357059327}, "Instant", "Gregorian", -5.], "ProcessorCount" -> 12, "ProcessorType" -> "ARM64", "OperatingSystem" -> "MacOSX", "SystemWordLength" -> 64, "Evaluations" -> {}]]] ``` Predict new examples: ```wl In[3]:= p[{1.2, 2.3, 7.8}] Out[3]= {3.16314, 4.2667, 9.7845} ``` --- Train a prediction object to find the predictor for a function whose domain is defined by an interval: ```wl In[1]:= apo = ActivePrediction[(# - 4) ^ 2&, Interval[{0, 8}]] Out[1]= ActivePredictionObject[…] ``` Extract the predictor: ```wl In[2]:= p = apo["PredictorFunction"] Out[2]= PredictorFunction[«1»] ``` Predict new examples: ```wl In[3]:= p[{-0.5, 2.3, 16.9}] Out[3]= {14.9373, 2.7652, 15.6751} ``` --- Train a prediction object to find the predictor for the ``Det`` function, with the domain defined by a configuration generator: ```wl In[1]:= apo = ActivePrediction[Det, RandomReal[1, {2, 2}]&] Out[1]= ActivePredictionObject[…] ``` Extract the predictor: ```wl In[2]:= p = apo["PredictorFunction"] Out[2]= PredictorFunction[Association["ExampleNumber" -> 82, "Input" -> Association["Preprocessor" -> MachineLearning`MLProcessor["ToMLDataset", Association["Input" -> Association["f1" -> Association["Type" -> "NumericalTensor", "Dime ... e" -> DateObject[{2025, 5, 22, 15, 11, 9.155923`7.714277118368699}, "Instant", "Gregorian", -5.], "ProcessorCount" -> 12, "ProcessorType" -> "ARM64", "OperatingSystem" -> "MacOSX", "SystemWordLength" -> 64, "Evaluations" -> {}]]] ``` Predict new examples: ```wl In[3]:= matrices = Table[RandomReal[1, {2, 2}], 5]; p /@ matrices Out[3]= {-0.166665, 0.0627539, -0.324806, -0.224777, -0.3811} ``` ### Scope (3) Train a prediction object to find a predictor for the sine function in an interval: ```wl In[1]:= apo = ActivePrediction[Sin, Interval[{0, 10}]] Out[1]= ActivePredictionObject[…] ``` Obtain the list of available object properties: ```wl In[2]:= apo["Properties"] Out[2]= {"OracleFunction", "EvaluationHistory", "Method", "Properties", "LearningCurve", "TrainingHistory", "PredictorFunction", "PredictorMeasurementsObject"} ``` Obtain the history of explored configurations: ```wl In[3]:= apo["EvaluationHistory"] Out[3]= Dataset[{Association["Configuration" -> 5.219642502018771, "Value" -> -0.8740819825910249], Association["Configuration" -> 0.8622342695413243, "Value" -> 0.7592983912859621], Association["Configuration" -> 3.779129543048858, "Value" -> -0.595 ... ation["Configuration" -> 9.306212764757332, "Value" -> 0.11828759876418515], Association["Configuration" -> 6.9767069836059425, "Value" -> 0.6392493082894946], Association["Configuration" -> 7.61780331409375, "Value" -> 0.9722393027394842]}] ``` Obtain the predictors trained during active prediction, along with some of their properties: ```wl In[4]:= apo["TrainingHistory"] Out[4]= Dataset[<>] ``` Obtain the final predictor: ```wl In[5]:= p = apo["PredictorFunction"] Out[5]= PredictorFunction[Association["ExampleNumber" -> 24, "Input" -> Association["Preprocessor" -> MachineLearning`MLProcessor["ToMLDataset", Association["Input" -> Association["f1" -> Association["Type" -> "Numerical"]], "Output" -> A ... "Date" -> DateObject[{2025, 5, 22, 15, 11, 54.068016`8.485515408070897}, "Instant", "Gregorian", -5.], "ProcessorCount" -> 12, "ProcessorType" -> "ARM64", "OperatingSystem" -> "MacOSX", "SystemWordLength" -> 64, "Evaluations" -> {}]]] ``` Obtain the method used to choose configurations to add to the training set: ```wl In[6]:= apo["Method"] Out[6]= "MaxEntropy" ``` Obtain some other properties: ```wl In[7]:= apo[{"OracleFunction", "PredictorMeasurementsObject"}] Out[7]= {Sin, PredictorMeasurementsObject[«1»]} ``` Display the performances of the predictors trained during active prediction: ```wl In[8]:= apo["LearningCurve"] Out[8]= [image] ``` Visualize the predictions of the predictor on new examples: ```wl In[9]:= Plot[{Sin[x], p[x]}, {x, 0, 10}, Exclusions -> False, Frame -> True, PlotLegends -> {"Function", "Model"}] Out[9]= [image] ``` --- Train a prediction object to find the predictor for a function that computes the color distance between a given color and the red color, with the domain defined by a random color generator: ```wl In[1]:= apo = ActivePrediction[ColorDistance[Red, #]&, RandomColor[LCHColor[0.5, _, _]]&] Out[1]= ActivePredictionObject[…] ``` Obtain the predictor: ```wl In[2]:= p = apo["PredictorFunction"] Out[2]= PredictorFunction[«1»] ``` Display the performances of the predictors trained during active prediction: ```wl In[3]:= apo["LearningCurve"] Out[3]= [image] ``` Display the comparison plot of the predictor for a test set: ```wl In[4]:= colors = RandomColor[LCHColor[0.5, _, _], 30]; testset = # -> ColorDistance[Red, #]& /@ colors; apM = PredictorMeasurements[p, testset]; apM["ComparisonPlot"] Out[4]= [image] ``` --- Define a nontrivial function, with the domain defined by a neighborhood configuration generator: ```wl In[1]:= f[x_] := -x Sin[x] + 3 Cos[x] + x / 2; nsampler[x_] := x + RandomReal[{-0.1, 0.1}]; ``` Train a prediction object to find a predictor for the function, starting with some initial configurations: ```wl In[2]:= apo = ActivePrediction[f, {-0.3, 0.5} -> nsampler] Out[2]= ActivePredictionObject[…] ``` Obtain the predictor: ```wl In[3]:= p = apo["PredictorFunction"] Out[3]= PredictorFunction[«1»] ``` Visualize the predictions of the predictor. It provides a good model of the function in the neighborhood of the initial configurations: ```wl In[4]:= Plot[{f[x], p[x]}, {x, -1, 1}, Exclusions -> False, Frame -> True, PlotLegends -> {"Function", "Model"}] Out[4]= [image] ``` ### Options (3) #### InitialEvaluationHistory (1) Define a quadratic function whose domain is defined by a set of numbers: ```wl In[1]:= f[x_] := x ^ 2 - 3 * x + 2; configset = RandomReal[{0, 10}, 25]; ``` Construct an initial "training set": ```wl In[2]:= evalhist = # -> f[#]& /@ RandomReal[{0, 10}, 5] Out[2]= {1.97609 -> -0.0233417, 7.07402 -> 30.8197, 7.94729 -> 41.3176, 3.46156 -> 3.5977, 7.07662 -> 30.8487} ``` Train a prediction object to find a predictor for the function using the preceding information: ```wl In[3]:= apo = ActivePrediction[f, configset, InitialEvaluationHistory -> evalhist] Out[3]= ActivePredictionObject[…] ``` The examples in the first row in the training history now correspond to the initial training set: ```wl In[4]:= apo["TrainingHistory"] Out[4]= Dataset[<>] ``` #### MaxIterations (1) Define a nontrivial two-dimensional function: ```wl In[1]:= f[{x_, y_}] := Cos[x] - Exp[(x - 0.5) * y]; ``` Train a prediction object to find a predictor for the function within a unit disk: ```wl In[2]:= apo = ActivePrediction[f, Disk[]] Out[2]= ActivePredictionObject[…] ``` Obtain the number of function evaluations: ```wl In[3]:= apo["TrainingHistory"][[-1]]["# Training Examples"] Out[3]= 43 ``` Specify the maximum number of iterations: ```wl In[4]:= apo2 = ActivePrediction[f, Disk[], MaxIterations -> 17] Out[4]= ActivePredictionObject[…] ``` Check the number of function evaluations now: ```wl In[5]:= apo2["TrainingHistory"][[-1]]["# Training Examples"] Out[5]= 17 ``` #### Method (1) Define a nontrivial stochastic function: ```wl In[1]:= f[{x_, y_}] := Cos[x * y] + RandomReal[{-.1, .1}]; ``` Train a prediction object by specifying the method as an association, choosing the evaluation strategy and the prediction method: ```wl In[2]:= apo = ActivePrediction[f, Rectangle[{-2, -2}, {2, 2}], Method -> <|"EvaluationStrategy" -> "MaxEntropy", "PredictionMethod" -> "NearestNeighbors"|>] Out[2]= ActivePredictionObject[…] ``` Obtain the predictor: ```wl In[3]:= p = apo["PredictorFunction"] Out[3]= PredictorFunction[Association["ExampleNumber" -> 2000, "Input" -> Association["Preprocessor" -> MachineLearning`MLProcessor["ToMLDataset", Association["Input" -> Association["f1" -> Association["Type" -> "NumericalVector", "Le ... "Date" -> DateObject[{2025, 5, 22, 15, 18, 17.810298`8.003246170615979}, "Instant", "Gregorian", -5.], "ProcessorCount" -> 12, "ProcessorType" -> "ARM64", "OperatingSystem" -> "MacOSX", "SystemWordLength" -> 64, "Evaluations" -> {}]]] ``` Specify a different method for active prediction: ```wl In[4]:= apo2 = ActivePrediction[f, Rectangle[{-2, -2}, {2, 2}], Method -> <|"EvaluationStrategy" -> "MaxEntropy", "PredictionMethod" -> "GaussianProcess"|>] Out[4]= ActivePredictionObject[…] ``` Obtain the predictor again: ```wl In[5]:= p2 = apo2["PredictorFunction"] Out[5]= PredictorFunction[Association["ExampleNumber" -> 15, "Input" -> Association["Preprocessor" -> MachineLearning`MLProcessor["ToMLDataset", Association["Input" -> Association["f1" -> Association["Type" -> "NumericalVector", "Leng ... "Date" -> DateObject[{2025, 5, 22, 15, 18, 33.983633`8.283844785166332}, "Instant", "Gregorian", -5.], "ProcessorCount" -> 12, "ProcessorType" -> "ARM64", "OperatingSystem" -> "MacOSX", "SystemWordLength" -> 64, "Evaluations" -> {}]]] ``` Visualize the predictions of the two predictors. ``"GaussianProcess"`` produces a smoother predictor relative to ``"NearestNeighbors"`` : ```wl In[6]:= Plot3D[{p[{x, y}], p2[{x, y}]}, {x, -2, 2}, {y, -2, 2}, Exclusions -> False, PlotLegends -> {"NearestNeighbors", "GaussianProcess"}] Out[6]= [image] ``` ### Applications (2) Efficiently train a model to predict the elevation at a specific location: ```wl In[1]:= apo = ActivePrediction[QuantityMagnitude @* GeoElevationData, RandomGeoPosition[Entity["AdministrativeDivision", {"Massachusetts", "UnitedStates"}]]&]; ``` Sample 1000 random locations to test the quality of the model: ```wl In[2]:= points = RandomGeoPosition[Entity["AdministrativeDivision", {"Massachusetts", "UnitedStates"}], 1000]; ``` Compare the predicted elevation versus the actual elevation obtained via a server call: ```wl In[3]:= PointValuePlot[points -> apo["PredictorFunction"][points]] Out[3]= [image] In[4]:= PointValuePlot[points -> GeoElevationData[points]] Out[4]= [image] ``` #### LogLikelihood Function Predictor (1) Load Fisher's *Iris* dataset and divide it into a training set and a test set: ```wl In[1]:= sample = RandomSample[ExampleData[{"MachineLearning", "FisherIris"}, "Data"]]; trainingsample = sample[[1 ;; 100]]; testsample = sample[[101 ;; ]]; ``` Construct a ``LogLikelihood`` function that trains a classifier on the training sample and then gives the ``"LogLikelihoodRate"`` on the test sample for a given choice of the hyperparameters: ```wl In[2]:= f[{l1_, l2_}] := Module[ {class}, class = Classify[trainingsample, Method -> {"LogisticRegression", "L1Regularization" -> Exp[l1], "L2Regularization" -> Exp[l2]}]; ClassifierMeasurements[class, testsample, "LogLikelihood"] ]; reg = Rectangle[{-3., -3.}, {3., 3.}]; ``` Train a prediction object to find a predictor for the ``LogLikelihood`` function over a rectangular domain: ```wl In[3]:= apo = ActivePrediction[f, reg] Out[3]= ActivePredictionObject[…] ``` Obtain the predictor: ```wl In[4]:= p = apo["PredictorFunction"] Out[4]= PredictorFunction[«1»] ``` Obtain the coordinates of the explored configurations and their values: ```wl In[5]:= configurations = Join[Normal[apo["EvaluationHistory"][[#, "Configuration"]]], {Normal[apo["EvaluationHistory"][[#, "Value"]]]}]& /@ Range[Length[apo["EvaluationHistory"]]]; ``` Visualize the prediction of the predictor together with the explored configurations: ```wl In[6]:= Show[Plot3D[p[{x, y}], {x, -3, 3}, {y, -3, 3}, Exclusions -> False], ListPointPlot3D[configurations]] Out[6]= [image] ``` ## See Also * [`Predict`](https://reference.wolfram.com/language/ref/Predict.en.md) * [`ActivePredictionObject`](https://reference.wolfram.com/language/ref/ActivePredictionObject.en.md) * [`BayesianMinimization`](https://reference.wolfram.com/language/ref/BayesianMinimization.en.md) * [`BayesianMaximization`](https://reference.wolfram.com/language/ref/BayesianMaximization.en.md) ## Related Guides * [Supervised Machine Learning](https://reference.wolfram.com/language/guide/SupervisedMachineLearning.en.md) * [Machine Learning](https://reference.wolfram.com/language/guide/MachineLearning.en.md) ## History * [Introduced in 2017 (11.1)](https://reference.wolfram.com/language/guide/SummaryOfNewFeaturesIn111.en.md) \| [Updated in 2017 (11.2)](https://reference.wolfram.com/language/guide/SummaryOfNewFeaturesIn112.en.md)