SynthesizeMissingValues

SynthesizeMissingValues[{example1,example2,…}]

replaces missing values in each example by generated values.

SynthesizeMissingValues[dist,data]

uses the distribution dist to generate values.

Details and Options

  • SynthesizeMissingValues is used to synthesize missing elements in a dataset, inferring them from the known part of the data.
  • SynthesizeMissingValues can be used on many types of data, including numerical, nominal and images.
  • Each examplei can be a single data element, a list of data elements or an association of data elements. Examples can also be given as a Dataset object.
  • The distribution dist must be a LearnedDistribution[…].
  • The following options can be given:
  • FeatureNames Automaticfeature names to assign for input data
    FeatureTypes Automaticfeature types to assume for input data
    Method Automaticwhich modeling algorithm to use
    MissingValuePattern _Missing|Indeterminatethe pattern of the assumed missing values
    PerformanceGoal Automaticaspects of performance to optimize
    RandomSeeding1234what seeding of pseudorandom generators should be done internally
    TimeGoalAutomatichow long to spend for training
    TrainingProgressReporting Automatichow to report progress during training
    ValidationSetAutomaticthe set of data on which to evaluate the model during training
  • Possible settings for PerformanceGoal include:
  • "Quality"maximize the synthesis quality
    "Speed"maximize the synthesis speed
    Automaticautomatic tradeoff between speed and quality
  • Possible settings for Method include:
  • Automaticautomatically choose distribution method and synthesis strategy
    Nonedo not use any missing synthesizer
    methoduse the specified method
    strategyhow to synthesize from the distribution
    assocspecify both distribution method and synthesis strategy
  • Possible settings for method include:
  • "Multinormal"use a multivariate normal (Gaussian) distribution
    "ContingencyTable"discretize data and store each possible probability
    "KernelDensityEstimation"use a kernel mixture distribution
    "DecisionTree"use a decision tree to compute probabilities
    "GaussianMixture"use a mixture of Gaussian (normal) distributions
  • Possible settings for strategy include:
  • Automaticautomatically choose the synthesis strategy
    "MarginalSampling"sample from each feature's marginal distribution
    "ModeFinding"find the mode of the distribution conditioned on known values
    "RandomSampling"sample from the distribution conditioned on known values (default)
  • In the form Methodassoc, the association assoc should be of the form <|"LearningMethod"method,"EvaluationStrategy"strategy|>.
  • The following settings for TrainingProgressReporting can be used:
  • "Panel"show a dynamically updating graphical panel
    "Print"periodically report information using Print
    "ProgressIndicator"show a simple ProgressIndicator
    "SimplePanel"dynamically updating panel without learning curves
    Nonedo not report any information
  • Possible settings for RandomSeeding include:
  • Automaticautomatically reseed every time the function is called
    Inheriteduse externally seeded random numbers
    seeduse an explicit integer or strings as a seed
  • SynthesizeMissingValues[…,FeatureExtractor"Minimal"] indicates that the internal preprocessing should be as simple as possible.

Examples

open allclose all

Basic Examples  (2)

Fill in missing values in a numeric dataset:

Train a distribution on a two-dimensional dataset:

Fill in missing values based on the learned distribution:

Scope  (5)

Fill in missing values in a vector:

Fill in missing values in a list of vectors:

Fill in missing values in a list of associations:

Fill in missing values in a dataset:

Fill in values using a pretrained LearnedDistribution[…]:

This can be used to synthesize a single example:

Options  (9)

FeatureTypes  (1)

Specify that the first feature should be interpreted as a nominal variable, while the others should be determined automatically:

FeatureNames  (1)

Replace missing values and specify that the feature named "gender" should be considered nominal:

Method  (2)

Replace Missing[] values using "Multinormal" method for computing the distribution:

Use "KernelDensityEstimation" method for replacing the missing values:

Specify the method as an association, choosing the evaluation strategy and the learning method for computing the distribution:

MissingValuePattern  (2)

Specify values that should be assumed missing using MissingValuePattern:

Use a pattern to specify the values:

Specify missing values with Condition:

PerformanceGoal  (1)

Synthesize missing values by specifying the PerformanceGoal:

Compare the missing imputation time with the default PerformanceGoal:

TrainingProgressReporting  (2)

Print the training progress periodically during training:

Show training progress interactively without the plots:

Applications  (2)

Obtain a dataset of images:

Train a distribution on the images:

Replace the value that should be considered missing with the samples that are generated from the learned distribution:

Obtain a dataset related to features of moons of Jupiter that contains missing values:

Replace missing values in the dataset:

Properties & Relations  (1)

The "EvaluationStrategy" setting affects the type of sampling that will be performed.

Define some random data:

Learn the data distribution:

Try different strategies:

Compare the different sample points distributions:

Plot the sampled points together with the original data:

Possible Issues  (1)

Single values cannot be synthesized without a reference distribution:

Wolfram Research (2019), SynthesizeMissingValues, Wolfram Language function, https://reference.wolfram.com/language/ref/SynthesizeMissingValues.html.

Text

Wolfram Research (2019), SynthesizeMissingValues, Wolfram Language function, https://reference.wolfram.com/language/ref/SynthesizeMissingValues.html.

CMS

Wolfram Language. 2019. "SynthesizeMissingValues." Wolfram Language & System Documentation Center. Wolfram Research. https://reference.wolfram.com/language/ref/SynthesizeMissingValues.html.

APA

Wolfram Language. (2019). SynthesizeMissingValues. Wolfram Language & System Documentation Center. Retrieved from https://reference.wolfram.com/language/ref/SynthesizeMissingValues.html

BibTeX

@misc{reference.wolfram_2024_synthesizemissingvalues, author="Wolfram Research", title="{SynthesizeMissingValues}", year="2019", howpublished="\url{https://reference.wolfram.com/language/ref/SynthesizeMissingValues.html}", note=[Accessed: 05-November-2024 ]}

BibLaTeX

@online{reference.wolfram_2024_synthesizemissingvalues, organization={Wolfram Research}, title={SynthesizeMissingValues}, year={2019}, url={https://reference.wolfram.com/language/ref/SynthesizeMissingValues.html}, note=[Accessed: 05-November-2024 ]}