LearnDistribution

✖
`LearnDistribution`

Updatedshow changeshide changes

✖

LearnDistribution[{example₁,example₂,…}]

generates a LearnedDistribution[…] that attempts to represent an underlying distribution for the examples given.

Details and Options

LearnDistribution can be used on many types of data, including numerical, nominal and images.
Each example_i can be a single data element, a list of data elements or an association of data elements. Examples can also be given as a Dataset or a Tabular object.
LearnDistribution effectively assumes that each of the example_i is independently drawn from an underlying distribution, which LearnDistribution attempts to infer.
LearnDistribution[examples] yields a LearnedDistribution[…] on which the following functions can be used:

	PDF[dist,…]	probability or probability density for data
	RandomVariate[dist]	random samples generated from the distribution
	SynthesizeMissingValues[dist,…]	fill in missing values according to the distribution
	RarerProbability[dist,…]	compute the probability to generate a sample with lower PDF than a given example

The following options can be given:

FeatureExtractor	Identity	how to extract features from which to learn
FeatureNames	Automatic	feature names to assign for input data
FeatureTypes	Automatic	feature types to assume for input data
Method	Automatic	which modeling algorithm to use
PerformanceGoal	Automatic	aspects of performance to try to optimize
RandomSeeding	1234	what seeding of pseudorandom generators should be done internally
TimeGoal	Automatic	how long to spend training the classifier
TrainingProgressReporting	Automatic	how to report progress during training
ValidationSet	Automatic	the set of data on which to evaluate the model during training

Possible settings for PerformanceGoal include:

	"DirectTraining"	train directly on the full dataset, without model searching
	"Memory"	minimize storage requirements of the distribution
	"Quality"	maximize the modeling quality of the distribution
	"Speed"	maximize speed for PDF queries
	"SamplingSpeed"	maximize speed for generating random samples
	"TrainingSpeed"	minimize time spent producing the distribution
	Automatic	automatic tradeoff among speed, quality and memory
	{goal₁,goal₂,…}	automatically combine goal₁, goal₂, etc.

Possible settings for Method include:

	"ContingencyTable"	discretize data and store each possible probability
	"DecisionTree"	use a decision tree to compute probabilities
	"GaussianMixture"	use a mixture of Gaussian (normal) distributions
	"KernelDensityEstimation"	use a kernel mixture distribution
	"Multinormal"	use a multivariate normal (Gaussian) distribution

The following settings for TrainingProgressReporting can be used:

	"Panel"	show a dynamically updating graphical panel
	"Print"	periodically report information using Print
	"ProgressIndicator"	show a simple ProgressIndicator
	"SimplePanel"	dynamically updating panel without learning curves
	None	do not report any information

Possible settings for RandomSeeding include:

	Automatic	automatically reseed every time the function is called
	Inherited	use externally seeded random numbers
	seed	use an explicit integer or strings as a seed

Only reversible feature extractors can be given in the option FeatureExtractor.
LearnDistribution[…,FeatureExtractor"Minimal"] indicates that the internal preprocessing should be as simple as possible.
All images are first conformed using ConformImages.
Information[LearnedDistribution[…]] generates an information panel about the distribution and its estimated performances.

Examples

open allclose all

Basic Examples (3)Summary of the most common use cases

Train a distribution on a numeric dataset:

In[1]:=1

✖

https://wolfram.com/xid/0ywjl2umsqsa-5hhlm7

Out[1]=1

Generate a new example based on the learned distribution:

In[2]:=2

✖

https://wolfram.com/xid/0ywjl2umsqsa-38oizs

Out[2]=2

Compute the PDF of a new example:

In[3]:=3

✖

https://wolfram.com/xid/0ywjl2umsqsa-ed58y3

Out[3]=3

Train a distribution on a nominal dataset:

In[1]:=1

✖

https://wolfram.com/xid/0ywjl2umsqsa-1giaj9

Out[1]=1

Generate a new example based on the learned distribution:

In[2]:=2

✖

https://wolfram.com/xid/0ywjl2umsqsa-67g5d3

Out[2]=2

Compute the probability of the examples "A" and "B":

In[3]:=3

✖

https://wolfram.com/xid/0ywjl2umsqsa-wmcrl5

Out[3]=3

Train a distribution on a two-dimensional dataset:

In[1]:=1

✖

https://wolfram.com/xid/0ywjl2umsqsa-9eb8g

Out[1]=1

Generate a new example based on the learned distribution:

In[2]:=2

✖

https://wolfram.com/xid/0ywjl2umsqsa-mxqvzs

Out[2]=2

Compute the probability of two examples:

In[3]:=3

✖

https://wolfram.com/xid/0ywjl2umsqsa-q48rx3

Out[3]=3

Impute the missing value of an example:

In[4]:=4

✖

https://wolfram.com/xid/0ywjl2umsqsa-u4nvqc

Out[4]=4

Scope (3)Survey of the scope of standard use cases

Train a distribution on a dataset containing numeric and nominal variables:

In[1]:=1

✖

https://wolfram.com/xid/0ywjl2umsqsa-4k23aj

Out[1]=1

Generate a new example based on the learned distribution:

In[5]:=5

✖

https://wolfram.com/xid/0ywjl2umsqsa-632fcq

Out[5]=5

Impute the missing value of an example:

In[7]:=7

✖

https://wolfram.com/xid/0ywjl2umsqsa-7pqpk0

Out[7]=7

Train a distribution on colors:

In[1]:=1

✖

https://wolfram.com/xid/0ywjl2umsqsa-uggcn4

Out[1]=1

Generate 100 new examples based on the learned distribution:

In[2]:=2

✖

https://wolfram.com/xid/0ywjl2umsqsa-hk1n8f

Out[2]=2

Compute the probability density of some colors:

In[3]:=3

✖

https://wolfram.com/xid/0ywjl2umsqsa-w1cs1m

Out[3]=3

Train a distribution on dates:

In[1]:=1

✖

https://wolfram.com/xid/0ywjl2umsqsa-88g0s5

Out[1]=1

Generate 10 new examples based on the learned distribution:

In[2]:=2

✖

https://wolfram.com/xid/0ywjl2umsqsa-pj6kpg

Out[2]=2

Compute the probability density of some new dates:

In[3]:=3

✖

https://wolfram.com/xid/0ywjl2umsqsa-85tleo

Out[3]=3

Options (6)Common values & functionality for each option

FeatureTypes (1)

Specify that the data is nominal:

In[1]:=1

✖

https://wolfram.com/xid/0ywjl2umsqsa-9he65q

Out[1]=1

In[2]:=2

✖

https://wolfram.com/xid/0ywjl2umsqsa-rlcxrs

Out[2]=2

Without specification, the data is considered numerical:

In[3]:=3

✖

https://wolfram.com/xid/0ywjl2umsqsa-165wx1

Out[3]=3

In[4]:=4

✖

https://wolfram.com/xid/0ywjl2umsqsa-ge4l6x

Out[4]=4

Method (2)

Train a "Multinormal" distribution on a numeric dataset:

In[1]:=1

✖

https://wolfram.com/xid/0ywjl2umsqsa-y6t5a

Out[1]=1

Plot the PDF along with the training data:

In[2]:=2

✖

https://wolfram.com/xid/0ywjl2umsqsa-jfmxrj

Out[2]=2

Train a distribution on a two-dimensional dataset with all available methods ("Multinormal", "ContingencyTable", "KernelDensityEstimation", "DecisionTree" and "GaussianMixture"):

In[1]:=1

✖

https://wolfram.com/xid/0ywjl2umsqsa-xaxv0z

In[2]:=2

✖

https://wolfram.com/xid/0ywjl2umsqsa-uro8uq

In[3]:=3

✖

https://wolfram.com/xid/0ywjl2umsqsa-4q6tc5

Visualize the probability density of these distributions:

In[4]:=4

✖

https://wolfram.com/xid/0ywjl2umsqsa-fgbu12

Out[4]=4

TimeGoal (2)

Learn a distribution while specifying a total training time of 5 seconds:

In[1]:=1

✖

https://wolfram.com/xid/0ywjl2umsqsa-idsajs

Out[1]=1

In[2]:=2

✖

https://wolfram.com/xid/0ywjl2umsqsa-c4k8t4

Out[2]=2

Load 1000 images of the "MNIST" dataset:

In[1]:=1

✖

https://wolfram.com/xid/0ywjl2umsqsa-1w6ors

In[2]:=2

✖

https://wolfram.com/xid/0ywjl2umsqsa-qpozuj

Out[2]=2

Learn its distribution while specifying a target training time of 3 seconds:

In[3]:=3

✖

https://wolfram.com/xid/0ywjl2umsqsa-8yeble

Out[3]=3

The loss value obtained (cross-entropy) is about -0.43:

In[4]:=4

✖

https://wolfram.com/xid/0ywjl2umsqsa-zarmrq

Out[4]=4

Learn its distribution while specifying a target training time of 30 seconds:

In[5]:=5

✖

https://wolfram.com/xid/0ywjl2umsqsa-ljkkil

Out[5]=5

The loss value obtained (cross-entropy) is about -0.978:

In[6]:=6

✖

https://wolfram.com/xid/0ywjl2umsqsa-hd7qga

Out[6]=6

Compare the learning curves for both trainings:

In[7]:=7

✖

https://wolfram.com/xid/0ywjl2umsqsa-1xo0an

Out[7]=7

TrainingProgressReporting (1)

Load the "UCILetter" dataset:

In[1]:=1

✖

https://wolfram.com/xid/0ywjl2umsqsa-pac3cx

Show training progress interactively during training:

In[2]:=2

✖

https://wolfram.com/xid/0ywjl2umsqsa-735k7e

Out[2]=2

Show training progress interactively without plots:

In[3]:=3

✖

https://wolfram.com/xid/0ywjl2umsqsa-fy4ogh

Out[3]=3

Print training progress periodically during training:

In[4]:=4

✖

https://wolfram.com/xid/0ywjl2umsqsa-me00oh

Show a simple progress indicator:

In[5]:=5

✖

https://wolfram.com/xid/0ywjl2umsqsa-jyd4wa

Out[5]=5

Do not report progress:

In[6]:=6

✖

https://wolfram.com/xid/0ywjl2umsqsa-uhwsza

Applications (4)Sample problems that can be solved with this function

Obtain a dataset of images:

In[1]:=1

✖

https://wolfram.com/xid/0ywjl2umsqsa-hamat6

In[9]:=9

✖

https://wolfram.com/xid/0ywjl2umsqsa-qzzjzs

Out[9]=9

Train a distribution on the images:

In[2]:=2

✖

https://wolfram.com/xid/0ywjl2umsqsa-mwubth

Out[2]=2

Generate 50 new examples based on the learned distribution:

In[16]:=16

✖

https://wolfram.com/xid/0ywjl2umsqsa-8e6w9u

Out[16]=16

Compare the probability density for an image of the training set, an image of a test set, a sample from the learned distribution, an image of another dataset and a random image:

In[4]:=4