WOLFRAM

Wolfram Language & System Documentation Center
Unsupervised Learning with Neural Networks

Unsupervised Learning with Neural Networks

Manifold LearningMNIST Digit Autoencoder
Manifold Learning
Create a synthetic two-dimensional manifold:
Create a net with a "bottleneck" layer in order to learn a parameterization of the manifold:
Create a loss network that computes a loss based on the "reconstruction error" on the input manifold:
Train the loss network on the manifold and extract the original net from the loss network:
Visualize how the net projects arbitrary points onto the manifold:
Split the net into "encoder" and "decoder" networks (the encoder parameterizes points using a single scalar value, whereas the decoder reconstructs the point from this parameterization):
Color each point in the original manifold by its parameterization under the encoder:
Obtain the range of the parameterization by applying the encoder to the manifold:
Show the reconstruction over this range along with the original manifold:
MNIST Digit Autoencoder
First, obtain the training data and select images corresponding to digits 0 through 4:
An autoencoder has the same shape for its input and output, and has a bottleneck in the middle of the net to prevent the net simply memorizing the inputs.
Create a net that takes an input with dimensions {1,28,28} and returns an output with the same dimensions {1,28,28}:
The preceding net takes as input 784 real numbers and compresses this to a vector of 40 real numbers (the bottleneck). The net then needs to reconstruct the original image from these 40 real numbers.
Create an encoder that converts images to arrays of dimension {1,28,28}:
Attach a loss to the autoencoder net that penalizes the input being different from the output:
Train the network on the reconstruction error:
Obtain a subnetwork that performs only reconstruction:
Reconstruct some sample images:
Obtain a subnetwork that produces the code vector:
Compute codes for all of the test images:
Project the code vectors to three dimensions and visualize them along with the original labels (not seen by the network). The digit classes tend to cluster together:
Perform automatic clustering directly on the code vectors and show a sample taken from each cluster:
Visualize a hierarchical clustering of random representatives from each class: