"LLE" (Machine Learning Method)

Details & Suboptions

  • "LLE", which stands for locally linear embedding, is a nonlinear neighborhood-preserving dimensionality reduction method.
  • "LLE" is able to learn nonlinear manifolds; however, it can fail if data has high-density variations and tends to collapse large portions of the data close together.
  • The following plots (see FeatureSpacePlot) show two-dimensional embeddings learned by the "LLE" method applied to the benchmark datasets Fisher's Irises, MNIST and FashionMNIST:
  • "LLE" seeks to find a low-dimensional embedding that locally preserves the intrinsic geometry (such as angles and relative distances) of the data. To do so, "LLE" first defines the neighborhood of each data point by its nearest neighbors. Then, it computes the optimal "reconstruction weights" Wij by minimizing the reconstruction error (xj is a neighbor of xi): ∑ Ni=1|xi-∑jWijxj|2, subject to the constraint ∑jWij=1 (this constraint enforces the invariance of Wij to rotations, rescalings and translations of the data). Once Wij are computed, the lower-dimensional embeddings are computed by minimizing the embedding cost: ∑ Ni=1|yi-∑jWijyj|2.
  • "LLE" attempts to have the same reconstruction weights in the original and embedding space, which is why the local geometric properties of the data points are approximately preserved.
  • The following suboption can be given:
  • "NeighborsNumber" Automaticnumber of nearest neighbors

Examples

open allclose all

Basic Examples  (1)

Create and visualize a "Swiss roll" dataset:

Train a nonlinear dimension reducer using "LLE" on the dataset to map to two-dimensional space:

Find and visualize the data coordinates in the reduced space:

Visualize the dataset in the original space, with each point colored according to its reduced variable:

Scope  (1)

Dataset Visualization  (1)

Load the Fisher Iris dataset from ExampleData and perform a train/test split:

Generate a reducer function using "LLE" with the features of each training example:

Group test examples by their species:

Reduce the dimension of the features:

Visualize the reduced dataset:

Options  (1)

"NeighborsNumber"  (1)

Generate a dataset of different head poses from 3D geometry data with random viewpoints:

Visualize different head poses:

Reduce the dataset to a two-dimensional representation by specifying the "NeighborsNumber" in the neighborhood graph for performing the locally linear embedding:

Visualize the original images in the reduced space, in which the up-down and front-side poses are disentangled: