Class: Rumale::Manifold::LocallyLinearEmbedding

Inherits:
Base::Estimator show all
Includes:
Base::Transformer
Defined in:
rumale-manifold/lib/rumale/manifold/locally_linear_embedding.rb

Overview

LocallyLinearEmbedding is a class that implements Locally Linear Embedding.

Reference

  • Roweis, S., and Saul, L., “Nonlinear Dimensionality Reduction by Locally Linear Embedding,” J. of Science, vol. 290, pp. 2323-2326, 2000.

Examples:

require 'numo/linalg/autoloader'
require 'rumale/manifold/locally_linear_embedding'

lem = Rumale::Manifold::LocallyLinearEmbedding.new(n_components: 2, n_neighbors: 15)
z = lem.fit_transform(x)

Instance Attribute Summary collapse

Attributes inherited from Base::Estimator

#params

Instance Method Summary collapse

Constructor Details

#initialize(n_components: 2, n_neighbors: 10, reg_param: 1e-3) ⇒ LocallyLinearEmbedding

Create a new transformer with Locally Linear Embedding.

Parameters:

  • n_components (Integer) (defaults to: 2) —

    The number of dimensions on representation space.

  • n_neighbors (Integer) (defaults to: 10) —

    The number of nearest neighbors for k-nearest neighbor graph construction.

  • reg_param (Float) (defaults to: 1e-3) —

    The reguralization parameter for local gram matrix.



33
34
35
36
37
38
39
40
 
# File 'rumale-manifold/lib/rumale/manifold/locally_linear_embedding.rb', line 33

def initialize(n_components: 2, n_neighbors: 10, reg_param: 1e-3)
  super()
  @params = {
    n_components: n_components,
    n_neighbors: [1, n_neighbors].max,
    reg_param: reg_param
  }
end

Instance Attribute Details

#embedding ⇒ Numo::DFloat (readonly)

Return the data in representation space.

Returns:

  • (Numo::DFloat) —

    (shape: [n_samples, n_components])



26
27
28
 
# File 'rumale-manifold/lib/rumale/manifold/locally_linear_embedding.rb', line 26

def embedding
  @embedding
end

Instance Method Details

#fit(x) ⇒ LocallyLinearEmbedding

Fit the model with given training data.

Returns The learned transformer itself.

Parameters:

  • x (Numo::DFloat) —

    (shape: [n_samples, n_features]) The training data to be used for fitting the model.

Returns:



47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
 
# File 'rumale-manifold/lib/rumale/manifold/locally_linear_embedding.rb', line 47

def fit(x, _y = nil)
  raise 'LocallyLinearEmbedding#fit requires Numo::Linalg but that is not loaded' unless enable_linalg?(warning: false)

  x = Rumale::Validation.check_convert_sample_array(x)

  n_samples = x.shape[0]
  tol = @params[:reg_param].fdiv(@params[:n_neighbors])
  distance_mat = Rumale::PairwiseMetric.squared_error(x)
  neighbor_ids = neighbor_ids(distance_mat, @params[:n_neighbors], true)

  affinity_mat = Numo::DFloat.eye(n_samples)
  n_samples.times do |n|
    x_local = x[neighbor_ids[n, true], true] - x[n, true]
    gram_mat = x_local.dot(x_local.transpose)
    gram_mat += tol * gram_mat.trace * Numo::DFloat.eye(@params[:n_neighbors])
    weights = Numo::Linalg.solve(gram_mat, Numo::DFloat.ones(@params[:n_neighbors]))
    weights /= weights.sum + 1e-8
    affinity_mat[n, neighbor_ids[n, true]] -= weights
  end

  kernel_mat = affinity_mat.transpose.dot(affinity_mat)
  _, eig_vecs = Numo::Linalg.eigh(kernel_mat, vals_range: 1...(1 + @params[:n_components]))

  @embedding = @params[:n_components] == 1 ? eig_vecs[true, 0].dup : eig_vecs.dup
  @x_train = x.dup

  self
end

#fit_transform(x) ⇒ Numo::DFloat

Fit the model with training data, and then transform them with the learned model.

Returns (shape: [n_samples, n_components]) The transformed data.

Parameters:

  • x (Numo::DFloat) —

    (shape: [n_samples, n_features]) The training data to be used for fitting the model.

Returns:

  • (Numo::DFloat) —

    (shape: [n_samples, n_components]) The transformed data



81
82
83
84
85
86
87
 
# File 'rumale-manifold/lib/rumale/manifold/locally_linear_embedding.rb', line 81

def fit_transform(x, _y = nil)
  unless enable_linalg?(warning: false)
    raise 'LocallyLinearEmbedding#fit_transform requires Numo::Linalg but that is not loaded'
  end

  fit(x).transform(x)
end

#transform(x) ⇒ Numo::DFloat

Transform the given data with the learned model.

Parameters:

  • x (Numo::DFloat) —

    (shape: [n_samples, n_features]) The data to be transformed with the learned model.

Returns:

  • (Numo::DFloat) —

    (shape: [n_samples, n_components]) The transformed data.



93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
 
# File 'rumale-manifold/lib/rumale/manifold/locally_linear_embedding.rb', line 93

def transform(x)
  x = Rumale::Validation.check_convert_sample_array(x)

  n_samples = x.shape[0]
  tol = @params[:reg_param].fdiv(@params[:n_neighbors])
  distance_mat = Rumale::PairwiseMetric.squared_error(x, @x_train)
  neighbor_ids = neighbor_ids(distance_mat, @params[:n_neighbors], false)
  weight_mat = Numo::DFloat.zeros(n_samples, @x_train.shape[0])

  n_samples.times do |n|
    x_local = @x_train[neighbor_ids[n, true], true] - x[n, true]
    gram_mat = x_local.dot(x_local.transpose)
    gram_mat += tol * weight_mat.trace * Numo::DFloat.eye(@params[:n_neighbors])
    weights = Numo::Linalg.solve(gram_mat, Numo::DFloat.ones(@params[:n_neighbors]))
    weights /= weights.sum + 1e-8
    weight_mat[n, neighbor_ids[n, true]] = weights
  end

  weight_mat.dot(@embedding)
end