Class: Rumale::Manifold::HessianEigenmaps

Inherits:
Base::Estimator show all
Includes:
Base::Transformer
Defined in:
rumale-manifold/lib/rumale/manifold/hessian_eigenmaps.rb

Overview

HessianEigenmaps is a class that implements Hessian Eigenmaps.

Reference

  • Donoho, D. L., and Grimes, C., “Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data,” Proc. Natl. Acad. Sci. USA, vol. 100, no. 10, pp. 5591–5596, 2003.

Examples:

require 'numo/linalg/autoloader'
require 'rumale/manifold/hessian_eigenmaps'

hem = Rumale::Manifold::HessianEigenmaps.new(n_components: 2, n_neighbors: 15)
z = hem.fit_transform(x)

Instance Attribute Summary collapse

Attributes inherited from Base::Estimator

#params

Instance Method Summary collapse

Constructor Details

#initialize(n_neighbors: 5, n_components: 2, reg_param: 1e-6) ⇒ HessianEigenmaps

Create a new transformer with Hessian Eigenmaps.

Parameters:

  • n_components (Integer) (defaults to: 2) —

    The number of dimensions on representation space.

  • n_neighbors (Integer) (defaults to: 5) —

    The number of nearest neighbors for k-nearest neighbor graph construction.

  • reg_param (Float) (defaults to: 1e-6) —

    The reguralization parameter for local gram matrix in transform method.



33
34
35
36
37
38
39
40
 
# File 'rumale-manifold/lib/rumale/manifold/hessian_eigenmaps.rb', line 33

def initialize(n_neighbors: 5, n_components: 2, reg_param: 1e-6)
  super()
  @params = {
    n_neighbors: n_neighbors,
    n_components: n_components,
    reg_param: reg_param
  }
end

Instance Attribute Details

#embedding ⇒ Numo::DFloat (readonly)

Return the data in representation space.

Returns:

  • (Numo::DFloat) —

    (shape: [n_samples, n_components])



26
27
28
 
# File 'rumale-manifold/lib/rumale/manifold/hessian_eigenmaps.rb', line 26

def embedding
  @embedding
end

Instance Method Details

#fit(x) ⇒ HessianEigenmaps

Fit the model with given training data.

Returns The learned transformer itself.

Parameters:

  • x (Numo::DFloat) —

    (shape: [n_samples, n_features]) The training data to be used for fitting the model.

Returns:



47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
 
# File 'rumale-manifold/lib/rumale/manifold/hessian_eigenmaps.rb', line 47

def fit(x, _y = nil) # rubocop:disable Metrics/AbcSize
  raise 'HessianEigenmaps#fit requires Numo::Linalg but that is not loaded' unless enable_linalg?(warning: false)

  x = Rumale::Validation.check_convert_sample_array(x)

  n_samples = x.shape[0]
  distance_mat = Rumale::PairwiseMetric.squared_error(x)
  neighbor_ids = neighbor_ids(distance_mat, @params[:n_neighbors], true)

  tri_n_components = @params[:n_components] * (@params[:n_components] + 1) / 2
  hessian_mat = Numo::DFloat.zeros(n_samples * tri_n_components, n_samples)
  ones = Numo::DFloat.ones(@params[:n_neighbors], 1)
  n_samples.times do |i|
    tan_coords = tangent_coordinates(x[neighbor_ids[i, true], true])
    xi = Numo::DFloat.zeros(@params[:n_neighbors], tri_n_components)
    @params[:n_components].times do |m|
      offset = Array.new(m + 1) { |v| v }.sum
      (@params[:n_components] - m).times do |n|
        xi[true, m * @params[:n_components] - offset + n] = tan_coords[true, m] * tan_coords[true, m + n]
      end
    end

    xt, = Numo::Linalg.qr(Numo::DFloat.hstack([ones, tan_coords, xi]))
    pii = xt[true, (@params[:n_components] + 1)..-1]
    tri_n_components.times do |j|
      pj_sum = pii[true, j].sum
      normalizer = pj_sum <= 1e-8 ? 1 : 1.fdiv(pj_sum)
      hessian_mat[i * tri_n_components + j, neighbor_ids[i, true]] = pii[true, j] * normalizer
    end
  end

  kernel_mat = hessian_mat.transpose.dot(hessian_mat)
  _, eig_vecs = Numo::Linalg.eigh(kernel_mat, vals_range: 1...(1 + @params[:n_components]))

  @embedding = @params[:n_components] == 1 ? eig_vecs[true, 0].dup : eig_vecs.dup
  @x_train = x.dup

  self
end

#fit_transform(x) ⇒ Numo::DFloat

Fit the model with training data, and then transform them with the learned model.

Returns (shape: [n_samples, n_components]) The transformed data.

Parameters:

  • x (Numo::DFloat) —

    (shape: [n_samples, n_features]) The training data to be used for fitting the model.

Returns:

  • (Numo::DFloat) —

    (shape: [n_samples, n_components]) The transformed data



92
93
94
95
96
97
98
99
100
 
# File 'rumale-manifold/lib/rumale/manifold/hessian_eigenmaps.rb', line 92

def fit_transform(x, _y = nil)
  unless enable_linalg?(warning: false)
    raise 'HessianEigenmaps#fit_transform requires Numo::Linalg but that is not loaded'
  end

  fit(x)

  @embedding.dup
end

#transform(x) ⇒ Numo::DFloat

Transform the given data with the learned model. For out-of-sample data embedding, the same method as Locally Linear Embedding is used.

Parameters:

  • x (Numo::DFloat) —

    (shape: [n_samples, n_features]) The data to be transformed with the learned model.

Returns:

  • (Numo::DFloat) —

    (shape: [n_samples, n_components]) The transformed data.



107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
 
# File 'rumale-manifold/lib/rumale/manifold/hessian_eigenmaps.rb', line 107

def transform(x)
  x = Rumale::Validation.check_convert_sample_array(x)

  n_samples = x.shape[0]
  tol = @params[:reg_param].fdiv(@params[:n_neighbors])
  distance_mat = Rumale::PairwiseMetric.squared_error(x, @x_train)
  neighbor_ids = neighbor_ids(distance_mat, @params[:n_neighbors], false)
  weight_mat = Numo::DFloat.zeros(n_samples, @x_train.shape[0])

  n_samples.times do |n|
    x_local = @x_train[neighbor_ids[n, true], true] - x[n, true]
    gram_mat = x_local.dot(x_local.transpose)
    gram_mat += tol * weight_mat.trace * Numo::DFloat.eye(@params[:n_neighbors])
    weights = Numo::Linalg.solve(gram_mat, Numo::DFloat.ones(@params[:n_neighbors]))
    weights /= weights.sum + 1e-8
    weight_mat[n, neighbor_ids[n, true]] = weights
  end

  weight_mat.dot(@embedding)
end