Class: Rumale::SVM::LocallyLinearSVC

Inherits:
Base::Estimator
  • Object
show all
Includes:
Base::Classifier
Defined in:
lib/rumale/svm/locally_linear_svc.rb

Overview

LocallyLinearSVC is a class that implements Locally Linear Support Vector Classifier with the squared hinge loss. This classifier requires Numo::Linalg (or Numo::TinyLinalg) and Lbfgsb gems, but they are listed in the runtime dependencies of Rumale::SVM. Therefore, you should install and load Numo::Linalg and Lbfgsb gems explicitly to use this classifier.

Reference

  • Ladicky, L., and Torr, P H.S., “Locally Linear Support Vector Machines,” Proc. ICML’11, pp. 985–992, 2011.

Examples:

require 'numo/linalg/autoloader'
require 'lbfgsb'
require 'rumale/svm'

estimator = Rumale::SVM::LocallyLinearSVC.new(reg_param: 1.0, n_anchors: 128)
estimator.fit(training_samples, traininig_labels)
results = estimator.predict(testing_samples)

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(reg_param: 1.0, reg_param_local: 1e-4, max_iter: 100, tol: 1e-4, n_anchors: 128, n_neighbors: 10, fit_bias: true, bias_scale: 1.0, random_seed: nil) ⇒ LocallyLinearSVC

Create a new classifier with Locally Linear Support Vector Machine.

Parameters:

  • reg_param (Float) (defaults to: 1.0) —

    The regularization parameter for weight vector.

  • reg_param_local (Float) (defaults to: 1e-4) —

    The regularization parameter for local coordinate.

  • max_iter (Integer) (defaults to: 100) —

    The maximum number of iterations.

  • tol (Float) (defaults to: 1e-4) —

    The tolerance of termination criterion for finding anchors with k-means algorithm.

  • n_anchors (Integer) (defaults to: 128) —

    The number of anchors.

  • n_neighbors (Integer) (defaults to: 10) —

    The number of neighbors.

  • fit_bias (Boolean) (defaults to: true) —

    The flag indicating whether to fit bias term.

  • bias_scale (Float) (defaults to: 1.0) —

    The scale parameter for bias term.

  • random_seed (Integer) (defaults to: nil) —

    The seed value using to initialize the random generator.



57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
 
# File 'lib/rumale/svm/locally_linear_svc.rb', line 57

def initialize(reg_param: 1.0, reg_param_local: 1e-4, max_iter: 100, tol: 1e-4,
               n_anchors: 128, n_neighbors: 10, fit_bias: true, bias_scale: 1.0, random_seed: nil)
  raise 'LocallyLinearSVC requires Numo::Linalg but that is not loaded' unless enable_linalg?(warning: false)

  super()
  @params = {
    reg_param: reg_param,
    reg_param_local: reg_param_local,
    max_iter: max_iter,
    n_anchors: n_anchors,
    tol: tol,
    n_neighbors: n_neighbors,
    fit_bias: fit_bias,
    bias_scale: bias_scale,
    random_seed: random_seed || srand
  }
  @rng = Random.new(@params[:random_seed])
end

Instance Attribute Details

#anchors ⇒ Numo::DFloat (readonly)

Return the anchor vectors.

Returns:

  • (Numo::DFloat) —

    (shape: [n_anchors, n_features])



36
37
38
 
# File 'lib/rumale/svm/locally_linear_svc.rb', line 36

def anchors
  @anchors
end

#bias_term ⇒ Numo::DFloat (readonly)

Return the bias term (a.k.a. intercept).

Returns:

  • (Numo::DFloat) —

    (shape: [n_classes, n_anchors])



44
45
46
 
# File 'lib/rumale/svm/locally_linear_svc.rb', line 44

def bias_term
  @bias_term
end

#classes ⇒ Numo::Int32 (readonly)

Return the class labels.

Returns:

  • (Numo::Int32) —

    (size: n_classes)



32
33
34
 
# File 'lib/rumale/svm/locally_linear_svc.rb', line 32

def classes
  @classes
end

#weight_vec ⇒ Numo::DFloat (readonly)

Return the weight vector.

Returns:

  • (Numo::DFloat) —

    (shape: [n_classes, n_anchors, n_features])



40
41
42
 
# File 'lib/rumale/svm/locally_linear_svc.rb', line 40

def weight_vec
  @weight_vec
end

Instance Method Details

#decision_function(x) ⇒ Numo::DFloat

Calculate confidence scores for samples.

Parameters:

  • x (Numo::DFloat) —

    (shape: [n_samples, n_features]) The samples to compute the scores.

Returns:

  • (Numo::DFloat) —

    (shape: [n_samples, n_classes]) Confidence score per sample.



122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
 
# File 'lib/rumale/svm/locally_linear_svc.rb', line 122

def decision_function(x)
  x = Rumale::Validation.check_convert_sample_array(x)
  n_samples = x.shape[0]

  if multiclass_problem?
    n_classes = @classes.size
    df = Numo::DFloat.zeros(n_samples, n_classes)
    n_samples.times do |i|
      xi = x[i, true]
      coeff = local_coordinates(xi)
      n_classes.times do |j|
        df[i, j] = coeff.dot(@weight_vec[j, true, true]).dot(xi) + coeff.dot(@bias_term[j, true])
      end
    end
  else
    df = Numo::DFloat.zeros(n_samples)
    n_samples.times do |i|
      xi = x[i, true]
      coeff = local_coordinates(xi)
      df[i] = coeff.dot(@weight_vec).dot(xi) + coeff.dot(@bias_term)
    end
  end
  df
end

#fit(x, y) ⇒ LocallyLinearSVC

Fit the model with given training data.

Parameters:

  • x (Numo::DFloat) —

    (shape: [n_samples, n_features]) The training data to be used for fitting the model.

  • y (Numo::Int32) —

    (shape: [n_samples]) The labels to be used for fitting the model.

Returns:



81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
 
# File 'lib/rumale/svm/locally_linear_svc.rb', line 81

def fit(x, y)
  x = Rumale::Validation.check_convert_sample_array(x)
  y = Rumale::Validation.check_convert_label_array(y)
  Rumale::Validation.check_sample_size(x, y)
  raise 'LocallyLinearSVC#fit requires Lbfgsb but that is not loaded' unless defined?(Lbfgsb)

  @classes = Numo::Int32[*y.to_a.uniq.sort]

  find_anchors(x)
  n_samples, n_features = x.shape
  @coeff = Numo::DFloat.zeros(n_samples, @params[:n_anchors])
  n_samples.times do |i|
    xi = x[i, true]
    @coeff[i, true] = local_coordinates(xi)
  end

  x = expand_feature(x) if fit_bias?

  if multiclass_problem?
    n_classes = @classes.size
    @weight_vec = Numo::DFloat.zeros(n_classes, @params[:n_anchors], n_features)
    @bias_term = Numo::DFloat.zeros(n_classes, @params[:n_anchors])
    n_classes.times do |n|
      bin_y = Numo::Int32.cast(y.eq(@classes[n])) * 2 - 1
      w, b = partial_fit(x, bin_y)
      @weight_vec[n, true, true] = w
      @bias_term[n, true] = b
    end
  else
    negative_label = @classes[0]
    bin_y = Numo::Int32.cast(y.ne(negative_label)) * 2 - 1
    @weight_vec, @bias_term = partial_fit(x, bin_y)
  end

  self
end

#predict(x) ⇒ Numo::Int32

Predict class labels for samples.

Parameters:

  • x (Numo::DFloat) —

    (shape: [n_samples, n_features]) The samples to predict the labels.

Returns:

  • (Numo::Int32) —

    (shape: [n_samples]) Predicted class label per sample.



151
152
153
154
155
156
157
158
159
160
161
162
163
 
# File 'lib/rumale/svm/locally_linear_svc.rb', line 151

def predict(x)
  x = Rumale::Validation.check_convert_sample_array(x)
  n_samples = x.shape[0]

  if multiclass_problem?
    df = decision_function(x)
    predicted = Array.new(n_samples) { |n| @classes[df[n, true].max_index] }
  else
    df = decision_function(x).ge(0.0).to_a
    predicted = Array.new(n_samples) { |n| @classes[df[n]] }
  end
  Numo::Int32.asarray(predicted)
end