Class: Rumale::SVM::RandomRecursiveSVC

Inherits:
Base::Estimator
  • Object
show all
Includes:
Base::Classifier
Defined in:
lib/rumale/svm/random_recursive_svc.rb

Overview

RandomRecursiveSVC is a class that implements Random Recursive Support Vector Classifier.

Reference

  • Vinyals, O., Jia, Y., Deng, L., and Darrell, T., “Learning with Recursive Perceptual Representations,” In Proc. NIPS’12, pp. 2825–2833, 2012.

Examples:

require 'rumale/svm'

estimator = Rumale::SVM::RandomRecursiveSVC.new(n_hidden_layers: 2, beta: 0.5, random_seed: 1)
estimator.fit(training_samples, training_labels)
results = estimator.predict(testing_samples)

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(n_hidden_layers: 2, beta: 0.5, penalty: 'l2', loss: 'squared_hinge', dual: true, reg_param: 1.0, fit_bias: true, bias_scale: 1.0, tol: 1e-3, verbose: false, random_seed: nil) ⇒ RandomRecursiveSVC

Create a new classifier with Random Recursive Support Vector Machine.

Parameters:

  • n_hidden_layers (Integer) (defaults to: 2) —

    The number of hidden layers.

  • beta (Float) (defaults to: 0.5) —

    The weight parameter for the degree of moving the original data.

  • penalty (String) (defaults to: 'l2') —

    The type of norm used in the penalization (‘l2’ or ‘l1’).

  • loss (String) (defaults to: 'squared_hinge') —

    The type of loss function (‘squared_hinge’ or ‘hinge’). This parameter is ignored if penalty = ‘l1’.

  • dual (Boolean) (defaults to: true) —

    The flag indicating whether to solve dual optimization problem. When n_samples > n_features, dual = false is more preferable. This parameter is ignored if loss = ‘hinge’.

  • reg_param (Float) (defaults to: 1.0) —

    The regularization parameter.

  • fit_bias (Boolean) (defaults to: true) —

    The flag indicating whether to fit the bias term.

  • bias_scale (Float) (defaults to: 1.0) —

    The scale of the bias term. This parameter is ignored if fit_bias = false.

  • tol (Float) (defaults to: 1e-3) —

    The tolerance of termination criterion.

  • verbose (Boolean) (defaults to: false) —

    The flag indicating whether to output learning process message

  • random_seed (Integer/Nil) (defaults to: nil) —

    The seed value using to initialize the random generator.



49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
 
# File 'lib/rumale/svm/random_recursive_svc.rb', line 49

def initialize(n_hidden_layers: 2, beta: 0.5, penalty: 'l2', loss: 'squared_hinge', dual: true, reg_param: 1.0,
               fit_bias: true, bias_scale: 1.0, tol: 1e-3, verbose: false, random_seed: nil)
  super()
  @params = {
    n_hidden_layers: n_hidden_layers,
    beta: beta,
    penalty: penalty == 'l1' ? 'l1' : 'l2',
    loss: loss == 'hinge' ? 'hinge' : 'squared_hinge',
    dual: dual,
    reg_param: reg_param.to_f,
    fit_bias: fit_bias,
    bias_scale: bias_scale.to_f,
    tol: tol.to_f,
    verbose: verbose,
    random_seed: random_seed || Random.rand(4_294_967_295)
  }
  @rng = Random.new(@params[:random_seed])
end

Instance Attribute Details

#classifiers ⇒ Array<LinearSVC> (readonly)

Return the classifiers for each layer.

Returns:



26
27
28
 
# File 'lib/rumale/svm/random_recursive_svc.rb', line 26

def classifiers
  @classifiers
end

#random_matrices ⇒ Array<Numo::DFloat> (readonly)

Return the random matrices for each hidden layer.

Returns:

  • (Array<Numo::DFloat>) —

    (shape: [n_classes, n_features])



30
31
32
 
# File 'lib/rumale/svm/random_recursive_svc.rb', line 30

def random_matrices
  @random_matrices
end

Instance Method Details

#decision_function(x) ⇒ Numo::DFloat

Calculate confidence scores for samples.

Parameters:

  • x (Numo::DFloat) —

    (shape: [n_samples, n_features]) The samples to compute the scores.

Returns:

  • (Numo::DFloat) —

    (shape: [n_samples, n_classes]) Confidence score per sample.



82
83
84
 
# File 'lib/rumale/svm/random_recursive_svc.rb', line 82

def decision_function(x)
  @classifiers.last.decision_function(transform(x))
end

#fit(x, y) ⇒ RandomRecursiveSVC

Fit the model with given training data.

Parameters:

  • x (Numo::DFloat) —

    (shape: [n_samples, n_features]) The training data to be used for fitting the model.

  • y (Numo::Int32) —

    (shape: [n_samples]) The labels to be used for fitting the model.

Returns:



73
74
75
76
 
# File 'lib/rumale/svm/random_recursive_svc.rb', line 73

def fit(x, y)
  partial_fit(x, y)
  self
end

#predict(x) ⇒ Numo::Int32

Predict class labels for samples.

Parameters:

  • x (Numo::DFloat) —

    (shape: [n_samples, n_features]) The samples to predict the labels.

Returns:

  • (Numo::Int32) —

    (shape: [n_samples]) Predicted class label per sample.



90
91
92
 
# File 'lib/rumale/svm/random_recursive_svc.rb', line 90

def predict(x)
  @classifiers.last.predict(transform(x))
end

#transform(x) ⇒ Numo::DFloat

Transform the given data with the learned model.

Parameters:

  • x (Numo::DFloat) —

    (shape: [n_samples, n_features]) The data to be transformed with the learned model.

Returns:

  • (Numo::DFloat) —

    (shape: [n_samples, n_features]) The output of last hidden layer.



98
99
100
101
102
103
104
105
106
107
 
# File 'lib/rumale/svm/random_recursive_svc.rb', line 98

def transform(x)
  d = x
  s = Numo::DFloat.zeros(x.shape)
  @random_matrices.each_with_index do |w, n|
    o = @classifiers[n].predict_proba(d)
    s += o.dot(w)
    d = sigmoid(x + @params[:beta] * s)
  end
  d
end