Class: Rumale::LinearModel::LogisticRegression

Inherits:
BaseEstimator show all
Includes:
Base::Classifier
Defined in:
rumale-linear_model/lib/rumale/linear_model/logistic_regression.rb

Overview

Note:

Rumale::SVM provides Logistic Regression based on LIBLINEAR. If you prefer execution speed, you should use Rumale::SVM::LogisticRegression. github.com/yoshoku/rumale-svm

LogisticRegression is a class that implements (multinomial) Logistic Regression.

Examples:

require 'rumale/linear_model/logistic_regression'

estimator = Rumale::LinearModel::LogisticRegression.new(reg_param: 1.0)
estimator.fit(training_samples, traininig_labels)
results = estimator.predict(testing_samples)

Instance Attribute Summary collapse

Attributes inherited from BaseEstimator

#bias_term, #weight_vec

Attributes inherited from Base::Estimator

#params

Instance Method Summary collapse

Methods included from Base::Classifier

#score

Constructor Details

#initialize(reg_param: 1.0, fit_bias: true, bias_scale: 1.0, max_iter: 1000, tol: 1e-4, n_jobs: nil, verbose: false) ⇒ LogisticRegression

Create a new classifier with Logisitc Regression.

Parameters:

  • reg_param (Float) (defaults to: 1.0) —

    The regularization parameter.

  • fit_bias (Boolean) (defaults to: true) —

    The flag indicating whether to fit the bias term.

  • bias_scale (Float) (defaults to: 1.0) —

    The scale of the bias term. If fit_bias is true, the feature vector v becoms [v; bias_scale].

  • max_iter (Integer) (defaults to: 1000) —

    The maximum number of epochs that indicates how many times the whole data is given to the training process.

  • tol (Float) (defaults to: 1e-4) —

    The tolerance of loss for terminating optimization.

  • n_jobs (Integer) (defaults to: nil) —

    The number of jobs for running the predict methods in parallel. If nil is given, the methods do not execute in parallel. If zero or less is given, it becomes equal to the number of processors. This parameter is ignored if the Parallel gem is not loaded.

  • verbose (Boolean) (defaults to: false) —

    The flag indicating whether to output loss during iteration. ‘iterate.dat’ file is generated by lbfgsb.rb.



48
49
50
51
52
53
54
55
56
57
58
59
 
# File 'rumale-linear_model/lib/rumale/linear_model/logistic_regression.rb', line 48

def initialize(reg_param: 1.0, fit_bias: true, bias_scale: 1.0, max_iter: 1000, tol: 1e-4, n_jobs: nil, verbose: false)
  super()
  @params = {
    reg_param: reg_param,
    fit_bias: fit_bias,
    bias_scale: bias_scale,
    max_iter: max_iter,
    tol: tol,
    n_jobs: n_jobs,
    verbose: verbose
  }
end

Instance Attribute Details

#classes ⇒ Numo::Int32 (readonly)

Return the class labels.

Returns:

  • (Numo::Int32) —

    (shape: [n_classes])



31
32
33
 
# File 'rumale-linear_model/lib/rumale/linear_model/logistic_regression.rb', line 31

def classes
  @classes
end

Instance Method Details

#decision_function(x) ⇒ Numo::DFloat

Calculate confidence scores for samples.

Parameters:

  • x (Numo::DFloat) —

    (shape: [n_samples, n_features]) The samples to compute the scores.

Returns:

  • (Numo::DFloat) —

    (shape: [n_samples, n_classes]) Confidence score per sample.



81
82
83
84
85
 
# File 'rumale-linear_model/lib/rumale/linear_model/logistic_regression.rb', line 81

def decision_function(x)
  x = Rumale::Validation.check_convert_sample_array(x)

  x.dot(@weight_vec.transpose) + @bias_term
end

#fit(x, y) ⇒ LogisticRegression

Fit the model with given training data.

Parameters:

  • x (Numo::DFloat) —

    (shape: [n_samples, n_features]) The training data to be used for fitting the model.

  • y (Numo::Int32) —

    (shape: [n_samples]) The labels to be used for fitting the model.

Returns:



66
67
68
69
70
71
72
73
74
75
 
# File 'rumale-linear_model/lib/rumale/linear_model/logistic_regression.rb', line 66

def fit(x, y)
  x = Rumale::Validation.check_convert_sample_array(x)
  y = Rumale::Validation.check_convert_label_array(y)
  Rumale::Validation.check_sample_size(x, y)

  @classes = Numo::Int32[*y.to_a.uniq.sort]
  @weight_vec, @bias_term = partial_fit(x, y)

  self
end

#predict(x) ⇒ Numo::Int32

Predict class labels for samples.

Parameters:

  • x (Numo::DFloat) —

    (shape: [n_samples, n_features]) The samples to predict the labels.

Returns:

  • (Numo::Int32) —

    (shape: [n_samples]) Predicted class label per sample.



91
92
93
94
95
96
97
98
99
100
101
102
 
# File 'rumale-linear_model/lib/rumale/linear_model/logistic_regression.rb', line 91

def predict(x)
  x = Rumale::Validation.check_convert_sample_array(x)

  n_samples, = x.shape
  decision_values = predict_proba(x)
  predicted = if enable_parallel?
                parallel_map(n_samples) { |n| @classes[decision_values[n, true].max_index] }
              else
                Array.new(n_samples) { |n| @classes[decision_values[n, true].max_index] }
              end
  Numo::Int32.asarray(predicted)
end

#predict_proba(x) ⇒ Numo::DFloat

Predict probability for samples.

Parameters:

  • x (Numo::DFloat) —

    (shape: [n_samples, n_features]) The samples to predict the probailities.

Returns:

  • (Numo::DFloat) —

    (shape: [n_samples, n_classes]) Predicted probability of each class per sample.



108
109
110
111
112
113
114
115
116
117
118
119
 
# File 'rumale-linear_model/lib/rumale/linear_model/logistic_regression.rb', line 108

def predict_proba(x)
  x = Rumale::Validation.check_convert_sample_array(x)

  proba = 1.0 / (Numo::NMath.exp(-decision_function(x)) + 1.0)
  return (proba.transpose / proba.sum(axis: 1)).transpose.dup if multiclass_problem?

  n_samples, = x.shape
  probs = Numo::DFloat.zeros(n_samples, 2)
  probs[true, 1] = proba
  probs[true, 0] = 1.0 - proba
  probs
end