Class: Rumale::Ensemble::VotingClassifier

Inherits:
Base::Estimator show all
Includes:
Base::Classifier
Defined in:
rumale-ensemble/lib/rumale/ensemble/voting_classifier.rb

Overview

VotingClassifier is a class that implements classifier with voting ensemble method.

Reference

  • Zhou, Z-H., “Ensemble Methods - Foundations and Algorithms,” CRC Press Taylor and Francis Group, Chapman and Hall/CRC, 2012.

Examples:

require 'rumale/ensemble/voting_classifier'

estimators = {
  lgr: Rumale::LinearModel::LogisticRegression.new(reg_param: 1e-2),
  mlp: Rumale::NeuralNetwork::MLPClassifier.new(hidden_units: [256], random_seed: 1),
  rnd: Rumale::Ensemble::RandomForestClassifier.new(random_seed: 1)
}
weights = { lgr: 0.2, mlp: 0.3, rnd: 0.5 }

classifier = Rumale::Ensemble::VotingClassifier.new(estimators: estimators, weights: weights, voting: 'soft')
classifier.fit(x_train, y_train)
results = classifier.predict(x_test)

Instance Attribute Summary collapse

Attributes inherited from Base::Estimator

#params

Instance Method Summary collapse

Methods included from Base::Classifier

#score

Constructor Details

#initialize(estimators:, weights: nil, voting: 'hard') ⇒ VotingClassifier

Create a new ensembled classifier with voting rule.

Parameters:

  • estimators (Hash<Symbol,Classifier>) —

    The sub-classifiers to vote.

  • weights (Hash<Symbol,Float>) (defaults to: nil) —

    The weight value for each classifier.

  • voting (String) (defaults to: 'hard') —

    The voting rule for the predicted results of each classifier. If ‘hard’ is given, the ensembled classifier predicts the class label by majority vote. If ‘soft’ is given, the ensembled classifier uses the weighted average of predicted probabilities for the prediction.



46
47
48
49
50
51
52
53
 
# File 'rumale-ensemble/lib/rumale/ensemble/voting_classifier.rb', line 46

def initialize(estimators:, weights: nil, voting: 'hard')
  super()
  @estimators = estimators
  @params = {
    weights: weights || estimators.each_key.with_object({}) { |name, w| w[name] = 1.0 },
    voting: voting
  }
end

Instance Attribute Details

#classes ⇒ Numo::Int32 (readonly)

Return the class labels.

Returns:

  • (Numo::Int32) —

    (size: n_classes)



37
38
39
 
# File 'rumale-ensemble/lib/rumale/ensemble/voting_classifier.rb', line 37

def classes
  @classes
end

#estimators ⇒ Hash<Symbol,Classifier> (readonly)

Return the sub-classifiers that voted.

Returns:

  • (Hash<Symbol,Classifier>) 


33
34
35
 
# File 'rumale-ensemble/lib/rumale/ensemble/voting_classifier.rb', line 33

def estimators
  @estimators
end

Instance Method Details

#decision_function(x) ⇒ Numo::DFloat

Calculate confidence scores for samples.

Parameters:

  • x (Numo::DFloat) —

    (shape: [n_samples, n_features]) The samples to compute the scores.

Returns:

  • (Numo::DFloat) —

    (shape: [n_samples, n_classes]) The confidence score per sample.



77
78
79
80
81
82
83
84
85
86
87
88
89
 
# File 'rumale-ensemble/lib/rumale/ensemble/voting_classifier.rb', line 77

def decision_function(x)
  x = ::Rumale::Validation.check_convert_sample_array(x)

  return predict_proba(x) if soft_voting?

  n_samples = x.shape[0]
  n_classes = @classes.size
  z = Numo::DFloat.zeros(n_samples, n_classes)
  @estimators.each do |name, estimator|
    estimator.predict(x).to_a.each_with_index { |c, i| z[i, c] += @params[:weights][name] }
  end
  z
end

#fit(x, y) ⇒ VotingClassifier

Fit the model with given training data.

Parameters:

  • x (Numo::DFloat) —

    (shape: [n_samples, n_features]) The training data to be used for fitting the model.

  • y (Numo::Int32) —

    (shape: [n_samples]) The labels to be used for fitting the model.

Returns:



60
61
62
63
64
65
66
67
68
69
70
71
 
# File 'rumale-ensemble/lib/rumale/ensemble/voting_classifier.rb', line 60

def fit(x, y)
  x = ::Rumale::Validation.check_convert_sample_array(x)
  y = ::Rumale::Validation.check_convert_label_array(y)
  ::Rumale::Validation.check_sample_size(x, y)

  @encoder = ::Rumale::Preprocessing::LabelEncoder.new
  y_encoded = @encoder.fit_transform(y)
  @classes = Numo::NArray[*@encoder.classes]
  @estimators.each_key { |name| @estimators[name].fit(x, y_encoded) }

  self
end

#predict(x) ⇒ Numo::Int32

Predict class labels for samples.

Parameters:

  • x (Numo::DFloat) —

    (shape: [n_samples, n_features]) The samples to predict the labels.

Returns:

  • (Numo::Int32) —

    (shape: [n_samples]) The predicted class label per sample.



95
96
97
98
99
100
101
102
103
 
# File 'rumale-ensemble/lib/rumale/ensemble/voting_classifier.rb', line 95

def predict(x)
  x = ::Rumale::Validation.check_convert_sample_array(x)

  n_samples = x.shape[0]
  n_classes = @classes.size
  z = decision_function(x)
  predicted = z.max_index(axis: 1) - Numo::Int32.new(n_samples).seq * n_classes
  Numo::Int32.cast(@encoder.inverse_transform(predicted))
end

#predict_proba(x) ⇒ Numo::DFloat

Predict probability for samples.

Parameters:

  • x (Numo::DFloat) —

    (shape: [n_samples, n_features]) The samples to predict the probabilities.

Returns:

  • (Numo::DFloat) —

    (shape: [n_samples, n_classes]) Predicted probability of each class per sample.



109
110
111
112
113
114
115
116
117
118
119
120
 
# File 'rumale-ensemble/lib/rumale/ensemble/voting_classifier.rb', line 109

def predict_proba(x)
  x = ::Rumale::Validation.check_convert_sample_array(x)

  n_samples = x.shape[0]
  n_classes = @classes.size
  z = Numo::DFloat.zeros(n_samples, n_classes)
  sum_weight = @params[:weights].each_value.sum
  @estimators.each do |name, estimator|
    z += @params[:weights][name] * estimator.predict_proba(x)
  end
  z /= sum_weight
end