When did I face the problem?
I am trying to develop a soft voting classifier by VotingClassifier of sklearn. To tune the hyperparameters of each model used in the VotingClassifier, I want to use the hyperopt library. However, I do not find the way to set the hyperparameters in the objective function and thus, I failed to tune the hyperparameters of that classifier. I just know how to do it (using hyperopt) separately for each model (e.g., just for Decision Tree classifier).
Hyperparameters
dic_clf_params = {'DT__max_depth': hp.choice('DT__max_depth', range(2, 20)), # Decision Tree
'DT__criterion': hp.choice('DT__criterion', ['gini', 'entropy']),
'DT__class_weight': hp.choice('DT__class_weight', ['balanced']),
'AB__n_estimators': hp.choice('n_estimators', range(2, 1000)), # Adaptive Boosting
'AB__base_estimator': DecisionTreeClassifier(max_depth=20),
'AB__learning_rate': hp.uniform('learning_rate', 0.2, 1)}
Objective function
def objective_function(params):
clf = model(**params)
f1 = cross_val_score(clf, x_data, y_data.values.tolist(), scoring="f1", cv=10).mean()
return {"loss": -f1, "status": STATUS_OK}
How did I try to solve the problem?
I have checked the official documents (e.g., this document describing the objective function of hyperopt. Also, I searched a lot to find the tutorials and Q&A (in SO) regarding the usage of hyperopt in voting classifier, stacking classifier. However, I failed to find a single tutorial or Q&A.
Note: I want to use the hyperopt since each individual model's hyperparameters were tuned by that library.