AttributeError: 'GridSearchCV' object has no attribute 'best_params_'

Question

Grid search is a way to find the best parameters for any model out of the combinations we specify. I have formed a grid search on my model in the below manner and wish to find best parameters identified using this gridsearch.

from sklearn.model_selection import GridSearchCV
# Create the parameter grid based on the results of random search 
param_grid = {
    'bootstrap': [True],'max_depth': [20,30,40, 100, 110],
    'max_features': ['sqrt'],'min_samples_leaf': [5,10,15],
    'min_samples_split': [40,50,60], 'n_estimators': [150, 200, 250]
}
# Create a based model
rf = RandomForestClassifier()
# Instantiate the grid search model
grid_search = GridSearchCV(estimator = rf, param_grid = param_grid, 
                          cv = 3, n_jobs = -1, verbose = 2)

Now I want to find the best parameters of the gridsearch as the output

grid_search.best_params_

Error:

----> grid_search.best_params_
AttributeError: 'GridSearchCV' object has no attribute 'best_params_'

What am I missing?

Towandatoward

Answer 1

You cannot get best parameters without fitting the data.

Fit the data

grid_search.fit(X_train, y_train)

Now find the best parameters.

grid_search.best_params_

grid_search.best_params_ will work after fitting on X_train and y_train.

Towandatoward

Answer 2

If we look into the source code, best_params_ is defined in the fit() method, so it must be called on the GridSearchCV or RandomizedSearchCV object, just as @noob said.

If you got this error even after you fitted the data, then you probably passed multiple scoring metrics. Choose one of them to be the refitting metric and the error will go away. This is because if it's not chosen, it's not clear which metric to use to determine best_params_.

gs = GridSearchCV(
    rf, param_grid, 
    scoring=['precision', 'recall'], refit=False,
    n_jobs=-1)
gs.fit(X_train, y_train)
gs.best_params_                                       # <---- error

gs = GridSearchCV(
    rf, param_grid, 
    scoring=['precision', 'recall'], refit='recall',
#                                    ^^^^^^^^^^^^^^   # <---- choose refit metric
    n_jobs=-1)
gs.fit(X_train, y_train)
gs.best_params_                                       # <---- OK

---

If you arrived here because you got the following error:

AttributeError: 'GridSearchCV' object has no attribute 'best_estimator_'

then, again, because best_estimator_ is defined in the fit() method, check if you fitted the data. If you got the error even after you fitted the data, then refit is again the culprit. To set best_estimator_, refit=True has to hold so that the entire data can be fit again on the estimator (in the example below, Random Forest) using best_params_.

best_ = GridSearchCV(rf, param_grid, refit=False, n_jobs=-1).fit(X_train, y_train).best_estimator_  # <---- error
best_ = GridSearchCV(rf, param_grid, refit=True, n_jobs=-1).fit(X_train, y_train).best_estimator_   # <---- OK
#                                    ^^^^^^^^^^

Dikmen

Answer 3

My problem was that while adapting, it was giving an error because there was line code, markdown, and line by line information.

Certify