Loading XGBoost model from pickle file. Error: 'XGBClassifier' object has no attribute 'use_label_encoder'

Question

I am trying to load a serialized xgboost model from a pickle file.

import pickle
def load_pkl(fname):
    with open(fname, 'rb') as f:
        obj = pickle.load(f)
    return obj

model = load_pkl('model_0_unrestricted.pkl')

while printing the model object, I am getting the following error in linux(AWS Sagemaker Notebook)

    ~/anaconda3/envs/python3/lib/python3.6/site-packages/xgboost/sklearn.py in get_params(self, deep)
    436             if k == 'type' and type(self).__name__ != v:
    437                 msg = 'Current model type: {}, '.format(type(self).__name__) + \
--> 438                       'type of model in file: {}'.format(v)
    439                 raise TypeError(msg)
    440             if k == 'type':

~/anaconda3/envs/python3/lib/python3.6/site-packages/sklearn/base.py in get_params(self, deep)
    193         out = dict()
    194         for key in self._get_param_names():
--> 195             value = getattr(self, key)
    196             if deep and hasattr(value, 'get_params'):
    197                 deep_items = value.get_params().items()

AttributeError: 'XGBClassifier' object has no attribute 'use_label_encoder'

Can you please help to fix the issue?

It is working fine in my local mac.

Ref: xgboost:1.4.1 installation log (Mac)

    Collecting xgboost
  Downloading xgboost-1.4.1-py3-none-macosx_10_14_x86_64.macosx_10_15_x86_64.macosx_11_0_x86_64.whl (1.2 MB)
     

But not working on AWS

Ref: xgboost:1.4.1 installation log (SM Notebook, linux machine)

Collecting xgboost
  Using cached xgboost-1.4.1-py3-none-manylinux2010_x86_64.whl (166.7 MB)

Thanks

Jeramey

Answer 1

Looks like you upgraded xgboost. You may consider downgrading to 1.2.0 by:

pip install xgboost==1.2.0

Airedale

I was facing the same problem, it was because of version mismatch. – Zarf

Answer 2

I tried testing on notebook running on ubuntu, it seems to work fine, however can you check how are you initializing your classifier ? This is what I tried :

import numpy as np
import pickle
from scipy.stats import uniform, randint

from sklearn.datasets import load_breast_cancer, load_diabetes, load_wine
from sklearn.metrics import auc, accuracy_score, confusion_matrix, mean_squared_error
from sklearn.model_selection import cross_val_score, GridSearchCV, KFold,RandomizedSearchCV, train_test_split

import xgboost as xgb
cancer = load_breast_cancer()
X = cancer.data
y = cancer.target
xgb_model = xgb.XGBClassifier(objective="binary:logistic", random_state=45)
xgb_model.fit(X, y)
pickle.dump(xgb_model, open("xgb_model.pkl", "wb"))

Load the model back using your function and output it :

def load_pkl(fname):
    with open(fname, 'rb') as f:
        obj = pickle.load(f)
    return obj

model = load_pkl('xgb_model.pkl')
model

Below is the output :

XGBClassifier(base_score=0.5, booster='gbtree', colsample_bylevel=1,
          colsample_bynode=1, colsample_bytree=1, gamma=0, gpu_id=-1,
          importance_type='gain', interaction_constraints='',
          learning_rate=0.300000012, max_delta_step=0, max_depth=6,
          min_child_weight=1, missing=nan, monotone_constraints='()',
          n_estimators=100, n_jobs=8, num_parallel_tree=1, random_state=45,
          reg_alpha=0, reg_lambda=1, scale_pos_weight=1, subsample=1,
          tree_method='exact', validate_parameters=1, verbosity=None)

​

Calcic

Hi @vbhatt, thanks for your answer. As mentioned above, it is working in my local the way you have mentioned above, but the same is not working on AWS. To solve this, I got all the dependencies(requirements.txt) from the DS who created the pickle file and then installed those in a venv before loading the model. Then it started working on AWS too. It proves that there were some dependency-related issues(probably with NumPy) but I am not sure about it. – Jeramey

Answer 3

I suspect the pickled model you are loading in was modified in someway to have that additional method prior to being saved. Either that or as @vbhatt said, you may be modifying some aspect of your classifier prior to loading it in. This has happened to me before when using custom models in Pytorch Lightning.

If you haven't modified the base model at all, please ensure that you are using the same version from within the notebook as well, could be the venv in the notebook has a different version?

Anthocyanin

Thanks @Raed Shabbir for your suggestion. I didn't modify the pickle file. It has been working in my local but not on AWS. The version of Xgboost was also same(1.4.1) but the only difference was the system. xgboost-1.4.1-py3-none-macosx vs xgboost-1.4.1-py3-none-manylinux2010_x86_64. After creating a venv, and then install all dependencies the problem was solved but I am not sure about the root cause. Will do some more debugging in the weekend. Thanks – Jeramey