ModuleNotFoundError: No module named 'huggingface_hub.utils' using Anaconda

Question

I'm trying to execute the example code of the huggingface website:

from transformers import GPTJTokenizer, TFGPTJModel
import tensorflow as tf

tokenizer = GPTJTokenizer.from_pretrained("EleutherAI/gpt-j-6B")
model = TFGPTJModel.from_pretrained("EleutherAI/gpt-j-6B")

inputs = tokenizer("Hello, my dog is cute", return_tensors="tf")
outputs = model(inputs)

last_hidden_states = outputs.last_hidden_state

I'm using anaconda and I installed the transformers package beforehand with conda install -c huggingface transformers as explained in the documentation. But I still get this error, when I'm trying to execute the code. Following error message pops up: ModuleNotFoundError: No module named 'huggingface_hub.utils'

How to resolve this error?

Saratov

Answer 1

Please try:

conda install -c conda-forge huggingface_hub

Akiko

Why do you think this would work? Could you add some explanation and if possible a link to a reference? I'm not familiar with conda or with this specific package, but this command seems to install huggingface_hub, which is already correctly installed on the machine of the OP. So I doubt this would work, but maybe this does something "magic", in case an explanation would be really useful. – Legislate

Answer 2

I had the same problem. In my case, it was related to the fact that work with libssl library was messed up in a newer version of a hugging face. Downgrading a little bit resolved the issue for me.

This would downgrade tokenizers to 0.10.3 and transformers to a last version that suitable for the tokenizers.

conda install -c huggingface transformers==4.14.1 tokenizers==0.10.3 -y

In case you afterwards get the error Import Error : cannot import name 'create_repo' from 'huggingface_hub' you should also update your huggingface_hub version by using:

conda install -c huggingface huggingface_hub

Centuple

It now works with tokeniers=0.13.3 but currently it's only available via conda-forge, not huggingface. This may be updated in the future. – Bainmarie