XLNetTokenizer requires the SentencePiece library but it was not found in your environment

Question

I am trying to implement the XLNET on Google Collaboratory. But I get the following issue.

ImportError: 
XLNetTokenizer requires the SentencePiece library but it was not found in your environment. Checkout the instructions on the
installation page of its repo: https://github.com/google/sentencepiece#installation and follow the ones
that match your environment.

I have also tried the following steps:

!pip install -U transformers
!pip install sentencepiece

from transformers import XLNetTokenizer
tokenizer = XLNetTokenizer.from_pretrained('xlnet-base-cased-spiece.model')

Thank you for your help in advance.

Gratulate

Answer 1

After the
!pip install transformers and !pip install sentencepiece
please restart your runtime and then execute all other codes.

Deficient

Yes, it works, but I get this message "Calling XLNetTokenizer.from_pretrained() with the path to a single file or url is deprecated" – Gratulate

Try this --> (1) from transformers import TFXLNetModel, XLNetTokenizer (2) xlnet_model = 'xlnet-large-cased' (3) xlnet_tokenizer = XLNetTokenizer.from_pretrained(xlnet_model) – Deficient

Perfect. It works. Thank you @ Anoop kottappuram – Gratulate

it doesn't work on windows, i did exactly the same – Gragg

I tried pip install simpletransformers instead and it worked for my case. – Goulet

Answer 2

I got the same error in google colab. Restarting the runtime did it for me.

Bipod

Answer 3

Restart the runtime in colab did it for me. like the one mention aboved

Chirm

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. – Chansoo