NameError: name 'PartialState' is not defined error while training hugging face wave2vec model

Question

Here is the code block which caused the error

training_args = TrainingArguments(
    output_dir="my_awesome_mind_model",
    evaluation_strategy="epoch",
    save_strategy="epoch",
    learning_rate=3e-5,
    per_device_train_batch_size=32,
    gradient_accumulation_steps=4,
    per_device_eval_batch_size=32,
    num_train_epochs=10,
    warmup_ratio=0.1,
    logging_steps=10,
    load_best_model_at_end=True,
    metric_for_best_model="accuracy",
    push_to_hub=True,
)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=dataset["train"],
    # eval_dataset=encoded_minds["test"],
    tokenizer=feature_extractor,
    compute_metrics=compute_metrics,
)

trainer.train()

getting the following error

NameError Traceback (most recent call last) in <cell line: 1>() 1 training_args = TrainingArguments( 2 output_dir="my_awesome_mind_model", 3 evaluation_strategy="epoch", 4 save_strategy="epoch", 5 learning_rate=3e-5,

4 frames /usr/local/lib/python3.10/dist-packages/transformers/training_args.py in _setup_devices(self) 1629 self._n_gpu = 1 1630 else: 1631 self.distributed_state = PartialState(backend=self.ddp_backend) 1632 self._n_gpu = 1 1633 if not is_sagemaker_mp_enabled():

NameError: name 'PartialState' is not defined

I am trying to follow the audio classification guide of hugging face(link on another dataset but upon running the training args code i am getting name "PartialState" not defined error.

Danzig

Answer 1

As of 2023-05-11:

The error seems to be caused by an issue in the huggingface/accelerate library.

You can try following solutions:

Reinstall transformers & accelerate

pip uninstall -y transformers accelerate
pip install transformers accelerate

If you are using colab/Jupyter, make sure to restart the notebook's Runtime.

Install dev version of accelerate

pip install git+https://github.com/huggingface/accelerate

Reverse to previous version of transformers (4.28.0)

# You might also need to uninstall transformers first: pip uninstall -y transformers
pip install transformers==4.28.0

Heteronym

I had a similar problem, just restarting my colab notebook's Runtime solved the problem. – Phoneme

falling back to 4.28.0 worked for me – Doriadorian

Answer 2

pip install --upgrade accelerate and restart your notebook's runtime.

Problem

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. – Obstipation