Failed to rename ; Access denied Tensorflow

Question

I am trying to build a machine learning code with tensorflow 2.0 in jupyter using the mnist dataset . The data set is being taken from tensorflow-datasets but during the initial downloading of the dataset , after it downloads its showing an error that it is unable to rename the dataset and then aborting the whole process.

this is the line used for loading the dataset

mnist_dataset, mnist_info = tfds.load(name='mnist', with_info=True, as_supervised=True)

and this is the error

Downloading and preparing dataset mnist (11.06 MiB) to C:\Users\Main\tensorflow_datasets\mnist\1.0.0...
Dl Completed...:
0/0 [00:00<?, ? url/s]
Dl Size...:
0/0 [00:00<?, ? MiB/s]
Extraction completed...:
0/0 [00:00<?, ? file/s]






WARNING:tensorflow:From C:\Python38\Anaconda3\envs\py3-TF2.0\lib\site-packages\tensorflow_datasets\core\file_format_adapter.py:209: tf_record_iterator (from tensorflow.python.lib.io.tf_record) is deprecated and will be removed in a future version.
Instructions for updating:
Use eager execution and: 
`tf.data.TFRecordDataset(path)`
WARNING:tensorflow:From C:\Python38\Anaconda3\envs\py3-TF2.0\lib\site-packages\tensorflow_datasets\core\file_format_adapter.py:209: tf_record_iterator (from tensorflow.python.lib.io.tf_record) is deprecated and will be removed in a future version.
Instructions for updating:
Use eager execution and: 
`tf.data.TFRecordDataset(path)`
---------------------------------------------------------------------------
UnknownError                              Traceback (most recent call last)
<ipython-input-2-6bf2983938fb> in <module>
----> 1 mnist_dataset, mnist_info = tfds.load(name='mnist', with_info=True, as_supervised=True)

C:\Python38\Anaconda3\envs\py3-TF2.0\lib\site-packages\tensorflow_datasets\core\api_utils.py in disallow_positional_args_dec(fn, instance, args, kwargs)
     50     _check_no_positional(fn, args, ismethod, allowed=allowed)
     51     _check_required(fn, kwargs)
---> 52     return fn(*args, **kwargs)
     53 
     54   return disallow_positional_args_dec(wrapped)  # pylint: disable=no-value-for-parameter

C:\Python38\Anaconda3\envs\py3-TF2.0\lib\site-packages\tensorflow_datasets\core\registered.py in load(name, split, data_dir, batch_size, in_memory, shuffle_files, download, as_supervised, decoders, with_info, builder_kwargs, download_and_prepare_kwargs, as_dataset_kwargs, try_gcs)
    298   if download:
    299     download_and_prepare_kwargs = download_and_prepare_kwargs or {}
--> 300     dbuilder.download_and_prepare(**download_and_prepare_kwargs)
    301 
    302   if as_dataset_kwargs is None:

C:\Python38\Anaconda3\envs\py3-TF2.0\lib\site-packages\tensorflow_datasets\core\api_utils.py in disallow_positional_args_dec(fn, instance, args, kwargs)
     50     _check_no_positional(fn, args, ismethod, allowed=allowed)
     51     _check_required(fn, kwargs)
---> 52     return fn(*args, **kwargs)
     53 
     54   return disallow_positional_args_dec(wrapped)  # pylint: disable=no-value-for-parameter

C:\Python38\Anaconda3\envs\py3-TF2.0\lib\site-packages\tensorflow_datasets\core\dataset_builder.py in download_and_prepare(self, download_dir, download_config)
    305         self.info.size_in_bytes = dl_manager.downloaded_size
    306         # Write DatasetInfo to disk, even if we haven't computed the statistics.
--> 307         self.info.write_to_directory(self._data_dir)
    308     self._log_download_done()
    309 

C:\Python38\Anaconda3\envs\py3-TF2.0\lib\contextlib.py in __exit__(self, type, value, traceback)
    117         if type is None:
    118             try:
--> 119                 next(self.gen)
    120             except StopIteration:
    121                 return False

C:\Python38\Anaconda3\envs\py3-TF2.0\lib\site-packages\tensorflow_datasets\core\file_format_adapter.py in incomplete_dir(dirname)
    198   try:
    199     yield tmp_dir
--> 200     tf.io.gfile.rename(tmp_dir, dirname)
    201   finally:
    202     if tf.io.gfile.exists(tmp_dir):

C:\Python38\Anaconda3\envs\py3-TF2.0\lib\site-packages\tensorflow\python\lib\io\file_io.py in rename_v2(src, dst, overwrite)
    506   """
    507   _pywrap_file_io.RenameFile(
--> 508       compat.as_bytes(src), compat.as_bytes(dst), overwrite)
    509 
    510 

UnknownError: Failed to rename: C:\Users\Main\tensorflow_datasets\mnist\1.0.0.incompleteI3ZU6X to: C:\Users\Main\tensorflow_datasets\mnist\1.0.0 : Access is denied.
; Input/output error

Waterloo

possibly because of the anaconda restrictions. #60309513 – Carmagnole

The problem is solved after running pip install -U tensorflow-datasets in the preferred conda environment . But if someone could point out whether -U in that line represents update or not would rather be more helpful . – Waterloo

-U indicates upgrading "all specified packages to the newest available version. The same problem occurred to me, but pip install -U tensorflow-datasets didn't solve my problem. – Carmagnole

Answer 1

A GitHub issue has been open in order to solve it.

In that thread, a temporal solution is presented, although it implies modifying some TensorFlow functions by hand. I am not sure about whether it will have an impact in model performance, but I will let the link posted below in case you want to give it a try:

Windows - tensorflow.python.framework.errors_impl.UnknownError: Failed to rename: #41380

Consumer

Answer 2

I was facing the same problem. Just change the data_dir attribute like data_dir=r"D:\data" and make sure it should not be in C: drive. You are good to go.

Jovita

Answer 3

Changing disk or data directory didn't work for me (Win10).

However, specifying the version of the dataset worked.

Example: tfds.load(name='celeb_a:2.0.0',... )

Mullin

Answer 4

I had the same error while running locally. I assume the error is due to read and write permissions of the folder in which the data is being downloaded and stored.

I fixed it by specifying the data_dir parameter in the tfds.load() function to a folder where special user permissions won't be required.

here is my function call:

mnist_dataset, mnist_info = tfds.load(name='mnist', data_dir='C:/', with_info=True, as_supervised=True)

this will create a folder for the data in the root directory instead of the default one.

Heddie

Answer 5

Tensorflow/models/MODEL_NAME /

the solution for me was to make sure there is no "checkpoint" in there beforehand, there was a checkpoint folder that is in there for obvious reasons but the script im using(someone elses) they're calling that the ckpt.0 in a different location so i basically moved it to that new location and removed the checkpoint folder from *above path. im an inexperience coder, ESPECIALLY new to python but im currently running the Train Script successfully.

Gibbosity

Answer 6

I just ran into this issue today. I solved it by updating permissions on the destination folder to include read/write privileges. Initially the folder was read-only.

Churchwell

Answer 7

I tried to get the Tensorflow API 2.0 working and got the same error as above. I specified the Model folder the same as the Checkpoint folder. To fix the error mentioned above I had to separate the Model and the Checkpoint folder.

Matrilineage