Why do I get "Pickle - EOFError: Ran out of input" reading an empty file?

Question

I am getting an interesting error while trying to use Unpickler.load(), here is the source code:

open(target, 'a').close()
scores = {};
with open(target, "rb") as file:
    unpickler = pickle.Unpickler(file);
    scores = unpickler.load();
    if not isinstance(scores, dict):
        scores = {};

Here is the traceback:

Traceback (most recent call last):
File "G:\python\pendu\user_test.py", line 3, in <module>:
    save_user_points("Magix", 30);
File "G:\python\pendu\user.py", line 22, in save_user_points:
    scores = unpickler.load();
EOFError: Ran out of input

The file I am trying to read is empty. How can I avoid getting this error, and get an empty variable instead?

Simplehearted

Don't close the file – Pisa

The first line open(...).close() is here to ensure the file exists – Simplehearted

Why not just do os.path.isfile(target)? – Epaminondas

Comments that start with "why not just" really miss the point... people ask questions because they don't know. You should elaborate. – Immaterialism

Answer 1

I would check that the file is not empty first:

import os

scores = {} # scores is an empty dict already

if os.path.getsize(target) > 0:      
    with open(target, "rb") as f:
        unpickler = pickle.Unpickler(f)
        # if file is not empty scores will be equal
        # to the value unpickled
        scores = unpickler.load()

Also open(target, 'a').close() is doing nothing in your code and you don't need to use ;.

Stephainestephan

open(target, 'a').close() is here to make sure the file exists ;-) + I don't need to use ; but I just came from C, and not using ; at the end of my lines make my cry T.T – Simplehearted

ok, but the issinstance is unnecessary as I imagine you are only going to be pickling a dict, checking for an empty file will suffice – Stephainestephan

furthermore, checking the file is not empty will not always mean I can unpickle it... raising an exception... That's why I don't think your answer is not the best, even thought it is not bad. – Simplehearted

catching an EOF exception won't save you from all the other potential errors. – Stephainestephan

You are right, that's why I will also check for other errors... catching EOFError only is good here, but catching all other error types are better, and this is more clever to do it this way, as I am sure the program will not be stopped by a dictionnary-reading function which is not that important in the program. – Simplehearted

you can check if a file exists using the os module also, might be better than opening and closing a file every time . – Stephainestephan

Answer 2

Most of the answers here have dealt with how to mange EOFError exceptions, which is really handy if you're unsure about whether the pickled object is empty or not.

However, if you're surprised that the pickle file is empty, it could be because you opened the filename through 'wb' or some other mode that could have over-written the file.

for example:

filename = 'cd.pkl'
with open(filename, 'wb') as f:
    classification_dict = pickle.load(f)

This will over-write the pickled file. You might have done this by mistake before using:

...
open(filename, 'rb') as f:

And then got the EOFError because the previous block of code over-wrote the cd.pkl file.

When working in Jupyter, or in the console (Spyder) I usually write a wrapper over the reading/writing code, and call the wrapper subsequently. This avoids common read-write mistakes, and saves a bit of time if you're going to be reading the same file multiple times through your travails

Extravagancy

file locking necessity - This answer would help many people, I was trying to read the file while it was open for writing. – Interweave

Man went through the same thing today, opened a pickle with 'wb' when I actually intended to read it ;( Is there anyway to go back? – Obvolute

Thanks, this was exactly my issue! copy-pasted some old code and ended up emptying my pickle file since it was set to wb instead of rb – Radiance

People who have this problem should probably be directed to one of the canonicals about file modes instead. – Principality

Answer 3

It is very likely that the pickled file is empty.

It is surprisingly easy to overwrite a pickle file if you're copying and pasting code.

For example the following writes a pickle file:

pickle.dump(df,open('df.p','wb'))

And if you copied this code to reopen it, but forgot to change 'wb' to 'rb' then you would overwrite the file:

df=pickle.load(open('df.p','wb'))

The correct syntax is

df=pickle.load(open('df.p','rb'))

Insufficient

The last two code examples should be swapped, right? – Zinciferous

Yes, I made the same mistake and all the results are ruined which had to rerun all the previous calculations and wait for one day to get the results. What a pity! – Granulocyte

IT doesn't work on python 3.5 – Burkhardt

@ShadabK I'm on 3.7.X and it works fine. – Insufficient

looks like this code examples leaves files opened which can be bad in some cases – Overnight

Answer 4

As you see, that's actually a natural error ..

A typical construct for reading from an Unpickler object would be like this ..

try:
    data = unpickler.load()
except EOFError:
    data = list()  # or whatever you want

EOFError is simply raised, because it was reading an empty file, it just meant End of File ..

Helotism

Answer 5

Had the same issue. It turns out when I was writing to my pickle file I had not used the file.close(). Inserted that line in and the error was no more.

Graphy

This does not really answer the question. If you have a different question, you can ask it by clicking Ask Question. To get notified when this question gets new answers, you can follow this question. Once you have enough reputation, you can also add a bounty to draw more attention to this question. - From Review – Skite

I liked this answer. I had the same error and was scrambling to figure out how this question applied to my situation. It didn't, but this answer did. So useful if only to put me on the right track. – Negate

Answer 6

I have encountered this error many times and it always occurs because after writing into the file, I didn't close it. If we don't close the file the content stays in the buffer and the file stays empty. To save the content into the file, either file should be closed or file_object should go out of scope.

That's why at the time of loading it's giving the ran out of input error because the file is empty. So you have two options :

1. file_object.close()
2. file_object.flush(): if you don't wanna close your file in between the program, you can use the flush() function as it will forcefully move the content from the buffer to the file.

Expropriate

also in my case I see from a jupyter notebook execution that, if size of object serialised is too big for memory it raises the same error. – Floor

The with open syntax should prevent this, correct? – Bonnibelle

Answer 7

You can catch that exception and return whatever you want from there.

open(target, 'a').close()
scores = {};
try:
    with open(target, "rb") as file:
        unpickler = pickle.Unpickler(file);
        scores = unpickler.load();
        if not isinstance(scores, dict):
            scores = {};
except EOFError:
    return {}

Oram

The problem with this is that it's going to silently hide corrupted files. – Sententious

Answer 8

if path.exists(Score_file):
      try : 
         with open(Score_file , "rb") as prev_Scr:

            return Unpickler(prev_Scr).load()

    except EOFError : 

        return dict() 

Endocentric

Hellow and welcome to Stackoverflow. Can you explain this code a bit please? – Bilharziasis

Answer 9

This error comes when your pickle file is empty (0 Bytes). You need to check the size of your pickle file first. This was the scenario in my case. Hope this helps!

Handkerchief

In my case, the file was empty because... it had not been correctly downloaded earlier: the status code of the response was ERROR 403: Forbidden. It is nice to know that the issue was not in the Python code. – Actuality

Answer 10

Note that the mode of opening files is 'a' or some other have alphabet 'a' will also make error because of the overwritting.

pointer = open('makeaafile.txt', 'ab+')
tes = pickle.load(pointer, encoding='utf-8')

Pudens

Answer 11

temp_model = os.path.join(models_dir, train_type + '_' + part + '_' + str(pc))
# print(type(temp_model)) # <class 'str'>
filehandler = open(temp_model, "rb")
# print(type(filehandler)) # <class '_io.BufferedReader'>
try:
    pdm_temp = pickle.load(filehandler)
except UnicodeDecodeError:
    pdm_temp = pickle.load(filehandler, fix_imports=True, encoding="latin1")

Terret

pdm_temp = pickle.load(filehandler) EOFError: Ran out of input – Its

I have this problem and I want to ask you?File pdm_temp = pickle.load(file handler) EOFError: Ran out of input – Its

Answer 12

Would suggest to create a dummy/temporary file save data into it and move it to the desired location

import pickle
import os

data = [1, 2, 3, 4, 5]
file_name = "data.pkl"  # Specify the file name

# Write data to the file using pickle
with open(file_name, 'wb') as file:
    pickle.dump(data, file)

print(f"Data has been written to '{file_name}' using pickle.")

# Step 2: Read data from the file using pickle
# Specify the new path and name where you want to move and rename the file
new_path = "/path/to/new/directory/final_data.pkl"  

# Use os.rename to move and rename the file
os.rename(file_name, new_path)

print(f"File '{file_name}' has been renamed and moved to '{new_path}'.")

# Step 3: Read the data back from the new path
with open(new_path, 'rb') as file:
    loaded_data = pickle.load(file)

print("Data loaded from the new path:", loaded_data)

In this way, while reading the file we won't end up in EOF: Ran out of input

Fiorenze

Answer 13

from os.path import getsize as size
from pickle import *
if size(target)>0:
    with open(target,'rb') as f:
        scores={i:j for i,j in enumerate(load(f))}
else: scores={}

#line 1. we importing Function 'getsize' from Library 'OS' sublibrary 'path' and we rename it with command 'as' for shorter style of writing. Important is hier that we loading only one single Func that we need and not whole Library! line 2. Same Idea, but when we dont know wich modul we will use in code at the begining, we can import all library using a command '*'. line 3. Conditional Statement... if size of your file >0 ( means obj is not an empty). 'target' is variable that schould be a bit earlier predefined. just an Example : target=(r'd:\dir1\dir.2..\YourDataFile.bin') Line 4. 'With open(target) as file:' an open construction for any file, u dont need then to use file.close(). it helps to avoid some typical Errors such as "Run out of input" or Permissions rights. 'rb' mod means 'rea binary' that u can only read(load) the data from your binary file but u cant modify/rewrite it. Line5. List comprehension method in applying to a Dictionary.. line 6. Case your datafile is empty, it will not raise an any Error msg, but return just an empty dictionary.

Gnarled

Could you provide a short explanation as to how your code works? – Anguilla

What is exactly not clear for You? Which line in code is? We have a limited size of answer-area, i cannt write hier to much. – Gnarled

That's not true: you can use as much space as you need to write your answer. It's worth making it clear, if you want others to benefit from it in the future. – Resurrectionism

BrokenBenchmark, look my edited Answer with code . Regards – Gnarled