Copy a file, but don't overwrite, without TOCTTOU issues in Python

Question

I know that if I want to copy a file in Python but not overwrite the destination I can use code like this:

if os.path.exists(dest):
    raise Exception("Destination file exists!")
else:
    shutil.copy2(src, dest)

But the state of the world could change between the time I call os.path.exists and the time I call copy2. Is there a more preferred way to copy without overwriting, presumably wherein the copy operation will raise an exception if the destination already exists?

Cloison

Answer 1

You can use the lower-level os.open and then os.fdopen to copy the file:

import os
import shutil

# Open the file and raise an exception if it exists
fd = os.open(filename, os.O_CREAT | os.O_EXCL | os.O_WRONLY)

# Copy the file and automatically close files at the end
with os.fdopen(fd) as f:
    with open(src_filename) as sf:
        shutil.copyfileobj(sf, f)

Recti

os.fdopen(fd) perhaps? – Truckage

Using shutil.copyfileobj(sf, f) would be preferable to f.write(sf.read()) as it wouldn't read the entire file into memory – Whipstitch

Thank you both! I made those changes. – Recti

This certainly does it, thanks. Two notes: filename is the input filename; and someone who wants to duplicate the behavior of shutil.copy2 would want to run shutil.copystat at the end (though this was not explicit in the question) – Cloison

As a security measure, isn't it important to copy the file mode before copying data into it? I don't see that in this code. Even if you do add shutil.copystat afterwards as suggested above. I think you need to set a minimal mode argument. – Trevor

@Trevor can you be more specific, perhaps even write out the code? Is it just a matter of doing copystat immediately before copyfileobj? – Cloison