Checking when shutil.copyfile is done

Question

I have a such code:

for file in file_list:
    shutil.copyfile(file,newpath)
#do further actions

And here is the question, at #do further actions I use the copied f iles thus I need to make sure the shutil.copyfile functions finish their task. How can I make sure of this ?

Spool

Answer 1

The shutil functions should return only after the operation is finished. If nothing funny is going on on the OS level it should be safe this way.

If you know the size of the file you could check if that is correct.

Inharmonic

Modern OSes do heavy caching on the filesystem level (in modern Linux distributions, it can take up to one minute until the data actually gets written), and the disk controllers do additional caching. Of course, the latter can't be controlled by applications. – Staircase

But the correct (cached) data should be returned if an application requests the file. I was more thinking about NFS with the async option or something similar. – Inharmonic

In my case I was running dependent subprocesses without .wait()ing for them to finish. – Katey

Answer 2

copyfile is a blocking function. By the time you get to #do further actions it should always be done. Are you having issues where it isn't?

Chapeau