Permission denied doing os.mkdir(d) after running shutil.rmtree(d) in Python

Question

Very often in the Windows 7 console if I run a python program twice very quickly that does

if os.path.isdir(d):
    shutil.rmtree(d)
if not os.path.exists(d):
    os.mkdir(d)

where d is the name of a directory with many files, I get a "Permission denied" for the mkdir command. But if I run once, then wait some seconds, then run again I do not get such error. What is the problem here?

Proselytism

FYI some related discussion thread here: groups.google.com/forum/#!topic/comp.lang.python/sVnSa5iWWzg – Dummy

Answer 1

There are three things that come to mind:

1. Windows itself delays some file operations in order to preserve metadata. If you for example rename a file and create another one in its location, Windows has a time-window where things like ACLs are transferred to the new file. This is a "feature" to preserve this metadata even for programs that write a new file before deleting the old one, in order to not loose data when something fails in the middle.

2. Malware scanners sometimes hook into filesystem operations and perform a scan on files, searching for malware (or government-critic texts, if you're paranoid, and maybe even if you're not paranoid). During that scan, some other accesses to the file are blocked.

3. Lastly, I'm not sure how shutil.rmtree() is implemented, but under Windows, some tree operations are actually implemented not by the OS core but by the shell (i.e. Explorer) and they could be executed asynchronously, which would explain a short time window in which the path is still blocked even though the call already returned.

I believe that e.g. Subversion or rather the Apache Portable Runtime stumbled across the same problem and worked around it by simply retrying with a delay. This solution doesn't win a beauty contest, but it seems to Do The Job(tm).

Belsky

Definitely, it seems that using a delay and then retry is the only solution. Thanks – Proselytism

There is easier solution. Just "os.rename" folder before deleting. Renaming is immediate. – Himes

Definitely not (2), since this behavior is perfectly reproducible. To me (3) sounds like the most plausible explanation. File operations being async makes me reconsider using shutil altogether. – Olomouc

@Olomouc - another possible explanation is mentioned under the discussion section in Eryk Sun's answer here: https://mcmap.net/q/56187/-os-mkdir-can-fail-with-permissionerror-after-successful-shutil-rmtree-on-windows. Basically, under Windows "delete" marks a file for deletion, but does not unlink it, until the last file handle is closed. – Defraud