Deleting read-only directory in Python

Question

shutil.rmtree will not delete read-only files on Windows. Is there a python equivalent of "rm -rf" ? Why oh why is this such a pain?

Mirage

"Why oh why is this such a pain?" Maybe nobody has taken the five minutes to file a bug at bugs.python.org ...You could be the first! Just sayin'. – Homoousian

I know I'm late to the party, but I just filed bugs.python.org/issue22040 – Topaz

Answer 1

shutil.rmtree can take an error-handling function that will be called when it has problem removing a file. You can use that to force the removal of the problematic file(s).

Inspired by http://mail.python.org/pipermail/tutor/2006-June/047551.html and http://techarttiki.blogspot.com/2008/08/read-only-windows-files-with-python.html:

import os
import stat
import shutil

def remove_readonly(func, path, excinfo):
    os.chmod(path, stat.S_IWRITE)
    func(path)

shutil.rmtree(top, onerror=remove_readonly)

(I haven't tested that snippet out, but it should be enough to get you started)

Parimutuel

This worked for me like a charm, even on Python unit tests. Thanks! – Srini

One-liner: shutil.rmtree('mypath', onerror=lambda func, path, _: (os.chmod(path, stat.S_IWRITE), func(path))) – Whilom

How this method could handle if the directory is not exist, i.e. containing the ignore_errors=True in itself. – Horsy

Answer 2

If you import win32api from PyWin32, you can use:

win32api.SetFileAttributes(path, win32con.FILE_ATTRIBUTE_NORMAL)

To make files cease to be read-only.

Isogamete

Answer 3

Another way is to define rmtree on Windows as

rmtree = lambda path: subprocess.check_call(['cmd', '/c', 'rd', '/s', '/q', path])

Interlaken

Answer 4

There's a comment at the ActiveState site that says:

shutil.rmtree has its shortcomings. Although it is true you can use shutil.rmtree() in many cases, there are some cases where it does not work. For example, files that are marked read-only under Windows cannot be deleted by shutil.rmtree().

By importing the win32api and win32con modules from PyWin32 and adding line like "win32api.SetFileAttributes(path, win32con.FILE_ATTRIBUTE_NORMAL" to the rmgeneric() function, this obstacle can be overcome. I used this approach to fix the hot-backup.py script of Subversion 1.4 so it will work under Windows. Thank you for the recipe.

I don't use Windows so can't verify whether this works or not.

Helminthology

Answer 5

This will presumably be fixed with the release of Python 3.5 (currently - June 2015 - still in development) in the sense of giving a hint about this in the documentation.

You can find the bugreport here. And this is the according changeset.

See the newly added example from the Python 3.5 docs:

import os, stat
import shutil

def remove_readonly(func, path, _):
    "Clear the readonly bit and reattempt the removal"
    os.chmod(path, stat.S_IWRITE)
    func(path)

shutil.rmtree(directory, onerror=remove_readonly)

Instigation

Answer 6

Here is a variant of what Steve posted, it uses the same basic mechanism, and this one is tested :-)

What user do python scripts run as in windows?

Gaylenegayler

Answer 7

I had this issue on Python 3.7 when invoking rmtree() and worked around it by explicitly fixing the permissions before exiting the TemporaryDirectory() context manager. See akaihola/darker#453 for details. Here's a copy of the implementation:

import os
import sys
from pathlib import Path
from typing import Union

WINDOWS = sys.platform.startswith("win")

def fix_py37_win_tempdir_permissions(dirpath: Union[str, Path]) -> None:
    """Work around a `tempfile` clean-up issue on Windows with Python 3.7

    Call this before exiting a ``with TemporaryDirectory():`` block or in teardown for
    a Pytest fixture which creates a temporary directory.

    See discussion in https://github.com/akaihola/darker/pull/393
    Solution borrowed from https://github.com/python/cpython/pull/10320

    :param dirpath: The root path of the temporary directory

    """
    if not WINDOWS or sys.version_info >= (3, 8):
        return
    for root, dirs, files in os.walk(dirpath):
        for name in dirs + files:
            path = os.path.join(root, name)
            try:
                os.chflags(path, 0)  # type: ignore[attr-defined]
            except AttributeError:
                pass
            os.chmod(path, 0o700)

and here's how to use it in Pytest unit tests or when creating a temporary directory using tempfile:

import pytest

from my_utils import fix_py37_win_tempdir_permissions

@pytest.fixture
def myfixture(tmp_path):
    # setup code here
    yield tmp_path
    fix_py37_win_tempdir_permissions(tmp_path)


def myfunc():
    with TemporaryDirectory() as tmpdir:
        # work on the temporary directory here
        fix_py37_win_tempdir_permissions(tmp_path)

Catcall