Python3 project remove __pycache__ folders and .pyc files

Question

What is the BEST way to clear out all the __pycache__ folders and .pyc/.pyo files from a python3 project. I have seen multiple users suggest the pyclean script bundled with Debian, but this does not remove the folders. I want a simple way to clean up the project before pushing the files to my DVS.

Etem

It may also be worth considering changing the cache directory to avoid this issue alltogether. For example, export PYTHONPYCACHEPREFIX=/tmp/pycache. – Confluent

Answer 1

I found the answer myself when I mistyped pyclean as pycclean:

No command 'pycclean' found, did you mean:
  Command 'py3clean' from package 'python3-minimal' (main)
  Command 'pyclean' from package 'python-minimal' (main)
  pycclean: command not found

Running py3clean . cleaned it up very nicely.

Etem

py3clean is more conservative than @V.Gamula's command: it does not delete .pyc files generated from source files that have been deleted since. – Insincere

As of may/2017, in macos/homebrew, I had to use the package pycleaner from pip, and the command to execute it is pycleaner. Python 2 only. – Stultify

python3-minimal seems to be available on Debian only (not finding it on RHEL, CentOS, brew, MacOS) – Hale

oddly enough, my installed py3clean (3.5.1-3) does nothing actually, clears none of those files and simply silently returns (python 3.6.5). – Consolidate

This only seems to work if the executable is available under the specific name. This is not a general solution but more like Debian/Ubuntu only. – Witten

It does not delete __pycache__ folders, nor .pyc files (tested on debian) – Chasechaser

Answer 2

You can do it manually with the next command:

find . | grep -E "(/__pycache__$|\.pyc$|\.pyo$)" | xargs rm -rf

This will remove all .pyc and .pyo files as well as __pycache__ directories recursively starting from the current directory.

Nunatak

The command is dirty. It will also delete *__pycache__, *.pyc* and it doesn't distinguish between files and directories. Most of the generated rm commands will also be superfluous. – Mud

This is just wrong. That's not how you use find nor grep nor rm and it could be dangerous for any current working directory that might contain useful.pycle.json or important_deployment_setting_about__pycache__.txt etc. Which I wouldn't rule out on the entire user base of SO. Not to mention how (dangerously rm -rf) broken xargs would be by a file with whitespace in the name. So I edited it… – Jello

@dwanderson I think you're talking about a different answer; or does SO edit history no longer show the complete history EG there was a version in 2017 that isn't shown there? – Jello

Doesn't work on OSX grep: parentheses not balanced – Witten

is it safe to delete these files ? – Euphoria

this command is not safe. make sure about what you are going to do here, if you are inside the python virtual environment and if you executes this command it will remove the entire .pyc in the virtual environment as well. – Isoniazid

@Jello So is it correct now? If so, please delete the comment, it kind of makes me afraid to use it as is. – Gracegraceful

@Mud Is it still true after the edits? – Gracegraceful

@Gracegraceful it still doesn't distinguish between files and directories. – Mud

@Gracegraceful also note other comments – Mud

Test with find . -regex '^.*\(__pycache__\|\.py[co]\)$' then add -delete i.e. find . -regex '^.*\(__pycache__\|\.py[co]\)$' -delete – Cham

Answer 3

macOS & Linux

BSD's find implementation on macOS is different from GNU find - this is compatible with both BSD and GNU find. Start with a globbing implementation, using -name and the -o for or - Put this function in your .bashrc file:

pyclean () {
    find . -type f -name '*.py[co]' -delete -o -type d -name __pycache__ -delete
}

Then cd to the directory you want to recursively clean, and type pyclean.

GNU find-only

This is a GNU find, only (i.e. Linux) solution, but I feel it's a little nicer with the regex:

pyclean () {
    find . -regex '^.*\(__pycache__\|\.py[co]\)$' -delete
}

Any platform, using Python 3

On Windows, you probably don't even have find. You do, however, probably have Python 3, which starting in 3.4 has the convenient pathlib module:

python3 -Bc "import pathlib; [p.unlink() for p in pathlib.Path('.').rglob('*.py[co]')]"
python3 -Bc "import pathlib; [p.rmdir() for p in pathlib.Path('.').rglob('__pycache__')]"

The -B flag tells Python not to write .pyc files. (See also the PYTHONDONTWRITEBYTECODE environment variable.)

The above abuses list comprehensions for looping, but when using python -c, style is rather a secondary concern. Alternatively we could abuse (for example) __import__:

python3 -Bc "for p in __import__('pathlib').Path('.').rglob('*.py[co]'): p.unlink()"
python3 -Bc "for p in __import__('pathlib').Path('.').rglob('__pycache__'): p.rmdir()"

Critique of an answer

The top answer used to say:

find . | grep -E "(__pycache__|\.pyc|\.pyo$)" | xargs rm -rf

This would seem to be less efficient because it uses three processes. find takes a regular expression, so we don't need a separate invocation of grep. Similarly, it has -delete, so we don't need a separate invocation of rm —and contrary to a comment here, it will delete non-empty directories so long as they get emptied by virtue of the regular expression match.

From the xargs man page:

find /tmp -depth -name core -type f -delete

Find files named core in or below the directory /tmp and delete them, but more efficiently than in the previous example (because we avoid the need to use fork(2) and exec(2) to launch rm and we don't need the extra xargs process).

Exhilarative

You could probably get the python version to one line by using shutil.rmtee on the __pycache__ directories instead of emptying them first; a one-liner might be nice for an easy alias. I'm a big fan of Path right now, though, so I think I'll use that version in my script. – Melbamelborn

python -Bc "import pathlib; import shutil; [shutil.rmtree(p) for p in pathlib.Path('.').rglob('__pycache__')]" – Hectic

For some reason the -delete flags on find don't seem to be working for me on macOS inside of a Makefile, even when explicitly setting SHELL to bash. But piping to xargs to delete worked fine. Not really sure what that's about. – Kaylil

For powershell user Get-ChildItem -Recurse | where {$_.name -Match '__pycache__'} | Remove-Item -Force -Recurse – Bamberger

Answer 4

Since this is a Python 3 project, you only need to delete __pycache__ directories -- all .pyc/.pyo files are inside them.

find . -type d -name __pycache__ -exec rm -r {} \+

_{(Note: using -delete instead of -exec rm -r {} \+ won't work because it can't delete non-empty directories)}

---

Alternatively, if you're doing this in a directory that's under revision control, you can tell the RCS to ignore __pycache__ directories recursively. Then, at the required moment, just clean up all the ignored files. This will likely be more convenient because there'll probably be more to clean up than just __pycache__.

Mud

No, not all .pyc files are inside it. – Contributory

@V.Gamula This is a considerably more safe command than the one ~114 people were lucky enough did not bite them with permanent deletion of a mistaken match. – Jello

@MartinThoma in a python3 only project? – Jello

Am I the only one getting Directory not empty with the second variant here? – Consolidate

find ./ -type f -name '*.pyc' -delete -print && find ./ -type d -name "pycache" -delete -print – Pender

Answer 5

If you need a permanent solution for keeping Python cache files out of your project directories:

Starting with Python 3.8, you can use the environment variable PYTHONPYCACHEPREFIX to define a cache directory for Python.

From the Python docs:

If this is set, Python will write .pyc files in a mirror directory tree at this path, instead of in __pycache__ directories within the source tree. This is equivalent to specifying the -X pycache_prefix=PATH option.

Example

If you add the following line to your ~/.profile shell configuration file on Linux:

export PYTHONPYCACHEPREFIX="${HOME}/.cache/Python"

Python won't create the annoying __pycache__ directories in your project directory; instead, it will put all of them under ~/.cache/Python

If you work Linux and macOS, you could add these lines to ~/.bash_profile / ~/.profile for a cross-platform shell configuration (which also creates the folder as required):

# set python cache folder for .pyc files:
#   https://stackoverflow.com/a/57415054
case "${OSTYPE}" in
    darwin*)
        folder="${HOME}/Library/Caches/Python"
        if [[ ! -d "${folder}" ]]; then mkdir -p "${folder}"; fi
        export PYTHONPYCACHEPREFIX="${folder}"
        ;;
    linux-*)
        folder="${HOME}/.cache/Python"
        if [[ ! -d "${folder}" ]]; then mkdir -p "${folder}"; fi
        export PYTHONPYCACHEPREFIX="${folder}"
        ;;
    *)
        printf "WARNING: unsupported operating system '%s'; "`
              `'not setting PYTHONPYCACHEPREFIX' "${OSTYPE}" >&2
        return 1
        ;;
esac

Rhoden

Yes. It's a great feature! All OS distros should enable it by default. Such as adding PYTHONPYCACHEPREFIX=/var/cache/python into /etc/environment – Sino

@Keelung: I am not sure if it is advisable to use a system-wide folder such as /var/cache: This might clash with the cache of other users on the system - especially when configuring this on a "distribution level" (i.e. making that setting the default for all users). – Pigeon

Answer 6

The command I've used:

find . -type d -name "__pycache__" -exec rm -r {} +

Explains:

1. First finds all __pycache__ folders in current directory.

2. Execute rm -r {} + to delete each folder at step above ({} signify for placeholder and + to end the command)

Edited 1:

I'm using Linux, to reuse the command I've added the line below to the ~/.bashrc file

alias rm-pycache='find . -type d -name  "__pycache__" -exec rm -r {} +'

Edited 2: If you're using VS Code, you don't need to remove __pycache__ manually. You can add the snippet below to settings.json file. After that, VS Code will hide all __pycache__ folders for you

"files.exclude": {
     "**/__pycache__": true
}

Hope it helps !!!

Sangria

Thanks this "ind . -type d -name "pycache" -exec rm -r {} +" worked for me, was trying to find an easy way to delete all the pycache folder in my django project – Exsanguine

Answer 7

I'm running Python3 and pip3 on a Mac. For me, the solution was as follows (In the root directory of my project):

pip3 install pyclean
pyclean .

I'd like to emphasize, since I see many answers which involve bash scripting, it is best practice in software to favour tested solutions to problems (which is exactly what an established python package represents) over a hand-rolled approach.

Fibrovascular

Assuming the established python package has tested all possible scenarios... – Snarl

That seems an impossibility. It is also established practice to be selective about the problems we solve. If your use-case is genuinely unsupported by the common solutions, of course it makes sense to solve your unique use case. In such a case, It may be that others share the same problem and your solution should be published. – Fibrovascular

Answer 8

There is a nice pip package.

pip install pyclean
pyclean --verbose .

done

Pruter

Answer 9

This is my alias that works both with Python 2 and Python 3 removing all .pyc .pyo files as well __pycache__ directories recursively.

alias pyclean='find . -name "*.py[co]" -o -name __pycache__ -exec rm -rf {} +'

Numbat

Answer 10

Using PyCharm

To remove Python compiled files

1. In the Project Tool Window, right-click a project or directory, where Python compiled files should be deleted from.

2. On the context menu, choose Clean Python compiled files.

The .pyc files residing in the selected directory are silently deleted.

Rochelle

Answer 11

From the project directory type the following:

Deleting all .pyc files

find . -path "*/*.pyc" -delete

Deleting all .pyo files:

find . -path "*/*.pyo" -delete

Finally, to delete all '__pycache__', type:

find . -path "*/__pycache__" -type d -exec rm -r {} ';'

If you encounter permission denied error, add sudo at the begining of all the above command.

Arita

Better add "-print" after "-delete". – Pender

Answer 12

Thanks a lot for the other answers, based on them this is what I used for my Debian package's prerm file:

#!/bin/sh
set -e

deb_package='package-name'
python_package='package_name'

if which pyclean >/dev/null 2>&1; then
    py3clean -p $deb_package
else
    dpkg -L $deb_package | grep ${python_package}$ | while read file
    do
        find ${file} -type d -name __pycache__ -exec rm -r {} \+
    done
fi

Publius

Answer 13

Most of the time your project is under source control; in case if git is used, one can just run git clean -d -f . to clean up non-version controlled files; in case if Python cache files/directories are in .gitignore, you'll also need -x (also remove ignored files) or -X (remove only ignored files) command line switches.

Please note that if you have any temporary files that have to be saved, move them outside local repository or add to index tree before running git clean without -X.

Alaynaalayne

Answer 14

Empty the directories first and then remove them:

find ./ -type f -name '*.pyc' -delete -print && find ./ -type d -name '__pycache__' -delete -print

Pender

Answer 15

Why not just use rm -rf __pycache__? Run git add -A afterwards to remove them from your repository and add __pycache__/ to your .gitignore file.

Adlai

He's not doing this manually. He's trying to add it to a script used in a Debian package. A debian package is not used with git but with the debian package manager or the apt installer. – Publius

Answer 16

As simple as it can be

#!/bin/bash

project_root_dir=$(pwd)

# 
# function _dcache deletes __pycache__ folders floating around python modules
# 
function _dcache() {
  find "$project_root_dir/inb" -name "__pycache__" > pycache

  while IFS= read -r cache_file; do
    rm -r $cache_file
  done < pycache

  rm pycache
}

This is what I use for my projects. You can give it a try.

Oof

have you considered the "pyclean" pip package? It does the job and likely covers edge cases that this script would not. – Fibrovascular

Answer 17

If you have a nested directory with multiple junk files, you can give the root directory to the following function and it will clear all subfolders from the junkies:

import os, shutil
def clean_all_from_junk(directory):
    def delete_junk(f):
        try:
            shutil.rmtree(f)
        except:
            os.unlink(f)
    msk_dirs = os.listdir(directory)  
    for f in msk_dirs:
        if f.startswith('.') or f=='__pycache__' or f.endswith('txt'):
            junk_p = os.path.join(directory, f)
            print('deleted-->', junk_p)
            delete_junk(junk_p)
        else:
            if os.path.isdir(f):
                subfolder_p = os.path.join(directory, f)
                clean_all_from_junk(subfolder_p)
    return 'All cleaned!'

 
clean_all_from_junk('path/to/root/directory')

Bandler

Answer 18

If you deleted all the __pycache__/ stuff and you still get HINT: remove __pycache__/.pyc files and/or use a unique basename for your test file modules, it may be caused by the second part of the error message - just check that you don't have the two files with the exact names.

Spoof

Answer 19

Please just go to your terminal then type:

$rm __pycache__

and it will be removed.

Nebuchadnezzar

What you mean is if somebody has thousands of pycache folders floating around a python project then (s)he can go with your approach and it will be fine? The questioner wants to automate this task so it will be helpful if you can provide a script that will perform the cleanup automatically. Thanks! – Oof

The bare minimum would be a rm -r __pycache__ – Burschenschaft