Python does not detect .pyc files

Question

I am using Python 3.2 (both for building and executing), and here is my question.

I intend to ship my python application with the following setup:

There is a main script (say, Main.py), that is using a compiled module, say Module1.pyc). To be precise, the directory structure is:

.\Main.py
.\__pycache__\Module1.cpython-32.pyc

When I use the python interpreter to run the main script, it fails to find the module with the following error:

Traceback (most recent call last):
  File "Main.py", line 10, in <module>
    import Module1
ImportError: No module named Module1

---

Note that I have added the current directory to PYTHONPATH environment variable, and is part of sys.path. Also, the inner __pycache__ directory is also added, and is visible in sys.path.

Not sure why Module1 is not found. Am guessing, it could be because of the different file name - Module1.cpython-32.pyc? But, then that is how the Python 3.2 interpreter generates it.

Saturable

Make sure your filenames are the same. You initially state the module name is Module1, and show so from your dir structure. Then you import Module and talk about a different filename. So it's hard to know if it's just the fact that you mistyped the module name or if it's actually incorrect. – Gormley

Sorry that was just a typo. The file names are consistent. – Saturable

Does this answer your question? How to run a Python project using __pycache__ folder? – Almoner

Answer 1

Have a look at PEP-3147. They describe how the python-lookup mechanism works.

So in your concrete case: Put the file Module1.pyc directly in the root folder.

Heckle

Tried copying Module.cpython-32.pyc to the same folder, but again it fails. Now, because the file name has 'cpython-32.' as part of it. If i rename the file to Module.pyc, it works like a charm. – Saturable

While this just solves the problem, it requires one more step of renaming the pyc files. Is there a way to control the pyc filenames created? – Saturable

Have a look at the module compileall, this could be what you are looking for... (compileall.compile_dir) – Heckle

Answer 2

As stated below, two steps resolved the issue: Step 1: Copy the Module.cpython-32.pyc file from .__pycache__ directory to .\ Step 2: Rename the file to Module.pyc

PS: Thanks to gecco for sharing the detail.

Saturable

Hint: Your answer is not answering your question (Why pyc not found): Your answer could be down-voted by severe SO users... – Heckle

@gecco, there are two parts to fix this problem, which i have mentioned above. Unless both steps are performed, it does not work. The flowchart you have shared pertains to step 1. So, not sure why this should be down voted. Because, i have just tried, and found both steps are needed to get it working – Saturable

As far as I can read in your question, you did nowhere ask how to get from ./__pycache__/Module.cpython-32.py to ./Module.pyc. You asked why it didn't work and how to fix it: Both answers are provided in my answer... – Heckle

Answer 3

This problem may depend on which version of python you are using, as the build/compile/packaging systems have changed over the years.

For python 3.9, I found this command works:

# recursively compile all modules
python -m compileall -b .

# run the compiled version of your `Main.py`
python Main.pyc

Explanation

The main issue is that in recent versions of python /package_dir/my_module.py gets compiled to /package_dir/__pycache__/my_module.cpython-64.pyc (or some variation). The module discovery system apparently does not look in __pycache__, resulting in a module not found error at the my_module import statement. The -b flag tells the compiler to write compiled files to same location as source .py files, so they are discoverable.

Illuse