No module named 'scipy.spatial.transform._rotation_groups after compile python script with pyinstaller

Question

After days of looking for an answer on internet and, of course, in overflow I make this post. I hope someone could help me. I made a little program that fits some data that I got from chemistry experimentation with a math model using scipy.optimize. Then I got a plot of the points from the data and the fit. When I run it on the IDLE everything goes okay and when I run from windows command, too. The thing is that a want to make an .exe file so I could share my program with my university classmates. For this I used pyinstaller. When I run pyinstaller to make mi .exe file evertything is ok, but when I want to run it from windows command I get the next output:

Traceback (most recent call last):
  File "solver_tp2fq_ver2.py", line 9, in <module>
  File "c:\users\lenovo\appdata\local\programs\python\python39\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 493, in exec_module
    exec(bytecode, module.__dict__)
  File "scipy\optimize\__init__.py", line 421, in <module>
  File "c:\users\lenovo\appdata\local\programs\python\python39\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 493, in exec_module
    exec(bytecode, module.__dict__)
  File "scipy\optimize\_shgo.py", line 9, in <module>
  File "c:\users\lenovo\appdata\local\programs\python\python39\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 493, in exec_module
    exec(bytecode, module.__dict__)
  File "scipy\spatial\__init__.py", line 107, in <module>
  File "c:\users\lenovo\appdata\local\programs\python\python39\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 493, in exec_module
    exec(bytecode, module.__dict__)
  File "scipy\spatial\transform\__init__.py", line 19, in <module>
  File "rotation.pyx", line 5, in init scipy.spatial.transform.rotation
ModuleNotFoundError: No module named 'scipy.spatial.transform._rotation_groups'
[10152] Failed to execute script solver_tp2fq_ver2 

As you can see I get the next error: ModuleNotFoundError: No module named 'scipy.spatial.transform._rotation_groups'

I don't understand why I get this error. I installed all the modules, checked that path is correct, upgraded the modules and also reinstalled python (Because this, actually I'm using python 3.9). I don't know what else I can do.

Well, I think it's all. I will be very thankfull if someone helps me.

Renz

Answer 1

I've already found and answer in this post: What is the recommended way to persist (pickle) custom sklearn pipelines?

I solved my problem, compiling my python file with the next command:

pyinstaller --hidden-import scipy.spatial.transform._rotation_groups --onefile solver_tp2fq_ver2.py

I'll leave it for someone who maybe have the same problem.

Renz

Your workaround works. However, this could probably be addressed by the scipy developers by adding a pyinstaller hook, if I understand correctly: pyinstaller.readthedocs.io/en/stable/hooks.html – Cautionary

Thanks for asking. I tried to solve adding a pyinstaller hook, but a i didn't understood if i have to create a new hook or if a have to modify the scipy one. In the post that i suggested (What is the recommended way to persist (pickle) custom sklearn pipelines?) there is a comment which talks about modifying a .spec file. It confused me more. Do you know where to find exactly this file? – Renz

I've created an issue over at the scipy repo: github.com/scipy/scipy/issues/13378 – Cautionary

Answer 2

I had exactly the same error. I solved it making some changes into hook-scipy.py and hook-sklearn.metrics.cluster.py (you have to look for them into pyistaller folder) as shown here: scipy import error with pyinstaller

Once the scipy import issue was solved, a sklearn one arised. So, I also had to explicitly add

--hidden-import=sklearn.utils._weight_vector 

while running pyinstaller since previous modifies were not sufficient for my script.

Clypeate

sorry, but i think that if you had to writte that specification when you where running pyinstaller, you didn't do the modifications in the hook file correctly. The modification of this files is still confusing for me. – Renz

Answer 3

I solved exactly the same issue (Issue with cx_Freeze) just by putting import scipy.spatial.transform._rotation_groups in the top of my main.py file. Probably not a clean solution though...

Roundabout

Answer 4

I was running my script inside of a virtualenv and ran into this error. I tried --hiddenimports and .spec files with no luck. I decided to just:

1. run pip freeze > requirements
2. rm -f venv (This is where I seen this error again)
3. Pushed the requirements.txt to github.
4. Pulled from github into a new folder. (so now libraries are no installed yet)
5. Reinstalled Libraries with pip install -r requirements.txt

For whatever reason the scipy folder was installed wrong and starting from scratch in my virtual environment worked.

Zecchino

Answer 5

It well very well be an issue with the latest scipy version, which might be incompatible with the latest pyinstaller version. Now, using hidden import be can by pass the issue, but recently I faced an issue of going into an infinite loop when generating the one file exe file using pyinstaller and hidden import. So, what I did, downloaded the scipy version

pip install "scipy==1.4.1"

and then :

pyinstaller --clean --onefile myscript.py

this seems to work for me fine.

Sabotage