Can you use other python modules like pandas and numpy when you compile a stand alone executable with Cython?

Question

I'm trying to convert a python script into Cython executable binary. It also uses other modules like Tensorflow, pandas, numpy, matplotlib, seaborn, and pickle. Here is my compile command.

cython --embed -o foo.c foo.pyx

gcc -Os -o FOOEXE foo.c -I /usr/include/python3.6m/ -L/workdirectory/somePythonEnviroment-venv/lib -lpython3.6m -lpthread -lm -lutil -ldl

I downloaded all the required modules with pip and running it as a python script is fine. When I compile it into a C binary using cython magic. The first thing it says
import pandas as pd No module named 'pandas'

Which I agree. I had assume it would compile all included modules too and put it in the binary. How do I INCLUDE the module pandas? How do I include any python module I install with pip into a cython project. Does cython only work with basic Python code and libraries from python.h? Do I need to have the original python module files with the excutable?

Saavedra

It is unclear what you mean by "using cython magic". If this means embed - your program will need some setup to make it work with site-packages, e.g. https://mcmap.net/q/49212/-importerror-after-cython-embed/5769463 and https://mcmap.net/q/48835/-minimal-set-of-files-required-to-distribute-an-embed-cython-compiled-code-and-make-it-work-on-any-machine/5769463 – Tinny

Yes, you are correct, I did do cython --embed -o foo.c foo.pyx first. I compiled successfully, but its asking for modules in some way, which I am unsure how to "include" these modules. – Saavedra

Are you trying to get it to use the pandas module installed on your computer (which is fine) or are you trying to make a genuinely self-contained exe that includes pandas (Cython isn't designed for this and you almost certainly won't get it to work)? – Keldon

Anyway to get the self-contained executable to run. If I need just copy the pandas folder to where its located, I'll do that, I just don't know if thats how it works or not. I would prefer it was all self contained if thats possible. – Saavedra

I just keep getting this message. ImportError: C Extension: No module named 'pandas._libs.interval' not built. If you want to import pandas from the source directory. you may need to run 'python setup.py build_ext --force' to build the C extensions first. I have no idea what it means by this. I ran setup.py script to get the .SO file. – Saavedra

I see a similar question here, #37008444 but that can't be right, does pandas no longer work with cython? – Saavedra

I got it working, but my question is still unanswered. It involved uninstalling anaconda3 and manually reinstall all the pip packages, and then set all the include and lib files around anaconda's folders. I have no coherent answer. – Saavedra

This was probably a misconfiguration in the notebook where you were calling cython magic. – Wilkey

@Saavedra were you running this line by line to produce the gcc? If so do you remember what -I and -L you used to get pandas found? – Lecturer