How to create C-extension/embed Python with MinGW-w64 on Windows

Question

Is it possible (and how) to use MinGW-w64 for building of C-extensions for Python or embeding Python on Windows?

Let's take as example the following cython-extension foo.pyx:

print("foo loaded")

from which the C-code can be generated either via cython -3 foo.pyx or cython -3 --embed foo.pyx if interpreter should be embedded.

Infatuated

Answer 1

While mingw-w64-compiler is not really supported (the only supported windows compiler is MSVC), it can be used to create C-extensions or to embed Python. There are however no guarantee, this won't break in the future versions.

distutils does not support mingw-w64, so there is no gain in setting up a setup.py-file - the steps must be performed manually.

First we need some information usually provided by distutils:

• Headers: We need the path to the Python includes. For a way to find them see this SO-post.
• DLL: mingw-w64's linker works differently than MSVC's: python-dll and not python-lib is needed. So we need the path to the pythonXY.dll which is usually next the the python.exe.

Once the C-code is created/generated, the extension can be build via

x86_64-w64-mingw32-gcc -shared foo.c -DMS_WIN64 -O2 <other_options> -I <path_to_python_include> -L <path_to_python_dll> -lpython37 -o foo.pyd

The important details are:

• it is probably Ok to use only use -O2 for optimization and leave <other_options> empty-
• It is important to define MS_WIN64-macro (e.g. via -DMS_WIN64). In order to build for x64 on windows it must be set, but it works out of the box only for MSVC (defining _WIN64 could have slightly different outcomes):

#ifdef _WIN64
#define MS_WIN64
#endif

if it is not done, at least for files generated by Cython the following error message will be generated by the compiler:

 error: enumerator value for ‘__pyx_check_sizeof_voidp’ is not an integer constant
  201 |     enum { __pyx_check_sizeof_voidp = 1 / (int)(SIZEOF_VOID_P == sizeof(void*)) };

• pyd is just a dll in disguise, thus we need the -shared option, which means a dynamic library (i.e. shared-object in Linux-world) will be created.

• It is important, that the python-library (pythonXY) should be the dll itself and not the lib (see this SO-post). Thua we use the path to pythonXY.dll (in my case python37) and not pythonXY.lib, as it would be the case for MSVC.

One probably should add the proper suffix to the resulting pyd-file, I use the old convention for simplicity here.

---

Embeded Python:

In this case an executable should be build (e.g. the C-file is generated by Cython with --embed option: cython -3 --embed foo.pyx) and thus the command line looks as follows:

x86_64-w64-mingw32-gcc foo.c -DMS_WIN64 -O2 <other_options> -I <path_to_python_include> -L <path_to_python_dll> -lpython37 -o foo.exe -municode

There are two important differences:

• -shared should no longer be used, as the result is no longer a dynamic library (that is what *.pyd-file is after all) but an executable.
• -municode is needed, because for Windows, Cython defines int wmain(int argc, wchar_t **argv) instead of int main(int argc, char** argv). Without this option, an error message like

/build/mingw-w64-_1w3Xm/mingw-w64-4.0.4/mingw-w64-crt/crt/crt0_c.c:18: undefined reference to 'WinMain'
collect2: error: ld returned 1 exit status

would appear (see this SO-post for more information).

Note: for the resulting executable to run, a whole python-distribution (and not only the dll) is needed (see also this SO-post), otherwise the resulting executable will abort with error (either the python dll wasn't found or the python installation or the site packages - depending on the configuration of the machine on which the exe has to run).

---

mingw-w64 can also be used on Linux for cross-compilation for Windows, see this SO-post.

Infatuated