Tidying Up the Cython Build-Flags used by gcc

Question

I currently use 'setuptools' to automatically cythonize and compile my Cython modules on Linux using gcc. From now on, I need more control over the build flags supplied to gcc. If I use the following in my setup.py:

cythonize(
    [Extension("*", ["project/*.pyx"])
    nthreads=4
)

I get build flags, that look like:

gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -fno-plt -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -fno-plt -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -fno-plt -fPIC -I./fastmat/core -I/home/seb/.local/lib/python3.6/site-packages/numpy/core/include -Iproject/core -Ifastmat/inspect -Iutil -I/usr/include/python3.6m -c project/BlockDiag.c -o build/temp.linux-x86_64-3.6/project/BlockDiag.o

Here I am totally flabbergasted by the fact that several build flags occur multiple times and without issuing this in any (to me obvious) way.

How can I clean up these build flags, such that they look like the ones suggested here? I hope to learn something about setuptools along the way to ultimately get full control over the build process without having to use a self-maintained makefile.

Cam

For anyone interested in 32-bit cross-compilation: https://mcmap.net/q/1010513/-how-to-build-32bit-python-2-6-on-64bit-linux, unix.stackexchange.com/a/352784/43390, askubuntu.com/a/454254/173666, askubuntu.com/a/905083/173666, https://mcmap.net/q/1020161/-macports-osx-10-9-2-is-m32-compile-possible – Lais

Answer 1

The flags GCC gets come from one of the env variables. Enter

$ python -c "from distutils import sysconfig;\
print(sysconfig.get_config_vars('CC', 'CXX', 'OPT', 'CFLAGS',\
'BASECFLAGS', 'LDFLAGS', 'CCSHARED', 'LDSHARED', 'SO'))"

to print them. This is what distutils uses by default for extension compilation. Now check which env var introduces which flag and override the env vars accordingly, for example

$ CC="gcc-7.3.0" CFLAGS="-Ofast" python setup.py build_ext

to use the specific compiler version and turn on O3 optimizations.

Also, it looks like you're using numpy.distutils instead of vanilla distutils, so be aware of extra include/link flags numpy adds under the hood.

Methaemoglobin

Thanks! I got it cleaned up to gcc -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I./fastmat/core [...] -o build/temp.linux-x86_64-3.6/fastmat/BlockDiag.o by using CC="gcc" CXX="g++" OPT="" CFLAGS="" BASECFLAGS="" LDFLAGS="" CCSHARED="" LDSHARED="gcc -shared" PY_CORE_CFLAGS="" PY_CFLAGS="" SO="" python setup.py build_ext --inplace. Now I am still wondering where the remaining flags come from, because I searched the dictionary, which is returned by sysconfig.get_config_vars() for any key containing -O3 for example and set it to "", but still it continues to pop up. – Cam

And for clarification, I am just linking against some numpy libs, but I am using the standard distutils lib. – Cam

If you can't find the flags being set in one of the env vars (for reference, here's the spot in distutils source where the flags are read from env), then they're probably baked into gcc by the vendor or distro maintainer (you can check gcc -dumpspecs for that). If you need a vanilla gcc, build one from source and pass the custom executable via CC var. – Methaemoglobin