Distutils compiler options configuration

Question

Maybe a stupid question, but I was wondering where Python's distutils get the compiler options from? It gets some linked directories wrong and I want to correct that once and for all.

I know there should be a prefix/lib/pythonver/distutils/distutils.cfg but I can't find any distutils.cfg anywhere on the computer. Obviously I haven't got a local setup.cfg or any $HOME/.pydistutils.cfg.

I'm using the Enthought 64-bit distribution, version 7.3 (Python 2.7) on Mac OS X 10.8.3

Cheers, U.

Janusfaced

Answer 1

I actually export them to the environment, just like for autotools' configure:

export CC=/usr/local/bin/clang
export CFLAGS=-I${HOME}/include
export LDFLAGS=-lboost

If you also need to override the linker separately:

export LDSHARED=/usr/local/bin/clang -shared

And if you don't like exporting the settings to your environment, do something like this for a one-time setting:

CC=/usr/local/bin/clang CFLAGS=-I${HOME}/include python setup.py build

If you want to find out what the default options were when python was build, use python-config --<flag>. Some flags are cflags, ldflags, libs or includes.

Coffee

Alright, that would set the variables, but they aren't set right now, so where does distutils get them from? I haven't found any hint in the python files that distutils would read it out somewhere else. – Janusfaced

See also Eric's answer; if there's no config file (you say it isn't there), the compiler flags indeed follow the original configure/make flags, and thus can also be seen with python-config --cflags etc. – Coffee

Ah, there they are! Since I haven't compiled Python but took the EPD distribution I wouldn't have the Makefile I guess. But now I know where to look and what to do. Cheers! – Janusfaced

Finally got enough time to try the above and it sort of works. I included the correct paths and options into my .bashrc in the form export CFGLAGS='...loads of options and paths...'. However, distutils now gets both sets of flags and some of them contradict each other, such as -O3 and -O2. Programme compiles fine though... – Janusfaced

@Mulle: if I have it correctly, for -L and -I options, they are interpreted in the order of appearance. So if your local include directory is specified before the default ones (which I think happens when using CFLAGS and LDFLAGS), your local include files get picked up first. For the -O` options, however, the gcc manual page says: "If you use multiple -O options, with or without level numbers, the last such option is the one that is effective.". If that works incorrectly for you, you could use -fno-... options to turn off optimisations. – Coffee

See also this question. – Coffee

you may want to try export CPATH=$(env | grep _INC | cut -d= -f2 | paste -d: -s) and export LIBRARY_PATH=$(env | grep _LIB | cut -d= -f2 | paste -d: -s) – Pyrogenic

On my system default values are stored in /usr/lib/python3.7/_sysconfigdata_m_linux_x86_64-linux-gnu.py. And actually python-config --cflags is original CFLAGS + python-config --includes. So, ideally I've got to take CFLAGS from _sysconfigdata_-file, add my custom options, and pass all that to distuils (CFLAGS=... pip install ...). Isn't there an easier way (from the user's standpoint)? – Gerkman

Well, the better way is probably using --global-option thing. Additionally, --global-option disables reusing already built wheel. With env vars, you've got to add --no-cache-dir, or delete the wheel, for your env vars to take effect. E.g., find ~/.cache/pip/wheels -name 'cryptography-*.whl' -delete – Gerkman

Answer 2

Compiler options are taken from CPython’s Makefile. IOW they are the same as the ones used to compile Python. You can override most of them on the command line as Evert described.

The global distutils.cfg is something that a sysadmin can create to set default options, not a file that is installed with Python.

Menial