How to remove -pthread compiler flag from cython setup file

Question

In linux environment, when I run the setup script for cython, I get

gcc -pthread -B /apps/.../compiler_compat -Wl,-sysroot=/ -Wsign-compare 
-DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/ap......  

for my case, I want to remove the pthread option. How do I do that thru the cython setup file? I see there are options to add compiler flags but none to remove. My setup file:

from distutils.core import setup
from Cython.Build import cythonize

from distutils.extension import Extension

extensions = [Extension("foo",sources = ["foo.pyx"])]                                 

setup(ext_modules = cythonize(extensions))

Somite

This question is related but I don't think helps; it's possible the third option from this answer might help, but it's a bit extreme losing all the arguments. – Sigman

Answer 1

As @DavidW has pointed out, for many options there is also an option which negates/overrides it and the last option on the command line "wins". So for example adding extra compile option -Os would overrule the default setting -O2 or -O3, because -Os would come after -O2 on the command line (-fwrapv/-fno-wrapv another example for such pairs).

However, -pthread has no such "partner" and the only chance to disable it, is to prevent its appearance on the command line altogether. The ways to do it are somewhat hacky, but is this hackiness not the reason we all use python?

distutils uses distutils.sysconfig to find the right compile/link flags. One possibility would be modify its functionality, so that -pthread is filtered out.

I choose get_config_vars, but there are also other options of course. The plan is simple:

1. wrap distutils.sysconfig.get_config_vars, so that -pthread is filtered out
2. replace distutils.sysconfig.get_config_vars with the wrapper
3. otherwise, setup.py is unchanged

Which leads to the following setup.py:

# manipulate get_config_vars:
# 1. step: wrap functionality and filter
from distutils.sysconfig import get_config_vars as default_get_config_vars

def remove_pthread(x):
    if type(x) is str:
        # x.replace(" -pthread ") would be probably enough...
        # but we want to make sure we make it right for every input
        if x=="-pthread":
            return ""
        if x.startswith("-pthread "):
            return remove_pthread(x[len("-pthread "):])
        if x.endswith(" -pthread"):
            return remove_pthread(x[:-len(" -pthread")])
        return x.replace(" -pthread ", " ")
    return x

def my_get_config_vars(*args):
  result = default_get_config_vars(*args)
  # sometimes result is a list and sometimes a dict:
  if type(result) is list:
     return [remove_pthread(x) for x in result]
  elif type(result) is dict:
     return {k : remove_pthread(x) for k,x in result.items()}
  else:
     raise Exception("cannot handle type"+type(result))

# 2.step: replace    
import distutils.sysconfig as dsc
dsc.get_config_vars = my_get_config_vars


# 3.step: normal setup.py

from distutils.core import setup
from Cython.Build import cythonize

from distutils.extension import Extension

extensions = [Extension("foo",sources = ["foo.pyx"])]                                 

setup(ext_modules = cythonize(extensions))

---

I'm not entirely sure, it is a great idea to build without -pthread and to load the resulting module in a python-interpreter which is built with -pthread - not sure it will not break in some subtle ways and work as intended.

Errantry

Works so beautifully like a charm. I am also learning new python tricks from your post besides able to remove the pthread. Thank you so much @Errantry – Somite