What is the best way to deal with "_d" suffix for C extensions when using debug build?

Question

I'm trying to debug my C extension for Python 2.7. I use python2.7 debug build. I build my project with setuptools and my setup.py has such lines:

ext_modules=[Extension("my.extension",
                       ["my/_extension.c"])]

When I invoke python setup.py install for some reason extension compiles to a file with _d suffix, and after that, in Python, I can't do import my.extension, I can only do import my.extension_d. And what I get is:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "build/bdist.linux-x86_64/egg/my/extension_d.py", line 7, in <module>
  File "build/bdist.linux-x86_64/egg/my/extension_d.py", line 6, in __bootstrap__
ImportError: dynamic module does not define init function (initextension_d)

Of course my extension does not have initextension_d, it only has initextension function.

This is very incovenient, because I have to change some code and add this _d suffix to imports and other stuff.

Is it possible to turn off prepending of this suffix? Or how to deal with that problem in other way? Maybe there are some "official" ways?

UPDATE #0

I use Ubuntu Linux.

Menhaden

On what platform do you see this? Your traceback mentions linux. Windows is the only platform that uses _d for debug builds. See my attempt below to give an answer for the Windows platform (which you may not have meant). – Alginate

You should post the output of python setup.py install. – Alginate

And maybe setup.py itself and the value of os.name. I'm also wondering about the .py extension. It should by .pyd or .so ... – Eyla

Answer 1

Just disable debug mode when build C Extension. Or, if you do like to keep debug info, temporary disable _DEBUG macro:

#ifdef _DEBUG
# ifndef BOOST_DEBUG_PYTHON
#  ifdef _MSC_VER  
    // VC8.0 will complain if system headers are #included both with
    // and without _DEBUG defined, so we have to #include all the
    // system headers used by pyconfig.h right here.
#   include <stddef.h>
#   include <stdarg.h>
#   include <stdio.h>
#   include <stdlib.h>
#   include <assert.h>
#   include <errno.h>
#   include <ctype.h>
#   include <wchar.h>
#   include <basetsd.h>
#   include <io.h>
#   include <limits.h>
#   include <float.h>
#   include <string.h>
#   include <math.h>
#   include <time.h>
#  endif
#  undef _DEBUG // Don't let Python force the debug library just because we're debugging.
#  define DEBUG_WRAP_PYTHON_H
# endif
#endif

#include <Python.h>

#ifdef DEBUG_WRAP_PYTHON_H
# define _DEBUG
#endif

For full code sample you may take a look on full version of how boost.python includes python.h.

Preeminent

Answer 2

A comment from the distutils source file of build_ext.py:

# extensions in debug_mode are named 'module_d.pyd' under windows

It's the same for C Extensions, so there shouldn't be any problem. But since there's one, it's also possible to remove the _d suffix:

import os.path
from setuptools.command.build_ext import build_ext as _build_ext

class build_ext(_build_ext):
    def get_ext_filename(self, ext_name):
        fn = _build_ext.get_ext_filename(self, ext_name)
        fn = os.path.splitext(fn)
        if fn[0].endswith('_d'):
            fn[0] = fn[0][:-2]
        return fn[0] + fn[1]

Or just disable debug for a while:

from setuptools.command.build_ext import build_ext as _build_ext

class build_ext(_build_ext):
    def get_ext_filename(self, ext_name):
        debug, self.debug = self.debug, False
        fn = _build_ext.get_ext_filename(self, ext_name)
        self.debug = debug
        return fn

Don't forget to set the cmdclass within setup:

setup(
    ...
    cmdclass={'build_ext': build_ext},
    ...
)

I'm not using Windows myself, so it's just a wild guess, but maybe you're mixing debug and release parts of Python.

Eyla

Answer 3

To solve this problem, you can define in your C module function

void initextension_d()
{ initextension(); }

Derose

@Alginate Why you think so? – Derose

because you will end up having loaded a debug build of the extension into a non-debug build of the interpreter which use mutually incompatible runtime libraries. I have tried to explain that in my answer. – Alginate

@Alginate Quote from your answer: All that said, it is not necessary to use a debug build of Python to debug Python extensions, unless you plan to use the debugging features of msvcrtd.dll Why did you decide that the author works with Visual C++ compiler? – Derose

Apparently because Windows is the only platform where Python extension modules are named differently for debug builds (adding _d). OTOH, you are right, the question mentions linux-x86 in the traceback ... perhaps we should ask the questioner? – Alginate

... and maybe "Gill Bates" made me think "Windows", too ;-) – Alginate

BTW, simply providing multiple entry points is not addressing the root cause of this problem on Linux either. – Alginate