How to build a Python C Extension so I can import it from a module

Question

I have a Python project with many sub-modules that I package up with distutils. I would like to build some Python extensions in C to live in some of these sub-modules but I don't understand how to get the Python extension to live in a submodule. What follows is the simplest example of what I'm looking for:

Here is my Python extension c_extension.c:

#include <Python.h>

static PyObject *
get_answer(PyObject *self, PyObject *args)
{
    return Py_BuildValue("i", 42);
}

static PyMethodDef Methods[] = {
    {"get_answer",  get_answer, METH_VARARGS, "The meaning of life."},
    {NULL, NULL, 0, NULL}
};

PyMODINIT_FUNC
initc_extension(void) {
  (void) Py_InitModule("c_extension", Methods);
}

And here is a setup.py that works:

from distutils.core import setup
from distutils.extension import Extension

setup(name='c_extension_demo',
      ext_modules = [Extension('c_extension', sources = ['c_extension.c'])])

After installing in an virtualenv I can do this:

>>> import c_extension
>>> c_extension.get_answer()
42

But I would like to have c_extension live in a sub-module, say foo.bar. What do I need to change in this pipeline to be able to get the behavior in the Python shell to be like this:

>>> import foo.bar.c_extension
>>> foo.bar.c_extension.get_answer()
42

Fastening

Related: https://mcmap.net/q/552793/-how-to-build-a-python-c-extension-so-i-can-import-it-from-a-module/7954504 – Ultun

For python >= 3.10 , distutils is deprecated, please see my answer in another thread for detail – Rationale

Answer 1

Just change

Extension('c_extension', ...)

to

Extension('foo.bar.c_extension', ...)

You will need __init__.py files in each of the foo and bar directories, as usual. To have these packaged with the module in your setup.py, you need to add

packages = ['foo', 'foo.bar'],

to your setup() call, and you will need the directory structure

setup.py
foo/
    __init__.py
    bar/
        __init__.py

in your source directory.

Hothead

Thanks for the answer but that didn't work for me. I made foo/bar directory with an __init__.py in each plus added the foo.bar. prefix in the setup command but in the python shell this happens: >>> import foo.bar.c_extension Traceback (most recent call last): File "<stdin>", line 1, in <module> ImportError: No module named c_extension – Fastening

What version of Python are you using? I'm testing on 2.7.3. – Hothead

Don't test from your source directory. That will try to import from the foo/bar in the source dir which doesn't have c_extension. – Hothead

Oh RIGHT. That was it. Thank you for the help! – Fastening