Can I use C++ features while extending Python?

Question

The Python manual says that you can create modules for Python in both C and C++. Can you take advantage of things like classes and templates when using C++? Wouldn't it create incompatibilities with the rest of the libraries and with the interpreter?

Rauwolfia

Take a look at answers given to https://mcmap.net/q/1769596/-is-the-python-c-api-entirely-compatible-with-c/151641 – Firedrake

Answer 1

It doesn't matter whether your implementation of the hook functions is implemented in C or in C++. In fact, I've already seen some Python extensions which make active use of C++ templates and even the Boost library. No problem. :-)

Superordinate

Answer 2

The boost folks have a nice automated way to do the wrapping of C++ code for use by python.

It is called: Boost.Python

It deals with some of the constructs of C++ better than SWIG, particularly template metaprogramming.

Grossularite

Answer 3

What you're interested in is a program called SWIG. It will generate Python wrappers and interfaces for C++ code. I use it with templates, inheritance, namespaces, etc. and it works well.

Exultant

If you're primarily targeting C++, SWIG is definitely not the tool you want to use. It's convenient that it understands C++ well enough to wrap it if it's a small part of what you're working with, but it can't touch Boost.Python for C++ support. – Vendace

@Nick: I've looked at the Boost::Python library, and I'm not convinced it's superior. All the templating and macro magic that Boost uses can be fragile, and it limits what can be done with the code. – Exultant

Why the downvote? This is a more useful answer than the "answer." – Exultant

@Seth: What parts have you found fragile? And what limits have you run into? I've found Boost.Python very flexible and robust. Don't get me wrong - SWIG is a good library but I would always prefer Boost.Python if I was only interfacing C++ and Python... – Bessbessarabia

Answer 4

You should be able to use all of the features of the C++ language. The Extending Python Documentation (2.6.2) says that you may use C++, but mentions the followings caveats:

It is possible to write extension modules in C++. Some restrictions apply. If the main program (the Python interpreter) is compiled and linked by the C compiler, global or static objects with constructors cannot be used. This is not a problem if the main program is linked by the C++ compiler. Functions that will be called by the Python interpreter (in particular, module initialization functions) have to be declared using extern "C". It is unnecessary to enclose the Python header files in extern "C" {...} — they use this form already if the symbol __cplusplus is defined (all recent C++ compilers define this symbol).

The first restriction, "global or static objects with constructors cannot be used", has to do with the way most C++ compiler initialize objects with this type of storage duration. For example, consider the following code:

class Foo { Foo() { } };

static Foo f;

int main(int argc, char** argv) {}

The compiler has to emit special code so that the 'Foo' constructor gets invoked for 'f' before main gets executed. If you have objects with static storage duration in your Python extension and the Python interpreter is not compiled and linked for C++, then this special initialization code will not be created.

The second restriction, "Functions that will be called by the Python interpreter (in particular, module initialization functions) have to be declared using extern "C"", has to do with C++ name mangling. Most C++ compilers mangle their names so that they can use the same linkers provided for C toolchains. For example say you had:

void a_function_python_calls(void* foo);

the C++ compiler may convert references to the name 'a_function_python_calls' to something like 'a_function_python_calls@1vga'. In which case you may get an unresolved external when trying to link with the Python library.

Customhouse

Of course, these problems are consistent with any C to C++ linkage, and not unique to Python. If you control the build process of the Python interpreter (e.g. you are embedding and extending, not just extending), then you don't need to worry about this, as you can build Python with a C++ compiler and linker. – Vendace