Hello world with boost python and python 3.2

Question

So I'm trying to interface python 3.2 and c++ using boost python, and have come across many many issues. I've finally gotten it to compile using the 2.7 libraries and it works, but I can't seem to make it work with python 3.2.

Here's the c++ code

#include <iostream>

using namespace std;

void say_hello(const char* name) {
    cout << "Hello " <<  name << "!\n";
}

int main(){return 0;}

#include <boost/python/module.hpp>
#include <boost/python/def.hpp>
using namespace boost::python;

BOOST_PYTHON_MODULE(hello)
{
    def("say_hello", say_hello);
}

If I compile it using the 2.7 libraries it works just fine, but when I use the 3.2 libraries I get tons of undefined references from libboost_python.so

Otherwise I wrote a little bit of python to make it work:

from distutils.core import setup
from distutils.extension import Extension

setup(name="PackageName",
    ext_modules=[
        Extension("hello", ["testBoost.cpp"],
        libraries = ["boost_python"])
    ])

and this will create an so using python 3.2 or 2.7 build, but when I open the python 3 interpreter and attempt to import the so it give me the error undefined symbol PyClass_Type from libboost_python.so again. Any ideas? Is boost python compatible with python 3.x?

If the information is useful, here is my attempted compile using 3.2:

   $ g++ testBoost.cpp -I/usr/include/python3.2 -I/usr/local/include/boost/python -lboost_python -lpython3.2mu 
    /tmp/ccdmU1Yu.o: In function `PyInit_hello':
    testBoost.cpp:(.text+0xc2): undefined reference to `boost::python::detail::init_module(PyModuleDef&, void (*)())'
    /usr/local/lib/libboost_python.so: undefined reference to `PyString_Size'
    /usr/local/lib/libboost_python.so: undefined reference to `PyFile_FromString'
    /usr/local/lib/libboost_python.so: undefined reference to `PyString_Type'
    /usr/local/lib/libboost_python.so: undefined reference to `PyInt_Type'
    /usr/local/lib/libboost_python.so: undefined reference to `PyString_FromString'
    /usr/local/lib/libboost_python.so: undefined reference to `PyString_FromStringAndSize'
    /usr/local/lib/libboost_python.so: undefined reference to `Py_InitModule4_64'
    /usr/local/lib/libboost_python.so: undefined reference to `PyString_FromFormat'
    /usr/local/lib/libboost_python.so: undefined reference to `PyNumber_Divide'
    /usr/local/lib/libboost_python.so: undefined reference to `PyNumber_InPlaceDivide'
    /usr/local/lib/libboost_python.so: undefined reference to `PyInt_AsLong'
    /usr/local/lib/libboost_python.so: undefined reference to `PyString_InternFromString'
    /usr/local/lib/libboost_python.so: undefined reference to `PyClass_Type'
    /usr/local/lib/libboost_python.so: undefined reference to `PyString_AsString'
    /usr/local/lib/libboost_python.so: undefined reference to `PyInt_FromLong'
    /usr/local/lib/libboost_python.so: undefined reference to `PyFile_AsFile'
    collect2: ld returned 1 exit status

And the error from the python 3 interpreter is

File "<stdin>", line 1, in <module>
ImportError: /usr/local/lib/libboost_python.so.1.47.0: undefined symbol: PyClass_Type

Thanks for any help!

Schlenger

You might want to consider looking into SWIG rather than Boost.Python, if you are able. It requires much less boilerplate code and I've gotten it to work with Python3 pretty easily before. – Disadvantage

@Disadvantage I'm not sure what boilerplate code you're talking about; my boost/python works swimmingly with only 5 lines of extra code. – Koziel

Answer 1

Although this discussion old, just for the record: Modify project-config.jam to change the python version to your setup

# Python configuration
using python : 3.4 : /usr ;

Then build boost:

./b2 clean
./b2 --with-python link=static cxxflags="-std=c++11 -fPIC" variant=release stage
./b2 --with-python link=static cxxflags="-std=c++11 -fPIC" variant=release install

The later command requires super user privileges. Then move to the folder containing C++ code for the extension:

g++ -std=c++11 hellopy.cpp -I/usr/include/python3.4 -I/usr/local/include/boost/python -lboost_python3  -o hello.so -shared -fPIC

You can then import hello into your python environment.

Endogen

Answer 2

I had the exact same problem, with Ubuntu 12.04. I installed the 1.48 version of the library and had to link with libboost_python-py32.so instead of libboost_python.so After this the linker errors was gone.

Brnaby

Answer 3

The above c++ code compiles into a module with

$ g++ testBoost.cpp -I/usr/include/python3.2 -I/usr/local/include/boost/python -lboost_python3 -lpython3.2mu -o hello.so -shared

This compile command adds -lboost_python3, and -shared, and also the naming convention for python extension modules. You should also install the python3-dev package, and configure/build/install boost with python3, if you haven't already.

In python 3, I can then do the following:

$ python3
Python 3.2 (r32:88445, Mar 25 2011, 19:28:28) 
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import hello
>>> hello.say_hello('bill')
Hello bill!
>>>

You should be off to the races at that point.

Yt

I tried using -lboost_python3 but still get the same error, and I've already changed my jam file to point to python 3 – Schlenger

Okay so I compiled with your instructions, and it told me I needed to recompile with -fpic, which worked, and I ended up with the .so. Now however, when I try to import in python, I get an error that says: ImportError: /usr/local/lib/libboost_python3.so.1.47.0: undefined symbol: PyClass_Type – Schlenger

Answer 4

Linking in the python libraries (for e.g. -L/usr/lib/x86_64-linux-gnu -lpython2.7 on linux or find_package(PythonLibs) in CMake¹) will make this linker issue go away.

Here is a more detailed explanation of the issue below. On the command line,

$ nm --dynamic <path-to>/libboost_python.so | grep PyString_Size

If you are feeling lazy and assuming your libboost_python is linking to python2.7, just run this

$ nm --dynamic `locate libboost_python27.so | awk 'NR==1'` | grep PyString_Size

You should see something like

U PyString_Size

So PyString_Size is undefined (U) in libboost_python27.so. This is what the linker was complaining about. We've confirmed that. Now let's look for this symbol in libpython.

$ nm --dynamic `locate libpython2.7.so | awk 'NR==1'` | grep PyString_Size

On my machine, I saw something like this:

00000000000f0530 T PyString_Size

The T indicates that the text for this symbol is at the address indicated. So this is proof that we were not linking in libpython in addition to libboost_python.

¹ Why aren't you using CMake? :)

Ferret

Have any idea on how to get libboost_python to link statically with libpython during a boost build ?? – Somnambulate