Exposing C++ interface in boost python

Question

Sample code to illustrate:

struct Base
{
  virtual int foo() = 0;
};

struct Derived : public Base
{
  virtual int foo()
  {
    return 42;
  }
};

Base* get_base()
{
  return new Derived;
}

BOOST_PYTHON_MODULE(libTestMod)
{
  py::class_<Base>("Base", py::no_init)
    .def("foo", py::pure_virtual(&Base::foo));

  py::def("get_base", get_base, py::return_internal_reference<>()); //ignore mem leak
}

• Base::foo will not be overridden in python
• Base:foo will be implemented in c++ but that should not be exposed to python

Tried the above code but fails to compile.

update: Compilation Error:

/path/to/boostlib/boost/1.53.0-0/common/include/boost/python/object/value_holder.hpp:66:11: error: cannot declare field 'boost_1_53_0::python::objects::value_holder<Base>::m_held' to be of abstract type 'Base'
Main.C:59:8: note:   because the following virtual functions are pure within 'Base':
Main.C:61:15: note:         virtual int Base::foo()

Condescending

Would you mind to show the compilation errors, or what's your question? – Manley

Included compilation errors. – Condescending

Obviously py::class_<Base>("Base", py::no_init) wants to create an instance of Base, which isn't possible because Base is abstract. I'm not experienced with boost python though, can't tell what's wrong in detail. – Manley

Answer 1

Abstract C++ classes cannot be exposed in this manner to Boost.Python. The Boost.Python tutorial gives examples as to how to expose pure virtual functions. In short, when decorating methods with boost::python::pure_virtual, a wrapper type needs to be created to allow C++ to polymorphic resolve the virtual function, and the virtual function implementation will delegate resolving the function polymorphically in the Python object's hierarchy.

struct BaseWrap : Base, boost::python::wrapper<Base>
{
  int foo()
  {
    return this->get_override("foo")();
  }
};

...

boost::python::class_<BaseWrap>("Base", ...)
  .def("foo", boost::python::pure_virtual(&Base::foo))
  ;

For details, when a type is exposed via boost::python::class_, HeldType defaults to the type being exposed, and the HeldType is constructed within a Python object. The class_ documentation states:

Template Parameter:

• T: The class being wrapped
• HeldType: Specifies the type that is actually embedded in a Python object wrapping a T instance [...]. Defaults to T.

Hence, the boost::python::class_<Base> will fail, because T = Base and HeldType = Base, and Boost.Python will try to instantiate an object of HeldType into a Python object that represents an instance of Base. This instantiation will fail as Base is an abstract class.

---

Here is a complete example showing the use of a BaseWrap class.

#include <boost/python.hpp>

struct Base
{
  virtual int foo() = 0;
  virtual ~Base() {}
};

struct Derived : public Base
{
  virtual int foo()
  {
    return 42;
  }
};

Base* get_base()
{
  return new Derived;
}

namespace python = boost::python;

/// @brief Wrapper that will provide a non-abstract type for Base.
struct BaseWrap : Base, python::wrapper<Base>
{
  BaseWrap() {}

  BaseWrap(const Base& rhs)
    : Base(rhs)
  {}

  int foo()
  {
    return this->get_override("foo")();
  }
};

BOOST_PYTHON_MODULE(example)
{
  python::class_<BaseWrap>("Base")
    .def("foo", python::pure_virtual(&Base::foo));
    ;

  python::def("get_base", &get_base,
              python::return_value_policy<python::manage_new_object>());
}

and its usage:

>>> import example
>>> class Spam(example.Base):
...     pass
... 
>>> Spam().foo()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: Pure virtual function called
>>> class Egg(example.Base):
...     def foo(self):
...         return 100
... 
>>> e = Egg()
>>> e.foo()
100
>>> d = example.get_base()
>>> d.foo()
42

---

It is possible to expose an abstract class in Boost.Python by exposing it with no default initializer (boost::python::no_init) and non-copyable (boost::noncopyable). The lack of an initializer prevents Python types from deriving from it effectively preventing overriding. Additionally, the implementation detail that Base::foo() is implemented within C++ by Derived is inconsequential. If Python should not know about a foo() method at all, then omit exposing it via def().

#include <boost/python.hpp>

struct Base
{
  virtual int foo() = 0;
  virtual ~Base() {}
};

struct Derived
  : public Base
{
  virtual int foo()
  {
    return 42;
  }
};

struct OtherDerived
  : public Base
{
  virtual int foo()
  {
    return 24;
  }
};

Base* get_base()
{
  return new Derived;
}

Base* get_other_base()
{
  return new OtherDerived;
}

BOOST_PYTHON_MODULE(example)
{
  namespace python = boost::python;
  python::class_<Base, boost::noncopyable>("Base", python::no_init)
    ;

  python::class_<Derived, python::bases<Base> >("Derived", python::no_init)
    .def("foo", &Base::foo)
    ;

  python::class_<OtherDerived, python::bases<Base> >(
      "OtherDerived", python::no_init)
    ;

  python::def("get_base", &get_base,
              python::return_value_policy<python::manage_new_object>());

  python::def("get_other_base", &get_other_base,
              python::return_value_policy<python::manage_new_object>());
}

Interactive usage:

>>> import example
>>> b = example.get_base()
>>> b.foo()
42
>>> b = example.get_other_base()
>>> b.foo()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'OtherDerived' object has no attribute 'foo'

Iyeyasu

The above solution works only if all of Base's pure virtual functions are exposed in python. Is there a way to expose only a subset of Base's functions? – Condescending

@balki: The type exposed above has no pure virtual functions, and exposing a type neither requires nor exposes members functions. If foo() should not be exposed, then do not expose it via def() on the class_. If the type returned from get_base() needs a narrower interface than the one provided by BaseWrap, then expose Derived as a type with only the desired methods, and the default Bases template parameter. – Iyeyasu

I see that we can avoid exposing in python by not specifying in def. However it requires BaseWrap to implement all the pure virtual functions. i.e rewrite Base's interface with dummy impl. I guess there is no better way. – Condescending

Answer 2

Abstract classes actually CAN be exposed to Boost.Python without wrappers. The trick is defining your class with boost::noncopyable and avoiding pure_virtual method wrappers.

Here is the corrected code (tested with Boost.Python 1.47.0 and Python 2.7.6):

#include <boost/python/class.hpp>
#include <boost/python/def.hpp>
#include <boost/python/module.hpp>

struct Base
{
  virtual int foo() = 0;
};

struct Derived : public Base
{
  virtual int foo()
  {
    return 42;
  }
};

Base* get_base()
{
  return new Derived;
}

BOOST_PYTHON_MODULE(libTestMod)
{
    namespace py = boost::python;

    py::class_<Base, boost::noncopyable>("Base", py::no_init)
        .def("foo", &Base::foo);

    py::def("get_base", get_base,
        py::return_value_policy<py::reference_existing_object>()); //ignore mem leak
}

Testing:

$ python
Python 2.7.6 (default, Mar 31 2014, 16:04:58) 
[GCC 4.7.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import libTestMod
>>> base = libTestMod.get_base()
>>> print base.foo()
42

Ricercar

How to I also export the derived class? If I do it with: py::class_<Derived, boost::noncopyable>("Derived", py::init<>()).def("foo", &Derived::foo); I get Traceback (most recent call last): File "code.py", line 3, in <module> print base.foo() Boost.Python.ArgumentError: Python argument types in Derived.foo(Derived) did not match C++ signature: foo(Derived {lvalue}) – Foulness

Export derived classes as: py::class_<Derived, py::bases<Base> >("Derived"); The foo() method will be inherited automatically. – Ricercar