Wrap C++ template class with boost python

Question

I am try to wrap a C++ template class with booth python. I get errors with the current wrapper. The program is basically for creating customized vector's and use it in python.

#include <boost/python.hpp>

template <class T> class allocator{
  public:
  T* allocate(size_t);
  void deallocate(T* , size_t);
  void construct(T*,T);
  void destroy(T*);
};


template <class T> class vec{

  typedef T* iterator;
  typedef const T* const_iterator;
  typedef size_t size_type;
  typedef T value_type;


  vec(){ create();  }
  explicit vec(size_type n, const T& t =T()){ create (n,t);}
  vec(const vec& v){ create(v.begin(),v.end());}
  vec& operator= (const vec&);

  ~vec(){uncreate;}


  T& operator[](size_type i){return data[i];}
  const T& operator[]( size_type i) const { return data[i];}

  void push_back(const T& t){
    if (avail==limit)
      grow();
    unchecked_append(t);
  }
  size_type size() const{return avail-data;}

  iterator begin () {return data;}
  const_iterator begin () const {return data;}

  iterator end () {return avail;}
  const_iterator end () const {return avail;}

  iterator data;
  iterator limit;
  iterator avail;

  allocator<T> alloc;

  void uninitialized_fill(T*, T*, const T&);
  T* uninitialized_copy(T*,T*,T*);

  void create();
  void create(size_type,const T&);
  void create (const_iterator, const_iterator);
  void uncreate();

  void grow();
  void unchecked_append(const T&);
};

// create
template <class T> void vec<T>:: create(){
  data = avail = limit = 0;
}

template <class T> void vec<T>::create(size_type n, const T& val){
  data =alloc.allocate(n);
  limit = avail = data + n;
  uninitialized_fill(data, limit, val);
}

template <class T>
void vec<T>::create(const_iterator i, const_iterator j){
  data =  alloc.allocate(j - i);
  limit = avail = uninitialized_copy(i,j, data);
}

//uncreate
template < class T> void vec<T>::uncreate(){
  if (data){
    iterator it = avail;
    while (it != data)
      alloc.destroy(--it);

    alloc.deallocate(data, limit - data);
  }
  data = limit =avail = 0;
}

//grow
template <class T> void vec<T>::grow(){
  size_type new_size = max(2 * (limit-data), ptrdiff_t(1));

  iterator new_data = alloc.allocate(new_size);
  iterator new_avail = uninitialized_copy(data, avail, new_data);

  uncreate();

  data = new_data;
  avail = new_avail;
  limit = data + new_size;
}

template <class T> void vec<T>::unchecked_append(const T& val){
  alloc.construct(avail++,val);
}

the Boost python wrap is as follows:

BOOST_PYTHON_MODULE(vbox_shmem_box_vec_ext)
{
  using namespace boost::python;
  class_<vec>("vec");
    .def("size",&std_item<vec>::size,
      return_value_policy<copy_non_const_reference>());
    .def("begin",&std_item<vec>::begin,
      return_value_policy<copy_non_const_reference>());
    .def("end",&std_item<vec>::end,
      return_value_policy<copy_non_const_reference>());
}

Here are the errors

vec_ext.cpp:113:13: error: type/value mismatch at argument 1 in template parameter list for 'template<class T, class X1, class X2, class X3> class boost::python::class_'
vec_ext.cpp:113:13: error:   expected a type, got 'vec'
vec_ext.cpp:114:5: error: expected primary-expression before '.' token
vec_ext.cpp:114:18: error: 'std_item' was not declared in this scope
vec_ext.cpp:114:30: error: missing template arguments before '>' token
vec_ext.cpp:114:31: error: '::size' has not been declared
vec_ext.cpp:114:31: note: suggested alternative:
size_fwd.hpp:20:38: note:   'boost::mpl::size'
vec_ext.cpp:116:5: error: expected primary-expression before '.' token
vec_ext.cpp:116:31: error: missing template arguments before '>' token
vec_ext.cpp:116:32: error: '::begin' has not been declared
vec_ext.cpp:116:32: note: suggested alternative:
begin_end_fwd.hpp:22:38: note:   'boost::mpl::begin'
vec_ext.cpp:118:5: error: expected primary-expression before '.' token
vec_ext.cpp:118:29: error: missing template arguments before '>' token
vec_ext.cpp:118:30: error: '::end' has not been declared
vec_ext.cpp:118:30: note: suggested alternative:
begin_end_fwd.hpp:23:38: note:   'boost::mpl::end'

Purposely

Please attach the errors to the question. – Collimator

Michael, I attached the errors. – Purposely

Are you #includeing the first file (where you define class vec) in the file where you have your BOOST_PYTHON_MODULE line? – Ambuscade

its all in the same file – Purposely

I kind of wonder how you think that would work? Templates are compile-time (as in C++ compile-time, not Python). Do you expect to be able to write vec<int> in Python? And that that would work? – Frayne

I am trying to create a customized vector class that uses unix shared memory. I am trying to write the vector class in C++ and call it from python – Purposely

Answer 1

The easiest way for your task is to inherit from STL vector interface and use Boost Python vector_indexing_suite, otherwise you had to manually implement slice operations for Python interface that is not so fast and trivial.

But as for the original question Wrap C++ template class then I faced with the next problem when wrapped templates:

template<typename LinksT>
class Base {
public:
  virtual ~Base()  {}
  virtual Base* x() = 0;
};

template<typename LinksT>
class BaseWrap : public Base<LinksT>, public wrapper<Base<LinksT>> {
public:
  virtual Base<LinksT>* x() { return this->get_override("x")(); }
};

BOOST_PYTHON_MODULE(xxx)
{
    class_<BaseWrap<LinksT>, noncopyable>("Base", no_init)
      .def("x", pure_virtual(&Base<LinksT>::x), return_internal_reference<>())
      ;
}

fails to compile on GCC 4.8 under Linux, thought for direct classes it works fine:

class Base {
public:
  virtual ~Base()  {}
  virtual Base* x() = 0;
};

class BaseWrap : public Base, public wrapper<Base> {
public:
  virtual Base* x() { return this->get_override("x")(); }
};

BOOST_PYTHON_MODULE(xxx)
{
    class_<BaseWrap, noncopyable>("Base", no_init)
      .def("x", pure_virtual(&Base::x), return_internal_reference<>())
      ;
}

There were 2 issues:

1. I had to specify -ftemplate-backtrace-limit=64 (default it's 10) compiler flag to allow instantiate more inner templates
2. I had to instantiate template before the BOOST_PYTHON_MODULE(xxx) declaration:

template class BaseWrap<SimpleLinks>;
BOOST_PYTHON_MODULE(xxx)
{ ...

then it works fine.

Martainn

should add boost::python::wrapper to make the code compilable. – Inapt

Answer 2

You can also add another template for your class so that you don't have to write/instantiate for each int/float type.

template<typename LinksT>
class Base {
public:
  virtual ~Base()  {}
  virtual Base* x() = 0;
};

#include <boost/python.hpp>
using namespace boost::python;

template<typename LinksT>
class BaseWrap : public Base<LinksT>, public wrapper<Base<LinksT> > {
public:
  virtual Base<LinksT>* x() { return this->get_override("x")(); }
};

template<typename LinksT>
void export_virtualfunction()
{
    class_<BaseWrap<LinksT>, boost::noncopyable>("Base", no_init)
        .def("x", pure_virtual(&Base<LinksT>::x), return_internal_reference<>())
        ;
}

BOOST_PYTHON_MODULE(test_template_python)
{
    export_virtualfunction<int>();
}

Pay attention to different type of the exported class. Give different names to it. Refer to link

Inapt