std::vector to boost::python::list

Question

I have a method in c++ that gets called from python and needs to return a python list object.

I have already created the method, and its attached to an exposed class and callable from python right now... (it returns void).

So the question is, how do I create a python list from this:

std::vector<std::string> results;

I am not really understanding how the constructor works from this documentation:

http://www.boost.org/doc/libs/1_37_0/libs/python/doc/v2/list.html

Also... I don't really want to return kind of wrapped vector... I just want to create a new python list with the string values from the vector.

My apologies if this is a duplicate... I found quite a few list to vector questions but I couldn't find any about creating a new python list.

I could expand this question to include some other questions like:

Creating a new python dictionary from a: std::map<std::string, std::string> and so on.

Noumenon

I saw this question: #3241471 But that was talking about returning a wrapped vector... I just want to create a new python object and return that. – Noumenon

Possible duplicate: https://mcmap.net/q/373241/-how-to-export-std-vector/198633 – Ronaronal

Answer 1

I have this function using iterators to convert std::vector to py::list:

namespace py = boost::python;

template<class T>
py::list std_vector_to_py_list(const std::vector<T>& v)
{
    py::object get_iter = py::iterator<std::vector<T> >();
    py::object iter = get_iter(v);
    py::list l(iter);
    return l;
}

Windywindzer

doesn't have this a massive overhead? It is really a question – Helfant

I tried this, I get an exception on py::object iter = get_iter(v) : No to_python (by-value) converter found for C++ type: class std::vector<int,class std::allocator<int> > as of Boost 1.51 – Pub

likewise 'No to_python (by-value) converter found for C++ type: std::vector<std::string, std::allocator<std::string> >', boost 1.48 – Bamberg

Same error as well, <type 'exceptions.TypeError'>: No to_python (by-value) converter found for C++ type: std::vector<double, std::allocator<double> > – Cognate

The same error here: No to_python (by-value) converter found for C++ type: std::vector<int, std::allocator<int> > – Flypaper

Same error here with Boost 1.58. Any idea how to solve this? – Bandanna

Answer 2

boost::python already includes functionality for wrapping vectors and maps. Here's sample code for vectors, as you can see both passing and returning lists is quite simple:

// C++ code
typedef std::vector<std::string> MyList;
class MyClass {
  MyList myFuncGet();
  void myFuncSet(const Mylist& list);
  //       stuff
};

// Wrapper code

#include <boost/python/suite/indexing/vector_indexing_suite.hpp>

using namespace boost::python;


BOOST_PYTHON_MODULE(mymodule)
{
    class_<MyList>("MyList")
        .def(vector_indexing_suite<MyList>() );

    class_<MyClass>("MyClass")
        .def("myFuncGet", &MyClass::myFuncGet)
        .def("myFuncSet", &MyClass::myFuncSet)
        ;
}

Maps are very similar to vectors and are described in this post: Boost::Python- possible to automatically convert from dict --> std::map?

Unfortunately boost::python does not currently include facilities for wrapping lists. You can create the wrapper manually, but I'm out of time for this answer. I can post it today or tomorrow. I'd appreciate a new question about this particular problem, because the answer will be quite extensive and is probably outside of the scope of this post. I'd just avoid lists and use vectors instead.

Sludge

hello @Aleksey, I think I have a related question for you link – Wichman

Thank you, by the way, does this method has a large overhead? Sorry, I am a newbie in C++ and boost.python. – Pedestrian

And can you share the link of the corresponding docs of vector_indexing_suite. – Pedestrian

All vector_indexing_suite does is create a python wrapper for std::vector. This means that the std::vector is accessed directly. It's basically the lowest overhead you can hope for. Here's the link: boost.org/doc/libs/1_68_0/libs/python/doc/html/reference/topics/…. – Sludge

Answer 3

If you only want to create python list manually (and have the function return py::list rather than vector), do it like this:

/* using namespace std; namespace py=boost::python;
   #define FOREACH BOOST_FOREACH
*/
vector<string> ss;
py::list ret;
FOREACH(const string& s, ss) ret.append(s);
return s;

For automatic conversions, define the converter for vector from python list to c++ and from c++ to python list -- I just wrote about that at Instantiating shared_ptr's in boost::python (the second part of the reply); that way, you get realy python lists.

Another possibility for automatic conversion (which I have no experience with) is to use indexing_suite, which will wrap vector<string> as a special class in python, as a colleague mentioned here already.

Childs

Answer 4

From http://gist.github.com/octavifs/5362272:

// Converts a C++ vector to a python list
template <class T>
boost::python::list toPythonList(std::vector<T> vector) {
    typename std::vector<T>::iterator iter;
    boost::python::list list;
    for (iter = vector.begin(); iter != vector.end(); ++iter) {
        list.append(*iter);
    }
    return list;
}

Rosaceous

Yes, it works, but I'm not sure if there is too much overhead by having to iterate the vector and append each element. Maybe using a numpy array is more efficient? – Cairn

Answer 5

FWIW, here's a templated function in the same vein as eudoxos' solution:

namespace py = boost::python;

template<class T>
py::list std_vector_to_py_list(const std::vector<T>& v)
{
  py::list l;
  typename std::vector<T>::const_iterator it;
  for (it = v.begin(); it != v.end(); ++it)
    l.append(*it);   
  return l;  
}

Bamberg

This copies the values as it appends to the list, right? Is there any way to avoid that (assuming the values are of some class type), so in python get_list(0) == get_list(0) returns true? Something like bp::reference_existing_object but within the function? – Sweetener

Answer 6

I use following utility functions to convert from/to stl containers. The trivial sum function illustrates how they are used. Also I found following opensource package which has quite a few conversion utilities: https://github.com/cctbx/cctbx_project/tree/master/scitbx/boost_python

#include <vector>
#include <boost/python.hpp>
#include <boost/python/object.hpp>
#include <boost/python/stl_iterator.hpp>

namespace bpy = boost::python;

namespace fm {

template <typename Container>
bpy::list stl2py(const Container& vec) {
  typedef typename Container::value_type T;
  bpy::list lst;
  std::for_each(vec.begin(), vec.end(), [&](const T& t) { lst.append(t); });
  return lst;
}

template <typename Container>
void py2stl(const bpy::list& lst, Container& vec) {
  typedef typename Container::value_type T;
  bpy::stl_input_iterator<T> beg(lst), end;
  std::for_each(beg, end, [&](const T& t) { vec.push_back(t); });
}

bpy::list sum(const bpy::list& lhs, const bpy::list& rhs) {
  std::vector<double> lhsv;
  py2stl(lhs, lhsv);

  std::vector<double> rhsv;
  py2stl(rhs, rhsv);

  std::vector<double> result(lhsv.size(), 0.0);
  for (int i = 0; i < lhsv.size(); ++i) {
    result[i] = lhsv[i] + rhsv[i];
  }
  return stl2py(result);
}

} // namespace fm

BOOST_PYTHON_MODULE(fm)
{
  bpy::def("sum", &fm::sum);
}

Caducous