Boost.Serialization and Boost.Python two-way pickle

Question

I have a C++ library that uses Boost.Serialization. I'm creating Python bindings for this library using Boost.Python. It's fairly clear how to make a pickle suite for Boost.Python that uses Boost.Serialization (save to a string using Boost.Serialization, and return that string to Python).

What I want is the reverse: given a boost::python::object, I want to have a serialize(...) function that would call Python's pickle.dumps() function and serialize the resulting string. (Imagine an std::vector<boost::python::object>. As I serialize this vector, Boost.Serialization would call the auxiliary serialize() function.) Is this possible? Better yet, is it possible to use cPickle and bypass giving the control to the Python interpreter?

Mont

Have you found a solution? I am having the same problem now. I think defining a namespace boost{namespace serialization{ template<class Archive> void serialize(Archive& a, boost::python::object& obj, const unsigned version){ /*...*/; } }} should do the trick, but before I try... Thanks! – Orchestrion

For me the bigger issue was how to use the pickle module itself. You can import it using boost::python::import. I eventually opted for a different solution to my specific problem that did not involve this question. The boost::serialization::serialize() function setup should work exactly as you describe. – Mont

Answer 1

Here is the code I use to pickle/unpickle instance of boost::mersenne_twister

typedef boost::mt19937 rng_t;

struct mt_pickle_suite : bp::pickle_suite {

  static bp::object getstate (const rng_t& rng) {
    std::ostringstream os;
    boost::archive::binary_oarchive oa(os);
    oa << rng;
    return bp::str (os.str());
  }

static void
  setstate(rng_t& rng, bp::object entries) {
    bp::str s = bp::extract<bp::str> (entries)();
    std::string st = bp::extract<std::string> (s)();
    std::istringstream is (st);

    boost::archive::binary_iarchive ia (is);
    ia >> rng;
  }
};

Luminesce

That's the wrong way; I understand how to pickle a C++ class through a stringstream. I want the other way around: use bp::object in Boost.Serialization (using Python's pickle). – Mont

Answer 2

If your object is a native python object, then pickle.dumps() will do the trick.

In the other hand, if you have a std::vector, then you are mixing c++ and python. For that you will need to register the std::vector like:

using PyVec = std::vector<boost::python::object>;
boost::python::_class<PyVec, boost::noncopyable>
    ("PyVec", "My vector of PyObjects", boost::python::init<>() )
    .enable_pickling()
    .def_pickle( py_vec_pickle_suit() );

Then, of course you will need to define a pickle suit. But that is too cumbersome.

TL;DR: Use boost::python::list instead and call dump over it.

Foppish