Direct boost serialization to char array

Question

Boost serialization doc's assert that the way to serialize/deserialize items is using a binary/text archive with a stream on the underlying structure. This works fine if I wan't to use the serialized data as an std::string, but my intention is to convert it directly to a char* buffer. How can I achieve this without creating a temporary string?

Solved! For the ones that wanted a example:

char buffer[4096];

boost::iostreams::basic_array_sink<char> sr(buffer, buffer_size);  
boost::iostreams::stream< boost::iostreams::basic_array_sink<char> > source(sr);

boost::archive::binary_oarchive oa(source);

oa << serializable_object; 

Spinoza

On the other hand, why would you relinquish the free memory management and risk leak and overrun :) ? – Mendive

Sadly, because of performance issues :( – Spinoza

Answer 1

IIUC, you would like to write to a preallocated array of fixed size.

You could use a boost::iostreams::array_sink (wrapped with stream to give it an std::ostream interface) for that.

Strawberry

@op: Thanks, that worked like a charm! I just wanted that boost had more documentation :( – Spinoza

Answer 2

If you do not know the size of the data you are sending in advance, this is a generic way to serialize into an std::string:

// serialize obj into an std::string
std::string serial_str;
boost::iostreams::back_insert_device<std::string> inserter(serial_str);
boost::iostreams::stream<boost::iostreams::back_insert_device<std::string> > s(inserter);
boost::archive::binary_oarchive oa(s);

oa << obj;

// don't forget to flush the stream to finish writing into the buffer
s.flush();

// now you get to const char* with serial_str.data() or serial_str.c_str()

To deserialize, use

// wrap buffer inside a stream and deserialize serial_str into obj
boost::iostreams::basic_array_source<char> device(serial_str.data(), serial_str.size());
boost::iostreams::stream<boost::iostreams::basic_array_source<char> > s(device);
boost::archive::binary_iarchive ia(s);
ia >> obj;

This works like a charm, I use this to send data around with MPI.

This can be done very fast if you keep the serial_str in memory, and just call serial_str.clear() before you serialize into it. This clears the data but does not free any memory, so no allocation will happen when your next serialization data size does not require it.

Liliuokalani

Great answer! I'm attemping to keep the buffer in memory as you suggest, but it seems calling serial_str.clear() is not enough - do I also need to reset to binary_orachive or the back_inserter_device somehow, to tell it to go back to the writing in the beginning of serial_str? – Adena

@omer, in my code I have just std::string serial_str as a member variable, and re-create everything else every time I send something. – Liliuokalani

How do i get the size of serial_str.data()? serial_str.size() does not work, because it cuts off at the first NULL character right? – Simms

Nope, std::string is not null terminated, so size() works as intended. – Scandic

@Liliuokalani boost::iostreams::stream does not even resolve. Is this answer too outdated perhaps? – Atheist

@Atheist sorry, I have no idea, I didn't need this since 2011 – Liliuokalani

Check my answer below – Glabrate

Answer 3

Simpler version than accepted answer using std::stringstream :

// access data with .data() and size with .size()
using RawDataBuffer = std::string;

RawDataBuffer serialize(const Foo &obj) {
    std::stringstream ss;
    boost::archive::binary_oarchive oa(ss);

    oa << obj;

    return ss.str();
}

Foo deserialize(const RawDataBuffer &data) {
    std::stringstream ss(data);
    boost::archive::binary_iarchive ia(ss);

    Foo obj; // Foo must be default-constructible
    ia >> obj;

    return obj;
}

---

---

Full working example compiled with boost 1.66 :

#include <boost/archive/binary_iarchive.hpp>
#include <boost/archive/binary_oarchive.hpp>
#include <boost/serialization/access.hpp>
#include <sstream>
#include <iostream>


class Foo {
public:
    Foo() = default;
    Foo(int i) : _i(i)
    {}

    int get() const
    { return _i; }

protected:
    friend class boost::serialization::access;

    template<class Archive>
    void serialize(Archive &ar, const unsigned int /* version */ )
    {
        ar & _i;
    }

private:
    int _i;
};


// access data with .data() and size with .size()
using RawDataBuffer = std::string;

RawDataBuffer serialize(const Foo &obj) {
    std::stringstream ss;
    boost::archive::binary_oarchive oa(ss);

    oa << obj;

    return ss.str();
}

Foo deserialize(const RawDataBuffer &data) {
    std::stringstream ss(data);
    boost::archive::binary_iarchive ia(ss);

    Foo obj; // Foo must be default-constructible
    ia >> obj;

    return obj;
}


int main()
{
    RawDataBuffer buff;

    {
        Foo fortyTwo(42);

        buff = serialize(fortyTwo);
    }
    {
        Foo reborn;

        reborn = deserialize(buff);

        std::cout << "Reborn from " << reborn.get() << std::endl;
    }
}

Glabrate

ss.str() returns a copy of the serialised data. It's very inefficient. That changes in C++20 – Roslynrosmarin