Direct boost serialization to char array

Question

Boost serialization doc's assert that the way to serialize/deserialize items is using a binary/text archive with a stream on the underlying structure. This works fine if I wan't to use the serialized data as an std::string, but my intention is to convert it directly to a char* buffer. How can I achieve this without creating a temporary string?

Solved! For the ones that wanted a example:

char buffer[4096];

boost::iostreams::basic_array_sink<char> sr(buffer, buffer_size);  
boost::iostreams::stream< boost::iostreams::basic_array_sink<char> > source(sr);

boost::archive::binary_oarchive oa(source);

oa << serializable_object;

On the other hand, why would you relinquish the free memory management and risk leak and overrun :) ? — Matthieu M.
– Matthieu M., Commented Jun 10, 2010 at 15:10

martinus · Accepted Answer · 2017-09-22 09:01:57Z

35

If you do not know the size of the data you are sending in advance, this is a generic way to serialize into an std::string:

// serialize obj into an std::string
std::string serial_str;
boost::iostreams::back_insert_device<std::string> inserter(serial_str);
boost::iostreams::stream<boost::iostreams::back_insert_device<std::string> > s(inserter);
boost::archive::binary_oarchive oa(s);

oa << obj;

// don't forget to flush the stream to finish writing into the buffer
s.flush();

// now you get to const char* with serial_str.data() or serial_str.c_str()

To deserialize, use

// wrap buffer inside a stream and deserialize serial_str into obj
boost::iostreams::basic_array_source<char> device(serial_str.data(), serial_str.size());
boost::iostreams::stream<boost::iostreams::basic_array_source<char> > s(device);
boost::archive::binary_iarchive ia(s);
ia >> obj;

This works like a charm, I use this to send data around with MPI.

This can be done very fast if you keep the serial_str in memory, and just call serial_str.clear() before you serialize into it. This clears the data but does not free any memory, so no allocation will happen when your next serialization data size does not require it.

edited Sep 22, 2017 at 9:01

answered Apr 9, 2011 at 11:37

martinus

18.1k18 gold badges75 silver badges92 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

Omer Raviv Over a year ago

Great answer! I'm attemping to keep the buffer in memory as you suggest, but it seems calling serial_str.clear() is not enough - do I also need to reset to binary_orachive or the back_inserter_device somehow, to tell it to go back to the writing in the beginning of serial_str?

martinus Over a year ago

@omer, in my code I have just std::string serial_str as a member variable, and re-create everything else every time I send something.

Prasanna Over a year ago

How do i get the size of serial_str.data()? serial_str.size() does not work, because it cuts off at the first NULL character right?

Stacker Over a year ago

Nope, std::string is not null terminated, so size() works as intended.

Yanis.F Over a year ago

Check my answer below

|

Éric Malenfant · Accepted Answer · 2010-06-10 15:30:50Z

6

IIUC, you would like to write to a preallocated array of fixed size.

You could use a boost::iostreams::array_sink (wrapped with stream to give it an std::ostream interface) for that.

answered Jun 10, 2010 at 15:30

Éric Malenfant

14.2k42 silver badges42 bronze badges

1 Comment

scooterman Over a year ago

@op: Thanks, that worked like a charm! I just wanted that boost had more documentation :(

Yanis.F · Accepted Answer · 2019-09-26 17:16:36Z

Simpler version than accepted answer using `std::stringstream` :

// access data with .data() and size with .size()
using RawDataBuffer = std::string;

RawDataBuffer serialize(const Foo &obj) {
    std::stringstream ss;
    boost::archive::binary_oarchive oa(ss);

    oa << obj;

    return ss.str();
}

Foo deserialize(const RawDataBuffer &data) {
    std::stringstream ss(data);
    boost::archive::binary_iarchive ia(ss);

    Foo obj; // Foo must be default-constructible
    ia >> obj;

    return obj;
}

Full working example compiled with boost 1.66 :

#include <boost/archive/binary_iarchive.hpp>
#include <boost/archive/binary_oarchive.hpp>
#include <boost/serialization/access.hpp>
#include <sstream>
#include <iostream>


class Foo {
public:
    Foo() = default;
    Foo(int i) : _i(i)
    {}

    int get() const
    { return _i; }

protected:
    friend class boost::serialization::access;

    template<class Archive>
    void serialize(Archive &ar, const unsigned int /* version */ )
    {
        ar & _i;
    }

private:
    int _i;
};


// access data with .data() and size with .size()
using RawDataBuffer = std::string;

RawDataBuffer serialize(const Foo &obj) {
    std::stringstream ss;
    boost::archive::binary_oarchive oa(ss);

    oa << obj;

    return ss.str();
}

Foo deserialize(const RawDataBuffer &data) {
    std::stringstream ss(data);
    boost::archive::binary_iarchive ia(ss);

    Foo obj; // Foo must be default-constructible
    ia >> obj;

    return obj;
}


int main()
{
    RawDataBuffer buff;

    {
        Foo fortyTwo(42);

        buff = serialize(fortyTwo);
    }
    {
        Foo reborn;

        reborn = deserialize(buff);

        std::cout << "Reborn from " << reborn.get() << std::endl;
    }
}

ss.str() returns a copy of the serialised data. It's very inefficient. That changes in C++20

Collectives™ on Stack Overflow

Direct boost serialization to char array

3 Answers 3

7 Comments

1 Comment

Simpler version than accepted answer using `std::stringstream` :

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

7 Comments

1 Comment

Simpler version than accepted answer using std::stringstream :

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related

Simpler version than accepted answer using `std::stringstream` :