C/C++ Serialize Fast : Boost vs Cpickle vs Json vs Protocol buffer

Question

Hi,

I need to serialize C/C++ structure in a binary string,very fast.

Env = Windows,Boost 1.44,Python 2.4.

We have 3 structures differents to serialize: Basic : int,double,long,float,string (or char*) Vector: - Each element can be : Basic or Vector or a Map --> vector< Basic,Vector,Map >
Map: - Each Value element can be : Basic or Vector or a Map --> map Basic or List or Map >

I try with cPickle to serialize structures CPython define as above. Time to serialize : 1,5 s I try the same thing with boost 1.44 with different class: - with polymorphisme (pointer) and virtual function -> 35 s - with boost::variant --> 7s

I can't explain how the difference between Boost en Cpickle is too big. I note, the serialize time of boost to serialize an vector and vector is a factor 10. ( TimeSerialize (vector) = 10 * TimeSerialize(vector) So my idea, was to use boost::variant to avoid pointer. But is slow.

I didn't try protocole Buffer and JsonCpp. I try to developp all in C++ and not use Python. But for the moment Python is 5 faster than C++ to serialize.

If someone can help me.

Answer 1

Boost's design goals don't include being the fastest. I would guess protobuf would be faster but it's harder to use. I just did serialization code for my own project. I did something similar to what was implemented in MFC. It's reasonably speedy without a lot of overhead. If you really need speed roll your own like this.