Performance of Serialized Objects in C++

Question

Hi Everyone,

I'm wondering if there is a fast way to dump an STL set to disk and then read it back later.

The internal structure of a set is a binary tree, so if I serialize it naively, when I read it back the program will have to go though the process of inserting each element again. I think this is slow even if it is read back in correct order, correct me if I am wrong.

Is there a way to "dump" the memory containing the set into disk and then read it back later? That is, keep everything in binary format, thus avoiding the re-insertion.

Do the boost serialization tools do this?

Thanks!

EDIT: oh I should probably read, http://www.parashift.com/c++-faq-lite/serialization.html I will read it now... no it doesn't really help

Answer 1

No, and if you are actually reading it back from a hard disk (or probably any permanent storage), the mechanical part will be the bottleneck.

If you put the container in a contiguous block of memory, that block must have some free space, and reading that wasted space from the disk wastes time… and disk space.

This is classic premature optimization.

If you really find yourself needing it, Boost Interprocess has (relatively) serialization-friendly containers.

Answer 2

As each set element is somewhere on the heap, you cannot just dump the structure to disk. You therefore need a proper serialization routine that goes through each element.

To read the element back again, you can use "hints", which allow you to hint to the insert method where the element should be inserted. This can bring the construction of the set back to linear complexity instead of n log n.