Binary parser or serialization ?

Question

I want to store a graph of different objects for a game, their classes may or may not be related, they may or may not contain vectors of simple structures.

• I want parsing operation to be fast, data can be pretty big.
• Adding new things should not be hard, and it should not break backward compatibility.
• Smaller file size is kind of important
• Readability counts

By serialization I mean, making objects serialize themselves, which is effective, but I will need to write different serialization methods for different objects for that.

By binary parsing/composing I mean, creating a new tree of parsers/composers that holds and reads data for these objects, and passing this around to have my objects push/pull their data.

I can also use json, but it can be pretty slow for reading, and it is not very size effective when it comes to pretty big sets of matrices, and numbers.

Answer 1

Check out protocol buffers from Google or thrift from Apache. Although billed as a way to write wire protocols easily, it's basically an object serialization mechanism that can create bindings in a dozen languages, has efficient binary representation, easy versioning, fast performance, and is well-supported.

Answer 2

We're using Boost.Serialization. Don't know how it performs next to those offered by samkass.

Answer 3

Point by point:

• Fast Parsing: binary (since you don't necessarily have to "parse", you can just deserialize)
• Adding New Things: text
• Smaller: text (even if gzipped text is larger than binary, it won't be much larger).
• Readability: text

So that's three votes for text, one point for binary. Personally, I'd go with text for everything except images (and other data which is "naturally" binary). Then, store everything in a big zip file (I can think of several games do this or something close to it).

Good reads: The Importance of Being Textual and Power Of Plain Text.