I'm looking into a mechanism for serialize data to be passed over a socket or shared-memory in a language-independent mechanism. I'm reluctant to use XML since this data is going to be very structured, and encoding/decoding speed is vital. Having a good C API that's liberally licensed is important, but ideally there should be support for a ton of other languages. I've looked at google's protocol buffers and ASN.1. Am I on the right track? Is there something better? Should I just implement my own packed structure and not look for some standard?
views:
890answers:
9
+7
A:
Given your requirements, I would go with Google Protocol Buffers. It sounds like it's ideally suited to your application.
Greg Hewgill
2008-09-27 20:47:58
The problem I have with Protocol Buffers is that the C interface is positively awful. I'm leaning towards asn1c at the moment.
bmdhacks
2008-09-29 14:28:37
+2
A:
You could consider XDR. It has an RFC. I've used it and never had any performance problems with it. It was used in ONC RPC and has an and comes with a tool called rpcgen. It is also easy to create a generator yourself when you just want to serialize data (which is what I ended up doing for portability reasons, took me half a day). There is an open source C implementation, but it can already be in a system library, so you wouldn't need the sources.
ASN.1 always seemed a bit baroque to me, but depending on your actual needs might be more appropriate, since there are some limitations to XDR.
Hans van Eck
2008-09-27 21:21:25
I agree Sun XDR works well for all the platforms and OS we have used it on.
David Allan Finch
2008-12-23 11:14:55
A:
There is also Binary XML but it seems not stabilized yet. The article I link to gives a bunch of links which might be of interest.
PhiLho
2008-09-27 22:03:37
A:
JSON is really my favorite for this kind of stuff. I have no prior experience with binary stuff in it though. Please post your results if you are planning on using JSON!
SchizoDuckie
2008-09-27 23:14:11
I'm unfortunately going to have to rule JSON out due to the parsing overhead. I'm really looking for formats that store the data in binary that does not have to be parsed.
bmdhacks
2008-09-27 23:48:46
A:
Thrift is a binary format created by Facebook. Here's a comparison with google protocol buffers.
bmdhacks
2008-09-27 23:46:44
A:
Also check out Muscle. While it does quite a bit, it serializes to a binary format.
Joel Lucsy
2008-09-28 00:12:13
+1
A:
Just wanted to throw in ASN.1 into this mix. ASN.1 is a format standard, but there's libraries for most languages, and the C interface via asn1c is much cleaner than the C interface for protocol buffers.
bmdhacks
2008-09-29 17:12:19