protobuf serialization of language specific data structures

Question

Using google's Protocul Buffers, I have a service already written in Java which has its own data structures already. I'd like to use pb to delivering messages and I'm looking for a way to serialize the existing data structures that I have in Java to pb. I can start by defining all the data structures in pb from scratch, which is probably the right way to go but I'm too lazy. So, say I have a Person class in Java (or other supported languages) or a Plane class which has tens of attributes in it, is there a way to serialize that class to pb? Can I have a pb attribute of type Plane? (when Plane is not a pb, it's a Java class)

Answer 1

No, you can't. Fields in protobuf messages are always the primitives (numbers, strings and byte arrays, basically), protobuf enums (which are generated as Java enums) or protobuf messages - and repeated versions of all of those, of course.

You could potentially write a tool which used reflection to create a .proto file from a Java class, but I suspect you'd find it quicker just to do it by hand. In particular, if you did use reflection you'd want to make sure that the fields were always generated with the same name, to maintain compatibility. One thing you could do is annotate the Java classes and write code to generate the .proto file based on those annotations - or even potentially serialize directly to proto format using the annotations. Personally I'd recommend creating the .proto file in some way rather than effectively rewriting the PB project - otherwise there's a significant risk of introducing bugs where there's already thoroughly tested code.

If you do create an annotation system, I'm sure Kenton Varda (and the rest of the PB community) would be interested in seeing it.

Answer 2

One way I can think of is to have a string field in a protobuf and serialize a Java class to that field using Java's primitive serialization. That way, assuming the receiver of the message knows how to read/deserialize it, I can easily serialize Java to Java messages.

There are downsides to this technique, though. To name a few:

1. It's only Java to Java (no C++, Python or others)
2. It's not as efficient as native protobufs are (neither parsing/serializatin wise nor message size wise)
3. You have the logic of the data structures scattered around in several places, some are in the protobufs definition file, some in other Java classes and this makes things harder to maintain.

But - it gets the job done for the short term.