How to use protocol buffers ?

Question

Hi ,

Could someone please help and tell me how to use protocol buffers. Actually I want to exchange data through sockets between a program running on unix and anoother running on windows in order to run simulation studies.

The programs that use sockets to exchange data, are written in C/C++ and I would be glad if somneone could help me to use protocol buffers in order to exchange data in the form of :

struct snd_data{
    char *var="temp";
    int var1=1;
    float var2;
    double var2;
}

I tried several ways, but still data are not exchanged correctly. Any help would be very appreciated

Thanks for your help,

Answer 1

Are both machines x86? Otherwise you need to watch for big endian and little endian differences. Its also worth paying attention to struct packing. Passing pointer can be problematic too due to the fact pointer are different sizes on different platforms. All in there is far too little information in your post to say, for certain, what is going wrong ...

Answer 2

The answer lies in the endianess of the data being transmitted, this is something you need to consider very carefully and check. Look here to show what endianness can do and cause data to get messed up on both the receiver and sender. There is no such perfect measure of transferring data smoothly, just because data sent from a unix box guarantees the data on the windows box will be in the same order in terms of memory structure for the data. Also the padding of the structure on the unix box will be different to the padding on the windows box, it boils down to how the command line switches are used, think structure alignment.

Hope this helps, Best regards, Tom.

Answer 3

You start by defining your message in a .proto file:

package foo;

message snd_data {
  required string var= 1;
  required int32 var1 = 2;
  optional float var2 = 3;
  optional double var3 = 4;
}

(I guess the float and double actually are different variables...)

Then you compile it using protoc and then you have code implementing your buffer.

For further information see: http://code.google.com/apis/protocolbuffers/docs/cpptutorial.html

Answer 4

How are you writing your messages to the socket? Protobufs is not endian-sensitive itself, but neither does protobufs define a transport mechanism -- protobuf defines a mapping between a message and its serialized form (which is a sequence of (8-bit) bytes) and it is your responsibility to transfer this sequence of bytes to the remote host.

In our case, we define a very simple transport protocol; first we write the message size as an 32-bit integer (big endian), then comes the message itself. (Also remember that protobuf messages are not self-identifying, which means that you need to know which message you are sending. This is typically managed by having a wrapper message containing optional fields for all messages you want to send. See the protobuf website and mailing list archives for more info about this technique.)