views:

1515

answers:

8
+14  A: 

This is a very bad idea. Binary data should always be sent in a way that:

Don't ever write a whole struct in a binary way, not to a file, not to a socket.

Always write each field separately, and read them the same way.

You need to have functions like

unsigned char * serialize_int(unsigned char *buffer, int value)
{
  /* Write big-endian int value into buffer; assumes 32-bit int and 8-bit char. */
  buffer[0] = value >> 24;
  buffer[1] = value >> 16;
  buffer[2] = value >> 8;
  buffer[3] = value;
  return buffer + 4;
}

unsigned char * serialize_char(unsigned char *buffer, char value)
{
  buffer[0] = value;
  return buffer + 1;
}

unsigned char * serialize_temp(unsigned char *buffer, struct temp *value)
{
  buffer = serialize_int(buffer, value->a);
  buffer = serialize_char(buffer, value->b);
  return buffer;
}

unsigned char * deserialize_int(unsigned char *buffer, int *value);

Or the equivalent, there are of course several ways to set this up with regards to buffer management and so on. Then you need to do the higher-level functions that serialize/deserialize entire structs.

This assumes serializing is done to/from buffers, which means the serialization doesn't need to know if the final destination is a file or a socket. It also means you pay some memory overhead, but it's generally a good design for performance reasons (you don't want to do a write() of each value to the socket).

Once you have the above, here's how you could serialize and transmit a structure instance:

int send_temp(int socket, const struct sockaddr *dest, socklen_t dlen,
              const struct temp *temp)
{
  unsigned char buffer[32], *ptr;

  ptr = serialize_temp(buffer, temp);
  return sendto(socket, buffer, ptr - buffer, 0, dest, dlen) == ptr - buffer;
}

A few points to note about the above:

  • The struct to send is first serialized, field by field, into buffer.
  • The serialization routine returns a pointer to the next free byte in the buffer, which we use to compute how many bytes it serialized to
  • Obviously my example serialization routines don't protect against buffer overflow.
  • Return value is 1 if the sendto() call succeeded, else it will be 0.
unwind
And in this case, `int` can have different sizes on the different machines as well.
Douglas Leeder
@Douglas: Absolutely true, and added the list. Thanks!
unwind
@unwind - Are serialize_int and deserialize_int are standard functions ??
codingfreak
@codingfreak: No, you need to define them to do the kind of serialization you need.
unwind
@unwind - I did not get you ... ?? Should I send only individual members rather than sending whole structure at once via socket ??
codingfreak
@codingfreak If you want to be more practical, use some standard serialization format, e. g. JSON, ASN.1, XML. There are lots of serializaton/parsing libraries available for such formats.
Andrey Vlasovskikh
@codingfreak: You should *serialize* individual fields, then send the entire buffer containing the serialized data with a single write(). This way, you know exactly what is in each bit. If you write a whole struct at once, you don't know.
unwind
@unwind - Should I write serialize functions for each and every datatype ??
codingfreak
@codingfreak: Yes. :) It's either that, or use some library that does it for you. This will probably require you to describe your datastructures to that library, though.
unwind
@unwind - why should I serialize each and every variable i.e. int , char ? By using UDP sockets I can send integers, character and receive them properly on receiver end .... I am just using sendto and recvfrom calls ... and I am not doing any serialization in this case.What are you doing by doing right shifts ... converting hostbyte order to network byte order or viceversa ?? ... your method might get complex when my structure has enums and other types ...
codingfreak
@codingfreak: The bullet list first in my answer tells you why. If you write a whole struct using sendto(), you have no control over those things. Trying to receive the struct on a different machine, or just using code built by different compiler, will be difficult. You must treat each field separately, so you know what you send. Read the links I provided.
unwind
@unwind - I feel you are not getting my point - I understand why I should serialize a structure ... But since you are saying that I have to send each member of the structure individually ... Then I dont understand why I should again serialize each member of a structure --- Cant I send them as if I send a single int or char or string ...
codingfreak
@unwind - Ifpossible can you point me to a link where I can get the working simple code on passing a structure through sockets ... I am really so confused and not able to understand through conversation might be because of my communication gap ...
codingfreak
@codingfreak: I added another function, that shows how to combine the serialization functions with sending the result. It's really simple, I'm not sure what is confusing you so much about this. :|
unwind
A: 

Serialization is a good idea. You can also use "wireshark" to monitor the traffic and understand what is actually passed in the packets.

eyalm
+1  A: 

If you don't want to write the serialisation code yourself, find a proper serialisation framework, and use that.

Maybe Google's protocol buffers would be possible?

Douglas Leeder
That or XDR, which solved this problem decades ago.
Steve Emmerson
+2  A: 

There is no need to write own serialisation routines for short and long integer types - use htons()/htonl() POSIX functions.

qrdl
@qrdl - How should I use htonl for a structure ?? htonl(pkt); ???
codingfreak
@qrdl: So does my function, as documented. It will always serialize to big-endian. You can of course use htonX()/ntohX() functions too, but this tried to illustrate a more general approach.
unwind
@codingfreak I didn't say to use it on structure. It is proper way to serialise short or long integers, that's it
qrdl
@qrdl - But my question is how to send a structure via sockets ... even without using htons()/htonl() functions I am able to send and receive data properly using sendto() and recvfrom() functions .....
codingfreak
@qrdl: My functions assume nothing about the host's endianness.
unwind
@unwind My bad. I'm sorry, I keep forgetting that shift is endianness-aware. I'll change my post
qrdl
A: 

Instead of serialising and depending on 3rd party libraries its easy to come up with a primitive protocol using tag, length and value.

Tag: 32 bit value identifying the field
Length: 32 bit value specifying the length in bytes of the field
Value: the field

Concatenate as required. Use enums for the tags. And use network byte order...

Easy to encode, easy to decode.

Also if you use TCP remember it is a stream of data so if you send e.g. 3 packets you will not necessarily receive 3 packets. They maybe be "merged" into a stream depending on nodelay/nagel algorithm amongst other things and you may get them all in one recv... You need to delimit the data for example using RFC1006.

UDP is easier, you'll receive a distinct packet for each packet sent, but its a lot less secure.

Stuart
At present I am using recvfrom and sendto which are generally used in the case of UDP communication ....
codingfreak
Yes, it'll be fine for loopback or reliable connections.
Stuart
Is'nt there any other solution other than this TAGGING stuff ???
codingfreak
There are many. Google Protocols, Apache Thrift, ASN.1, CSN.1, JSON, XML ... Depending on your application you could get away with what you are doing and it'll work just fine... its just not very robust!
Stuart
I already mentioned in the Question... no third party stuff just libc libraries ...
codingfreak
ASN.1, CSN.1, JSON, XML only rely on third party stuff if you choose not to do it yourself. Tagging is the most basic building block of Google Protocols, Apache Thrift, ASN.1 and CSN.1 and its trivial to do. You could also look at this: liw.iki.fi/liw/texts/cpp-trick.html though it ignores byte order issues.
Stuart
A: 

If the format of the data you want to transfer is very simple then converting to and from an ANSI string is simple and portable.

the_mandrill
@mandrill - Let use suppose it is a complex one ---- a generic solution would be the best answer if possible ... ??
codingfreak
If the format is more complex then I defer to the superior solutions given elsewhere in this thread! Alternatively serialise to something more generic such as XML or a SOAP-like wrapper.
the_mandrill
Ultimately the aim is to serialise your data in a portable manner, so conversion to and from a string is portable, simple and readable. It may not be the most secure or efficient method, but it doesn't require any 3rd party libraries.
the_mandrill
How do I re-convert back the string into the data that cane be placed into a struct .... it would be more complex I feel so ??
codingfreak
To re-convert the string back into data you'd have to parse the string, using things like atof(), atoi(), sscanf(), etc. (well maybe not sscanf(), it's dangerous). You're right, parsing the string can get complex for non-simple data. I'd recommend using a 3rd party serialization library instead.
Jeremy Friesner
+1  A: 

Using the 'pragma' pack option did solved my problem but I am not sure if it has any dependencies ??

#pragma pack(1)   // this helps to pack the struct to 5-bytes
struct packet {
int i;
char j;
};
#pragma pack(0)   // turn packing off

Then the following lines of code worked out fine without any problem

n = sendto(sock,&pkt,sizeof(struct packet),0,&server,length);

n = recvfrom(sock, &pkt, sizeof(struct packet), 0, (struct sockaddr *)&from, &fromlen);
codingfreak
can some explain this more?
devin
@devin - http://www.cplusplus.com/forum/general/14659/ or http://gcc.gnu.org/onlinedocs/gcc/Structure_002dPacking-Pragmas.html
jschmier
A: 

here is a program...

#include <stdio.h>                 //H
#include <stdlib.h>                //E
#include <string.h>                //A
#include <unistd.h>                //D
#include <sys/types.h>             //E
#include <sys/socket.h>            //R
#include <netinet/in.h>            //FILE
#include <netdb.h>                 //S

#define BUFSIZE 1024

/* 
 * error - wrapper for perror
 */
void error(char *err) {
    perror(err);
    exit(0);
}
              struct student 
              {
                  char name[16];
                  unsigned short student_id;
                  char host[16];
                  unsigned short pid;
                  char email[32];

              }r1;

void main(int argc, char **argv) {
    int sockfd, portno, n;
    int serverlen;
    struct sockaddr_in serveraddr;
    struct hostent *server;
    char *hostname;
    char buf[BUFSIZE];
struct student r1 = {"Parikshit Tiwari", htons(0x6160), "Parikx-Laptop", htons(getpid()),"[email protected]"};

    if (argc != 3) {
       fprintf(stderr,"usage: %s <hostname> <port>\n", argv[0]);
       exit(0);
    }
    hostname = argv[1];
    portno = atoi(argv[2]);

    sockfd = socket(AF_INET, SOCK_DGRAM, 0);//socket created
    if (sockfd < 0) 
        error("ERROR opening socket");

    /* gethostbyname: get the server's DNS entry */
    server = gethostbyname(hostname);
    if (server == NULL) {
        fprintf(stderr,"ERROR, hostname %s\n does not exist", hostname);
        exit(0);
    }

    /* build the server's Internet address */
    bzero((char *) &serveraddr, sizeof(serveraddr));
    serveraddr.sin_family = AF_INET;
    bcopy((char *)server->h_addr, (char *)&serveraddr.sin_addr.s_addr, server->h_length);
    serveraddr.sin_port = htons(portno);


   /* send the message to the server */
    serverlen = sizeof(serveraddr);
    n = sendto(sockfd, (void *) &r1, sizeof(r1), 0, (struct sockaddr *)&serveraddr, serverlen);
    printf("%d", n);//Test to check the size of data send to the server 
    if (n < 0) 
      error("ERROR in sendto");

    close(sockfd);
}
Parikshit
How about using pragma ??
codingfreak