C: Addressing the case of partial headers when using select in the context of HTTP...

Question

This is in reference to a previous question of mine which was already answered:

http://stackoverflow.com/questions/2159628/c-using-a-select-call-when-i-am-reading-how-do-i-keep-track-of-the-data

My protocol actually sends how much data it is about send initially before it sends any other data. But yesterday, when testing out this code of mine using a browser, I had this one question:

A lot of people suggested that I check for message length but take the scenario of a browser. The browser's HTTP Request does not have a size when it is first sent to the server. Now, assuming that I used a 256 byte buffer, how am I supposed to manage the data structure of this client if I keep receiving partial headers between each multiplexing action? Keep using realloc as I keep getting more data and then when I encounter a termination sequence ('\r\n'), assume that all data has been received?

What I mean is, have something like this:

typedef struct {
   int fd;
   char *data;
} CLIENT;

And then keep using realloc on data? I was told to allocate a buffer size of the max protocol header but is that the only approach?

Answer 1

reallocing the buffer in sizes is fine. I'd pick a bit larger buffer than 256 - 1024 atleast or 4096. Allocating a max buffer size of a protocol like HTTP is not really feasible. You could also build an abstraction on top of your buffer that are able to read lines - much like fgets does on a FILE*

Keep in mind that reading a chunk might read the last part of the HTTP headers and a piece of the HTTP body if there's any , so you need to make sure you slice the buffer properly as you likely want to seperate the header from the body. You'll also have to look for the Content-Length: header, parse that, so you know how long the body will be.