Chunked responses in libevent2

Question

Hi

I am trying to do a chunked response (of large files) in libevent this way::

evhttp_send_reply_start(request, HTTP_OK, "OK");

int fd = open("filename", O_RDONLY);
size_t fileSize = <get_file_size>;
struct evbuffer *databuff = NULL;
for (off_t offset = 0;offset < fileSize;)
{
    databuff = evbuffer_new();

    size_t bytesLeft = fileSize - offset;
    size_t bytesToRead = bytesLeft > MAX_READ_SIZE ? MAX_READ_SIZE : bytesLeft;

    evbuffer_add_file(databuff, fd, offset, bytesToRead);
    offset += bytesToRead;

    evhttp_send_reply_chunk(request, databuff); // send it
    evbuffer_free(databuff);                    // destroy it
}

evhttp_send_reply_end(request);

fclose(fptr);

---

Problem is with this I have a feeling the add_file is asynchronous so the 3rd or so evhttp_send_reply_chunk gives me an error (or something similar):

[warn] evhttp_send_chain Closed(45): Bad file descriptor

I set MAX_READ_SIZE to be 8 to actually test out chunked transfer encoding.

I noticed there was a evhttp_request_set_chunked_cb (struct evhttp_request *, void(*cb)(struct evhttp_request *, void *)) method I could use but could not find any examples on how to use.

For instance, how could I pass an argument to the callback? The argument seems to be the same argument that was passed to the request handler which is not what I want, because I want to create an object that holds the file descriptor and the file offset I am sending out.

Appreciate all help.

Thanks in advance Sri

Answer 1

The libevent v2 documentation doesn't say that it's async, but it does say that it closes the file descriptor which your code doesn't account for.

I believe you need to move int fd = open("filename", O_RDONLY); inside your loop.

you can also test the chunk handling outside of your file code by just building a string buffer from scratch.

aside from that, (and the last line which should be fclose(fp); your example looks correct

Answer 2

Nice Nice mate. Thanks for that. I just realised that the only reason I wanted to chunked transfers were to avoid buffer reads. But since evbuffer_add_file already uses sendfile (if it finds it) this is not really an issue.

So I removed the loop completely and tried. But the contents are still not being sent. but atleast this time I dont have the bad file descriptor error (you are right - this was due to the file being closed - a check of file handles confirmed this!).