HTTP inputstream class (C++)

Question

I'm currently working on a project that does a lot of HTTP requests for data that it needs. It works by writing all the data into a buffer, then returning that buffer to the caller. It seems that waiting for the entire file to download data that can essentially streamed is a bad idea.

Question 1: Is there already a library / public code that can make HTTP and Authenticated HTTP requests that works as a stream?

Question 2: If there is no such library, why not? Is there a reason such a thing was never needed?

Answer 1

It wouldn't be too hard to get boost::asio to do this, but as far as I know, it doesn't have HTTP protocol built-in (and I can't think of a library that does, or why not), so you'd have to write header/body parsing (could be tedious and error-prone).

Some googling came up with some attempts at this, like so.

Answer 2

There are many C libraries that handle http requests, and the reason they're in C rather than C++ is simply because C++ can easily use the C libraries, but the converse isn't quite true.

Therefore, a C library is useful in both worlds...

libwww is the W3C library for http functions.

Curl also has a library with a C++ wrapper project too.

Answer 3

I'm not aware of an existing library, but it shouldn't be too hard to wrap Curl (or WinInet, if that's your cup of tea) with Boost.Iostreams.

Answer 4

This seems to be what you are looking for:

http://sourceforge.net/projects/urdl/