HttpSendRequest blocking when more than two downloads are already in progress

Question

In our program, a new thread is created each time an HTTP request needs to be made, and there can be several running simultaneously. The problem I am having is that if I've got two threads already running, where they are looping on reading from InternetReadFile() after having called HttpSendRequest(), any subsequent attempts to call HttpSendRequest() just hang on that call, so I end up with the previously mentioned two threads continuing to read from their connections just fine, but the third just blocks on HttpSendRequest() until it times out.

From what I've been able to find on my own, this seems like it could just be the way wininet works, as the HTTP spec recommends: "A single-user client SHOULD NOT maintain more than 2 connections with any server or proxy."

I've seen various programs handle multiple simultaneous downloads to the same server, but I'd imagine they need to do a lot of extra work to do that, in terms of managing the various connections, or writing their own http interface.

If it would require a lot of extra complexity to set it up to handle more than two active sessions, then I would just change things to only handle one or two files at a time, leaving the rest queued. However, if there were some low-complexity way to allow more than two at a time (off the top of my head, I'd guess using a new process per download might work, but would be messier), that would be preferable; it's not like it would be downloading more than 3-5 simultaneously anyway, and each download is at the user's request. I read some mentions of registry hacks to change the limit, but that's definitely not something I'd do. Any ideas?

Answer 1

If you call InternetOpen() mutliple times, you should be able to simultaneously download two files on each HINTERNET returned by InternetOpen().

Answer 2

The HTTP 1.1 standard mandates a maximum of 2 simultaneous connections per server. If you have IE5, IE6, or IE7 installed, the versions of WinInet they install allow you to use InternetSetOption() to increase the limit (look at INTERNET_OPTION_MAX_CONNS_PER_SERVER and INTERNET_OPTION_MAX_CONNS_PER_1_0_SERVER options). However, the version of WinInet that is installed by with IE8 apparently disables that functionality (see http://connect.microsoft.com/WNDP/feedback/ViewFeedback.aspx?FeedbackID=434396 and http://connect.microsoft.com/WNDP/feedback/ViewFeedback.aspx?FeedbackID=481485).