InternetOpenUrl intermittently times out

Question

I have some InternetOpenUrl requests that are strangely timing out. The endpoint is there and the URL is correct. This happens in a synchronous loop inside an activex control, and about the 6th time it executes, it times out without hitting the server.

HINTERNET hINet = InternetOpen(TEXT("InetURL/1.0"), INTERNET_OPEN_TYPE_PRECONFIG, NULL, NULL, 0 );
/*hangs*/ HINTERNET hFile = InternetOpenUrl( hINet, url.c_str(), headers, headersLen, dwFlags, dwContext);

GetLastError just returns 12002 operation timed out. Here's the full snippet: http://gist.github.com/559317

Interestingly, if I change the InternetOpen flag to INTERNET_OPEN_TYPE_DIRECT, and fiddler is NOT running, I get the failure (after a few successful requests), and if i run fiddler, the requests all succeed.

so far this has been replicated on Win7/IE8, and Vista64/IE8, XP/IE6

Answer 1

Per ivymike's comment, I looked at the InternetOpenUrl flags and see one I overlooked; INTERNET_FLAG_PRAGMA_NOCACHE

Update: This flag did not resolve the problem. I'm considering porting from WinINet (InternetOpenUrl) to WinHTTP to get around the caching behaviors.

(porting WinINet to WinHTTP) http://msdn.microsoft.com/en-us/library/aa384068(VS.85).aspx

Answer 2

You're almost certainly hitting the "Connections per host" limit in WinINET.

You can only have a certain number of requests to a single host running in parallel, in accordance with the connections-per-host SHOULD in RFC2616. When the cap is reached, WinINET will queue pending requests until an existing connection becomes available. If no connection becomes available before the timeout is reached, then the request times out.

WinINET implements this limit on a per-endpoint basis, so when you're running with a proxy (like Fiddler, ISA, SQUID, etc), you can have a maximum of "n" connections active at one time, while if no proxy is present, then you can have a maximum of "n" connections to each hostname.

If your ActiveX control does not properly free the connection when the response is completed, then you could hit this problem. Typically, running Fiddler shouldn't "fix" that for you, but you could use NetMon to check what's going on.

Answer 3

OK so I originally thought EricLaw was correct and commented: "my particular problem was that i had javascript ajax calls occuring after each control ajax call. this creates a race condition and eventually 4 javascript ajax calls hadn't returned when i made an ajax call inside the control. (yeah, i misstated my environment in the original question)"

this comment is incorrect.

the issue is actually the issue discussed here:

"Your ActiveX control has a common defect that single-threaded apartment (STA) ActiveX & COM objects must avoid: STA COM objects cannot perform blocking operations on the STA thread, unless the COM object also pumps Windows messages. Therefore, if your control needs to perform a synchronous blocking operation, it needs to implement a Windows message pump while waiting for the blocking operation to complete."

this solved the issue (and opened a whole new can of worms). if you run into this you're welcome to comment here and I'll explain better.