I have a hard time figuring out if there is a way to handle potential connectivity problems when using .NET's HttpWebRequest class to call a remote server (specifically a REST web service). From my investigations the behaviour of the WebClient class is the same, which is somewhat expected since it appears to only offer a more simple interface to the HttpWebRequest.
For simulation purposes, I've written a very simple HTTP server that does not behave according to the HTTP 1.1 RFC. What it does is it accepts a client connection, then sends appropriate HTTP 1.1 headers and a "Hello World!" payload back to the client and closes the socket, the thread accepting client connections on the server side looks as follows:
private const string m_defaultResponse = "<html><body><h1>Hello World!</h1></body></html>";
private void Listen()
{
while (true)
{
using (TcpClient clientConnection = m_listener.AcceptTcpClient())
{
NetworkStream stream = clientConnection.GetStream();
StringBuilder httpData = new StringBuilder("HTTP/1.1 200 OK\r\nServer: ivy\r\nContent-Type: text/html\r\n");
httpData.AppendFormat("Content-Length: {0}\r\n\r\n", m_defaultResponse.Length);
httpData.AppendFormat(m_defaultResponse);
Thread.Sleep(3000); // Sleep to simulate latency
stream.Write(Encoding.ASCII.GetBytes(httpData.ToString()), 0, httpData.Length);
stream.Close();
clientConnection.Close();
}
}
}
Since the HTTP 1.1 RFC states that HTTP 1.1 by default keeps connections alive and that a server must send a "Connection: Close" response header if it wants to close a connection this is unexpected behaviour for the client-side. The client uses HttpWebRequest in the following way:
private static void SendRequest(object _state)
{
WebResponse resp = null;
try
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://192.168.0.32:7070/asdasd");
request.Timeout = 50 * 1000;
DateTime requestStart = DateTime.Now;
resp = request.GetResponse();
TimeSpan requestDuration = DateTime.Now - requestStart;
Console.WriteLine("OK. Request took: " + (int)requestDuration.TotalMilliseconds + " ms.");
}
catch (WebException ex)
{
if (ex.Status == WebExceptionStatus.Timeout)
{
Console.WriteLine("Timeout occurred");
}
else
{
Console.WriteLine(ex);
}
}
finally
{
if (resp != null)
{
resp.Close();
}
((ManualResetEvent)_state).Set();
}
}
The above method is queued via ThreadPool.QueueUserWorkItem(waitCallback, stateObject). The ManualResetEvent is used to control queuing behavior so that not the entire thread pool gets filled up with waiting tasks (since the HttpWebRequest implicitly uses worker threads because it functions asynchronously internally to implement the timeout functionality).
The problem with all this is that once all connections of the HttpWebRequest's underlying ServicePoint are "used up" (i.e. closed by the remote server), there will no new ones be opened up. It does also not matter if the ConnectionLeaseTimeout of the ServicePoint is set to a low value (10 seconds). Once the system gets into this state, it will no longer function properly because it does not reconnect automatically and all subsequent HttpWebRequests will time out. Now the question really is if there is a way to solve this by somehow destroying a ServicePoint under certain conditions or closing underyling connections (I did not have any luck with ServicePoint.CloseConnectionGroup() yet, the method is also pretty undocumented in terms of how to properly use it).
Does anybody have any idea how I could approach this problem?