Reuse a HttpWebRequest?

Question

Am i able to use a HttpWebRequest?

It seems like the 3rd request to a site causes a operation timed out and it seems like each creates a new connection so i want to know if i can reuse a HttpWebRequest by changing the url and getting the request again. Code in question is below and this code is to check if a range of urls exist.

    static void storeList(TextWriter sw, string urlTemplate, int start, int end)
    {
        for (int i = start; i < end; i++)
        {
            var url = string.Format(urlTemplate, i);
            var req = (HttpWebRequest)HttpWebRequest.Create(url);
            {
                req.Method = "HEAD";
                tryHttpWebRequest
                {
                    var resp = req.GetResponse();
                    sw.WriteLine(i);
                }
                catch (Exception e)
                {
                }
            }
        }
        sw.Flush();
    }

Answer 1

You should be ok if you just call Close on your response. You are only allowed so many "open" connections, so the reason it is failing is because it can't open a new connection.

Once you are done with the response, you need to close it... no need to reuse anything.

From the MSDN article:

You must call either the Stream.Close or the HttpWebResponse.Close method to close the response and release the connection for reuse. It is not necessary to call both Stream.Close and HttpWebResponse.Close, but doing so does not cause an error.

Answer 2

Just create a new one. I wouldn't worry about being efficient in this scenario, as the .NET environment + HTTP keepalives should handle things for you from that perspective I believe.

Probably you're running into either 1) an issue with the number of open connections; kill the connection and the error will go away or 2) an issue with limits of number of requests per second on the server (anti-DoS stuff). I'd try #1 first, as it's simpler, and then if you still see the problem, check to see if the server is getting the request but denying it.

Answer 3

Wrap your response call in a using statement to ensure that the connection is always closed:

HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url)
// ...
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
  // ...
}

Also wrap any GetResponseStream() in a using statement.