Async HttpWebRequest failing to fetch all range requests

Question

I'm attempting to fetch a URI in parts by setting the Range header on my HttpWebRequest, and then requesting them asynchronously using Begin/EndGetResponse :

using System;
using System.Net;

namespace MultiGet
{
    class Program
    {        
        static void GetterCallback(IAsyncResult ip)
        {
            try
            {
                var req = (HttpWebRequest)ip.AsyncState;
                var response = req.EndGetResponse(ip);
                Console.WriteLine("In GetterCallback {0}",req.Headers["Range"]);
                response.Close();
            }
            catch
            {
                Console.WriteLine("Exception Thrown");
            }
        }

        static void Main(string[] args)
        {
            string URI = "http://www.microsoft.com";
            var req = (HttpWebRequest)WebRequest.Create(URI);
            req.Method = WebRequestMethods.Http.Head;
            var response = req.GetResponse();
            int TotalSize = Int32.Parse(response.Headers["Content-Length"]);

            Console.WriteLine("URI Content-Length {0}",TotalSize);
            Console.WriteLine("Now requesting the URI in 10 parts");                        

            Int32 start = 0;
            var splitreq = new HttpWebRequest[10];
            var results = new IAsyncResult[10];
            for (int i = 0; i < 10; i++)
            {
                Int32 end = start + TotalSize / 10 - 1;
                if (i == 9) end = TotalSize;

                splitreq[i] = (HttpWebRequest)WebRequest.Create(URI);
                splitreq[i].AddRange(start, end);             

                results[i] = splitreq[i].BeginGetResponse(new AsyncCallback(GetterCallback), splitreq[i]);

                start = end + 1;
            }
            Console.ReadKey(true);

            //This shows that some async calls did not complete
            foreach (var result in results)
            {
                Console.WriteLine(result.IsCompleted);                
            }

        }

    }
}

Even if I wait for a long time (no keypress for the Console.ReadKey(true)), allrequests do not complete. I'm not sure where I'm going wrong.

Answer 1

One possible problem is that you're not using using statements for the responses, so it's possible that they're holding onto HTTP connections in the pool.

So for example, I suggest you change your callback to:

static void GetterCallback(IAsyncResult ip)
{
    try
    {
        var req = (HttpWebRequest)ip.AsyncState;
        using (var response = req.EndGetResponse(ip))
        {
            Console.WriteLine("In GetterCallback {0}",req.Headers["Range"]);
        }
    }
    catch
    {
        Console.WriteLine("Exception Thrown");
    }
}

And earlier in your program, when you're finding the content length:

int TotalSize;
using (var response = (HttpWebResponse) req.GetResponse())
{                
    TotalSize = (int)response.ContentLength;
}

When you've made those changes, try it again... it's certainly working for me.