HttpWebRequest does has empty response requesting a search from Bing

Question

I have the following code that sends a HttpWebRequest to Bing. When I request the url below though it returns what appears to be an empty response when it should be returning a list of results.

            var response = string.Empty;
            var httpWebRequest = WebRequest.Create("http://www.bing.com/search?q=stackoverflow&count=100") as HttpWebRequest;

            httpWebRequest.Method = WebRequestMethods.Http.Get;
            httpWebRequest.Headers.Add("Accept-Language", "en-US");
            httpWebRequest.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Win32)";
            httpWebRequest.Headers.Add(HttpRequestHeader.AcceptEncoding, "gzip,deflate");

            using (var httpWebResponse = httpWebRequest.GetResponse() as HttpWebResponse)
            {
                Stream stream = null;
                using (stream = httpWebResponse.GetResponseStream())
                {
                    if (httpWebResponse.ContentEncoding.ToLower().Contains("gzip"))
                        stream = new GZipStream(stream, CompressionMode.Decompress);
                    else if (httpWebResponse.ContentEncoding.ToLower().Contains("deflate"))
                        stream = new DeflateStream(stream, CompressionMode.Decompress);

                    var streamReader = new StreamReader(stream, Encoding.UTF8);
                    response = streamReader.ReadToEnd();
                }
            }

Its pretty standard code for requesting and receiving a web page. Any ideas why the response is empty? Thanks in advance.

EDIT I left off a query string parameter in the url. I also had &count=100 which I have now corrected. It seems to work for values of 50 and below but returns nothing when larger. This works ok when in the browser, but not for this web request.

It makes me think the issue is that the response is large and HttpWebResponse is not handling that for me the way I have it set up. Just a guess though.

Answer 1

This works just fine on my machine. Perhaps you are IP banned from Bing?

Answer 2

Your code works fine on my machine.

I suggest you get yourself a copy of Fiddler and examine the actual HTTP sesssion occuring. May be a proxy or firewall thing.