views:

804

answers:

3

I am trying to use the System.Net.HttpWebRequest class to perform a HTTP GET request to a specific web server for web applications we have load balanced over numerous servers. In order to achieve this, I need to be able to set the Host header value for the request, and I have been able to achieve this by using the System.Net.WebProxy class.

However, this all breaks down when I try to perform a GET using SSL. When I attempt to do this, the call to HttpWebRequest.GetResponse throws a System.Net.WebException, with a HTTP status code of 400 (Bad Request).

Is what I'm trying to achieve possible with HttpWebRequest, or should I be looking for an alternative way to perform what I want?

Here is the code I've been using to try and get this all to work :-

using System;
using System.Web;
using System.Net;
using System.IO;

namespace UrlPollTest
{
    class Program
    {
        private static int suffix = 1;
        static void Main(string[] args)
        {
            PerformRequest("http://www.microsoft.com/en/us/default.aspx", "www.microsoft.com");
            PerformRequest("https://www.microsoft.com/en/us/default.aspx", "");
            PerformRequest("https://www.microsoft.com/en/us/default.aspx", "www.microsoft.com");

            Console.WriteLine("Press any key to continue");
            Console.ReadKey();
        }

        static void PerformRequest(string AUrl, string AProxy)
        {
            Console.WriteLine("Fetching from {0}", AUrl);
            try
            {
                HttpWebRequest request = WebRequest.Create(AUrl) as HttpWebRequest;
                if (AProxy != "")
                {
                    Console.WriteLine("Proxy = {0}", AProxy);
                    request.Proxy = new WebProxy(AProxy);
                }

                WebResponse response = request.GetResponse();
                using (Stream httpStream = response.GetResponseStream())
                {
                    using (StreamReader reader = new StreamReader(httpStream))
                    {
                        string s = reader.ReadToEnd();
                        File.WriteAllText(string.Format("D:\\Temp\\Response{0}.html", suffix++), s);
                    }
                }
                Console.WriteLine("  Success");
            }
            catch (Exception e)
            {
                Console.WriteLine("  " + e.Message);
            }
        }
    }
}
+2  A: 

I've used a hacky approach before - of having a test exe that modifies my "hosts" file, injecting an entry for the farm name to the specific server's address (and issuing an ipconfig /flushdns). After that, requests should get routed to the right server as though it were the only one there.

Obviously this requires admin access... I use it as part of an automated smoke-test to hit the farm as though it were individual machines.

The other thing you might try is something on the NLB, perhaps in TCL (in the case of F5) - maybe add a custom header that the NLB understands? (assuming it is doing SSL re-signing).

Marc Gravell
Thanks for the suggestion, but that won't work for us. We'll be running multiple threads at the same time, and there could definitely be HOSTS entry conflicts in that scenario.
John Kaster
A: 

In .NET 4.0 the Host header can be set independently of the URL that the request is going to so I would recommend that you use HttpwebRequest.Host to solve your problem. This (I think) will ship as part of .NET 4.0 beta2. There is NO way to re-write the host header to be different under previous versions of .Net without implementing your own custom subclass of WebRequest (believe me, it's been tried).

Jeff Tucker
A: 

Yes, not being able to set the Host header has been a major sticking point with the HttpWebRequest class. I had blogged about a workaround for this..

http://blogs.msdn.com/feroze_daud/archive/2005/03/31/404328.aspx

As Jeff has mentioned, .NET 4.0 has fixed this problem. Better late than never... If you can, upgrade to .Net 4.0, otherwise you can try the workaround.

Regarding your problem with the 400 status code, get a tracelog (see http://ferozedaud.blogspot.com/2009/08/tracing-with-systemnet.html). Probably something in the request is not being accepted by the server.

feroze