tags:

views:

125

answers:

3

Is there a way to get a System.Net.WebRequest or System.Net.WebClient to respect the hosts or lmhosts file?

For example: in my hosts file I have:

10.0.0.1  www.bing.com

When I try to load Bing in a browser (both IE and FF) it fails to load as expected.

Dns.GetHostAddresses("www.bing.com")[0]; // 10.0.0.1 
WebRequest.Create("http://10.0.0.1").GetResponse(); // throws exception (expected)
WebRequest.Create("http://www.bing.com/").GetResponse(); // unexpectedly succeeds

Similarly:

WebClient wc = new WebClient();
wc.DownloadString("http://www.bing.com"); //succeeds 

Why would System.Net.Dns respect the hosts file but System.Net.WebRequest ignore it? What do I need to change to make the WebRequest respect the hosts file?

Additional Info:

  • If I disable IPv6 and set my IPv4 DNS Server to 127.0.0.1, the above code works (fails) as expected. However if I add my normal DNS servers back as alternates, the unexpected behavior resumes.
  • I've reproduced this on 3 Win7 and 2 Vista boxes. The only constant is my company's network.
  • I'm using .NET 3.5 SP1 and VS2008

Edit

Per @Richard Beier's suggestion, I tried out System.Net tracing. With tracing ON the WebRequest fails as it should. However as soon as I turn tracing OFF the behavior reverts to the unexpected success. I have reproduced this on the same machines as before in both debug and release mode.

Edit 2

This turned out to be the company proxy giving us issues. Our solution was a custom proxy config script for our test machines that had "bing.com" point to DIRECT instead of the default proxy.

+2  A: 

I'm using VS 2010 on Windows 7, and I can't reproduce this. I made the same hosts-file change and ran the following code:

Console.WriteLine(Dns.GetHostAddresses("www.bing.com")[0]);             // 10.0.0.1     
var response = WebRequest.Create("http://www.bing.com/").GetResponse(); // * * *
Console.WriteLine(new StreamReader(response.GetResponseStream()).ReadToEnd());

I got an exception on the line marked "* * *". Here's the exception detail:

System.Net.WebException was unhandled
  Message=Unable to connect to the remote server
  Source=System
  StackTrace:
       at System.Net.HttpWebRequest.GetResponse()
       at ConsoleApplication2.Program.Main(String[] args) in c:\Data\Projects\ConsoleApplication2\ConsoleApplication2\Program.cs:line 17
  InnerException: System.Net.Sockets.SocketException
       Message=A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 10.0.0.1:80
       Source=System
       ErrorCode=10060

Maybe it's an issue with an earlier .NET version, that's now fixed in .NET 4 / VS 2010? Which version of .NET are you using?

I also found this thread from 2007, where someone else ran into the same problem. There are some good suggestions there, including the following:

  • Turn on system.net tracing

  • Work around the problem by using Dns.GetHostAddresses() to resolve it to an IP. Then put the IP in the URL - e.g. "http://10.0.0.1/". That may not be an option for you though.

In the above thread, mariyaatanasova_msft also says: "HttpWebRequest uses Dns.GetHostEntry to resolve the host, so you may get a different result from Dns.GetHostAddresses".

Richard Beier
I'm using VS2008 and 3.5 SP1. `GetHostEntry` gives me the same output as `GetHostAddress`. I'll look into tracing. Thanks.
Nate
Turned on System.Net tracing per your suggestion. With tracing on, the WebRequest fails as expected. With tracing off it goes back to unexpectedly succeeding. Thoughts?
Nate
That's very strange - I don't have a good answer. Hans Passant and Jonathan Stanton have good points regarding proxies. But if IE uses a proxy and still respects your hosts file, then the hosts file should also override the proxy when using WebRequest.The only thing I can suggest is to try disabling the proxy anyway, as mwalker's suggests. You could also try a packet sniffer such as Wireshark or Network Monitor to see what's happening on the wire - but I'm not sure that will help. If you see a DNS request for www.bing.com, that confirms it's not using the hosts file, but doesn't explain why.
Richard Beier
+3  A: 

Hi there,

I think that @Hans Passant has spotted the issue here. It looks like you have a proxy setup in IE.

Dns.GetHostAddresses("www.bing.com")[0]; // 10.0.0.1  

This works because you are asking the OS to get the IP addresses for www.bing.com

WebRequest.Create("http://www.bing.com/").GetResponse(); // unexpectedly succeeds

This works because because you are asking the framework to fetch a path from a server name. The framework uses the same engine and settings that IE frontend uses and hence if your company has specified by a GPO that you use a company proxy server, it is that proxy server that resolves the IP address for www.bing.com rather than you.

WebRequest.Create("http://10.0.0.1").GetResponse(); // throws exception (expected) 

This works/fails because you have asked the for the framework to fetch you a webpage from a specific server (by IP). Even if you do have a proxy set, this proxy will still not beable to connect to this IP address.

I hope that this helps.

Jonathan

Jonathan Stanton
You're right that I'm behind a company proxy but I'm not sure I follow your logic. I would expect that `WebRequest.Create()` would exhibit identical behavior to IE but it doesn't. Bing.com in IE times out trying to load 10.0.0.1
Nate
+1  A: 

This stackoverflow answer has the quick-and dirty configs to disable your proxy within .NET. Put it in your web.config or app.config and then System.Web won't look to the underlying system to get its proxy config.

mwalker