Why do I get a 401 Unauthorized when making a HttpWebRequest with the correct credentials?

Question

I have a url and a username and password to an external site. If I access the URL through a browser, an authentication box pops up. I give it the username and password, and I can get to the page.

I am trying to achieve the same thing through code, using a HttpWebRequest object:

var webRequest = (HttpWebRequest)WebRequest.Create(url);

webRequest.GetResponse() etc…

This used to work before the web sites owners added some protection to the site, and provided me with the username and password. The WebRequest has a credentials property, which I have been setting as follows:

webRequest.Credentials = new NetworkCredential("username", "password")

I have also tried:

webRequest.Credentials = new NetworkCredential("username", "password", “url domain”)

this always result in the error:

“The remote server returned an error: (401) Unauthorized.”

Am I missing something obvious?

Answer 1

Hi, try to debug your request with fiddler (www.fiddler2.com).

Therefore you must start Fiddler, and add the Proxy Settings to your Request:

webRequest.Proxy = New WebProxy("http://127.0.0.1:8888")

Answer 2

If you use HTTP, please simply use Microsoft Network Monitor or Wireshark to capture what kind of 401 message is returned from the web server. It can be 401.3 or another error code which has in fact another meaning.

Answer 3

Using System.Net.NetworkCredential might not be effective depending on the authentication model of the resource. It would be helpful to understand the model used by the remote site.

OpenID, Forms Authentication, and Integrated Windows Authentication all work differently. You can attempt to deduce what authentication method they use using tools as @Christoph and @Lex describe, or simply contact the remote web site provider.

If the remote site uses a negotiating protocol, such as Kerberos or NTLM, then the behavior you are experiencing is strange, indeed; however, some other protocols may require you to go about it another way.

Answer 4

If the site uses a custom authorization header then using NetworkCredential will not work. You need to find out whether this is the case, and if so, what format the Authorization header needs to be in. For example, a site that I recently had to connect to for one of my projects used the following:

Authorization: "Basic " + base64_encode(UTF8_encode(username + ":" + password))

Once you know the format you can set the header directly. For example:

webRequest.Headers[HttpRequestHeader.Authorization] = "Basic " + 
    Convert.ToBase64String(Encoding.UTF8.GetBytes(username + ":" + password));

Answer 5

If there is no privacy concern, you may wish to post the url you are trying to access.

If the URL is malformed, or references an invalid page (or an invalid page + parameters combination) you will sometimes see server errors like this.

I worked with a legacy web app from the state of Michigan and it would throw 400 style errors every time I tried to access a page with the wrong page parameters.