HttpWebRequest response produces HTTP 422. Why?

Question

Hi there.

I'm trying to programmatically send a POST-request to a web-server in order to login an then perform other requests that require a login.

This is my code:

    byte[] data = Encoding.UTF8.GetBytes(
    String.Format(
        "login={0}&password={1}&authenticity_token={2}"
        +"&login_submit=Entra&remember_me=1", 
        HttpUtility.UrlEncode(username), HttpUtility.UrlEncode(password),   
        HttpUtility.UrlEncode(token)));

    //Create HTTP-request for login
    HttpWebRequest request = 
          (HttpWebRequest)HttpWebRequest.Create("http://www.xxx.xx/xx/xx");

    request.Method = "POST";
    request.ContentType = "application/x-www-form-urlencoded";
    request.ContentLength = data.Length;
    request.CookieContainer = new CookieContainer();

    request.Accept = "application/xml,application/xhtml+xml,text/html;
                     +"q=0.9,text/plain ;q=0.8,image/png,*/*;q=0.5";
    request.Referer = "http://www.garzantilinguistica.it/it/session";
    request.Headers.Add("Accept-Language", "de-DE");
    request.Headers.Add("Origin", "http://www.xxx.xx");
    request.UserAgent = "C#";
    request.Headers.Add("Accept-Encoding", "gzip, deflate");

After sending the request

    //Send post request
    var requestStream = request.GetRequestStream();

    requestStream.Write(data, 0, data.Length);
    requestStream.Flush();
    requestStream.Close();

... I want to get the servers response:

    //Get Response
    StreamReader responseStreamReader = new
    StreamReader(
        request.GetResponse().GetResponseStream()); //WebException: HTTP 422!
    string content = responseStreamReader.ReadToEnd();

This piece of code fires the WebException, that tells me the server responded with HTTP 422 (unprocessable entity due to semantic errors)

Then I compared (using a TCP/IP sniffers) the requests of my program and the browser (which of course produces a valid POST-request and gets the right response).

(1) My program's request:

POST /it/session HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Accept: application/xml,application/xhtml+xml,text/html;
q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Referer: http://www.garzantilinguistica.it/it/session
Accept-Language: de-DE
Origin: http://www.garzantilinguistica.it
User-Agent: Test
Accept-Encoding: gzip, deflate
Host: www.garzantilinguistica.it
Content-Length: 148
Expect: 100-continue
Connection: Keep-Alive


HTTP/1.1 100 Continue


login=thespider14%40hotmail.com&password=xxxxx&authenticity_token=4vLgtwP3nFNg4NeuG4MbUnU7sy4z91Wi8WJXH0POFmg%3d&login_submit=Entra&remember_me=1

(2) The browser's request:

    POST /it/session HTTP/1.1
    Host: www.garzantilinguistica.it
    Referer: http://www.garzantilinguistica.it/it/session
    Accept: application/xml,application/xhtml+xml,text/html;q=0.9,
text/plain;q=0.8,image/png,*/*;q=0.5
    Accept-Language: de-DE
    Origin: http://www.garzantilinguistica.it
    User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; de-DE) AppleWebKit/531.22.7 (KHTML, like Gecko) Version/4.0.5 Safari/531.22.7
    Accept-Encoding: gzip, deflate
    Content-Type: application/x-www-form-urlencoded
    Cookie: __utma=244184339.652523587.1275208707.1275208707.1275211298.2; __utmb=244184339.20.10.1275211298; __utmc=244184339; __utmz=244184339.1275208707.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); _garzanti2009_session=BAh7CDoPc2Vzc2lvbl9pZCIlZDg4MWZjNjg2YTRhZWE0NDQ0ZTJmMTU2YWY4ZTQ1NGU6EF9jc3JmX3Rva2VuIjFqRWdLdll3dTYwOTVVTEpNZkt6dG9jUCtaZ0o4V0FnV2V5ZnpuREx6QUlZPSIKZmxhc2hJQzonQWN0aW9uQ29udHJvbGxlcjo6Rmxhc2g6OkZsYXNoSGFzaHsGOgplcnJvciIVbG9naW4gbm9uIHZhbGlkbwY6CkB1c2VkewY7CFQ%3D--4200fa769898dd156faa49e457baf660cf068d08
    Content-Length: 144
    Connection: keep-alive

authenticity_token=jEgKvYwu6095ULJMfKztocP%2BZgJ8WAgWeyfznDLzAIY%3D&login=thespider14%40hotmail.com&password=xxxxxx&remember_me=1&commit=Entra
HTTP/1.1 302 Found

Can someone help to understand which part of the request I am missing or what the main difference between the browser's and my request is? Why am I getting that 422?

EDIT:

I noticed that my request contains a Expect header with the value 100-continue, whereas the browser's doesn't. I set the request.Expect-property to null and to "". But I just couldn't get rid of it. Any suggestions? May this be the root of all evil?

EDIT:

Finally I removed the Expect-Header. But it didn't help. Any ideas? I activated the CookieContainer by setting

request.CookieContainer = new CookieContainer();

But I can't see the Cookie-Header in the HTTP-trace. Why?

Answer 1

You don't seem to be escaping the = and @ correctly. Not sure if that's the only issue.

You can try HttpUtility.UrlEncode.

Answer 2

Ok I've got it guys.

I had two problems.

1. HTTP 1.1/ 100 continue

   I solved this by setting

   System.Net.ServicePointManager.Expect100Continue = false;
   

2. The server repsonded with 422 because of a problem related to the cookies. The POST request needs to transmit a cookie container, that contains a cookie with the current session id. If this session id is not transmitted, the Server will respond with 422. In order to be able to have a cookie with the session id, I had to perform a simple HTTP-request on the login page. This request returned a cookie container with the needed session id. Then I passed the returned cookie container to POST request.

   //cookie container of previous request
   postRequest.CookieContainer = cookieContainer; 
   

With this settings the POST request could be sent successfully.

Thanks for your help.