WebRequest problem, response before request-body

Question

Hi,

I'am using the WebRequest class in .net and POST data to a server which is responding with a Response.

The wierd thing is that its working when I started fiddler to analyze my network traffic, but without fiddler it isn't.

So i started to analyze the package which is sent to and from my computer with WireShark. With in this program its simple to follow the TCP-stream. So when I had fiddler on, I can see the correct Request-header/body is sent, and gets the Response-header/body. The strange part is when i dont use fiddler the Request-header is sent, then i´ve got the Response-header/body, and finally the request-body in the end of the TCP-stream.

Here is my code i've been elaborating:

string lcUrl = "http://XX.XX.XXX.XX";

        // *** Establish the request 

        HttpWebRequest loHttp = (HttpWebRequest)WebRequest.Create(lcUrl);

        string lcPostData = testdata;

        loHttp.Method = "POST";

        byte [] lbPostBuffer = System.Text.Encoding.GetEncoding(1252).GetBytes(lcPostData);

        loHttp.ContentLength = lbPostBuffer.Length;

        loHttp.Credentials = CredentialCache.DefaultCredentials;

        //loHttp.SendChunked = true;

        loHttp.ServicePoint.Expect100Continue = false;

        Stream loPostData = loHttp.GetRequestStream();

        loPostData.Write(lbPostBuffer, 0, lbPostBuffer.Length);

        loPostData.Close();

        HttpWebResponse loWebResponse = (HttpWebResponse)loHttp.GetResponse();

        Encoding enc = System.Text.Encoding.GetEncoding(1252);

        StreamReader loResponseStream = new StreamReader(loWebResponse.GetResponseStream(), enc);

        string lcHtml = loResponseStream.ReadToEnd();

        loWebResponse.Close();

        loResponseStream.Close();

Answer 1

Please use following code. Seems that you have problems with time when underlying stream are send to remote server.

string lcUrl = "http://XX.XX.XXX.XX";
// *** Establish the request 
HttpWebRequest loHttp = (HttpWebRequest)WebRequest.Create(lcUrl);
string lcPostData = testdata;
loHttp.Method = "POST";
byte[] lbPostBuffer = System.Text.Encoding.GetEncoding(1252).GetBytes(lcPostData);
loHttp.ContentLength = lbPostBuffer.Length;
loHttp.Credentials = CredentialCache.DefaultCredentials;
//loHttp.SendChunked = true;
loHttp.ServicePoint.Expect100Continue = false;
using (Stream loPostData = loHttp.GetRequestStream())
{
     loPostData.Write(lbPostBuffer, 0, lbPostBuffer.Length);
}
string lcHtml;
using (HttpWebResponse loWebResponse = (HttpWebResponse)loHttp.GetResponse())
{
    Encoding enc = System.Text.Encoding.GetEncoding(1252);
    using (StreamReader loResponseStream = new StreamReader(loWebResponse.GetResponseStream(), enc))
    {
         lcHtml = loResponseStream.ReadToEnd();
    }               
}
// Perform processing of data here....

Also I could suggest you add following code in the app.config file for your application. This is helps when server returns response that not conforms with way how .NET handle HTTP request.

<configuration>
 <system.net>
 <settings>
<httpWebRequest
useUnsafeHeaderParsing="true"
/>
</settings>
</system.net>
</configuration>

Answer 2

I have a suspicion that the client is waiting for the "HTTP/1.1 100 continue" response from the server. This is how it works. When you are posting data to the server, sometimes the server might not be ready to accept the data just yet. For eg, it wants to authenticate the client first.

So, when you send a POST request, the client just sends the request headers, with an "Expect: 100-continue" appended.

POST /url HTTP/1.1 Server: Server-name/fqdn Content-Length: 100 Expect: 100-continue

If the server is ready to receive the data it responds with:

HTTP/1.1 100 continue Server: server-name/fqdn

Now, the client can send the data.

However if the server is not ready to receive the data, and wants to authenticate the client, it will respond with a different status code.

If you post your wireshark trace to pastebin.com I can verify, but I suspect this is what is happening.

The reason you dont see this in fiddler might be that fiddler is using HttpListener to listen to HTTP request, and HTTP listener hides the intermediate response like 100-continue from the app (in this case fiddler).