Can I tell WebRequest when to send a chunk from the request stream?

Question

I'm sending an HTTP PUT request from a WinForms application and I'd like to send a slow trickle of PUT data to the page that will write messages into a database as the PUT data arrives. I'm using WebRequest and I've set SendChunked to true, but it only seems to send a chunk after 8KB of data have been written to the request stream.

Even worse, the web page seems to stop receiving after about 42KB and the sender throws a WebException after about 77KB with the message, "The request was aborted: The request was canceled."

I'm actually sending very small amounts of data in each message, so if I could convince WebRequest to just send a small chunk containing each message, I'd be fine.

Here's what I'm experimenting with so far:

        var request = 
            (HttpWebRequest)WebRequest.Create("http://localhost/test.php");

        request.Method = "PUT";
        request.Timeout = 300 * 1000;
        request.SendChunked = true;
        request.AllowWriteStreamBuffering = false;
        request.ContentType = "application/octet-stream";
        using (var post = new StreamWriter(request.GetRequestStream()))
        {
            post.AutoFlush = true;
            for (int i = 0; i < 100; i++)
            {
                if (i > 0)
                {
                    // force flushing previous chunk
                    post.Write(new String(' ', 1048));
                    Thread.Sleep(2 * 1000);
                }
                Console.Out.WriteLine("Requesting {0} at {1}.", i, DateTime.Now);
                string chunk = i.ToString();
                post.WriteLine(i);
            }
        }

I'm writing 1KB of whitespace after each message to try and force the WebRequest to send a chunk sooner.

Answer 1

If nobody has any more elegant suggestions, I guess I could open a raw socket and implement the chunked encoding myself. There's a reasonable description in this answer.

Answer 2

If I understand your question correctly, you want to issue a single PUT request. As part of the request, you want to send chunked data, and you want the server to take action on each chunk that it sees.

I dont think this will be possible very easily by using HttpWebRequest. AS you have noticed, while HWR will send chunked data, it does not give you control over where exactly a chunk begins and ends.

Next, even if you could pull off the client side, by writing a custom Socket implementation as Don suggests, you still cannot guarantee that the webserver will give control to your app for each chunk that it receives. The webserver might decide to buffer a couple of chunks and give them all to the app at the same time.

I would suggest that you rethink your design, and have a simple Request/Response based model where the request+data constitutes an action that the server needs to take.