parts of an asynchronous httpwebrequest

Question

There are many pieces related to an httpwebrequest that can go asynchronous. I recall reading a question here about this very topic, but I can't seem to find it any more. So I'll re-ask the question. Which of the following get the most bang for the buck (so to speak).

BeginGetRequestStream/EndGetRequestStream

BeginWrite/EndWrite

BeginGetResponse/EndGetResponse

BeginRead/EndRead

I understand that BeginGetResponse must be paired with BeginGetRequestStream. So no need to reiterate that fact.

From the investigating I've done, it seems like BeginRead/EndRead may have the most potential. This came a little as a shock. It seems that EndGetResponse returns fairly quickly and there is significant delay after "first contact" and any real data coming to read. I'm pretty sure I am getting some headers immediately, and then a long delay, followed by the data I want.

I guess my real question is: Am I doing something wrong with BeginGetResponse or is the real valuable player here BeginRead?

As always, thanks in advance.

Answer 1

EndGetResposnse (or indeed, the synchronous GetResponse) returns upon receiving the beginning of data. This can be a small or large proportion of the total response time.

With a large response, the biggest portion of time is not going to be spent waiting for any of these, but on reading and processing the stream itself. Notably this can (especially if the webserver is sending chunked data) be parsed as the data comes in.

The biggest performance win can therefore be in processing this in a manner that allows efficent passing to the next level. One way is to make your entire processing asynch, rather than using the asynch methods of the httpwebrequest.

Another is that if you are producing a collection type from the response, to make use of a yield based ienumerable that allows for as-it-arrives processing (very powerful in tandem with the sort of delayed execution that is available by passing it to further yield based processing, or with a LINQ approach). The advantage of doing so can outweigh the benefits to be gained by taking an asych approach, though they can also be combined.