Processing packets in the async loop or not?

Question

In C#, when receiving network data with the BeginReceive/EndReceive methods, is there any reason you shouldn't process the packets as soon as you receive them? Some of the tasks can be decently cpu intensive. I ask because I've seen some implementations that push the packets off into a processing queue and then handle them there. To me this seems redundant because, as far as I know, the async methods also operate on a thread pool.

Answer 1

Generally, you need to receive 'enough' packets to have a data item that is 'processable'.

IMO, It's better to have one thread whose job is receiving data, and another to actually process it.

Answer 2

As Mitch points out, you need to be able to receive enough packets to have a complete message/frame . But there's no reason why you shouldn't start processing that frame immediately and issue another BeginReceive. In fact, if you believe your processing could take some time, you're better off handing it off to the worker thread-pool rather than block a thread from the i/o pool (which is where your callback will fire).

In addition, unless you're expecting a low number of connections, spawning a thread to handle each connection is not a very scalable approach, although it does have the benefit of some simplicity.

I recently wrote an article on pipelining data-processing off a network socket, which you can find here.