Batching messages in WCF?

Question

Here's my scenario ... I will be receiving a stream of individual messages from an outside source. The nature of this data and the processing I have to do to them is such that the more messages I can include in one batch, the more messages I can process on average. To use an example:

1. Messages 1 comes in, the WCF service starts processing the message
2. Messages 2 and 3 come in before step 1 is done processing. Process 2 and 3 together.

There are obviously a few assumptions here, that we can't start processing step 2 before step 1 is done, but we can process multiple messages in one run.

My initial thought was to simply use the msmq binding so that messages can queue up while one batch is processing. But then I realized that WCF will simply deliver each message one at a time. And I'm not even sure if it will wait until one message is done processing before pulling the next one and processing that one (perhaps in a thread?) which would be bad in this case.

So, I was just wondering if there might be some guidance someone could provide on how to accomplish such a system using WCF. thanks!

Answer 1

Maximizing Throughput

The solution is probably to not think in terms of WCF at all. You probably need a state machine that is running on a separate thread from the ServiceHost. That state machine will host, e.g., wrap, a Queue or similar type of class. The state machine will be supplied a delegate that will be used for processing the messages. When your WCF ServiceHost receives messages, it will push the messages onto the queue that is encapsulated within your state machine, and that is all it will care about.

As the state machine iterates at your designated interval, it will pop messages from the queue in whatever quantities you desire. Ideally, you should design this to be throttled from within your service's configuration settings. This will allow you to experiment to what batch sizes work best, similar to how it can be useful to batch together row updates to database tables for performance.

Batching Messages

Granted, you could design your messaging envelope (basically, the type you are passing as a parameter to an OperationContract method) to basically be a container for 1 to N messages. That would certain optimize the delivery of bytes by minimizing the number of messages. However, whether one message arrives or N messages arrive simultaneously would not change the logic needed to properly process these messages, except for adding a loop of some sort to iterate through the N messages. The fact that you are willing to consider MSMQ suggests that your messaging paradigm is already asynchronous, and so it should work well with the approach I describe for maximizing throughput.