High Throughput and Windows Workflow Foundation

Question

Can WWF handle high throughput scenarios where several dozen records are 'actively' being processed in parallel at any one time?

We want to build a workflow process which handles a few thousand records per hour. Each record takes up to a minute to process, because it makes external web service calls.

We are testing Windows Workflow Foundation to do this. But our demo programs show processing of each record appear to be running in sequence not in parallel, when we use parallel activities to process several records at once within one workflow instance.

Should we use multiple workflow instances or parallel activities?

Are there any known patterns for high performance WWF processing?

Answer 1

I think the common pattern is to use one workflow instance per record. The workflow runtime runs multiple instances in parallel.

One workflow instance runs one thread at a time. The parallel activity calls Execute method of each activity sequentially on this single thread. You may still get performance improvement from parallel activity however, if the activities are asynchronous and spend most of the time waiting for external process to finish its work. E.g. if activity calls an external web method, and then waits for a reply - it returns from Execute method and does not occupy this thread while waiting for the reply, so another activity in the Parallel group can start its job (e.g. also call to a web service) at the same time.

Answer 2

You should definitely use a new workflow per record. Each workflow only gets one thread to run in, so even with a ParallelActivity they'll still be handled sequentially.

I'm not sure about the performance of Windows Workflow, but from what I have heard about .NET 4 at Tech-Ed was that its Workflow components will be dramatically faster then the ones from .NET 3.0 and 3.5. So if you really need a lot of performance, maybe you should consider waiting for .NET 4.0.

Another option could be to consider BizTalk. But it's pretty expensive.

Answer 3

Hi, I want to create work flow , which should be continuously run on server and after some specific time it should run. I found delay activity use but I could not get how I can run continuously run.

Thanks in advance..

Answer 4

Hi! I have a question:

I think the common pattern is to use one workflow instance per record. The workflow runtime runs multiple instances in parallel.

Given this scenario, how can I limit the amount of instances? is there any queuing/scheduling mechanism in WF4?

thanks!