When to use Partitioner class?

Question

Can anyone suggest typical scenarios where Partitioner class introduced in .NET 4.0 can/should be used?

Answer 1

To parallelize an operation on a data source, one of the essential steps is to partition the source into multiple sections that can be accessed concurrently by multiple threads. PLINQ and the Task Parallel Library (TPL) provide default partitioners that work transparently when you write a parallel query or ForEach loop. For more advanced scenarios, you can plug in your own partitioner.

Read more here:

Answer 2

The Partitioner class is used to make parallel executions more chunky. If you have a lot of very small tasks to run in parallel the overhead of invoking delegates for each may be prohibitive. By using Partitioner, you can rearrange the workload into chunks and have each parallel invocation work on a slightly larger set. The class abstracts this feature and is able to partition based on the actual conditions of the dataset and available cores.

Example: Imagine you want to run a simple calculation like this in parallel.

Parallel.ForEach(Input, (value, loopState, index) => { Result[index] = value*Math.PI; });

That would invoke the delegate for each entry in Input. Doing so would add a bit of overhead to each. By using Partitioner we can do something like this

Parallel.ForEach(Partitioner.Create(0, Input.Length), range => {
   for (var index = range.Item1; index < range.Item2; index++) {
      Result[index] = Input[index]*Math.PI;
   }
});

This will reduce the number of invokes as each invoke will work on a larger set. In my experience this can boost performance significantly when parallelizing very simple operations.