Nesting Parallel Queries

Question

Are there any issues with running nested PLINQ queries?

For instance:

//Contains roughly 7000+ elements
mycollections.AsParallel().ForAll(x => { 

  //contains 12 elements
  anothercollection.AsParallel().ForAll(y => { 
     //download some data from the web and parse it
  });
});

Answer 1

There are no fundamental issues with using nested queries, so you can certainly do that and PLINQ will try to do its best to parallelize the code as efficiently as possible. However, it may be a thing to consider and you should definitely run some measurements if you want to get the best possible performance.

The best option depends on the number of elements in both of the collections and the time needed to run the processing.

• If the outer collection is small enough, then you'll need the inner loop too, so that you create enough potential for parallelisation (generate enough tasks for PLINQ, so that it can ballance the execution)

• If the outer collection contains a large number of elements, then the inner loop is likely unnecessary, because the single outer loop is enough to give PLINQ enough space for parallelization. In fact, the inner loop may be only adding overhead (though, from my experience, this happens only with a very large number of elements)

Also, if you're going to parallelize only a single loop, it should be the outer one. This way all work can be split at once and you'll incur the parallelization overhead only once.