Using Parallel.For to test SQL queries and comparison with the ThreadPool

Question

I'm looking for a way to easily load test and benchmark some of our SQL (using ADO.NET, nothing fancy using LINQ or PLINQ) that has to be performant when running under high parallel load.

I've thought of using the new parallel extensions CTP and specifically Parallel.For / Parallel.ForEach to simply run the SQL over 10k iterations or so - but I've not been able to find any data on what these have been optimized for.

Essentially I'm worried that because database access is inherently I/O bound, it won't create sufficient load. Does anyone know if Parallel.For is intelligent enough to use > x threads (where x = # of CPUs) if the tasks it is executing are not totally CPU bound? I.e. does it behave in a similar manner to the managed thread pool?

Would be rather cool if it was so!

EDIT: As CVertex has kindly referred to below, you can set the number of threads independently. Does anyone know if the parallel libraries by default are intelligent enough to keep adding threads if a job is I/O bound?

Answer 1

Sure can!

You can specify the max number of threads per CPU you want.

Before you Parallel.For whatever you're doing, you'll have to instantiate your own TaskManager from the System.Threading.Tasks namespace. Look at the ctor parameters to see how you can customize a task manager for your own purposes.

There should be an overload for Parallel.For that takes a task manager instance.

Answer 2

Another way is to define the PLINQ_DOP environment variable. From the docs:

PLINQ_DOP
DOP stands for degree of parallelism. Setting this environment variable defines the number of threads for PLINQ to use. 
E.g. PLINQ_DOP=1 means single-threaded, while PLINQ_DOP=8 means PLINQ should use 8 threads. 
If this is set to a value greater than the number of procs*cores available on the system, 
PLINQ will use more threads than processors. If one of them blocks, for instance, 
this allows other threads to make forward progress.