How to deal with OpenMP thread pool contention

Question

I'm working on an application that uses both coarse and fine grained multi-threading. That is, we manage scheduling of large work units on a pool of threads manually, and then within those work units certain functions utilize OpenMP for finer grain multithreading.

We have realized gains by selectively using OpenMP in our costliest loops, but are concerned about creating contention for the OpenMP worker pool as we add OpenMP blocks to cheaper loops. Is there a way to signal to OpenMP that a block of code should use the pool if it is available, and if not it should process the loop serially?

Answer 1

You may be able to do what you want by clever use of omp_get_num_threads, omp_set_num_threads, and if and num_threads clauses on parallel directives. OpenMP 3.0 also provides tasks which might be useful.