A generic C++ library that provides QtConcurrent functionality?

Question

QtConcurrent is awesome.

I'll let the Qt docs speak for themselves:

QtConcurrent includes functional programming style APIs for parallel list processing, including a MapReduce and FilterReduce implementation for shared-memory (non-distributed) systems, and classes for managing asynchronous computations in GUI applications.

For instance, you give QtConcurrent::map() an iterable sequence and a function that accepts items of the type stored in the sequence, and that function is applied to all the items in the collection. This is done in a multi-threaded manner, with a thread pool equal to the number of logical CPU's on the system.

There are plenty of other function in QtConcurrent, like filter(), filteredReduced() etc. The standard CompSci map/reduce functions and the like.

I'm totally in love with this, but I'm starting work on an OSS project that will not be using the Qt framework. It's a library, and I don't want to force others to depend on such a large framework like Qt. I'm trying to keep external dependencies to a minimum (it's the decent thing to do).

I'm looking for a generic C++ framework that provides the same/similar high-level primitives that QtConcurrent does, and that works with STL collections. AFAIK boost has nothing like this (I may be wrong though). boost::thread is very low-level compared to what I'm looking for (but if the requested lib used boost::thread for the low-level work, that would be great).

I know C# has something very similar with their Parallel Extensions so I know this isn't a Qt-only idea.

What do you suggest I use?

Answer 1

Can you use Boost? I don't think it provides quite as high-abstraction a layer as Qt, but it should be possible to make one as a reasonably thin facade on top of Boost's primitives (indeed, maybe some of the existing add-ons already provide what you require -- I have to admit I'm not familiar with them in detail, which is why I say "maybe";-).

If you find out that existing add-ons are unsuitable, your facade would be an excellent add-on to contribute to the Boost Vault (or other open-source repo) yourself, "giving back" a useful reusable open-source contribution... I hope this motivates you to do this work if needed!-)

Answer 2

I've heard good things about Intel's Threaded Building Blocks, though I haven't used it

As of Oct 2009, it doesn't seem to have map-reduce specifically. But people have expressed interest and suggested they were going to come up with something:

http://software.intel.com/en-us/forums/showthread.php?t=65053

"map reduce looks like a simple combination of a filter, a sort, and a reduction but it might need some magic to get it to be efficient"