Is it too early to start designing for Task Parallel Library?

Question

I have been following the development of the .NET Task Parallel Library (TPL) with great interest since Microsoft first announced it.

There is no doubt in my mind that we will eventually take advantage of TPL. What I am questioning is whether it makes sense to start taking advantage of TPL when Visual Studio 2010 and .NET 4.0 are released, or whether it makes sense to wait a while longer.

Why Start Now?

• The .NET 4.0 Task Parallel Library appears to be well designed and some relatively simple tests demonstrate that it works well on today's multi-core CPUs.
• I have been very interested in the potential advantages of using multiple lightweight threads to speed up our software since buying my first quad processor Dell Poweredge 6400 about seven years ago. Experiments at that time indicated that it was not worth the effort, which I attributed largely to the overhead of moving data between each CPU's cache (there was no shared cache back then) and RAM.
• Competitive advantage - some of our customers can never get enough performance and there is no doubt that we can build a faster product using TPL today.
• It sounds fun. Yes, I realize that some developers would rather poke themselves in the eye with a sharp stick, but we really enjoy maximizing performance.

Why Wait?

• Are today's Intel Nehalem CPUs representative of where we are going as multi-core support matures? You can purchase a Nehalem CPU with 4 cores which share a single level 3 cache today, and most likely a 6 core CPU sharing a single level 3 cache by the time Visual Studio 2010 / .NET 4.0 are released. Obviously, the number of cores will go up over time, but what about the architecture? As the number of cores goes up, will they still share a cache? One issue with Nehalem is the fact that, even though there is a very fast interconnect between the cores, they have non-uniform memory access (NUMA) which can lead to lower performance and less predictable results. Will future multi-core architectures be able to do away with NUMA?
• Similarly, will the .NET Task Parallel Library change as it matures, requiring modifications to code to fully take advantage of it?

Limitations

• Our core engine is 100% C# and has to run without full trust, so we are limited to using .NET APIs.

Answer 1

I would start now. I strongly suspect that we've seen the bulk of the changes - even if there are a few tweaks in the release candidate, I'm sure they'll be well documented in the PFX team blog, and easy to change. Even if chips change, I'd expect the TPL to adapt appropriate in future versions - and I would personally expect that the current TPL is still likely to do a better job of handling those new chips than any hand-crafted threading code mere mortals like us could write.

The one real downside I see to starting now is that the learning resources aren't really there yet. There's some documenation, some blog posts (some of which will be outdated by now) and some sample code - but no books dedicated to PFX. I'm sure those will come in time though - and if you're early in the game, you could even write one :)

Depending on your application, you might also want to look at Reactive Extensions, which works hand-in-hand with PFX.

Answer 2

In the end, it matters more if your core engine can benefit from parallelism in general. Does it have lots of shared state that needs to be guarded with locks? if that is true, can that be easily moved to a design centered around lock-free data structures?

I think those questions must be answered first, so that you can them have a clearer picture to assess if TPL may help down the road.

Answer 3

Well - your main reason against using the TPL today seems to be the following:
You don't know whether the TPL will take the maximum out of the future's multicore-CPUs.

I would say (that's only a guess - especially in computer science you will never be able to tell what happens next): Yes, they will change. And yes, the TPL will be changed at some points to maximize performance. However, some changes will be "under the hood" - you will profit from optimizations without actually changing a single line of code.

And even if there are changes in architectures which result in higher performance only in combination which changing your code: I don't think those changes will affect your entire code - maybe some percent were every single millisecond is very important.

And where are the alternatives? Using the Threadpool? Well - then the TPL is much more up to date. Therefore, your code will be more future-proof when using it IMHO. For example, the demos of VS 2010's debugging features look quite nice.

In addition, the TPL seems to be quite flexible in my eyes - if it doesn't fit in a specific situation you do not have to use it there. On the other hand it simplifies development mushc at other locations.

I think the most important thing today is to think about parallelization and include it into the architecture. The TPL makes this process much easier.

Therefore, my conclusion: Use it!

Answer 4

I'd go for it as well. The big change in my opinion is the "paradigm" shift from the "we did it this way for the last 8 years" development to a more functional/side-effect free programming.

If you start using PFX today I'd guess that it will take some time to get up to speed with it and literally port your code to get the most from it. If on the other hand PFX goes away in 2 years or brings major changes, I'd expect your code still to work far better using whatever you'll get there. We won't decrease the number of cores again and the big tasks to scale better won't be obsoleted for quite a while.

Regarding CPU architecture changes: I cannot really comment on those, except that in my opinion your investment now will result in a business advantage now+X month away. X is probably smaller than the time until major changes in CPU development happen -> You win.

Answer 5

I wouldn't wait either.

In fact, I would go further and say don't wait for VS2010/.NET 4.0. The TPL is available for .NET 3.5 now as part of the Reactive Extensions for .NET.