Threading paradigm?

Question

Are there any paradigm that give you a different mindset or have a different take to writing multi thread applications? Perhaps something that feels vastly different like procedural programming to function programming.

Answer 1

Concurrency has many different models for different problems. The Wikipedia page for concurrency lists a few models and there's also a page for concurrency patterns which has some good starting point for different kinds of ways to approach concurrency.

The approach you take is very dependent on the problem at hand. Different models solve various different issues that can arise in concurrent applications, and some build on others.

In class I was taught that concurrency uses mutual exclusion and synchronization together to solve concurrency issues. Some solutions only require one, but with both you should be able to solve any concurrency issue.

For a vastly different concept you could look at immutability and concurrency. If all data is immutable then the conventional approaches to concurrency aren't even required. This article explores that topic.

Answer 2

I don't really understand the question, but if you start doing some coding using CUDA give you some different way of thinking about multi-threading applications.

It differs from general multi-threading technics, like Semaphores, Monitors, etc. because you have thousands of threads concurrently. So the problem of parallelism in CUDA resides more in partitioning your data and mixing the chunks of data later.

Just a small example of a complete rethinking of a common serial problem is the SCAN algorithm. It is as simple as:

• Given a SET {a,b,c,d,e}

I want the following set:

{a, a+b, a+b+c, a+b+c+d, a+b+c+d+e}

Where the symbol '+' in this case is any Commutattive operator (not only plus, you can do multiplication also).

How to do this in parallel? It's a complete rethink of the problem, it is described in this paper.

Many more implementations of different algorithms in CUDA can be found in the NVIDIA website

Answer 3

Well, a very conservative paradigm shift is from thread-centric concurrency (share everything) towards process-centric concurrency (address-space separation). This way one can avoid unintended data sharing and it's easier to enforce a communication policy between different sub-systems.

This idea is old and was propagated (among others) by the Micro-Kernel OS community to build more reliable operating systems. Interestingly, the Singularity OS prototype by Microsoft Research shows that traditional address spaces are not even required when working with this model.

Answer 4

The relatively new idea I like best is transactional memory: avoid concurrency issues by making sure updates are always atomic.

Answer 5

The essays of Rich Hickey on concurrent programming in Clojure provide an interesting read.

Answer 6

Have a looksee at OpenMP for an interesting variation.