Concurrent programming techniques, pros, cons

Question

There is at least three well-known approaches for creating concurrent applications:

1. Multithreading and memory synchronization through locking(.NET, Java). Software Transactional Memory (link text) is another approach to synchronization.

2. Asynchronous message passing (Erlang).

I would like to learn if there are other approaches and discuss various pros and cons of these approaches applied to large distributed applications. My main focus is on simplifying life of the programmer.

For example, in my opinion, using multiple threads is easy when there is no dependencies between them, which is pretty rare. In all other cases thread synchronization code becomes quite cumbersome and hard to debug and reason about.

Answer 1

Read Herb Sutter's Effective Concurrency column, and you too will be enlightened.

Answer 2

I'd strongly recommend looking at this presentation by Rich Hickey. It describes an approach to building high performance, concurrent applications which I would argue is distinct from lock-based or message-passing designs.

Basically it emphasises:

• Lock free, multi-threaded concurrent applications
• Immutable persistent data structures
• Changes in state handled by Software Transactional Memory

And talks about how these principles influenced the design of the Clojure language.

Answer 3

With the Java 5 concurrency API, doing concurrent programming in Java doesn't have to be cumbersome and difficult as long as you take advantage of the high-level utilities and use them correctly. I found the book, Java Concurrency in Practice by Brian Goetz, to be an excellent read about this subject. At my last job, I used the techniques from this book to make some image processing algorithms scale to multiple CPUs and to pipeline CPU and disk bound tasks. I found it to be a great experience and we got excellent results.

Or if you are using C++ you could try OpenMP, which uses #pragma directives to make loops parallel, although I've never used it myself.