What are the developments going on in all languages in parallel programming area

Question

As we are nearing the 8-Core and 16-Core CPUs, what are the various advances going on in parallel programming area in all the languages? I am aware of a parallel task library in .Net, and also of a language Erlang which I think is inherently parallel. But what is the state of various other mainstream languages.

Answer 1

POSIX thread library has been the standard for C/C++ for some years now. MPI also serves as a parallel library, albeit low-level. Windows has its own threads library for C/C++ as well.

Ada has always supported concurrency.

Java supports threads pretty natively as well(I believe).

I believe there is a concurrent LISP out there, but don't quote me on that.

Answer 2

Have a look at Software Transactional Memory, which, while not exactly new, is getting renewed attention with the increasing prevalence of multicore architectures.

Simon Peyton-Jones and Tim Harris did a fun video on the topic for Channel 9 as well.

Answer 3

Cilk is an extension to ANSI C that tries to make parallel programming in C better.

Answer 4

Writing multi threaded applications that can truly make use of multiple cores/processors in C/C++ is like writing UIs in assembly. It's possible, but very hard.

Functional programming and DAGs is the way to go.

Answer 5

MultiLisp is a parallelized dialect of Lisp that's actually been around for quite awhile, but now it's much more useful.

Answer 6

Parallel Languages and Extensions:

• Parallel Computing Platform for .Net
• Erlang
• Parallel Haskell
• OpenML
• Unified Parallel C
• Chapel
• Fortress

Check out Computer Science theory on Futures and Promises relating to concurrent programming.

Also Wikipedia's entry on Parallel Programming Models.

I think all functional languages are becoming more mainstream to address the multi-core problem. They have some great inherent properties that make them more apt to parallel processing. Many of them use message passing instead of shared memory, which helps to make concurrent programming easier, but doesn't necessarily eliminate the problems that can occur like dead locks, etc.

Imperative languages like C, Java, and C# are a few of the popular languages used today due to the surge of Object-Oriented methodology, and in part because of the continual improvements in clock speed allowing programs written using these languages to get performance for free. The specific threading libraries like ParallelFX for the .Net platform are being created to help the programmer using the language and runtime a much easier abstraction over all of the complexity parallelism brings to the table.

Now, however we need to create programs that are as side-effect free as possible in order to utilize the parallel processing of many cores. This is where functional languages excel as they are an even higher level abstraction relative to imperative languages, allowing you to define what you want, not exactly how you want it done.

I think that compilers and runtimes will eventually be able to help parallize most code that can inherently be run in parallel. However, at least today there are many circumstances (mostly I/O - be it keyboard, screen, disk) where things cannot be strictly parallel, and must be completed in some order. In these cases many cores will not solve the problem.

An interesting thing is that Web Sites are developed as single threaded programs, that get run as multiple separate processes. This allows web servers to run in a very parallel fashion even though the code itself is developed in a single threaded mindset.

Answer 7

1. Map-Reduce libraries are becoming more and more common for data parallel processing.
2. Lock free data structures. Instead of using mutex and locks, use CAS (compare and swap) at appropriate places. They tend to scale much better w.r.t number of cores than lock based counter parts.

Answer 8

PARLANSE is an SMP capable language designed for fine-grain, symbolic computation. We've coded a very large program analysis/transformation tool in it. See http://www.semdesigns.com/products/parlanse/index.html

See this link for PARLANSE parallel iterative deepening search for a solution to the 15-puzzle, which is the 4x4 big-brother of the 8-puzzle.