views:

239

answers:

2

I think Erlang-style concurrency is the answer to exponential growth of core count. You can kind of fake it with other main stream languages. But the solutions always leave me wanting. I am not willing to give up multi-paradigm programming (C++/D) to switch to Erlang's draconian syntax.

What is Erlang-style concurrency:

From one of the language authors(What is Erlang's concurrency model actually ?):

  • Lightweight concurrency.
    Cheap to create threads and cheap to maintain insane numbers.
  • Asynchronous communication.
    Threads only communicate via messages.
  • Error handling.
  • Process isolation.

Or from an informed blogger (What is Erlang-Style Concurrency?):

  • Fast process creation/destruction
  • Ability to support >> 10 000 concurrent processes with largely unchanged characteristics.
  • Fast asynchronous message passing.
  • Copying message-passing semantics (share-nothing concurrency).
  • Process monitoring.
  • Selective message reception.

I think D's message passing can accomplish most of these features. The ones I wonder about are ">>10,000 concurrent processes(threads)" and "fast process creation/destruction".

How well does D handle these requirements?

I think that to support them correctly you'd have to use green threads. Can D's message passing features be used with green threads library?

+2  A: 

From whatever little I know about D: its message passing infrastructure is built on top its threading facilities. If the core threading library is a wrapper on OS threads, there is little chance that concurrency in D will reach the magnitude (>> 10000) of Erlang. Moreover D do not enforce immutability on objects, so it is easy to mess things up. So, Erlang is the best choice for heavy concurrency. Probably you can write the concurrency stuff in Erlang and the rest of the project in D. Still, it is possible to have efficient green threads in C like languages (C++, D etc) - have a look at Protothreads and ZeroMQ. You can implement very efficient messaging frameworks using these, and calling them via a C shim or directly from D.

Vijay Mathew
You *can* make an object (deep) immutable in D with the `immutable` modifier. It is just not the default.
KennyTM
@KennyTM Yes, that's why I said "D do not *enforce* immutability on objects..."
Vijay Mathew
@Vijay: Yes, so? You can't send mutable data via D's message passing interface. The compiler will stop you if you send an object that is not `immutable`.
KennyTM
@KennyTM Is it possible for two concurrent objects in D to *share* a mutable object, say a global variable?
Vijay Mathew
@Vijay: Mutable global variables are "thread local" by default. It is possible to make it shared by adding the `shared` modifier (ref: http://www.digitalmars.com/d/2.0/migrate-to-shared.html.)
KennyTM
In short, D doesn't prevent the use of thread hostile features (it has inline ASM and direct access to C after all), but it makes you go a bit out of your way to use them. (And that's a big chunk of the design philosophy of D.)
BCS
As to doing light weight threads in C and calling that; don't, unless your using a library that's already written, there is very little that C can do that can't be done just a well in D.
BCS
+8  A: 

Storage is thread-local by default in D, so nothing is shared between threads unless it is specifically marked as shared. If you mark a variable as shared, you can then use the traditional mutexes and conditions as well as synchronized objects and the like to deal with concurrency. However, the preferred means of communicating between threads is to use the message passing facilities in std.concurrency and let all data stay thread-local, only using shared when you must. All objects passed between threads using std.concurrency must either be passed by value or be immutable, so no sharing occurs there and it is completely thread-safe. However, it can currently be a bit of a pain to get an immutable reference type which isn't an array (idup generally makes it easy for arrays), so it can be a bit annoying to pass anything other than value types or arrays (though hopefully that situation improves soon as compiler and standard library bugs relating to const and immutable get fixed and more code is made const-correct).

Now, while message passing in D will definitely result in cleaner, safer code than what you'd get in languages like C++ or Java, it is built on top of normal, C threads (e.g. Linux uses pthreads), so it does not have the kind of light-weight threads that Erlang does, and so dealing with multiple threads is not going to be as efficient as Erlang.

Of course, I don't see any reason why a more efficient thread system could not be written using D, at which point you might be able to get thread efficiency similar to that of Erlang, and it could presumably use an API similar to that of std.concurrency, but all of D's standard threading stuff is built on top of normal, C threads, so you'd have to do all of that yourself, and depending on how you implemented it and depending on how exactly the thread-local/shared stuff is dealt with by the compiler and druntime, it could be difficult to get the type system to enforce that everything be thread-local with your "green" threads. I'm afraid that I don't know enough about exactly how shared is implemented or how "green" threads work to know for sure.

Regardless, D's message passing system will certainly result in dealing with threads being more pleasant than C++ or even Java, but it's not designed to be streamlined in the same way that Erlang is. D is a general purpose systems language, not a language specifically designed to use threads for everything and thus to use them absolutely as efficiently as possible. A large portion of D's standard facilities are built on top of C, so a lot of its efficiency characteristics will be similar to those of C.

Jonathan M Davis