I understand the differences between a multithreaded program and a program relying on inter-machine communication. My problem is that I have a nice multithreaded program written in 'C' that works and runs really well on an 8-core machine. There is now opportunity to port this program to a cluster to gain access to more cores. Is it worth the effort to rip out the pthread stuff and retrofit MPI (which I've never used) or are we better off recoding the whole thing (or most of it) from scratch? Assume we are "stuck" with C so a wholesale change of language isn't an option.
+1
A:
Depending on how your software is written, there may or may not be advantages to going to MPI over keeping your pthread implementation.
Unfortunately (or fortunately), message passing is a very different beast than pthreading - the basic assumption is quite different. I love this quote from Joshua Phillips of the Maestro team: "The difference between message-passing and shared-state communication is equivalent to the difference between sending a colleague an e-mail requesting her to complete a task and opening up her organizer to write down the task directly in her to-do list. More than just being rude, the latter is likely to confuse her – she might erase it, not notice it, or accidentally prioritize it incorrectly."
Unfortunately, the way you share data is very different. There is no direct access to data in other threads (since it can be on other machines), so it can be a very daunting task to migrate from pthreads to MPI. On the other hand, if the code is written so each thread is isolated, it can be an easy task, and definitely worthwhile.
In order to determine how useful this will be, you'll need to understand the code, and what you hope to achieve by switching. It can be worthwhile as a learning experience (you learn a LOT about synchronization and threading by working in MPI), but may not be practical if the gains will be minor.
Reed Copsey
2009-03-13 19:31:31
Thanks for the quick response. This program does have the advantage of a master-slave architecture. One thread launches many identical sub-threads. None of the sub-threads needs to communicate with any other. And the data exchanged between master and slave is small and infrequent.
Joey
2009-03-13 19:41:02
If that's the case, porting to MPI will probably be fairly straitforward. The other issue is how the threads communicate their results back to the main thread - is a lot of information passed back? Status and results will probably be your only worries, then.
Reed Copsey
2009-03-13 20:12:40
Thanks again. It's basically master setting making some data available to all subproblems, each of them doing some work based on that data, then each making a result available to the master. Master does some work and repeats. For coordination it's basically a "Sleeping Barber Problem".
Joey
2009-03-13 20:30:55
It should be a fairly easy port, then, and probably a good move if you have the cluster to run it on. What OS are you working with?
Reed Copsey
2009-03-13 21:16:36
The current 8-core box is RHEL 4, The cluster is SUSE-based. Everything is Linux which is nice. I had hoped that the simplicity of the program architecture would lead to a more straightforward port to MPI. If I didn't indicate before, it's all my own code using a couple libraries. Any other advice?
Joey
2009-03-13 21:32:27
If you're using linux, I'd recommend testing with OpenMPI (http://www.open-mpi.org/). It seems to work very well, and supports MPI-2, which has nice additions to the original MPI spec.
Reed Copsey
2009-03-13 21:54:41
A:
Re. your comment to Reed -- this sounds like an easy, low-overhead conversion to MPI. Just be careful: not all MPI APIs support dynamic creation of processes, i.e., you start your program with N processes (specified at startup) and you're stuck with N processes throughout the life-time of the program.
zvrba
2009-03-13 19:48:55
Thanks. I thought the comment might add detail that could help with suggestions. Your response speaks also to the potential ease of this conversion: everytime this program is run, it is known a priori how many threads (processes in MPI) will be needed. Thanks!
Joey
2009-03-13 20:07:14
Choosing the right MPI library for your target platform helps a lot here. Some MPI implementations are better than others for specific tasks, but programming to them should be very close, so it's usually fairly easy to try different options.
Reed Copsey
2009-03-13 20:13:38