Software needed to build a cluster

Question

I've been thinking about getting a little bit greener with my computers and using some lower power, mini-itx boards in my next computer. Some can generate under 10 watts and are pretty inexpensive.

So I though, if one is such low cost and low power, why not try to make a cluster out of them? However, I'm not really sure what I would need to do in terms of OS or management software to make this happen?

Can anyone provide advice on existing software to do this or any ideas as to how to design my own?

Answer 1

You can use Beowulf to run a cluster.

Answer 2

Take a look at beowulf to get started.

That being said, the best advice I can give is to carefully measure whether you are actually being more green with your cluster. I've been a little way down this road before, and in my experience, the losses involved in having many separate computers end up wiping out any energy savings. Keep in mind that every computer needs a power supply, which converts your household voltage down to a level that the computer wants. The conversion is inefficient, and wastes heat (this is why the power supplies have fans). The same can be said for each hard drive, RAM bank and motherboard that you need.

This isn't meant to discourage you from the project. Just be sure to profile. Exactly like writing software! :)

Answer 3

What do you want to actually do with your cluster kind of decides what software you will need.

• Do you need job scheduling?
• Monitoring tools?
• Do you need to deploy software across all nodes at once seamlessly?
• One file system across all nodes (recommended).

You could just as easily install a linux or *BSD on the boards and just use ssh to manage and run jobs across all the nodes. No other software really required.

Software you might find useful:

• PBS (mostly job scheduling, google)
• Kerrighed (Single System Image based, Linux distro)
• Rocks (cluster based distribution)
• Mosix ( cluster Management, openMosix also )
• Ganglia (Monitoring, probably over kill for you)
• Lustre (Super fast, opensource cluster filesytem from Sun)

Answer 4

There's a lot to this question.

First, if you just want to get a cluster up and running, there are many suggestions listed already here. Once you have the cluster up and running, though, you're just starting.

At that point, you need to have software that will work correctly across the cluster. If you are working on your own software, you'll need to design it to be parallelized across a cluster, using something like MPI.

Without software written to run across the cluster, though, the cluster is nothing but a highly customized box that doens't do anything special...