Open source distributed computing/cloud computing frameworks

Question

I was wondering if anyone knows of any good Open Source distributed computing projects? I have a computationally intensive program that could benefit from distributed computing (a la SETI@Home, etc.) and want to know if anyone has seen such a thing or will I be developing it from scratch?

Answer 1

Depends on if you want to run the machines or let others run your workers. I don't know o f the latter, but Hadoop and map-reduce would be a way to do the former.

Answer 2

Here's one for java and one for c# and here's an open source grid toolkit.

Answer 3

SETI@Home uses BOINC

Answer 4

MPAPI - Parallel and Distributed Applications Framework.

• Sector 0 Article: http://sector0.dk/?page%5Fid=15.

• Gives a good overview of the
  framework, architecture and the
  theory behind it.

• Works on a single machine to 'n'
  machines.

• Design distributed logic into the
  system.

• Focuses on message passing to isolate the state that each thread has access to i.e. no shared state only
  messages.

• Is Open Source =] and is MONO Compatible YAY!

Architecture in a Nutshell

Cluster

• Single Main Node
• Controls the cluster
• Numerous Sub-Nodes (one per machine) which are the work horses of the cluster
• Single Registration Server - Binds the cluster together by allowing nodes to register / unregister with cluster notifying existing nodes

Communication

• Node to Node directly. Each worker communicates with others through the node.
• The messages are not propagated down through the remoting layer unless two workers are on different nodes.

Answer 5

I see that this is over a year old but this is a new and relevant answer

http://openstack.org/