An approach towards a Distributed Computing problem using the .Net framework

Question

Im interested in programming a Project which distributes a certain computation on large files throughout several computers. The need for distributed computing arises from the crashy and unstable nature of the software I'm using to do the actual computing - so it might crash on some computers but others will surely do the job. The ideas I have so far Include: -Using several servers, each pulling a task from a master server whenever its possible -Using VMwares -Using a load-balancing Cluster

What is more suited for the job? Any other ideas I should be aware of?

Also, If you can recommend any reliable distributed computing C# framework, it will be helpful.

Answer 1

Have you looked at Hadoop MapReduce? It's an open-source implementation of Google's MapReduce framework. Though it's Java and not C#, it sounds like it could be perfectly suited to your scenario; the master server automatically handles load balancing and fault-tolerance in a distributed environment.

Answer 2

haven't used any of these myself (yet), but I bookmarked this question a little while ago. some good suggestions there.