I am developing an application that has the potential for massive parallel processing. Currently, I am using a single PC with 8 cores, but I need to support distributed computing. TPL Constructs such as PLINQ and Parallel.ForEach could be used to naturally distribute work load between different threads.
How could they be made to distribute workload between machines? If applicable, is there a favorable Distributed Computing framework for this specific task?
Edit:
I would like to clarify the requirements, given the suggestion to use the Dryad Project.
As far as I understand, Dryad is used for massively parallel manipulation of data.
The given application has many objects (to be distributed between machines), each object is connected to one or more of the other objects. Iteration over an object's collection alters the members of the collection. The topology is a graph that is quite complex. You could imagine a Neural Network as an example of a similar data structure. Now, I have two reservations re this solution.
The first is that the "resulting data" is a method call for each of the objects, and although "code is data", making such manipulation would be to the detriment of simplicity.
Given sufficient performance gains, simplicity could be waived, but in this case flattening the graph in order to enable such "stream processing" would also be highly inefficient - which is my second reservation.