Grid Computing and Java

Question

I couldn't seem to find a similar question to this.

I am currently looking at the best solution solving a grid computing problem.

The setup:

• I have a server/client situation where there clients [typically dumb of most logic] and recieve instructions from the server
• Have an authorization request
• Clients report back information on speed of completing the task (the task's difficult is judged by the task type)
• Clients recieve the best fit task for their previous performance (the best clients receive the worst problems)

Eventually the requirements would be:

• The client's footprint must be small and standalone - I can't have a client that requires lots to install and setup
• The client should be able to grab new jobs and job runtimes from the server (it would be nice to have the grid scale to new problems [and the new problems would be distributed by the server] that are introduced)
• I need to have an authentication layer (doesn't have to be complex or conform to an existing ldap) [easier requirement: clients can signup for a new "membership" and get access] (I'm not sure that RMI's strengths lie here)
• The clients would be able to run from the Internet rather in a networked environement
  • Which means encryption of the results requested

I'm currently using webservices to communicate between the clients and and the server. All of the information and results goes back to the hosting server (J2EE).

My question is there a grid system setup that matches all/most of these requirements, and is open source?

I'm not interested in doing a cloud because most of these tasks are small, but very frequent (once a day but the task may be easy, but performs maintenance).

All of the code for this system is in Java.

Answer 1

Take a look on Grid Beans

Answer 2

You may want to investigate space-based architectures, and in particular Jini and Javaspaces. What's Jini ? It is essentially RMI with a configurable discovery mechanism. You request an implementor of a Java interface, and the Jini subsystem finds current services implementing that interface and dynamically informs your service of these.

Briefly, you'd write the work items into a space. The grid nodes would be set up to read data transactionally from the space. Each grid node would take a work item, process it and write a result back into that space (or another space). The distributing node can monitor for results being written back (and.or for your projected result timings, as you've requested).

It's all Java, and will scale linearly. Because it's Jini, the grid nodes can dynamically load their classes from an HTTP server and so you can propagate code updates trivially.

Answer 3

BOINC sounds like it would work for your problem, though you have to wrap java for your clients. That, and it may be overkill for you.