Queueserver --> java appservers --> cluster of calculation servers

Question

For the problem at hand, see the title. A certain "calculation service" (cpu intensive) is served to the clients as follows: all client requests go to one machine that manages the queue and - when a server or cpu is free in the cluster - forwards the request to a java appserver. The appserver uses ejb for its workflow mangement and at one point in the workflow the calculation service must be invoked (*). The results of the calculation are managed in the workflow.

Questions: - the call to the calculation service must be a RMI or do other options exist? - which products support this architecture "out of the box" (so to speak).

(*) It is invoked as a standalone java program, that uses JNI internally.

Answer 1

Any form of RPC will do. RMI is a good solution, but I prefer to use Spring Remoting. It lets you define an interface, and inject an implementation of that interface that just so happens to do the work remotely. I think that'd suit what you want to do nicely.

Answer 2

The calculation server instances could all be listening on the same queue. When then are ready for a new calculation they read an item from the queue, do the work, and pop the answer on a response queue.

This is a very simple way to enable some number of engines to service requests. The availability characateristics of the queueing system need to be understood and controlled.

The Workflow engine would need to pop send the request to calculation server and then wait for its response. This implies that the workflow engine would have some kind of "pause for external event" capability. As workflows could also reasonably wait for humans to respond to questions this is quite a common feature.

There exists at least one commercial workflow engine that would naturally work like this.