Advice on building a distributed application?

Question

Hi Guys,

I'm looking into building a distributed application and would like some advice on the best way to go about it.

My system will comprise of a central management web application where tasks will be defined, and a number of task runners which will be installed on different machines to processes those tasks.

What I need to work out is how to distribute the tasks to the clients? Should I just create a queue of tasks, and the task runners constantly poll the queue to see if a new task is ready? What would be the best way to ensure a task is only processed by one task runner? How could I also ensure tasks get evenly distributed between the task runners, so that one task runner doesn't end up processing loads, and other only process a handfull?

The only other approach I could think of is that each task runner makes a TCP connection to the server and registers it's interest, then when a new task is ready, the web app chooses a task runner and pushes the task down to that task runner to process?

Anybody got any other ideas? pointers? or comments?

Many thanks

Matt

Answer 1

A general approach to this problem would be:

• Implement a set of WCF services that expose the "task running" functionality as operations;

• Deploy the WCF services on several hosts/machines hosted as Windows services (they all must use the same contract);

• Configure the WCF services to use the Microsoft Message Queuing binding (netMsmqBinding), all using the same queue, in transactional mode;

• Configure the individual management applications to send their requests to the message queue, using the same netMsmqBinding;

• Optionally, deploy the management apps/services in a load-balanced network such as an IIS Web Farm (if you want more scalability on the front-end).

This is probably the most frictionless approach as you can rely on proven tools to handle the load balancing (IIS/NLB), message security and serialization (WCF), and message delivery and transactional management (MSMQ). Most of the work is already done for you.

Of course, there's a lot of ground to cover with respect to the actual implementation and deployment, but that might be a bit too much material to cover in a single question/answer here. This should at least point you in the right general direction.

Answer 2

+1 for Aaronaught but I would also suggest looking at something like NServiceBus which can abstract a fair bit of the implementation details away for you. You can still use MSMQ and/or WCF to do the actual transport, just makes setting up publishers and subscribers easier.

Answer 3

I wrote something just like this a while back, and I chose what I felt was a simple method, pretty much what you described: each agent (task runner) registers itself with the "management server" for admin purposes. It then polls the server periodically for work. If there is anything in the queue, it takes the first (oldest) job. The server marks this job as in-progress so no other agents will take it. Load balancing wasn't an issue because each agent only ran 1 job at a time. WCF took care of the message queuing very nicely, such that there was no concern of two agents taking the same job. When an agent finishes a job, it tells the server and passes back any output.

I defined three WCF endpoints for my management server: one for the agents, one for the "console" used to submit jobs (a winforms app), and one for the admin tool. My management server was implemented as a web service hosted in a Windows service.

I know this doesn't answer all your questions, but this is an example of a working solution: Whether it will work for you depends on the complexity of your requirements.