.NET Distributed Code Execution Framework in C#

Question

We current operate a datacenter that aggregates a bunch of login data and stores it into a database.

We have several "jobs" that run periodically that operate on the data, perform statistics analysis, etc.

The current scheduler and job system is pretty basic, and I'd like to kick it in the nuts and send it into overdrive (just go with me on this).

I'd like to develop a scalable job execution and automated scheduler platform. I can handle the automated scheduling of resources no problem, but the scalable job execution engine is where I'd have to do some work.

I'd like to know if the .NET framework has support for this? I understand Azure will let developers produce distributed applications, but I believe these have to run on MSFT's datacenters.

Basically I'd like run my own mini windows azure environment.

Oh great S.O. community, is there any hope, or must I build my own?

Answer 1

It sounds like a distributed message queue would be good for you. MSMQ fits this case perfectly, as you can write to a public message queues, and distribute your actions (which you'd implement as windows services) across many servers.

Have you looked into this?