I am about to start working on a project where we need to process a large amount of data throughout the day. Basically, as changes happen in our system, we will need to reprocess all units associated with the change. When this happens, we want to give highest priority to the units with a date closest to today, then process this week’s units, then the rest of the units for the year. As soon as a change happens, if we are processing the units for next month, we want to pause that processing until all of today’s units are processed, then process the rest of units in order of priority/date. It is also preferable if we can load balance this across multiple servers. So before I go write a bunch of my own buggy code, is there some kind of open source or 3rd party library that will handle all of this?
A:
I wrote this as a proof of concept I never got to field test this to know if it's a good solution or not but I wrote a built in Scheduler for ASP.NET that can run processes at designated intervals. Feel free to take a look at my work: http://dotnetchris.wordpress.com/?s=scheduler
I'd love to hear anyones input on this either here or at my blog. I have no idea what the implications of this would be to roll this out in full force.
Chris Marisic
2009-09-08 18:35:08
+3
A:
Quartz.Net. Open source, robust and very flexible. Supports load-balancing and much more.
Mauricio Scheffer
2009-09-08 18:41:16