ASP.Net scheduling timed events for large scale systems

Question

What would be the best practice for having hundreds if not thousands of timed events?

I have a C# ASP.Net web based application, this application sits in-between Microsoft Exchange and another web based service. Exchange is checked once every hour for users appointments, these appointments then needed to be forwarded to another web based service but not immediately. They should be forwarded based on proximity to the appointment, so say 15 minutes before the appointment is scheduled to begin. I was thinking of using scheduled tasks for this, I am not sure how a system would hold up with a large number of users though?

The checking of Exchange is not really a problem, but firing the other event based on the data fetched from Exchange is a little more tricky. I have looked at Quartz.Net and maybe this could be of some help. I am curious as to what other design solutions people can think of and or problems with my current design. Of course an ideal world an Exchange plug-in would be the preferred solution, however it is not really an option t the moment.

Thanks for listening!

Answer 1

Perhaps a better solution would be to store the events in a staging area and query that table every minute and push the relevant events to users. That would mean your only querying once every min, but how you push back to the user I'm not sure.

It's far from a fully thought through option, but may be enough to get you thinking in a different direction.

Answer 2

App probably ought not be a web app - should be a windows service ideally, but I do appreciate that its not necessarily that simple, what you do need to make sure therefore is that all the code that you fire is self contained and minimally dependent on the hosting app.

I think in terms of sending data, I'd probably do this: 1) Get the exchange processing to determine the time that the forwarding should occur for each item - so you have a table with a column "send at" and another "sent". 2) You then run a process every minute that says send everything not sent that is due to be sent by now. Slight wrinkle is that if it overruns you need to run it again immediately - if it consistently overruns you have a capacity problem.

Point about this is that its very, very simple - you have an process for exchange that is performing a clearly defined task and you have a high frequency task that is not doing anything particularly complex, its a straightforward query (select from table where sent = false and send at < value) - probably into temporary table so that you can run a single query update after you've done the sends - that you can optimise the index for. You're not trying to queue up a huge pile of event triggers, just one that fires once a minute and processes things that are due.