What approach would you recommend for implementing web-based schedulers

Question

Hi,

I have found that I often have to implement some sort of a scheduler in the applications I develop. The applications can range from simple maintenance tasks to fairly complex.

So far my approach has been to set up cron jobs that essentially do batch processing of queued commands. For example, I have cron invoking a script (I am working in RubyOnRails so this is a runner script) every 5 minutes, which checks if there are any items that need to be processed, then delegates the tasks to appropriate handlers.

This works, but for some reason doesn't feel like the best approach. Can you recommend something, or have any comments on this?

I working in Ruby on Rails, but there is no reason this discussion should be limited only to RoR.

Thanks,

Answer 1

A number of ways to do this. For one project we recently developed we used delayed_job, an excellent async worker tool for Rails. We can set a job up to run every 5 minutes, and the job then creates another job once it's done, and so on.

Other tools like BackgroundRB actually support a cron-style worker by default.

I often find that doing a once-per-minute rake task to add new jobs to delayed_job works really well; you might want to try that. It's the most resilient and lets you keep all the timer logic in Ruby while not requiring ugly hacks or potentially breakable setups like my initial delayed_job example (eg you have to purge the work queue; you now have to reset all your jobs again to get them working properly).

Delayed_job is extremely easy to work on and very hackable, so if you're looking for somewhere to start that's probably as good a place as any.

Answer 2

Can't see why you wouldn't use cron and script/runner. The benefit is that you don't have your (additionally required) Rails environment always loaded, only loaded when you need to run the job.

Downside is that it depends on cron being present ...?

Answer 3

Another suggestion is to create your own custom daemon. This will give you the flexibility to do pretty much anything you need.

It was covered in a excellent Railscast screencast - http://railscasts.com/episodes/129-custom-daemon