Background processing in rails

Question

Hi,

This might seem like a FAQ on stackoverflow, but my requirements are a little different. While I have previously used BackgroundRB and DJ for running background processes in ruby, my requirement this time is to run some heavy analytics and mathematical computations on a huge set of data, and I need to do this only about the first 15 days of the month. Going by this, I am tempted to use cron and run a ruby script to accomplish this goal.

What I would like to know / understand is: 1 - Is using cron a good idea (cause I'm not a system admin, and so while I have basic idea of cron, I'm not overly confident of doing it perfectly) 2 - Can we somehow modify DJ to run only on the first 15 days of the month (with / without using cron), and then just stop and exit once all the jobs in the queue for the day are over (don't want it to ping the DB every time for a new job...whatever the jobs will be in the queue when DJ starts, that will be all).

I'm not sure if I have put the question in the right manner, but any help in this direction will be much appreciated.

Thanks

Answer 1

Yeah, why not? Go with cron. It's really well-tested in the wild, well suited for running periodical tasks and incredibly easy to use. You don't even need to learn the crontab syntax (although it's very easy) - just drop your script into /etc/cron.daily (this option might be available only on some Linux distros).

I'm not sure about the "only first fiteen days of the month" thing, but you can easily handle this condition inside your task, right?

EDIT:

Check out par's answer to see how to run the task only at a certain range of days.

Answer 2

With cron's "minute hour day month dayofweek" time specification, 3:33am 1st through fifteenth of every month would be "33 3 1-15 * *"

Answer 3

Using cron would be really easy, and you have a lot of example and it is reliable.

Anyway here are few screen casts from Railcasts you may want to look at:

Starling and Workling

Custom Daemon

Answer 4

I also had this requirement. I followed the "Automatic Periodic Tasks" recipe 75 in the Advanced Rails Recipes book. The recipe is written by David Bock. It has some code snippets and guidelines on how this can be achieved using cron and capistrano. However there is an unsolved (but mentioned) issue regarding users/permissions that has to be on the target machine. It is not really difficult to make it right, you just have to remember to do it and put it in you deployment capistrano scripts.

It seems that David Bock has continued to work on this and has now creaated a gem for use with cron: See his blog, and follow crondonkulous on github. Crondonkulous may very well take care of this user/permission thing and more, I haven't tried it.

Jarl