(RAILS) Looping a Custom Rake every 2 mins, Bad Idea? Better way to implement?

Question

I've currently created a custom rake file that does the following:

1.) Checks an External Feed for "NEW" Items

2.) For each new item in the feed array,it updates my Database with a new record

3.) I've currently got it on a WHILE loop. The while loop has an (@loopcheck) instance variable that is set to true initially, and if any exception is raised. Resets it to false (so the loop ends).

Here's an example:

   While(@loopcheck) do 
      begin
       ....(code here)...
      rescue
        Exception => e
        @loopcheck = false
      end
      sleep(120)
   End

Is this bad coding? Is there a better way to do this? Ideally, I just want to run a background task to simply check for a new feed every 2-3 mins. I looked into Starling/Workling, but that seemed a bit like overkill, and I wasn't sure about running script/runner via CRON, since it reloads the entire rails environment each time. BackgroundRB a bit overkill too? No?

Just wanted to get some ideas.

Answer 1

Combine cron with the sleep. It will ensure that things don't completely break down if you hit an exception. Loading "complete rails environment" is not all that bad: 3-6 secs worst case. so run your sleep loop 5 times. and cron the rake task to run every 12 minutes. Makes sense?

*/12 * * * * rake your task

## your rake task - valid for 10 mins -- 5 x 2 mins sleep worth.

@loopcheck = true

1.upto(5) do 
    begin
       ....(code here)...
    rescue
      Exception => e
      @loopcheck = false
    end
    break if not @loopcheck
    sleep(120)
end

Answer 2

Check out this recent Railscast on "Whenever"

the link is here and the comments discuss alternatives

http://railscasts.com/episodes/164-cron-in-ruby

Answer 3

This simplest thing that could possibly work, assuming you're on a Unix box, is to run the script once every two minutes from cron, without the loop/sleep. That lets you avoid having to monitor and restart your script if it fails. And if it does fail, cron will send you an email.

The downside of this approach, as you've noted, is that every two minutes you're taking a hit for starting up a Ruby process. Is that significant? Only you can say. I have several scripts like this on a lightly loaded box, and forget they're even there.