Use Delayed::Job to manage multiple job queues

Question

I want to use Delayed::Job (or perhaps a more appropriate job queue to my problem) to dispatch jobs to multiple background daemons.

I have several background daemons that carry out different responsibilities. Each one is interested in different jobs in the queue from the Rails app. Is this possible using Delayed::Job, or perhaps there is a different job queue that better fits this task?

Answer 1

DJ workers just grab the first job from the queue (there's only one) and do it. Each worker is the same as any other. You can run multiple workers, but they will all work from the same queue.

Answer 2

If you like the Delayed Job, it'd be simple enough to create multiple tables, one per queue

The worker gets it's table name from

class Job < ActiveRecord::Base
    MAX_ATTEMPTS = 25
    MAX_RUN_TIME = 4.hours
    set_table_name :delayed_jobs

So you could edit that and create one migration per table. And then when you start your jobs, you'll start one worker per queue.

Source: http://github.com/tobi/delayed_job

Answer 3

Resque is a job queue library that supports multiple queues.