How to deploy resque workers in production?

Question

The GitHub guys recently released their background processing app which uses Redis: http://github.com/defunkt/resque http://github.com/blog/542-introducing-resque

I have it working locally, but I'm struggling to get it working in production. Has anyone got a:

1. Capistrano recipe to deploy workers (control number of workers, restarting them, etc)
2. Deployed workers to separate machine(s) from where the main app is running, what settings were needed here?
3. gotten redis to survive a reboot on the server (I tried putting it in cron but no luck)
4. how did you work resque-web (their excellent monitoring app) into your deploy?

Thanks!

P.S. I posted an issue on Github about this but no response yet. Hoping some SO gurus can help on this one as I'm not very experienced in deployments. Thank you!

Answer 1

I just figured this out last night, for Capistrano you should use san_juan, then I like the use of God to manage deployment of workers. As for surviving a reboot, I am not sure, but I reboot every 6 months so I am not too worried.

Although he suggest different ways of starting it, this is what worked easiest for me. (Within your deploy.rb)

require 'san_juan'
after "deploy:symlink", "god:app:reload"
after "deploy:symlink", "god:app:start"

To manage where it runs, on another server, etc, he covers that in the README, search for Configuration.

I use Passenger on my slice, so it was relatively easy, I just needed to have a config.ru file like so:

require 'resque/server'

run Rack::URLMap.new \
  "/" => Resque::Server.new

For my VirtualHost file I have:

<VirtualHost *:80>
        ServerName resque.server.com
        DocumentRoot /var/www/server.com/current/resque/public

        <Location />
          AuthType Basic
          AuthName "Resque Workers"
          AuthUserFile /var/www/server.com/current/resque/.htpasswd
          Require valid-user
        </Location>
</VirtualHost>

Also, a quick note. Make sure you overide the resque:setup rake task, it will save you lots of time for spawning new workers with God.

I ran into a lot of trouble, so if you need any more help, just post a comment.

Answer 2

Garrett's answer really helped, just wanted to post a few more details. It took a lot of tinkering to get it right...

I'm using passenger also, but nginx instead of apache.

First, don't forget you need to install sinatra, this threw me for a while. sudo gem install sinatra

Then you need to make a directory for the thing to run, and it has to have a public and tmp folder. They can be empty but the problem is that git won't save an empty directory in the repo. The directory has to have at least one file in it, so I made some junk files as placeholders. This is a weird feature/bug in git.

I'm using the resque plugin, so I made the directory there (where the default config.ru is). It looks like Garrett made a new 'resque' directory in his rails_root. Either one should work. For me...

cd MY_RAILS_APP/vendor/plugins/resque/
mkdir public 
mkdir tmp
touch public/placeholder.txt
touch tmp/placeholder.txt

Then I edited MY_RAILS_APP/vendor/plugins/resque/config.ru so it looks like this:

#!/usr/bin/env ruby
require 'logger'

$LOAD_PATH.unshift File.expand_path(File.dirname(__FILE__) + '/lib')
require 'resque/server'

use Rack::ShowExceptions

# Set the AUTH env variable to your basic auth password to protect Resque.
AUTH_PASSWORD = "ADD_SOME_PASSWORD_HERE"
if AUTH_PASSWORD
  Resque::Server.use Rack::Auth::Basic do |username, password|
    password == AUTH_PASSWORD
  end
end

run Resque::Server.new

Don't forget to change ADD_SOME_PASSWORD_HERE to the password you want to use to protect the app.

Finally, I'm using Nginx so here is what I added to my nginx.conf

server {
  listen   80;
  server_name  resque.seoaholic.com;
  root /home/admin/public_html/seoaholic/current/vendor/plugins/resque/public;
  passenger_enabled on;
}

And so it gets restarted on your deploys, probably something like this in your deploy.rb

run "touch #{current_path}/vendor/plugins/resque/tmp/restart.txt"

I'm not really sure if this is the best way, I've never setup rack/sinatra apps before. But it works.

This is just to get the monitoring app going. Next I need to figure out the god part.

Answer 3

I'm a little late to the party, but thought I'd post what worked for me. Essentially, I have god setup to monitor redis and resque. If they aren't running anymore, god starts them back up. Then, I have a rake task that gets run after a capistrano deploy that quits my resque workers. Once the workers are quit, god will start new workers up so that they're running the latest codebase.

Here is my full writeup of how I use resque in production: http://blog.slicedsoftware.com/post/636319317/resque-in-production