Preventing delayed_job background jobs from consuming too much CPU on a single server

Question

My Rails application has a number of tasks which are offloaded into background processes, such as image resizing and uploading to S3. I'm using delayed_job to manage these processes.

These processes, particularly thumbnailing PDFs (using Ghostscript) and resizing images (using ImageMagick), are CPU intensive and often consume 100% CPU time. Since these jobs are running on the same (RedHat Linux) server as the web application itself, as well as the DB, they can lead to our web application being unresponsive.

One solution is to get another server on which to run only the background jobs. I guess this would be the optimal solution? However, since this isn't something I can do immediately I wonder whether it would be possible to somehow make the background jobs run at a lower operating system priority, and hence consume less CPU cycles in doing their work?

Thoughts appreciated.

Answer 1

If I'm not mistaken, delayed_job uses worker processes that will handle all the background jobs. It should be easily possible to alter the OS scheduling priority of the process when you start it.

So instead of, for example:

ruby script/delayed_job -e production -n 2 start

try:

nice -n 15 ruby script/delayed_job -e production -n 2 start