ruby/ruby on rails memory leak detection

Question

I wrote a small web app using ruby on rails, its main purpose is to upload, store, and display results from xml(files can be up to several MB) files. After running for about 2 months I noticed that the mongrel process was using about 4GB of memory. I did some research on debugging ruby memory leaks and could not find much. So I have two questions.

• Are there any good tools that can be used to find memory leaks in Ruby/rails?
• What type of coding patterns cause memory leaks in ruby?

Answer 1

Memory leak is a problem in the current ruby implementation a good place to start about this is http://whytheluckystiff.net/articles/theFullyUpturnedBin.html

for a more specific answer on problems with long running ruby processes see http://zdavatz.wordpress.com/2007/07/18/heap-fragmentation-in-a-long-running-ruby-process/

maybe you could give passenger (mod_rails) a try http://nubyonrails.com/articles/ask-your-doctor-about-mod_rails

Answer 2

Some tips to find memory leaks in Rails:

• use the Bleak House plugin
• implement Scout monitoring specifically the memory usage profiler
• implement FiveRuns monitoring
• try another simple memory usage logger

The first is a graphical exploration of memory usage by objects in the ObjectSpace.

The last two will help you identify specific usage patterns that are inflating memory usage, and you can work from there.

As for specific coding-patterns, from experience you have to watch anything that's dealing with file io, image processing, working with massive strings and the like.

I would check whether you are using the most appropriate XML library - ReXML is known to be slow and believed to be leaky (I have no proof of that!). Also check whether you can memoize expensive operations.

Answer 3

A super simple method to log memory usage after or before each request (only for Linux).

#Put this in applictation_controller.rb
before_filter :log_ram # or use after_filter
def log_ram
  logger.warn 'RAM USAGE: ' + `pmap #{Process.pid} | tail -1`[10,40].strip
end

You might want to load up script/console and try the statement out first to make sure it works on your box.

puts 'RAM USAGE: ' + `pmap #{Process.pid} | tail -1`[10,40].strip

Then just monitor top, when a request makes your memory usage jump, go check the logs. This, of course, will only help if you have a memory leak that occurs in large jumps, not tiny increments.

Answer 4

Switch to jruby and use the Eclipse Memory Analyzer. There's no comparable tool for Ruby at the moment.

Answer 5

Now, you can run the following to get the memory in a format that R can read. I am assuming that your log line looks like:

1234567890 RAM USAGE: 27456K

Run this (or modify to suite):

$ grep 'RAM USAGE' fubar.log | awk '{print s " " $1 " " $4; s++}' | sed 's/K//g' > mem.log

Then you can run this:

#!/bin/sh
rm -f mem.png
R --vanilla --no-save --slave <<RSCRIPT
    lst <- read.table("mem.log")
    attach(lst)
    m = memory / 1024.0
    summary(m)
    png(filename="mem.png", width=1024)
    plot(date, m, type='l', main="Memory usage", xlab="time", ylab="memory")
RSCRIPT

and get a nice graph.