Ruby Memory Management

Question

I have been using ruby for a while now and I find for bigger projects it can take up a fare amount of memory. What are ruby best practices for reducing memory usage?

• Please, let each answer have on "best practice" and let the community vote it up

Answer 1

Beware of C extensions which allocate large chunks of memory themselves.

As an example, when you load an image using RMagick, the entire bitmap gets loaded into memory inside the ruby process. This may be 30 meg or so depending on the size of the image.
However, most of this memory has been allocated by RMagick itself. All ruby knows about is a wrapper object, which is tiny(1).
Ruby only thinks it's holding onto a tiny amount of memory, so it won't bother running the GC. In actual fact it's holding onto 30 meg.
If you loop over a say 10 images, you can run yourself out of memory really fast.

The preferred solution is to manually tell the C library to clean up the memory itself - RMagick has a destroy! method which does this. If your library doesn't however, you may need to forcibly run the GC yourself, even though this is generally discouraged.

(1): Ruby C extensions have callbacks which will get run when the ruby runtime decides to free them, so the memory will eventually be successfully freed at some point, just perhaps not soon enough.

Answer 2

Don't abuse symbols.

Each time you create a symbol, ruby puts an entry in it's symbol table. The symbol table is a global hash which never gets emptied.
This is not technically a memory leak, but it behaves like one. Symbols don't take up much memory so you don't need to be too paranoid, but it pays to be aware of this.

A general guideline: If you've actually typed the symbol in code, it's fine (you only have a finite amount of code after all), but don't call to_sym on dynamically generated or user-input strings, as this opens the door to a potentially ever-increasing number

Answer 3

Don't do this:

def method(x)
  x.split( doesn't matter what the args are )
end

or this:

def method(x)
  x.gsub( doesn't matter what the args are )
end

Both will permanently leak memory in ruby 1.8.5 and 1.8.6. (not sure about 1.8.7 as I haven't tried it, but I really hope it's fixed.) The workaround is retarded and involves creating a local variable. You don't have to use the local, just create one...

Things like this are why I have lots of love for the ruby language, but no respect for MRI

Answer 4

When working with huge arrays of ActiveRecord objects be very careful... When processing those objects in a loop if on each iteration you are loading their related objects using ActiveRecord's has_many, belongs_to, etc. - the memory usage grows a lot because each object that belongs to an array grows...

The following technique helped us a lot (simplified example):

students.each do |student|
  cloned_student = student.clone
  ...
  cloned_student.books.detect {...}
  ca_teachers = cloned_student.teachers.detect {|teacher| teacher.address.state == 'CA'}
  ca_teachers.blah_blah
  ...
  # Not sure if the following is necessary, but we have it just in case...
  cloned_student = nil
end

In the code above "cloned_student" is the object that grows, but since it is "nullified" at the end of each iteration this is not a problem for huge array of students. If we didn't do "clone", the loop variable "student" would have grown, but since it belongs to an array - the memory used by it is never released as long as array object exists.

Different approach works too:

students.each do |student|
  loop_student = Student.find(student.id) # just re-find the record into local variable.
  ...
  loop_student.books.detect {...}
  ca_teachers = loop_student.teachers.detect {|teacher| teacher.address.state == 'CA'}
  ca_teachers.blah_blah
  ...
end

In our production environment we had a background process that failed to finish once because 8Gb of RAM wasn't enough for it. After this small change it uses less than 1Gb to process the same amount of data...