Tempfile and Garbage Collection

Question

I have this command in a Rails controller

  open(source) { |s| content = s.read }
  rss = RSS::Parser.parse(content, false)

and it is resulting in temporary files that are filling up the (scarce) disk space.

I have examined the problem to some extent and it turns out somewhere in the stack this happens:

io = Tempfile.new('open-uri')

but it looks like this Tempfile instance never gets explicitly closed. It's got a

def _close  # :nodoc:

method which might fire automatically upon garbage collection?

Any help in knowing what's happening or how to clean up the tempfiles would be helpful indeed.

Answer 1

it looks like _close closes the file and then waits for garbage collection to unlink (remove) the file. Theoretically you could force unlinking immediately by calling the Tempfile's close! method instead of close, or to call close(true) (which calls close! internally).

edit: But the problem is in open-uri, which is out of your hands - and that makes no promises for cleaning up after itself: it just assumes that the garbage collector will finalize all Tempfiles in due time.

In such a case, you are left with no choice but to call the garbage collector yourself using ObjectSpace.garbage_collect (see here). This should cause the removal of all temp files.

Answer 2

Definitely not a bug, but faulty error handling the IO. Buffer.io is either StringIO if the @size is less than 10240 bytes or a Tempfile if over that amount. The ensure clause in OpenURI.open_uri() is calling close(), but because it could be a StringIO object, which doesn't have a close!() method, it just can't just call close!().

The fix, I think, would be either one of these:

The ensure clause checks for class and calls either StringIO.close or Tempfile.close! as needed.

--or--

The Buffer class needs a finalizer that handles the class check and calls the correct method.

Granted, neither of those fix it if you don't use a block to handle the IO, but I suppose in that case, you can do your own checking, since open() returns the IO object, not the Buffer object.

The lib is a big chunk of messy code, imho, so it could use a work-over to clean it up. I think I might do that, just for fun. ^.^