Detect file handle leaks in python?

Question

My program appears to be leaking file handles, how can I find out where? - My program uses file handles in a few different places - output from child processes, call ctypes api (imagemagick) opens files, and they are copied.

It crashes in shutil.copyfile, but I'm pretty sure this is not the place it is leaking

Traceback (most recent call last):
  File "", line 1, in 
  File "C:\Python25\Lib\site-packages\magpy\magpy.py", line 874, in main
    magpy.run_all()
  File "C:\Python25\Lib\site-packages\magpy\magpy.py", line 656, in run_all
    [operation.operate() for operation in operations]
  File "C:\Python25\Lib\site-packages\magpy\magpy.py", line 417, in operate
    output_file = self.place_image(output_file)
  File "C:\Python25\Lib\site-packages\magpy\magpy.py", line 336, in place_image
    shutil.copyfile(str(input_file), str(self.full_filename))
  File "C:\Python25\Lib\shutil.py", line 47, in copyfile
    fdst = open(dst, 'wb')
IOError: [Errno 24] Too many open files: 'C:\\Documents and Settings\\stuart.axon\\Desktop\\calzone\\output\\wwtbam4\\Nokia_NCD\\nl\\icon_42x42_V000.png'
Press any key to continue . . .

Answer 1

Look at output from ls -l /proc/$pid/fd/ (substituting the PID of your process, of course) to see which files are open [or, on win32, use Process Explorer to list open files]; then figure out where in your code you're opening them, and make that close() is being called. (Yes, the garbage collector will eventually close things, but it's not always fast enough to avoid running out of fds).

Checking for any circular references which might be preventing garbage collection is also a good practice. (The cycle collector will eventually dispose of these -- but it may not run frequently enough to avoid file descriptor exhaustion; I've been bitten by this personally).

Answer 2

Use Process Explorer, select your process, View->Lower Pane View->Handles - then look for what seems out of place - usually lots of the same or similar files open points to the problem.

Answer 3

Doh, I couldn't accept the answer as I registered after asking + now I can't accept an answer.

For the record: Process explorer showed me that it is the temp files that are not being closed, and I'll try using .close to make sure they go - I'd accept the process explorer answer as it got me past the first stage, but the answer that mentioned it's worth calling .close is just as useful for the 2nd part.