Check the open FD limit for a given process in Linux

Question

Hi,

I recently had a Linux process which "leaked" file descriptors: It opened them and didn't properly close some of them.

If I had monitored this, I could tell - in advance - that the process was reaching its limit.

Is there a nice, Bash\Python way to check the FD usage ratio for a given process in a Ubuntu Linux system?

EDIT:

I now know how to check how many open file descriptors are there; I only need to know how many file descriptors are allowed for a process. Some systems (like Amazon EC2) don't have the /proc/pid/limits file.

Thanks,

Udi

Answer 1

You can try to write script which periodically call 'lsof' on given pid.

Answer 2

Count the entries in /proc/<pid>/fd/.

The bash builtin ulimit -n will show you the process open file descriptor limit for the current session, which is likely to be the same limit that applies to the session where your daemon is running.

Answer 3

You asked for bash/python methods. ulimit would be the best bash approach (short of munging through /proc/$pid/fd and the like by hand). For python, you could use the resource module.

--snip--

import resource

print resource.getrlimit(resource.RLIMIT_NOFILE)

--snip--

$ python test.py

(1024, 1024)

resource.getrlimit corresponds to the getrlimit call in a C program. The results represent the current and maximum values for the requested resource. In the above example, the current (soft) limit is 1024. The values are typical defaults on Linux systems these days.

Answer 4

The only interfaces provided by the Linux kernel to get resource limits are getrlimit() and /proc/pid/limits. getrlimit() can only get resource limits of the calling process. /proc/pid/limits allows you to get the resource limits of any process with the same user id, and is available on RHEL 5.2, RHEL 4.7, Ubuntu 9.04, and any distribution with a 2.6.24 or later kernel.

If you need to support older Linux systems then you will have to get the process itself to call getrlimit(). Of course the easiest way to do that is by modifying the program, or a library that it uses. If you are running the program then you could use LD_PRELOAD to load your own code into the program. If none of those are possible then you could attach to the process with gdb and have it execute the call within the process. You could also do the same thing yourself using ptrace() to attach to the process, insert the call in its memory, etc., however this is very complicated to get right and is not recommended.

With appropriate privileges, the other ways to do this would involve looking through kernel memory, loading a kernel module, or otherwise modifying the kernel, but I am assuming that these are out of the question.