How do I tell what a Linux process is waiting for?

Question

I'm trying to track down the cause of performance bottlenecks in an application I'm debugging under Linux. The various processes involved seem to spend a lot of their time blocking on I/O requests, and I was wondering if anybody knew any Linux tricks that let you see why a particular process is blocked/what resource it's waiting for? Is there anything useful in /proc for instance?

Answer 1

You could use the strace command to see what your process is up to. You might also find it useful to run lsof on your process to see what files and sockets it is using.

Answer 2

Whatever process it is, you use top to check the state. Then you could run it through truss or strace. That should detail what is going on. If that is not possible, hook it up to gdb. Tools like iostat might show you in general what is going on (e.g. if disk is bottleneck).

Answer 3

strace is your friend, because you can get the time spent into each system call, plus you can select the file descriptors and system call you are interested in. But you also can get support from the kernel : have a look at latencytop