How to detect which mutex gives largest amount of time to the OS?

Question

How to measure amount of time given by a mutex to the OS? The main goal is to detect a mutex, that blocks threads for largest amount of time.

PS: I tried oprofile. It reports 30% of time spent inside vmlinux/.poll_idle. This is unexpected, because the app is designed to take 100% of its core. Therefore, I'm suspecting, that the time is given back to the OS while waiting for some mutex, and oprofile reports it as the IDLE time.

Answer 1

Profile.

Whenever the question is "What really takes the [most|least] time?", the answer is always "Profile to find out.".

Answer 2

As suggested - profile, but decide before what do you want to measure - elapsed time (time threads were blocked), or user/kernel time (time it cost you to perform synchronization). In different scenarios you might want to measure one or another, or both.

Answer 3

You could profile your program, using say OProfile on linux. Then filter out your results to look at the time spent in pthread_mutex_lock() for each mutex, or your higher-level function that performs the locking. Since the program will block inside the lock function call until the mutex is aquired, profiling the time spent in that function should give you an idea of which mutexes are your most expensive.

Answer 4

start = GetTime();
Mutex.Lock();
stop = GetTime();

elapsedTime = stop - start;

elaspedTime is the amount of time it took to grab the mutex. If it is bigger than some small value, it's because another thread has the mutex. This won't show how long the OS has the mutex, only that another thread has it.