views:

124

answers:

4

Hi guys,

We're having the classic spring/hibernate/mysql stack running in Tomcat 5.5. Once in a while we're getting a deadlock when the attempt times out to lock a table row. Some kind of deadlock exception is thrown.

The exception is clear and the stack trace indicate what went wrong. But it doesn't show the other thread which is holding the actual lock. Unless I know what that thread is doing it's all just a needle in a haystack.

QUESTION: Is there a way to find the other thread ?

Thanks !

Jan

A: 

Try using the following command in MySQL next time you see a deadlock. This should show you the last deadlock.

SHOW INNODB STATUS

Typically when you see a deadlock on your application server the logs show only the victim thread (the one which was rolled back). Since the other thread has completed no exception is thrown. You need to go back to your DB to recreate the transactions.

Once you have a capture from your DB for where the deadlock occured then you can investigate further.

Sean
This is not going to work because I don't have access to the database (it's only on production of course). And even I could, I won't be able to backtrace fast enough to the Java code behind the various locked objects. I really need to find the other thread inside the JVM immediatly after the exception is thrown.
Jan Goyvaerts
whoever admins your production database should be able to gather the required information for you. Knowing which two java transactions are locking is normally useless. You need to see what is happening on your Database for this. This will tell you where in the transaction your lock happened, if it was in an index, primary key etc. There is a lot to gain by accessing this log
Sean
A: 

hey,

not sure if you've figured it out already but if it's a deadlock, thread dump would be of great help here. Depending on what OS the application is run and on your priviledges to access it, you can generate it in many ways.

  • on *nix sending QUIT signal to the process ('kill -3 pid') would do the work
  • using jconsole/jvisualvm has an option to get it
  • using standard jdk jstack (consider -F -l options) will do the trick
  • if you are lucky to be on solaris pstack will help a lot

Once you've got it, analyse locked/waiting threads to find a deadlock. You can do it manually or using some existing analyzers that utilize deadlock detection algorithms. Btw jvm has one builtin and it can give you the idea right in the thread dump.

If I can help more just let me know.

good luck.

regards, baz

bazeusz
A: 

if it's a code problem you could try to connect to the running process using jconsole and detect the deadlock.

punkers
A: 

If you need to find the thread that holds a lock, you can do this in Eclipse through the debug view. Have a look at http://archive.eclipse.org/eclipse/downloads/drops/R-3.1-200506271435/eclipse-news-part2b.html and scroll down to 'Debugging locks and deadlocks'.

The locks owned by a thread as well as the lock a thread is waiting for can both be displayed inline in the Debug view by toggling the Show Monitors menu item in the Debug view drop-down menu. Threads and locks involved in a deadlock are highlighted in red.

alt text

Chris Dennett