Hi,
One of our customers has a newish problem: the application grinds to halt. Thread dump shows that all threads are hanging on network IO in jdbc calls.
We/I have never seen these 'network IO' hangs. Typically a slow machine w/ DB problems has either a) one or two long-running queries or b) some type of lock/deadlock. In either of these cases the threads 'hang' on different methods. I have never seen all 30+ threads hanging on network IO.
Below I have included an excerpt from the thread dump. All 'http' threads are hanging on the same 'java.net.SocketInputStream.read' call.
I talked to their dba and sysadmin. According to them 'nothing has changed' in the environment recently which would cause this problem.
db environment
MSSQL 2005 64-bit Service Pack 2 Driver: sqljdbc.jar : 1.0 809 102
Note: they are running an older jdbc driver. AFAIK they tried upgrading from 1.0 to the 1.2 driver but had some other problem.
other environment issues
They're running both the app server and the db server in VMWare VM's. I don't know how this setup affects network performance.
Apparently this is the only application with this problem. I don't know anything else about their network architecture.
Questions * any insights on this problem? * if it is network, any next steps for analyzing?
Appendix A: Excerpt from Thread dump
All http connections are hanging on the same method:
"TP-Processor31" daemon prio=5 tid=0x04085b78 nid=0x970 runnable [0x0764d000..0x0764fd6c] at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at com.microsoft.sqlserver.jdbc.DBComms.receive(Unknown Source) at com.microsoft.sqlserver.jdbc.IOBuffer.sendCommand(Unknown Source) - locked <0x2313c318> (a com.microsoft.sqlserver.jdbc.DBComms) at com.microsoft.sqlserver.jdbc.SQLServerStatement.sendExecute(Unknown Source) at com.microsoft.sqlserver.jdbc.SQLServerStatement.doExecuteQuery(Unknown Source)