views:

204

answers:

3

A Java application I support that runs on JRE 1.4.2_12 is hanging near midnight every night. I'd like to try and record as much profiling information as I can to discover if there is an issue in the JVM or external to the app.

I'd like to use HPROF to collect as much information as possible.

  1. Is there a way to have HPROF dump its cpu sample and memory allocation report every minute instead of at the termination of the JVM?
  2. Is there a different, more appropriate profiler that can collect information like this?
+1  A: 

Rather than relying on dump files, I would try hooking up a profiler to the VM and leave it attached until the hang up occurs. Then use the profiler to introspect the state of the threads.

The use of Java 1.4 is a minor issue here, since 1.4's debug interface is not great, but some profilers still support it. I can particularly recommend YourKit, which is commercial, but offers an evaluation licence. It's the best profiler I've used, but some margin.

skaffman
You mean debugger?
Thorbjørn Ravn Andersen
YourKit is a wonderful profiler - we have used it to great success. You could also attempt remote debugging sessions with Eclipse, if you can figure out how to reproduce the bug (i.e. by setting the time of the server to midnight?)
aperkins
@Thorbjørn: debugger, profiler, potato, tomato.... there's some overlap here. A profiler will give more data on thread state, though, even if it doesn't tell you which code is executing.
skaffman
A: 

I would first try to determine if its actually your app or something else.

Are there any other apps on the box, if so do they run any batch around midnight. It could be a situation of your app suffering from a lack of resources due to other things running on the box or chewing up bandwidth.

Was this always the case or did it start recently. If this is new look at what changed on the box as a whole not just your own app.

Knife-Action-Jesus
There are a couple other people working on this aspect. Since it's at a customer site we don't have a whole lot of time in any particular day to diagnose, so we are trying to collect as much as possible at once.
magneticMonster
+1  A: 

First things first: did you analyze the thread dump when your application hangs? A lot of the time that has enough information to troubleshoot a hanging java app...

Ctrl-Break in the process window on Windows, or kill -QUIT [pid] on Linux.

ykaganovich
Can you suggest a way to send the Ctrl-Break signal every 60 seconds automatically? This way I don't have to stay up until 2am every night to test it.
magneticMonster
Just set up a job to run a script that loops for an hour or so?
ShabbyDoo
But how does one send the Ctrl-Break to a windows process?
magneticMonster
I don't know the specifics of your application, but if you can run it from the command line, and if it hangs in the middle of the night, then you can just ctrl-break it the next morning. It'll still be hung :)
ykaganovich