views:

966

answers:

6

I'm using Apache and Tomcat on a Windows server and since this morning, Tomcat stops working without any logs. It doesn't hang, it just shut down.
There's no log in Tomcat, the CPU/Memory are fines, there are no System.Exit in my code.

Anybody ever had this problem?

It happens at random, after 5-10 minutes. The application responds normally and sometime, boom.. stops working.

UPDATE : Still no clue. The Admin team will install the webapp on another box...

+1  A: 

things to look for in debugging an issue like this:

  • Look at the logs directory ($TOMCAT_ROOT/logs) to make sure none of the log files have any stack traces
  • Look at the tomcat startup script to make check the location of the log files to see if the logs are not being written to another directory.
  • Another reason could be some other user/process could be issuing a kill -9 that could kill tomcat without giving it any chance to log errors.
neesh
The logs are at "debug" and are logging stuff, but not when tomcat stops.
that is a strong indication that something below tomcat is causing this failure - it could be something *really* bad in the JVM or an external entity just killing the jvm using a kill -9 like signal
neesh
isn't kill a linux/unix command? Anyway, I get what you mean, but that's still weird :|
kill is a linux/unix command. The windows equivalent of this would be to kill the java process from the task manager
neesh
+1  A: 

Hi supersheep. This sounds like the JVM is crashing. Have you looked for a JVM crash log? It typically has a name like hs_err_pid*.log and is created in the JVM's working directory.

If you find a file like this and upload it, then we can probably help more.

Some questions:

  • Have you recently changed the version of Java you are using?
  • What is the exact version of Tomcat you are using?
  • Are you using Tomcat Native (the Apache Portable Runtime)?
Matt Solnit
I use to have it with a combination of Java 1.4.2 and FreeBSD 5.7 (don't ask me why we had this OS for running Java) - every time there was a NullPointerException, the JVM got segmentation fault.
David Rabinowitz
I'm using 6.0.18The Java version is the same since we installed the projectWe're not using Tomcat NativeI'm going to look for a hs_err_pid*.log (I don't have direct access to the prod server)... Where should it be on a Windows box?
no hs_err_pid*.* was found on the box :(
A: 

another possibility is that some process was started this morning on the box that is binding to a port that your server requires.

akf
I think he does say that the app runs normally for a few mins before tomcat dies. It wouldn't come up at all if the port is in use.
neesh
i agree this is an outlier. those elusive logs would be great, wouldnt they?
akf
If there was a "magic application" that was "stealing" the port, we would not be able to restart tomcat after it stops, but we can.
A: 

Are your servlets or one of it's dependencies allowed to call System.exit()? (Not sure how locked down Tomcat VMs are in that sense)

I've had developers thinking it's ok to use exit(666); on detecting a non-invertable matrix (which isn't good, but sure as heck not fatal). Arrgh. Perhaps you have some similar culprit in your system?

Marcus Lindblom
It's very unlikely because it worked for weeks before it happened and now it stops each 5 minutes... weird :| But we made test with the datas and war on another server and it seems OK. It looks like a configuration problem somewhere.
A: 
  1. Does Tomcat have enough memory (if it's possible, shows your memory and other JVM parameters - CATALINA_OPTS)?
  2. Can JVM allocate all necessary memory, providing for tomcat in CATALINA_OPTS?
  3. Do you have special classloaders?
FoxyBOA
A: 

It was probably an Apache issue. We rebooted, palyed a little with Apache config and it worked again after that... Weird