How to investigate excessive java garbage collection

Question

I have a Tomcat instance which is exhibiting the following behaviour:

• Accept a single http incoming request.
• Issue one request to a backend server and get back about 400kb of XML.
• Pass through this XML and transform it into about 400kb of JSON.
• Return the JSON response.

The problem is that in the course of handling the 400k request my webapp generates about 100mb of garbage which fills up the Eden space and triggers a young generation collection.

I have tried to use the built in java hprof functionality to do allocation sites profiling but Tomcat didn't seem to start up properly with that in place. It is possible that I was just a bit impatient as I imagine memory allocation profiling has a high overhead and therefore tomcat startup might take a long time

What are the best tools to use to do java memory profiling of very young objects/garbage? I can't use heap dumps because the objects I'm interested in are garbage.

Answer 1

You should be able to get heap dumps to work anyway by debugging the app, placing breakpoints at key points of the code and creating a heap dump while the app is paused at each breakpoint.

Answer 2

You can use the profiler in jvisualvm in the JDK to do memory profiling.

Also have a look at Templates to cache the XSLT transformer.

http://java.sun.com/j2se/1.5.0/docs/api/javax/xml/transform/Templates.html

Answer 3

You might want to try LambdaProbe, which is a profiler for Tomcat. It supports the following:

Overview

Lambda Probe (formerly Tomcat Probe) is a self sufficient web application, which helps to visualize various parameters of Apache Tomcat instance in real time. Lambda Probe is designed to work specifically with Tomcat so it is able to access far more information that is normally available to JMX agents. Here is a list of features available through Lambda Probe:

• New! Comprehensive JVM memory usage monitor.
• JBoss compatibility
• Display of deployed applications, their status, session count, session object count, context object count, datasource usage etc.
• Start, stop, restart, deploy and updeploy of applications
• Ability to view deployed JSP files
• Ability to compile all or selected JSP files at any time.
• Ability to pre-compile JSP files on application deployment.
• New! Ability to view auto-generated JSP servlets
• Display of list of sessions for a particular application
• Display of session attributes and their values for a particular application. Ability to remove session attributes.
• Ability to view application context attributes and their values.
• Ability to expire selected sessions
• Graphical display of datasource details including maximum number of connections, number of busy connections and configuration details
• New! Ability to group datasource properties by URL to help visualizing impact on the databases
• Ability to reset data sources in case of applications leaking connection
• Display of system information including System.properties, memory usage bar and OS details
• Display of JK connector status including the list of requests pending execution
• Real-time connector usage charts and statistics.
• Real-time cluster monitoring and clulster traffic charts
• New! Real time OS memory usage, swap usage and CPU utilisation monitoring
• Ability to show information about log files and download selected files
• Ability to tail log files in real time from a browser.
• Ability to interrupt execution of "hang" requests without server restart
• New! Ability to restart Tomcat/JVM via Java Serview Wrapper.
• Availability "Quick check"
• Support for DBCP, C3P0 and Oracle datasources
• Support for Tomcat 5.0.x and 5.5.x
• Support for Java 1.4 and Java 1.5

Answer 4

As to the actual problem: XML parsing can be very memory hogging when using a DOM based parser. Consider using a SAX or binary XML based parser (VTD-XML is a Java API based on that).

Actually, if the XML->JSON mapping is pure 1:1, then you can also consider to just read the XML and write the JSON realtime line by line using a little stack.

---

Back to the question: I suggest to use VisualVM for this. You can find here a blog article how to get it to work with Tomcat.

Answer 5

http://wiki.github.com/mchr3k/org.inmemprofiler/

InMemProfiler can be used to identify which objects are collected after a very short time.