Getting GCOverhead Limit exceeded

Question

Hi, The following code is throwing OutofMemoryError on Linux 3.5 enterprise box running jdk1.6.0_14 but running fine on JDK 1.6.0_20 I am clueless why its happening.

while (rs.next()) {
  for (TableMetaData tabMeta : metaList) {
rec.append(getFormattedString(rs, tabMeta));
  }
  rec.append(lf);
  recCount++;
  if (recCount % maxRecBeforWrite == 0) {
    bOutStream.write(rec.toString().getBytes());
    rec = null;
    rec = new StringBuilder();
  }
}
bOutStream.write(rec.toString().getBytes());

The getFormattedString() method goes here:

private String getFormattedString(ResultSet rs, TableMetaData tabMeta)
        throws SQLException, IOException {

    String colValue = null;
    // check if it is a CLOB column
    if (tabMeta.isCLOB()) {
        // Column is a CLOB, so fetch it and retrieve first clobLimit chars.
        colValue = String.format("%-" + clobLimit + "s", getCLOBString(rs,
                tabMeta));
    } else {
        colValue = String.format("%-" + tabMeta.getColumnSize() + "s", rs
                .getString(tabMeta.getColumnName()));
    }
    return colValue;
}

Below is the exception trace:

Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
        at java.util.Formatter$FormatSpecifier.justify(Formatter.java:2827)
        at java.util.Formatter$FormatSpecifier.print(Formatter.java:2821)
        at java.util.Formatter$FormatSpecifier.printString(Formatter.java:2794)
        at java.util.Formatter$FormatSpecifier.print(Formatter.java:2677)
        at java.util.Formatter.format(Formatter.java:2433)
        at java.util.Formatter.format(Formatter.java:2367)
        at java.lang.String.format(String.java:2769)
        at com.boa.cpal.cpal2repnet.main.CPALToReportNet.getFormattedString(Unknown Source)

I suspect that the use of String.format is the culprit, but not sure. How to overcome this issue?

Please note that this code has been written to query on the database that have huge tables to read the resultset and create extract files with specific formatting.

Answer 1

Garbage collection has been improved significantly with JDK 1.6.0_18:

In the Client JVM, the default Java heap configuration has been modified to improve the performance of today's rich client applications. Initial and maximum heap sizes are larger and settings related to generational garbage collection are better tuned.

A quick look at the details in this release notes makes me believe that this is why you have less problems with 1.6.0_20.

---

The following part of the code is not consistent with the comment:

// Column is a CLOB, so fetch it and retrieve first clobLimit chars.
colValue = String.format("%-" + clobLimit + "s", getCLOBString(rs, tabMeta));

colValue doesn't get the first clobLimit bytes from the CLOB, it is left-justfied at the column clobLimit. I wasn't sure and tried

 System.out.println(String.format("%-5s", "1234567890"));

and the output was

 1234567890

To achieve what you tell in the comment, you can use the simpler form:

colValue = getCLOBString(rs, tabMeta).substring(0, clobLimit);

Answer 2

The exception you are getting refers to the GC overhead limit that is enabled by this HotSpot option:

-XX:+UseGCOverheadLimit -Use a policy that limits the proportion of the VM's time that is spent in GC before an OutOfMemory error is thrown. (Introduced in 6.)

So, my best guess is that your application is simply running out of heap space. As @Andreas_D's answer says, the default heap sizes were changed between jdk1.6.0_14 and JDK 1.6.0_20, and that could explain the different behaviour. Your options are:

• Upgrade to the later JVM. You want to be at at least 1.6.0_20 for the latest security fixes, and 1.6.0_21 includes bug fixes and performance improvements.

• Explicitly set the heap dimensions -Xmx and -Xms options when launching the JVM. If you are already doing this (on the older JVM), increase the numbers so that the maximum heap size is larger.

You could also adjust the GC overhead limit, but that's probably a bad idea on a production server.

If this particular problem only happens after your server has been running for some time, then maybe you've got memory leaks.