Java Appengine APPSTATS causing java out of memory error

Question

I have several servlets in my java appengine app that do in memory sorting and take on the order of seconds to complete. These complete error free.

However, I recently enabled appstats for appengine and started receiving the following error:

java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Unknown Source)
    at java.lang.AbstractStringBuilder.expandCapacity(Unknown Source)
    at java.lang.AbstractStringBuilder.append(Unknown Source)
    at java.lang.StringBuilder.append(Unknown Source)
    at java.lang.StringBuilder.append(Unknown Source)
    at java.lang.StringBuilder.append(Unknown Source)
    at com.google.appengine.repackaged.com.google.protobuf.TextFormat$TextGenerator.write(TextFormat.java:344)
    at com.google.appengine.repackaged.com.google.protobuf.TextFormat$TextGenerator.print(TextFormat.java:332)
    at com.google.appengine.repackaged.com.google.protobuf.TextFormat.printUnknownFields(TextFormat.java:249)
    at com.google.appengine.repackaged.com.google.protobuf.TextFormat.print(TextFormat.java:47)
    at com.google.appengine.repackaged.com.google.protobuf.TextFormat.printToString(TextFormat.java:73)
    at com.google.appengine.tools.appstats.Recorder.makeSummary(Recorder.java:157)
    at com.google.appengine.tools.appstats.Recorder.makeSyncCall(Recorder.java:239)
    at com.google.apphosting.api.ApiProxy.makeSyncCall(ApiProxy.java:98)
    at com.google.appengine.api.datastore.DatastoreApiHelper.makeSyncCall(DatastoreApiHelper.java:54)
    at com.google.appengine.api.datastore.PreparedQueryImpl.runQuery(PreparedQueryImpl.java:127)
    at com.google.appengine.api.datastore.PreparedQueryImpl.asQueryResultList(PreparedQueryImpl.java:81)
    at org.datanucleus.store.appengine.query.DatastoreQuery.fulfillEntityQuery(DatastoreQuery.java:379)
    at org.datanucleus.store.appengine.query.DatastoreQuery.executeQuery(DatastoreQuery.java:289)
    at org.datanucleus.store.appengine.query.DatastoreQuery.performExecute(DatastoreQuery.java:239)
    at org.datanucleus.store.appengine.query.JDOQLQuery.performExecute(JDOQLQuery.java:89)
    at org.datanucleus.store.query.Query.executeQuery(Query.java:1489)
    at org.datanucleus.store.query.Query.executeWithArray(Query.java:1371)
    at org.datanucleus.jdo.JDOQuery.execute(JDOQuery.java:243)
    at com.poo.pooserver.dataaccess.DataAccessHelper.getPooStream(DataAccessHelper.java:204)
    at com.poo.pooserver.GetPooStreamServlet.doPost(GetPooStreamServlet.java:58)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:713)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:806)
    at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166)
    at com.google.appengine.tools.appstats.AppstatsFilter.doFilter(AppstatsFilter.java:92)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)

Answer 1

Appstats itself uses memory and can somewhat reduce the amount of available memory to your Java application, but the reduction should be slight. Appstats itself can also run out of memory (usually seen during a memcache call, link text), but that doesn't appear to be the case with your application. I'd see if you can do anything to break the work down into smaller pieces).