Java Appengine APPSTATS causing java out of memory error

Question

I have several servlets in my java appengine app that do in memory sorting and take on the order of seconds to complete. These complete error free.

However, I recently enabled appstats for appengine and started receiving the following error:

java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Unknown Source)
    at java.lang.AbstractStringBuilder.expandCapacity(Unknown Source)
    at java.lang.AbstractStringBuilder.append(Unknown Source)
    at java.lang.StringBuilder.append(Unknown Source)
    at java.lang.StringBuilder.append(Unknown Source)
    at java.lang.StringBuilder.append(Unknown Source)
    at com.google.appengine.repackaged.com.google.protobuf.TextFormat$TextGenerator.write(TextFormat.java:344)
    at com.google.appengine.repackaged.com.google.protobuf.TextFormat$TextGenerator.print(TextFormat.java:332)
    at com.google.appengine.repackaged.com.google.protobuf.TextFormat.printUnknownFields(TextFormat.java:249)
    at com.google.appengine.repackaged.com.google.protobuf.TextFormat.print(TextFormat.java:47)
    at com.google.appengine.repackaged.com.google.protobuf.TextFormat.printToString(TextFormat.java:73)
    at com.google.appengine.tools.appstats.Recorder.makeSummary(Recorder.java:157)
    at com.google.appengine.tools.appstats.Recorder.makeSyncCall(Recorder.java:239)
    at com.google.apphosting.api.ApiProxy.makeSyncCall(ApiProxy.java:98)
    at com.google.appengine.api.datastore.DatastoreApiHelper.makeSyncCall(DatastoreApiHelper.java:54)
    at com.google.appengine.api.datastore.PreparedQueryImpl.runQuery(PreparedQueryImpl.java:127)
    at com.google.appengine.api.datastore.PreparedQueryImpl.asQueryResultList(PreparedQueryImpl.java:81)
    at org.datanucleus.store.appengine.query.DatastoreQuery.fulfillEntityQuery(DatastoreQuery.java:379)
    at org.datanucleus.store.appengine.query.DatastoreQuery.executeQuery(DatastoreQuery.java:289)
    at org.datanucleus.store.appengine.query.DatastoreQuery.performExecute(DatastoreQuery.java:239)
    at org.datanucleus.store.appengine.query.JDOQLQuery.performExecute(JDOQLQuery.java:89)
    at org.datanucleus.store.query.Query.executeQuery(Query.java:1489)
    at org.datanucleus.store.query.Query.executeWithArray(Query.java:1371)
    at org.datanucleus.jdo.JDOQuery.execute(JDOQuery.java:243)
    at com.poo.pooserver.dataaccess.DataAccessHelper.getPooStream(DataAccessHelper.java:204)
    at com.poo.pooserver.GetPooStreamServlet.doPost(GetPooStreamServlet.java:58)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:713)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:806)
    at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166)
    at com.google.appengine.tools.appstats.AppstatsFilter.doFilter(AppstatsFilter.java:92)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)

Ethbun

I'm not sure what answer you want - you're running out of memory. You either need to reduce your memory consumption or disable appstats. – Gregory

well - just didn't understand why enabling appstats would use a lot more memory. How much memory is available for use within one request? How much of that is taken by appstats? – Ethbun

Answer 1

I wonder whether this can help you reducing the occurence of OutOfMemoryErrors: How to reduce the memory usage of Appstats on Google App Engine Java

<filter>
  <filter-name>appstats</filter-name>
    <filter-class>com.google.appengine.tools.appstats.AppstatsFilter</filter-class>
    <init-param>
      <param-name>maxLinesOfStackTrace</param-name>
      <param-value>16</param-value>
    </init-param>
</filter>

Angulate

Answer 2

Appstats itself uses memory and can somewhat reduce the amount of available memory to your Java application, but the reduction should be slight. Appstats itself can also run out of memory (usually seen during a memcache call, link text), but that doesn't appear to be the case with your application. I'd see if you can do anything to break the work down into smaller pieces).

Clustered