views:

473

answers:

2

The new release 1.3.1 of GAE has dropped the 1000 records limit:

No more 1000 result limit - That's right: with addition of Cursors and the culmination of many smaller Datastore stability and performance improvements over the last few months, we're now confident enough to remove the maximum result limit altogether.

However, the release notes link to a detail description which does not mention JPA, only JDO and the low level API:

In JDO, you can use an extension and the JDOCursorHelper class to use cursors with JDO queries.

JPA is not mentioned, and my tests using query.setFirstResult(n) still show the same error messages if n is greater than 1000.

java.lang.IllegalArgumentException: offset may not be above
1000
        at
com.google.appengine.api.datastore.DatastoreApiHelper.translateError(DatastoreApiHelper.java: 33)
        at
com.google.appengine.api.datastore.DatastoreApiHelper.makeSyncCall(DatastoreApiHelper.java:
60)
        at
com.google.appengine.api.datastore.PreparedQueryImpl.runQuery(PreparedQueryImpl.java:
115) ...

select count(*):

I also found that a select count(*) from MyEntity which has more than 1000 instances returns 1000.

A: 

JDO "extensions" in that "detail description" equates to JPA "hints" on the query, so why not set that same hint for a JPA query?

Cursor cursor = Cursor.fromWebSafeString(cursorString); query.setHint(JDOCursorHelper.CURSOR_EXTENSION, cursor);

DataNucleus
I will try this. So to solve the count(*) problem (see my update), this means I will have to iterate over Cursors, or use the low level API?
mjustin
A: 

The release 1.3.6 removed the limits:

Version 1.3.6 - August 17, 2010

Results of datastore count() queries and offsets for all datastore queries are no longer capped at 1000.

http://code.google.com/p/googleappengine/wiki/SdkReleaseNotes

mjustin