views:

47

answers:

2

Let's say I have 2500 MyModel entities in my datastore, and I run this code:

query = MyModel.all()
first_batch = query.fetch(2000)
len(first_batch) # 2000

next_query = MyModel.all().with_cursor(query.cursor())
next_batch = next_query.fetch(2000)

What do you think len(next_batch) is? 500, right? Nope - it's 1500. Apparently the query cursor never moves forward by more than 1000, even when the query itself returns more than 1000 entities.

Should I do something different or is it just an App Engine bug?

+2  A: 

It's an App Engine explicit limit, no more than 1000 in a fetch offset. Which kinda implies that Qql can't retrieve more than 1000 which I thought I saw documented but cannot find.

Oddly, Query.count() has a maximum value of 1000, also. They don't come out and say it, but queries returning more than 1000 entities seems antithetical to the GQL model.

msw
This is irrelevant because a query cursor is not the same thing as an offset. I agree that a user's page request should not require 1000 fetched entities, but the code I'm writing is for a large asynchronous operation.
Liron Shapira
GQL used to have a 1000 entity upper limit, but that has been removed in 1.3.1 or so. Maybe this issue is a corner case that cropped up with the interaction between cursors and the removal of the 1000 limit.
Peter Recore
A: 

Here's another Stack Overflow question with more info.

Steven Ourada
Nothing on that page addresses my very specific question about Query cursors.
Liron Shapira