Google AppEngine: how to count a database's entries beyond 1000?

Question

Duplicate of "how does one get a count of rows in a datastore model in google appengine?"

---

I'm having big trouble.

I want to know how many users I have... I did:

users = UserStore.all()
user_count=users.count()

quite simple... But now I have more than 1000 users and this method continues to return 1000.

Is there any other way of knowing how many users I have? (obviously, I want a dynamic way of doing so, I don't want to check the Dashboard)

Please help me! I'm very lost and panicked

Answer 1

Use pagination like these examples here.

Answer 2

It is indeed a duplicate and the other post describes how to theoretically do it, but I'd like to stress that you should really not be doing counts this way. The reason being that BigTable by it's distributed nature is really bad for aggregates. What you probably want to do is add a transactional counter to that entity, and if there are lots of transactions a sharded counter. See: http://code.google.com/appengine/articles/sharding_counters.html

UPDATE: Since 1.3.1 cursors make stuff like this a lot easier: http://code.google.com/appengine/docs/python/datastore/queriesandindexes.html#Query_Cursors

Answer 3

I have write this method to count a query, but how said Nick Johnson maybe its a bad idea...

def query_counter (q, cursor=None, limit=500):
  if cursor:
      q.with_cursor (cursor)
  count = q.count (limit=limit)
  if count == limit:
      return count + query_counter (q, q.cursor (), limit=limit)
  return count

The recipe http://goo.gl/c6cH

Answer 4

Since version 1.3.6 of the SDK the limit of 1000 on the count function has been removed. So a call to the count function will now return the exact number of entities, even if there are more than 1000. Only limitation would be if you had so many entities that the count function would not return before the request has a timeout.