How to get the number of rows in a table in a Datastore?

Question

Hi,

In many cases, it could be useful to know the number of rows in a table (a kind) in a datastore using Google Application Engine.
There is not clear and fast solution . At least I have not found one.. Have you?

Answer 1

You can efficiently get a count of all entities of a particular kind (i.e., number of rows in a table) using the Datastore Statistics API. Simple example:

from google.appengine.ext.db import stats
kind_stats = stats.KindStat().all().filter("kind_name =", "NameOfYourModel").get()
count = kind_stats.count

You can find a more detailed example of how to get the latest stats here (GAE may keep multiple copies of the stats - one for 5min ago, one for 30min ago, etc.).

Note that these statistics aren't constantly updated so they lag a little behind the actual counts. If you really need the actual count, then you could track counts in your own custom stats table and update it every time you create/delete an entity (though this will be quite a bit more expensive to do).

Answer 2

There's no concept of "Select count(*)" in App Engine. You'll need to do one of the following:

1. Do a "keys-only" (index traversal) of the Entities you want at query time and count them one by one. This has the cost of slow reads.
2. Update counts at write time - this has the benefit of extremely fast reads at a greater cost per write/update. Cost: you have to know what you want to count ahead of time. You'll pay a higher cost at write time.
3. Update all counts asynchronously using Task Queues, cron jobs or the new Mapper API. This has the tradeoff of being semi-fresh.