Google app engine entity groups

Question

Hey all,

As far as I understand from app engine tutorial, entity groups exist only for the purpose of transactions:

"Only use entity groups when they are needed for transactions" (from the tutorial)

The definition of being in the same entity group is to have the same root.. In that case, what is the use of having more than 1 hierarchy level? That is, why should I use "A -> B -> C" (A is the root, B his son, C his grandson) instead of "A -> B ; A -> C" ? (A, B and C are still in the same entity group since A is their root).

If the only purpose of entity groups in to make transaction possible between entities, why should I use more than 1 hierarchy level (what do I earn from Root -> Grandson linkage)?

Answer 1

When you store A -> B -> C, A has many Bs, and a B has many Cs. When you store A -> B and A -> C, A has many Bs, and many Cs. In other words, a C doesn't belong to a single B.

Which structure you use really depends on the data you're storing.

When using lots of write accesses, you might have to do unintuitive things to your entitygroups, see Sharding Counters for an example of this:

Answer 2

When you're doing queries, you can use ancestor() to restrict the query to children of a particular entity - in your example, you could look for only descendants of B, which you couldn't do if they were all at the top level.

There's more on Ancestor Queries in Programming Google App Engine

The Keys and Entity Groups doc also says that:

Entity group relationships tell App Engine to store several entities in the same part of the distributed network ... All entities in a group are stored in the same datastore node

edit: The same document also lists some of the reasons why you don't want your entity groups to grow too large:

The more entity groups your application has—that is, the more root entities there are—the more efficiently the datastore can distribute the entity groups across datastore nodes. Better distribution improves the performance of creating and updating data. Also, multiple users attempting to update entities in the same entity group at the same time will cause some users to retry their transactions, possibly causing some to fail to commit changes. Do not put all of the application's entities under one root.

Any transaction on an entity in a Group will cause any other writes to the same entity group to fail. If you have a large entity group with lots of writes, this causes lots of contention, and your app then has to handle the expected write failures. Avoiding datastore contention goes into more detail on the strategies you can use to minimse the contention.