How do I use GAE datastore transactions with many-to-many relationships?

Question

I have a data module that includes Vendor objects and VendorCategory objects. Each vendor has multiple categories, and each category can have multiple vendors. I need to list all vendors under a given category, and all categories under a given vendor.

The primary operations are on vendors, so I'm writing code to update/delete VendorCategory objects based on edit operations on vendors.

I'd like these operations to be transactional: i.e. a vendor category has its refcount updated iff the vendor creation/deletion/edit goes through, and not otherwise. But I've got things set up as an unowned many-to-many relationship, so AFAIK it's not possible to use transactions, since they're in different entity groups.

Is there a better way to model this relationship? Do I need to just suck it up and live with the nontransactional nature of the beast? I've thought about using the task queue to go through and clean up the relationships periodically, is that the best way?

Answer 1

Add a transactional task to update VendorCategory when you make the changes to Vendor. The task will be added only if the datastore write succeeds. Handle updating and deleting of the categories within their own transaction inside the task.

You might want to check out "Building high-throughput data pipelines with Google App Engine" from Google IO 2010. Specifically the stuff on materialized views.

Answer 2

I wrote a blog post recently describing different ways to do many-many relationships in App Engine. Which you choose depends on your situation, and on 'ownership' - for example, if a vendor 'owns' its list of categories, you might store it as a list on the Vendor entity most simply.