Adding mini caches to django models

Question

Hi,

I'm running into performance issues with Django because of m2m relationships. I have a list of Something objects that all have Something_instance.item_set, so I call Something_instance.item_set.all() many times. I'd like to cache that query into Something_instance so that I don't have to run so many queries. Is this possible? (This is essentially a hack for getting select_related() to work for m2m).

EDIT: The following 2 snippets shows the problem I'm having. In views.py, I'm querying the m2m relationship.

    for t in items:
          try:
              t.price = t.user_item_rel_set.get(user=u).payment_amount
          except:
              t.price = -1 * t.buyer_item_rel_set.get(buyer=u).payment_amount
    return items

Also, a function in my model:

def listBuyers(self):
    return self.buyer_item_rel_set.all()

I have this because I use it in my template to get information from those elements.

Querying the m2m relationships end up running twice: once in the views.py, and then once in the template. I'd like to get the queryset in views, attach it to the model instance, and then feed that to the template, so it (and the views.py code) uses the cached queryset, instead of fetching again.

Answer 1

The query is already cached. See the Django FAQ for information on how to view SQL queries that are generated during a page load.

The problem may be as simple as a missing index (although Django's ORM should generate needed indexes) or inefficient code using the results of the QuerySet. We would need much more information to help troubleshoot.

Answer 2

Yes, I do this all the time. Using your listBuyers method as an example (by the way, Pythonic convention would be to call it list_buyers...)

def listBuyers(self):
    if not hasattr(self, _buyers):
        self._buyers = self.buyer_item_rel_set.all()
    return self._buyers

This will hit the database the first time listBuyers is called for a particular instance, but not afterwards.

Answer 3

Hard to say without seeing your models and their relations (f.e. how 't' related to user 'u')

But the suggestion is hit all related to user('u') items at one query and than go one by one and calculate needed stuff...

BTW.. 't' and 'u' are not very good names for variables