Django, Borg pattern, API calls, caching results

Question

Hi everyone!

I'm using an API from a different site that returns a couple of 'pricepoint URL's that my users use to buy Virtual Goods.

I'm supposed to cache those results for at least an hour, since they don't change the price points on their system much. (And we want to save both our's and their bandwidth.)

After looking for singleton's in Python I discovered the borg pattern, which seems even cooler, so this is what i did:

def fetchPrices():
    #uses urllib2.urlopen() to fetch prices
    #parses results with ElementTree
    return prices

class PriceStore():
    __shared_state = {}

    def update(self):
        if self.lastUpdate is not None and (datetime.now() - self.lastUpdate).seconds >= 3600:
            self.prices = fetchPrices()
            self.lastUpdate = datetime.now()
        elif self.lastUpdate is not None:
            return
        else:
            self.lastUpdate = datetime.now() - timedelta(hours=1)
            self.update()

    def __init__(self):
        self.__dict__ = self.__shared_state
        self.lastUpdate = None
        self.update()

The idea is to use this in the following way:

store = PriceStore()
url = store.prices['2.9900']['url']

And the store should initialize correctly and only fetch new price point info, if the existing info is older than one hour.

I seem to be hitting their API with every time that PriceStore is initialized, though. Can anyone spot my problem? Can I use a global variable like __shared_state in django and expect it to still contain pricing info?

Thanks!

Answer 1

In __init__ you set self.lastUpdate = None. Don't do that.

Specifically, consider the following code:

A = PriceStore()

# some time later

B = PriceStore()

Now A.lastUpdate == None, which you didn't want! Instead, try

if "lastUpdate" not in self.__dict__:
    self.lastUpdate = None

That way you never overwrite it.

Answer 2

Your main problem is that when you create a new PriceStore, you set self.lastUpdate to None (in your second-to-last line). So although they all share state, every new object clobbers the state.

Instead do this:

class PriceStore():
    __shared_state = {'lastUpdate': None}

Another problem you might face is that depending on how your Django is deployed, you may be using more than one process. Each one would have its own copy of the shared state.

Answer 3

I seem to be hitting their API with every time that PriceStore is initialized, though. Can anyone spot my problem?

Yep, it's easy to spot:

def __init__(self):
    self.__dict__ = self.__shared_state
    self.lastUpdate = None

the self.lastUpdate = None absolutely guarantees that the immediately following call to self.update() will find self.lastUpdate's value to be None -- you just forced it to be so!

Remove that self.lastUpdate = None in the __init__ and, for example, use instead a

lastUpdate = None

at class body level, e.g. just after the __shared_state = {} assignment and with the same alignment as that assignment. That will make things work as you intend.