Converting to Twisted Asynchronous Design

Question

Ok I have had a problem expressing my problems with the code I am working on without dumping a ton of code; so here is what it would be synchronously (instead of asking it from the view of it being async).

Also for classes when should a variable be accessed through a method argument and when should it be accessed through a instance variable?

Synchronously it would look like so... Note: the actual server urls and parsing are different but just complicate things. Also in the following example the get_token method takes the session as a parameter, should it instead get the session by using self.session instead?

import urllib
import time

class SyncExampleClass(object):

    def __init__(self):
        self.session = None
        self.token = None
        self.session_time = -1


    def get_session(self):
        s = urllib.urlopen("http://example.com/session/").read()
        self.session_time = int(time.time())
        return s

    def get_token(self, session):
        t = urllib.urlopen("http://example.com/token/?session=%s" % session).read()
        return t

    def construct_api_call(self, api_method):
        # if the session is over an hour old (is that the correct sign?)
        if time.time() - 3600 > self.session_time or self.session is None:
            self.session = get_session()
            self.token = get_token(self.session)

        call = urllib.urlopen("http://example.com/api/?method=%s%session=%s&token=%s" % (api_method, self.session, self.token) ).read()
        return call

Answer 1

Given the circumstances, this just a skeleton of a solution which much implied. It seems to go against instinct to provide a solution with code where much is implied and untested...

However, if I was coding what I think you're trying to achieve, I might go about it something like this:

from twisted.internet import defer
from twisted.web import client
from twisted.python import log
from urllib import urlencode
import time

class APIException(Exception):
    pass

class ASyncExampleClass(object):
    def __init__(self):
        self.session = None
        self.token = None

    @defer.inlineCallbacks
    def api_call(self, api_method,tries=3,timeout=10):
        attempt = 1
        while attempt <= tries:
            attempt += 1
            if self.session = None:
                yield sess_data = client.getPage("http://example.com/session/",timeout=timeout)
                self.session = extractSessionFromData(sess_data)
            if self.token = None:
                yield token_data = client.getPage("http://example.com/token/?%s" % urlencode(dict(session=self.session)),timeout=timeout)
                self.token = extractTokenFromData(token_data)
            # Place "the" call
            yield api_result = client.getPage("http://example.com/api/?%s" % urlencode(dict(api_method=api_method,session=self.session,token=self.token)),timeout=timeout)
            #
            if sessionInvalid(api_result):
                log.msg("Request for %s failed because invalid session %s" % (api_method,self.session))
                self.session = None
                self.token = None
                continue
            if tokenInvalid(api_result):
                log.msg("Request for %s failed because invalid token %s" % (api_method,self.token))
                self.token = None
                continue
            # Any other checks for valid result
            returnValue(api_result)
            break # Not sure if this is needed, not in an position to test readily.
        else:
            raise APIException("Tried and failed %s times to do %s" % (attempt - 1, api_method))

The method that makes the external api uses inlineCallbacks and handles the logic of acquiring and renewing sessions and tokens itself. I have assumed that when a session is invalid, that any tokens acquired with it are also invalid. It implements a retry loop, which could also contain a try/except block for better handling of HTTP Exceptions.

You can use twisted.web.client.getPage with POST, the extra arguments you supply are handled by HTTPClientFactory.

Also, I wouldn't bother to time the session, just renew it when needed.