twisted.protocols.ftp.FTPClient and Deferreds

Question

Like most it's taking me a while to get used to using Deferreds but I'm slowly getting there. However, it's not clear to me how I can process a response and then call another FTP command using the processed response when using Twisted's FTP module. I'm using the the example FTP code as my jumping off point.

I want to connect to a FTP server, get all the top level directories, then enter each one and download all the files.

First I connect:

creator = ClientCreator(reactor, FTPClient, config.opts['username'], config.opts['password'], passive=config.opts['passive'])
creator.connectTCP(config.opts['host'], config.opts['port']).addCallback(connectionMade).addErrback(connectionFailed)
reactor.run()

It connects successfully, so my connectionMade function gets called:

def connectionMade(ftpClient):
    # Get a detailed listing of the current directory
    fileList = FTPFileListProtocol()
    d = ftpClient.list('.', fileList)
    d.addCallbacks(getSortedDirectories, fail, callbackArgs=(fileList,))
    d.addCallback(enterDirs)

As you see, I queue up getSortedDirectories and then enterDirs.

def getSortedDirectories(result, fileListProtocol):
    # Go through all directories from greatest to least
    dirs = [directory for directory in sorted(fileListProtocol.files, reverse=True) if directory['filetype'] == 'd']
    return dirs

My question is, how do I go through the directories in enterDirs?

def enterDirs(dirs):
    for directory in dirs:
        print "We'd be entering '%s' now." % directory

Should I be passing ftpClient to enterDirs like fileList is passed to getSortedDirectories and then make my download requests?

d.addCallback(enterDirs, callbackArgs=(ftpClient,))

def enterDirs(dirs, ftpClient):
    for directory in dirs:
        fileList = FTPFileListProtocol()
        d = ftpClient.list(directory, fileList)
        d.addCallbacks(downloadFiles, fail, callbackArgs=(directory, fileList, ftpClient))

def downloadFiles(result, directory, fileListProtocol, ftpClient):
    for f in fileListProtocol.files if f.filetype == '-':
        fileConsumer = FileConsumer()
        ftpClient.retrieveFile(os.path.join(directory['filename'], file['filename']), fileConsumer)

Is this the best approach?

Answer 1

Should I be passing ftpClient to enterDirs like fileList is passed to getSortedDirectories and then make my download requests? ... Is this the best approach?

I do think that passing the client object explicitly as an argument is indeed the best approach -- mostly, it's spare and elegant. The main alternative would be to code a class and stash the client object into an instance variable, which seems a bit more cumbersome; to use a global variable for the purpose would, in my opinion, be the least desirable alternative (the fewer globals around, the better!-).