Downloading a Directory Tree with FTPLIB

Question

This will not download the contents of sub-directories; how can I do so?

import ftplib
import configparser
import os

directories = []

def add_directory(line):
 if line.startswith('d'):
  bits = line.split()
  dirname = bits[8]
  directories.append(dirname)

def makeDir(archiveTo):
 for dir in directories:
  newDir = os.path.join(archiveTo, dir)
  if os.path.isdir(newDir) == True:
   print("Directory \"" + dir + "\" already exists!")
  else:
   os.mkdir(newDir)

def getFiles(archiveTo, ftp):
 files = ftp.nlst()
 for filename in files:
  try:
   directories.index(filename)
  except:
   ftp.retrbinary('RETR %s' % filename, open(os.path.join(archiveTo, filename), 'wb').write)

def runBackups():

 #Load INI
 filename = 'connections.ini'
 config = configparser.SafeConfigParser()
 config.read(filename)

 connections = config.sections()
 i = 0

 while i < len(connections):
  #Load Settings
  uri = config.get(connections[i], "uri")
  username = config.get(connections[i], "username")
  password = config.get(connections[i], "password")
  backupPath = config.get(connections[i], "backuppath")
  archiveTo = config.get(connections[i], "archiveto")

  #Start Back-ups
  ftp = ftplib.FTP(uri)
  ftp.login(username, password)
  ftp.cwd(backupPath)

  #Map Directory Tree
  ftp.retrlines('LIST', add_directory)

  #Make Directories Locally
  makeDir(archiveTo)

  #Gather Files
  getFiles(archiveTo, ftp)

  #End connection and increase counter.
  ftp.quit()
  i += 1

 print()
 print("Back-ups complete.")
 print()

Answer 1

this is an alternative. you can try using ftputil package. You can then use it to walk the remote directories and get your files

Answer 2

It is non-trivial at least. In the simplest case, you only assume you have files and directories. This isn't always the case, there are softlinks and hardlinks and Windows-style shortcut. Softlink and directory shortcut are particularly problematic since they make recursive directory possible, which would confuse naive-ly implemented ftp grabber.

How would you handle such recursive directory depends on your need; you might simply not follow softlinks or you might try to detect recursive links. Detecting recursive link is inherently tricky, you cannot do it reliably.