Process a set of files from a source directory to a destination directory in Python

Question

Being completely new in python I'm trying to run a command over a set of files in python. The command requires both source and destination file (I'm actually using imagemagick convert as in the example below).

I can supply both source and destination directories, however I can't figure out how to easily retain the directory structure from the source to the destination directory.

E.g. say the srcdir contains the following:

srcdir/
   file1
   file3
   dir1/
       file1
       file2

Then I want the program to create the following destination files on destdir: destdir/file1, destdir/file3, destdir/dir1/file1 and destdir/dir1/file2

So far this is what I came up with:

import os
from subprocess import call

srcdir = os.curdir # just use the current directory
destdir = 'path/to/destination'

for root, dirs, files in os.walk(srcdir):
    for filename in files:
        sourceFile = os.path.join(root, filename)
        destFile = '???'
        cmd = "convert %s -resize 50%% %s" % (sourceFile, destFile)
        call(cmd, shell=True)

The walk method doesn't directly provide what directory the file is under srcdir other than concatenating the root directory string with the file name. Is there some easy way to get the destination file, or do I have to do some string manipulation in order to do this?

Answer 1

While not pretty, this will preserve the directory structure of the tree:

_, _, subdirs = root.partition(srcdir)
destfile = os.path.join(destdir, subdirs[1:], filename)

Answer 2

Change your loop to:

for root, dirs, files in os.walk(srcdir):
    destroot = os.path.join(destdir, root[len(srcdir):])
    for adir in dirs:
        os.makedirs(os.path.join(destroot, adir))
    for filename in files:
        sourceFile = os.path.join(root, filename)
        destFile = os.path.join(destroot, filename)
        processFile(sourceFile, destFile)

Answer 3

There are a few relative path scripts out there that will do what you want -- namely find the relative path between two paths. E.g.:

• http://www.voidspace.org.uk/python/pathutils.html (relpath method)
• http://code.activestate.com/recipes/302594-another-relative-filepath-script/
• http://groups.google.com/group/comp.lang.python/browse_thread/thread/390d8d3e3ac8ef44/d8c74f96468c6a36?q=relative+path&rnum=1&pli=1

Unfortunately, I don't think this functionality has ever been added to core python.