In Python, how to find all the files under a directory, including the files in subdirectories ?

Question

Is there any built in functions to find all the files under a particular directory including files under subdirectories ? I have tried this code, but not working...may be the logic itself is wrong...

def fun(mydir):
    lis=glob.glob(mydir)
    length=len(lis)
    l,i=0,0
    if len(lis):
        while(l+i<length):
            if os.path.isfile(lis[i]):
                final.append(lis[i])
                lis.pop(i)
                l=l+1
                i=i+1
            else:
                i=i+1
            print final
        fun(lis)
    else:
        print final

Answer 1

os.walk() is what you need.

Answer 2

There is no built-in function, but using os.walk it's trivial to construct it:

import os
def recursive_file_gen(mydir):
    for root, dirs, files in os.walk(mydir):
        for file in files:
            yield os.path.join(root, file)

ETA: the os.walk function walks directory tree recursively; the recursive_file_gen function is a generator (uses yield keyword to produce next file). To get the resulting list do:

list(recursive_file_gen(mydir))

Answer 3

I highly recommend this path module, written by Jason Orendorff:

http://pypi.python.org/pypi/path.py/2.2

Unfortunately, his website is down now, but you can still download from the above link (or through easy_install, if you prefer).

Using this path module, you can do various actions on paths, including the walking files you requested. Here's an example:

from path import path

my_path = path('.')

for file in my_path.walkfiles():
    print file

for file in my_path.walkfiles('*.pdf'):
    print file

There are also convenience functions for many other things to do with paths:

In [1]: from path import path

In [2]: my_dir = path('my_dir')

In [3]: my_file = path('readme.txt')

In [5]: print my_dir / my_file
my_dir/readme.txt

In [6]: joined_path = my_dir / my_file

In [7]: print joined_path
my_dir/readme.txt

In [8]: print joined_path.parent
my_dir

In [9]: print joined_path.name
readme.txt

In [10]: print joined_path.namebase
readme

In [11]: print joined_path.ext
.txt

In [12]: joined_path.copy('some_output_path.txt')

In [13]: print path('some_output_path.txt').isfile()
True

In [14]: print path('some_output_path.txt').isdir()
False

There are more operations that can be done too, but these are some of the ones that I use most often. Notice that the path class inherits from string, so it can be used wherever a string is used. Also, notice that two or more path objects can easily be joined together by using the overridden / operator.

Hope this helps!