Use a Glob() to find files recursively in Python?

Question

This is what I have:

Glob(os.path.join('src','*.c'))

but I want to search the subfolders of src. Something like this would work:

Glob(os.path.join('src','*.c'))
Glob(os.path.join('src','*','*.c'))
Glob(os.path.join('src','*','*','*.c'))
Glob(os.path.join('src','*','*','*','*.c'))

But this is obviously limited and clunky.

Answer 1

You'll want to use os.walk to collect filenames that match your criteria. For example:

import os
cfiles = []
for root, dirs, files in os.walk('src'):
  for file in files:
    if file.endswith('.c'):
      cfiles.append(os.path.join(root, file))

Answer 2

Use os.walk to recursively walk a directory and fnmatch.filter to match against a simple expression:

import fnmatch
import os

matches = []
for root, dirnames, filenames in os.walk('src'):
  for filename in fnmatch.filter(filenames, '*.c'):
      matches.append(os.path.join(root, filename))

For Python versions older than 2.2, use the glob.glob() against each filename instead.

Answer 3

import os, fnmatch

def recursive_glob(treeroot, pattern):
  results = []
  for base, dirs, files in os.walk(treroot):
    goodfiles = fnmatch.filter(files, pattern)
    results.extend(os.path.join(base, f) for f in goodfiles)
  return results

fnmatch gives you exactly the same patterns as glob, so this is really an excellent replacement for glob.glob with very close semantics. An iterative version (e.g. a generator), IOW a replacement for glob.iglob, is a trivial adaptation (just yield the intermediate results as you go, instead of extending a single results list to return at the end).

Answer 4

Similar to above solutions, but using fnmatch.fnmatch instead of glob, since os.walk already listed the filenames:

import os, fnmatch


def find_files(directory, pattern):
    for root, dirs, files in os.walk(directory):
        for basename in files:
            if fnmatch.fnmatch(basename, pattern):
                filename = os.path.join(root, basename)
                yield filename


for filename in find_files('src', '*.c'):
    print 'Found C source:', filename

Also, using a generator alows you to process each file as it is found, instead of finding all the files and then processing them.