How to filter files (with known type) from os.walk?

Question

I have list from os.walk. But I want to exclude some directories and files. I know how to do it with directories:

for root, dirs, files in os.walk('C:/My_files/test'):
    if "Update" in dirs:
        dirs.remove("Update")

But how can I do it with files, which type I know. because this doesn't work:

if "*.dat" in files:
    files.remove("*.dat")

Answer 1

files = [file for file in files if os.path.splitext(file)[1] != '.dat']

Answer 2

A concise way of writing it, if you do this a lot:

def exclude_ext(ext):
    def compare(fn): return os.path.splitext(fn)[1] != ext
    return compare

files = filter(exclude_ext(".dat"), files)

Of course, exclude_ext goes in your appropriate utility package.

Answer 3

files = [ fi for fi in files if not fi.endswith(".dat") ]

Answer 4

Should be exactly what you need:

if thisFile.endswith(".txt"):

Answer 5

Try this:

import os

skippingWalk = lambda targetDirectory, excludedExtentions: (
    (root, dirs, [F for F in files if os.path.splitext(F)[1] not in excludedExtentions]) 
    for (root, dirs, files) in os.walk(targetDirectory)
)

for line in skippingWalk("C:/My_files/test", [".dat"]):
    print line

This is a generator expression generating lambda function. You pass it a path and some extensions, and it invokes os.walk with the path, filters out the files with extensions in the list of unwanted extensions using a list comprehension, and returns the result.

(edit: removed the .upper() statement because there might be an actual difference between extensions of different case - if you want this to be case insensitive, add .upper() after os.path.splitext(F)[1] and pass extensions in in capital letters.)