Running a python script on all the files in a directory.

Question

I have a Python script that reads through a text csv file and creates a playlist file. However I can only do one at a time, like:

python playlist.py foo.csv foolist.txt

However, I have a directory of files that need to be made into a playlist, with different names, and sometimes a different number of files.

So far I have looked at creating a txt file with a list of all the names of the file in the directory, then loop through each line of that, however I know there must be an easier way to do it.

Answer 1

for f in *.csv; do
  python playlist.py "$f" "${f%.csv}list.txt"
done

Will that do the trick? This will put foo.csv in foolist.txt and abc.csv in abclist.txt.

Or do you want them all in the same file?

Answer 2

if you have directory name you can use os.listdir

os.listdir(dirname)

if you want to select only a certain type of file, e.g., only csv file you could use glob module.

Answer 3

Just use a for loop with the asterisk glob, making sure you quote things appropriately for spaces in filenames

for file in *.csv; do
   python playlist.py "$file" >> outputfile.txt;
done

Answer 4

Is it a single directory, or nested?

Ex.

topfile.csv
topdir
   --dir1
       --file1.csv
       --file2.txt
       --dir2
           --file3.csv
           --file4.csv

For nested, you can use os.walk(topdir) to get all the files and dirs recursively within a directory.

You could set up your script to accept dirs or files:

python playlist.py topfile.csv topdir

import sys
import os

def main():
    files_toprocess = set()
    paths = sys.argv[1:]
    for p in paths:
        if os.path.isfile(p) and p.endswith('.csv'):
            files_toprocess.add(p)
        elif os.path.isdir(p):
            for root, dirs, files in os.walk(p):
                files_toprocess.update([os.path.join(root, f) 
                                        for f in files if f.endswith('.csv')])