ansaurus

Question

Using Python, how do I get an array of file info objects, based on a search of a file system?

Answer 1

+1 A:

You can use "find" through the "subprocess" module.
Afterwards, use the "split" string function to dissect each line
For each file, use the OS module (e.g. getmtime etc.) to get file information

or

Use the "walk" and "glob" modules to get the file paths in objects

jldupont 2009-10-24 11:08:54

Running subprocesses when not necessary is inefficient. Glob here isn't efficient either, because it would require double the number of operations on each directory (once for the walk, then a second operation for the glob.)

vy32 2009-10-24 12:04:54

I wasn't under the impression that there was that much optimization needed to be done here.

jldupont 2009-10-24 12:28:49

Answer 2

+2 A:

look into module os: os.walk is the function which walks the file system, os.path is the module which gives the file mtime and other file informations. also os.path defines a lot of functions for parsing and splitting filenames.

also of interest, module glob defines a functions for "globbing" strings (matching a string using unix wildcards rules)

from this, building a list of file matching some criterion should be easy.

Adrien Plisson 2009-10-24 11:10:50

Pretty much, however, now he needs to ask himself why does he need to code this in python other than "It's cool"

Daniel Goldberg 2009-10-24 11:13:02

python is more readable than a bash script and maybe far more efficient and powerful. also, you can build a whole application using python: in his case, he may be looking to mix the searching with a GUI to present the last shows in a user friendly fashion.

Adrien Plisson 2009-10-24 11:19:02

He already said this: he now wants to create symbolic links. This can get tedious in pure shell.

Martin v. Löwis 2009-10-24 11:19:43

Because coding anything at all complicated in shell script is an exercise in frustration and weird errors caused by incorrectly-escaped text?

bobince 2009-10-24 11:32:06

Answer 3

+3 A:

import os, time

allfiles = []
now = time.time()

# walk will return triples (current dir, list of subdirs, list of regular files)
# file names are relative to dir at first
for dir, subdirs, files in os.walk("/storage/disk-1/Media/Video/TV"):
    for f in files:
        if not f.endswith(".avi"):
            continue
        # compute full path name
        f = os.path.join(dir, f)
        st = os.stat(f)
        if st.st_mtime < now - 3600*24*7:
            # too old
            continue
        allfiles.append((f, st))

This will return all files that find also returned, as a list of pairs (filename, stat result).

Martin v. Löwis 2009-10-24 11:18:29

Awesome! Thanks for the code example, really helpful.

nbolton 2009-10-24 11:20:18

The interesting thing about this example is that the time is from when the search started, rather than "now." So if it takes longer than 7 days to search the file system, after 7 days no files will be returned. (Perhaps not an issue for 7 days, but an issue of you want all of the files less than 7 minutes old.)

vy32 2009-10-24 12:06:22

@simsong: if search takes a significant time, it is just not possible to get correct results merely by traversing the file system. If you assume that new files are created while the search is still in progress, then new files may also get created in parts of the tree that you have traversed already. The search will correctly return all files that were younger than 7 days when the search started.

Martin v. Löwis 2009-10-24 12:32:29

Note, that calling `os.stat()` for each file can be too slow for some applications such as file browser. Unfortunately `opendir()` is not available in Python, but it can be called using `ctypes` module.

Denis Otkidach 2009-10-24 13:42:30

@Denis: how would opendir help in finding out the last modification time of a file?

Martin v. Löwis 2009-10-24 13:52:31

ansaurus

tags:

views:

answers:

Using Python, how do I get an array of file info objects, based on a search of a file system?

related questions