What's the best way to get the size of a folder and all the files inside from python?

Question

If there will be a small number of files it should be easy with a recursive function to pass through all the files and add the size but what if there are lots of files, and by lots i really mean lots of files.

Answer 1

You mean something like this?

import os
for path, dirs, files in os.walk( root ):
    for f in files:
        print path, f, os.path.getsize( os.path.join( path, f ) )

Answer 2

There is no other way to compute the size than recursively invoking stat. This is independent of Python; the operating system just provides no other way.

The algorithm doesn't have to be recursive; you can use os.walk.

There might be two exceptions to make it more efficient:

1. If all the files you want to measure fill a partition, and the partition has no other files, then you can look at the disk usage of the partition.
2. If you can continuously monitor all files, or are responsible for creating all the files yourself, you can generate an incremental disk usage.

Answer 3

There is a recipe for that problem in the Python Cookbook (O'Reilly). You can read the full solution with an example online:

http://safari.oreilly.com/0596001673/pythoncook-CHP-4-SECT-24

or here:

http://books.google.com/books?id=yhfdQgq8JF4C&pg=PA152&dq=du+command+in+python