Python: Generating a MD5 checksum of a file?

Question

Is there any simple way of generating (and checking) MD5 checksums of a list of files in Python? (I have a small program I'm working on, and I'd like to confirm the checksums of the files).

Answer 1

There is a way that's pretty memory inefficient.

import hashlib
[(fname, hashlib.md5(file(fname, 'r').read()).digest()) for fname in fnamelst]

This will give you a list of tuples, each tuple containing the name of its file and its hash.

I strongly question your use of MD5. You should be at least using SHA1. MD5 is known broken, and shouldn't be used for any purpose, even if you don't think your purpose is security sensitive.

Here is a way that is more complex, but not memory inefficient:

import hashlib
def hashfile(afile, hasher, blocksize=65536):
    buf = afile.read(blocksize)
    while len(buf) > 0:
        hasher.update(buf)
        buf = afile.read(blocksize)
    return hasher.digest()

[(fname, hashfile(file(fname, 'r'), hashlib.md5()) for fname in fnamelst]

Answer 2

You can use hashlib.md5()

Note that sometimes you won't be able to fit the whole file in memory. In that case, you'll have to read chunks of 128 bytes sequentially and feed them to the Md5 function. See this question.