best way to download large files with python

Question

Which library/module is the best to use for downloading large 500mb+ files in terms of speed, memory, cpu? I was also contemplating using pycurl.

Answer 1

At sizes of 500MB+ one has to worry about data integrity, and HTTP is not designed with data integrity in mind.

I'd rather use python bindings for rsync (if they exist) or even bittorrent, which was initially implemented in python. Both rsync and bittorrent address the data integrity issue.