How do I download a file over HTTP using Python?

Question

I have a small utility that I use to download an MP3 from a website on a schedule and then builds/updates a podcast XML file which I've obviously added to iTunes.

The text processing that creates/updates the XML file is written in Python. I use wget inside a Windows .bat file to download the actual MP3 however. I would prefer to have the entire utility written in Python though. (It was the project I used to begin learning Python.)

I struggled though to find a way to actually down load the file in Python, thus why I resorted to wget.

So, how do I download the file using Python?

Answer 1

Use urllib2 which comes with the standard library.

import urllib2
response = urllib2.urlopen('http://www.example.com/')
html = response.read()

This is the most basic way to use the library, minus any error handling. You can also do more complex stuff such as changing headers. The documentation can be found here.

Answer 2

import urllib2
mp3file = urllib2.urlopen("http://www.example.com/songs/mp3.mp3")
output = open('test.mp3','wb')
output.write(mp3file.read())
output.close()

the 'wb' in open('test.mp3','wb') opens a (and erases any existing) file, binaraly, so you can save data with it, instead of just text.

Answer 3

I agree with Corey, urllib2 is more complete than urllib and should likely be the module used if you want to do more complex things, but to make the answers more complete, urllib is a simpler module if you want just the basics:

import urllib
response = urllib.urlopen('http://www.example.com/sound.mp3')
mp3 = response.read()

Will work fine. Or, if you don't want to deal with the "response" object you can call read() directly:

import urllib
mp3 = urllib.urlopen('http://www.example.com/sound.mp3').read()

Answer 4

One more, using urlretrieve

import urllib
urllib.urlretrieve ("http://www.example.com/songs/mp3.mp3", "mp3.mp3")

Answer 5

If you learning python, that look at http://www.java2s.com/Code/Python/Network/CatalogNetwork.htm Here you can find more beutiful snipets.