download mbox files over https using python

Question

I was trying to find the right module for downloading kernel patches from kernel.org site

For example,to download the file at https://patchwork.kernel.org/patch/62948/mbox/

I understand urlgrabber has a problem with https on debian. urllib2 seems to have problem with this url as well (says getaddrinfo failed, even though there are no problems reaching other urls)

Any help would be appreciated

Answer 1

Hmm, maybe you need redirect handling?

Answer 2

Curious, this url should work fine (though I've tried it on Mac OS X only). I used this very simple test in my code:

import urllib
get_url = lambda url : urllib.urlopen(url).read()
data = get_url('https://patchwork.kernel.org/patch/62948/mbox/')

Of course, this loads the result into memory - but it does work. What version of Python are you using? The only thing I can think of is that your Python socket module has been compiled without SSL support.