How can I route urllib requests through the TOR network? I have not been able to find any decent examples on the internet, can anyone help me?
+2
A:
Tor works as a proxy, right? So ask yourself "How do I use proxies in urllib?"
Now, when I look at the docs, first thing I see is
urllib.urlopen(url[, data[, proxies]])
which seems pretty suggestive to me...
dmckee
2009-04-02 20:15:02
I tried that, didnt work:>>> urllib.urlopen('http://www.google.com',proxies={'http':'http://127.0.0.1:9051'})<addinfourl at 61446104 whose fp = <socket._fileobject object at 0x03A82C30>>>>> _.read()
Lobe
2009-04-03 21:30:25
But Tor works for you from other apps? Can you use _any_ proxies with liburl?
dmckee
2009-04-04 02:15:30
Doesn't tor works on 127.0.0.1:8118?, I think 9051 is the control port.
jahmax
2010-07-24 01:03:31
No, TOR runs on 9050 by default, 9051 is control, and 8118 is for some extra privacy-helping proxies (edit: privoxy I think).
TomMD
2010-07-24 01:16:51
Oh yea, privoxy and polipo uses 8118 as default port.
jahmax
2010-07-24 01:21:40
A:
You should take a look at pycurl, the Python bindings to libcurl.
Deniz Dogan
2009-04-02 23:37:41
+1
A:
This works for me (using urllib2, haven't tried urllib):
def req(url):
proxy_support = urllib2.ProxyHandler({"http" : "127.0.0.1:8118"})
opener = urllib2.build_opener(proxy_support)
opener.addheaders = [('User-agent', 'Mozilla/5.0')]
return opener.open(url).read()
print req('http://google.com')
jahmax
2010-07-24 01:04:08