urllib2 connection timed out error

Question

I am trying to open a page using urllib2 but i keep getting connection timed out errors.
The line which i am using is: f = urllib2.urlopen(url)

exact error is:

URLError: <urlopen error [Errno 110] Connection timed out>

Answer 1

As a general strategy, open wireshark and watch the traffic generated by urllib2.urlopen(url). You may be able to see where the error is coming from.

Answer 2

urllib2 respects robots.txt. Many sites block the default User-Agent.

Try adding a new User-Agent, by creating Request objects & using them as arguments for urlopen:

import urllib2

request = urllib2.Request('http://www.example.com/')
request.add_header('User-agent', 'Mozilla/5.0 (Linux i686)')

response = urllib2.urlopen(request)

Several detailed walk-throughs are available, such as http://www.doughellmann.com/PyMOTW/urllib2/