urllib.urlopen isn't working. Is there a workaround?

Question

I'm getting a getaddress error and after doing some sleuthing, it looks like it might be my corporate intranet not allowing the connection (I'm assuming due to security, although it is strange that IE works but won't allow Python to open a url). Is there a safe way to get around this?

Here's the exact error:

Traceback (most recent call last):
  File "<pyshell#1>", line 1, in <module>
    b = urllib.urlopen('http://www.google.com')
  File "C:\Python26\lib\urllib.py", line 87, in urlopen
    return opener.open(url)
  File "C:\Python26\lib\urllib.py", line 203, in open
    return getattr(self, name)(url)
  File "C:\Python26\lib\urllib.py", line 342, in open_http
    h.endheaders()
  File "C:\Python26\lib\httplib.py", line 868, in endheaders
    self._send_output()
  File "C:\Python26\lib\httplib.py", line 740, in _send_output
    self.send(msg)
  File "C:\Python26\lib\httplib.py", line 699, in send
    self.connect()
  File "C:\Python26\lib\httplib.py", line 683, in connect
    self.timeout)
  File "C:\Python26\lib\socket.py", line 498, in create_connection
    for res in getaddrinfo(host, port, 0, SOCK_STREAM):
IOError: [Errno socket error] [Errno 11001] getaddrinfo failed

More info: I also get this error with urllib2.urlopen

Answer 1

Check you are using the correct proxy.
You can get the proxy information by using urllib.getproxies (note: getproxies does not work with dynamic proxy configuration, like when using PAC).

Update As per information about empty proxy list, I would suggest using an urlopener, with the proxy name and information.
Some good information about how use proxies urlopeners:

1. Urllib manual
2. Michael Foord's introduction to urllib

Answer 2

You probably need to fill in proxy information.

import urllib2
proxy_handler = urllib2.ProxyHandler({'http': 'http://yourcorporateproxy:12345/'})
proxy_auth_handler = urllib2.HTTPBasicAuthHandler()
proxy_auth_handler.add_password('realm', 'host', 'username', 'password')

opener = build_opener(proxy_handler, proxy_auth_handler)
opener.open('http://www.stackoverflow.com')

Answer 3

Possibly this is a DNS issue, try urlopen with the IP address of the web server you're accessing, i.e.

import urllib
URL="http://66.102.11.99"   # www.google.com
f = urllib.urlopen(URL)
f.read()

If this succeeds, then it's probably a DNS issue rather than a proxy issue (but you should also check your proxy setup).

Answer 4

Looks like a DNS problem.

Since you are using Windows, you can try run this command

nslookup www.google.com

To check if the web address can be resolved successfully.

If not, it is a network setting issue

If OK, then we have to look at possible alternative causes

Answer 5

Hello there

I was facing the same issue. In my system the proxy configuration is through a .PAC file. So i opended that file, took out the default proxy url, for me it was http://168.219.61.250:8080/

Following test code worked for me :

import urllib2

proxy_support = urllib2.ProxyHandler({'http': 'http://168.219.61.250:8080/'})
opener = urllib2.build_opener(proxy_support)
urllib2.install_opener(opener)
response = urllib2.urlopen('http://python.org/')
html = response.read()
print html

You might need to add some more code, if your proxy requires authentication

Hope this helps!!