How can I open a website with urllib via proxy? [Python]

Question

Hello there

I have this program that check a website, and I want to know how can I check it via proxy in Python...

this is the code, just for example

while True:
        try:
            h = urllib.urlopen(website)
            break
        except:
            print '['+time.strftime('%Y/%m/%d %H:%M:%S')+'] '+'ERROR. Trying again in a few seconds...'
            time.sleep(5)

Answer 1

By default, urlopen uses the environment variable http_proxy to determine which HTTP proxy to use:

$ export http_proxy='http://myproxy.example.com:1234'
$ python myscript.py  # Using http://myproxy.example.com:1234 as a proxy

If you instead want to specify a proxy inside your application, you can give a proxies argument to urlopen:

proxies = {'http': 'http://myproxy.example.com:1234'}
print "Using HTTP proxy %s" % proxies['http']
urllib.urlopen("http://www.google.com", proxies=proxies)

Edit: If I understand your comments correctly, you want to try several proxies and print each proxy as you try it. How about something like this?

candidate_proxies = ['http://proxy1.example.com:1234',
                     'http://proxy2.example.com:1234',
                     'http://proxy3.example.com:1234']
for proxy in candidate_proxies:
    print "Trying HTTP proxy %s" % proxy
    try:
        result = urllib.urlopen("http://www.google.com",
                                proxies={'http': proxies})
        print "Got URL using proxy %s" % proxy
        break
    except:
        print "Trying next proxy in 5 seconds"
        time.sleep(5)