Python urllib2: How to ignore HTTPError 401

Question

Hello,

I want to access a web page with urllib2 and I keep getting an HTTP Error 401: Unauthorized.

Now, my problem is that this page doesn't need any authentication when using browsers like Firefox. Only when I use Google Chrome an authentication dialog pops up. Though this happens only after the page is fully loaded. So I can just cancel the authentication and use the page as normal.

Does anyone know how I can open this webpage without authentication. Thank you very much!

P.S. the pages url is: http://61.19.248.15/$sitepreview/sampran.go.th/

Answer 1

Try using this code:

>>> try: 
...     s = urllib2.urlopen('http://61.19.248.15/$sitepreview/sampran.go.th/')
... except urllib2.HTTPError, x:
...     print 'Ignoring', x.code
>>> 

Answer 2

If the web server wants to returnin a 401 status instead of returning the web page, there is no way for a client to grab the content anyway. Instead, you must convince the server to serve the page. Perhaps it is expecting authentication credentials that you already have saved in Firefox. Perhaps it wants to see Firefox's User-Agent header in your http request. If you can't guess or ask the server admin, you might try using Wireshark to sniff the network conversation of a successful session and compare it to that of an unsuccessful one.

Answer 3

This is one of the best article out there on authentication.

http://www.voidspace.org.uk/python/articles/authentication.shtml