Python problems with FancyURLopener, 401, and "Connection: close"

Question

I'm new to Python, so forgive me if I am missing something obvious.

I am using urllib.FancyURLopener to retrieve a web document. It works fine when authentication is disabled on the web server, but fails when authentication is enabled.

My guess is that I need to subclass urllib.FancyURLopener to override the get_user_passwd() and/or prompt_user_passwd() methods. So I did:

class my_opener (urllib.FancyURLopener):

    # Redefine
    def get_user_passwd(self, host, realm, clear_cache=0):
        print "get_user_passwd() called; host %s, realm %s" % (host, realm)
        return ('name', 'password')

Then I attempt to open the page:

try:
    opener = my_opener()
    f = opener.open ('http://1.2.3.4/whatever.html')
    content = f.read()
    print "Got it:  ", content

except IOError:
    print "Failed!"

I expect FancyURLopener to handle the 401, call my get_user_passwd(), and retry the request.

It does not; I get the IOError exception when I call "f = opener.open()".

Wireshark tells me that the request is sent, and that the server is sending a "401 Unauthorized" response with two headers of interest:

WWW-Authenticate: BASIC
Connection: close

The connection is then closed, I catch my exception, and it's all over.

It fails the same way even if I retry the "f = opener.open()" after IOError.

I have verified that my my_opener() class is working by overriding the http_error_401() method with a simple "print 'Got 401 error'". I have also tried to override the prompt_user_passwd() method, but that doesn't happen either.

I see no way to proactively specify the user name and password.

So how do I get urllib to retry the request?

Thanks.