Why does WGET return 2 error messages before succeeding?

Question

I am using a script to pull down some XML data on a authentication required URL with WGET.

In doing so, my script produces the following output for each url accessed (IPs and hostnames changed to protect the guilty):

> Resolving host.name.com... 127.0.0.1
> Connecting to host.name.com|127.0.0.1|:80... connected.
> HTTP request sent, awaiting response... 401 Access denied
> Connecting to host.name.com|127.0.0.1|:80... connected.
> HTTP request sent, awaiting response... 401 Unauthorized
> Reusing existing connection to host.name.com:80.
> HTTP request sent, awaiting response... 200 OK

Why does WGET complain that accessing the URL fails twice before successfully connecting? Is there a way to shut it up, or get it to connect properly in the first attempt?

For reference, here's the line I am using to call WGET:

wget --http-user=USERNAME --password=PASSWORD -O file.xml http://host.name.com/file.xml

Answer 1

This appears to be by design. Following the advice of @Wayne Conrad, I added the -d switch and was able to observe the first attempt failing because NTLM was required, and the second attempt failing because the first NTLM attempt was only level 1, where a level 3 NTLM challenge-response was required. WGET finally provides the needed authentication at the third attempt.

WGET does get a cookie to prevent re-authenticating for the duration of the session, which would prevent this if the connection wasn't terminated between files. I would need to pass WGET a list of files for this to occur, however I am unable to because I do not know the file names in advance.