I'm using VBScript and the Microsoft.XMLHTTP object to scrape some web data. I have a list of URLs to check, but unfortunately some of them 301 redirect to others on the list, so I wind up with redundant data.
Is it at all possible to make the XMLHTTP object fail on 301 redirect? Or at least cache the original response header? Or otherwise just let me know what happened?
(notes: I have no control over the server I'm requesting data from; when I get new data, I could check if it's redundant, but I'd like to avoid that if possible).
Any ideas would be greatly appreciated.