views:

46

answers:

2

I'm using HttpWebRequest to scrape Wikipedia.org. A lot of times there will be links to topics on a page that have been consolidated and therefore they redirect you to the correct page.

for example

http://en.wikipedia.org/wiki/Polish_prisoners_of_war_in_Soviet_Union_(after_1939)

redirects you to the correct topic which is

http://en.wikipedia.org/wiki/Polish_prisoners_of_war_in_the_Soviet_Union_(after_1939)

Notice the addition of the word "the".

I need to determine at this point whether or not a redirect has happened. Can anyone suggest how I might do this?

Thanks!

UPDATE

I marked the response below as answered because technically that is how you tell if you have been redirected. The problem I am having is the Wikipedia is not actually doing a hard redirect with http response codes 3xx. They are doing soft redirects which serves up different content under the same Url. I'll have to find another solution.

+3  A: 

There is a property called "AllowAutoRedirects" on the HttpWebRequest object. If you turn that off you can follow the redirects yourself.

You could also try checking the HttpWebResponse.ResponseUri.

David
+1  A: 

Try this:

if(reponse.ResponseUri != request.RequestUri) {
    //You were redirected
}
SLaks