Using WebClient in C# is there a way to get the URL of a site after being redirected?

Question

Using the WebClient class I can get the title of a website easily enough:

WebClient x = new WebClient();    
string source = x.DownloadString(s);
string title = Regex.Match(source, 
    @"\<title\b[^>]*\>\s*(?<Title>[\s\S]*?)\</title\>",
    RegexOptions.IgnoreCase).Groups["Title"].Value;

I want to store the URL and the page title. However when following a link such as:

http://tinyurl.com/dbysxp

I'm clearly going to want to get the Url I'm redirected to.

QUESTIONS

Is there a way to do this using the WebClient class?

How would I do it using HttpResponse and HttpRequest?

Answer 1

With an HttpWebRequest, you would set the AllowAutoRedirect property to false. When this happens, any response with a status code between 300-399 will not be automatically redirected.

You can then get the new url from the response headers and then create a new HttpWebRequest instance to the new url.

With the WebClient class, I doubt you can change it out-of-the-box so that it does not allow redirects. What you could do is derive a class from the WebClient class and then override the GetWebRequest and the GetWebResponse methods to alter the WebRequest/WebResponse instances that the base implementation returns; if it is an HttpWebRequest, then set the AllowAutoRedirect property to false. On the response, if the status code is in the range of 300-399, then issue a new request.

However, I don't know that you can issue a new request from within the GetWebRequest/GetWebResponse classes, so it might be better to just have a loop that executes with HttpWebRequest/HttpWebResponse until all the redirects are followed.

Answer 2

The WebClient class has an option to follow redirects. Set that option and you should be fine.

Answer 3

HttpWebRequest.AllowAutoRedirect can be set to false. Then you'd have to manually http status codes in the 300 range.

// Create a new HttpWebRequest Object to the mentioned URL.
HttpWebRequest myHttpWebRequest=(HttpWebRequest)WebRequest.Create("http://www.contoso.com");    
myHttpWebRequest.MaximumAutomaticRedirections=1;
myHttpWebRequest.AllowAutoRedirect=true;
HttpWebResponse myHttpWebResponse=(HttpWebResponse)myHttpWebRequest.GetResponse();

Answer 4

Ok this is really hackish, but the key is to use the HttpWebRequest and then set the AllowAutoRedirect property to true.

Here's a VERY hacked together example

        HttpWebRequest req = (HttpWebRequest)WebRequest.Create("http://tinyurl.com/dbysxp");
        req.Method = "GET";
        req.AllowAutoRedirect = true;
        WebResponse response = req.GetResponse();

        response.GetResponseStream();
        Stream responseStream = response.GetResponseStream();

        // Content-Length header is not trustable, but makes a good hint.
        // Responses longer than int size will throw an exception here!
        int length = (int)response.ContentLength;

        const int bufSizeMax = 65536; // max read buffer size conserves memory
        const int bufSizeMin = 8192;  // min size prevents numerous small reads

        // Use Content-Length if between bufSizeMax and bufSizeMin
        int bufSize = bufSizeMin;
        if (length > bufSize)
            bufSize = length > bufSizeMax ? bufSizeMax : length;

        StringBuilder sb;
        // Allocate buffer and StringBuilder for reading response
        byte[] buf = new byte[bufSize];
        sb = new StringBuilder(bufSize);

        // Read response stream until end
        while ((length = responseStream.Read(buf, 0, buf.Length)) != 0)
            sb.Append(Encoding.UTF8.GetString(buf, 0, length));

        string source = sb.ToString();string title = Regex.Match(source, 
        @"\<title\b[^>]*\>\s*(?<Title>[\s\S]*?)\</title\>",RegexOptions.IgnoreCase).Groups["Title"].Value;

enter code here

Answer 5

If I understand the question, it's much easier than people are saying - if you want to let WebClient do all the nuts and bolts of the request (including the redirection), but then get the actual response URI at the end, you can subclass WebClient like this:

class MyWebClient : WebClient
{
    Uri _responseUri;

    public Uri ResponseUri
    {
        get { return _responseUri; }
    }

    protected override WebResponse GetWebResponse(WebRequest request)
    {
        WebResponse response = base.GetWebResponse(request);
        _responseUri = response.ResponseUri;
        return response;
    }
}

Just use MyWebClient everywhere you would have used WebClient. After you've made whatever WebClient call you needed to do, then you can just use ResponseUri to get the actual redirected URI. You'd need to add a similar override for GetWebResponse(WebRequest request, IAsyncResult result) too, if you were using the async stuff.