Cannot handle redirect from HTTP/HTTPS protocols to other dissimilar ones

Question

Basically, I'm trying to grab an EXE from CNet's Download.com

So i created web parser and so far all is going well.

Here is a sample link pulled directly from their site:

http://dw.com.com/redir?edId=3&siteId=4&oId=3001-20_4-10308491&ontId=20_4&spi=e6323e8d83a8b4374d43d519f1bd6757&lop=txt&tag=idl2&pid=10566981&mfgId=6250549&merId=6250549&pguid=PlvcGQoPjAEAAH5rQL0AAABv&destUrl=ftp%3A%2F%2F202.190.201.108%2Fpub%2Fryl2%2Fclient%2Finstaller-ryl2_v1673.exe

Here is the problem: When you attempt to download, it begins with HTTP, then redirects to an FTP site. I have tried .NET's WebClient and HttpWebRequest Objects, and it looks like Neither can support Redirects.

This Code Fails at GetResponse();

HttpWebRequest req = (HttpWebRequest)WebRequest.Create("http://dw.com.com/redir");
WebResponse response = req.GetResponse();

Now, I also tried this:

HttpWebRequest req = (HttpWebRequest)WebRequest.Create("http://dw.com.com/redir");
req.AllowAutoRedirect = false;
WebResponse response = req.GetResponse();
string s = new StreamReader(response.GetResponseStream()).ReadToEnd();

And it does not throw the error anymore, however variable s turns out to be an empty string.

I'm at a loss! Can anyone help out?

Answer 1

You can get the value of the "Location" header from the response.headers, and then create a new FtpWebRequest to download that resource.

Answer 2

in your first code snippet you will be redirected to a link using a different protocol (i.e it's no longer Http as in *Http*WebRequest) so it fails du to a malformed http response.

In the second part you're no longer redirected and hence you don't receive a FTP response (which is not malform when interpreted as HTTP response).

You need to acquire FTP link,as ferozo wrote you can do this by getting the value of the header "location", and use a FtpWebRequest to access the file