views:

96

answers:

2
+1  Q: 

WebRequest bug?

EDIT: Solved, the problem was server-side.

I'm using C# and .NET2 and I wonder is that a WebRequest bug.. I do several good requests with this method and all is fine, but after that every time I get "The operation has timed out.". I really don't understand why is that.

public string RequestPage(string url) {
        HttpWebRequest req = null;
        string line = "";
        string site = "";

        try {
            req = (HttpWebRequest) WebRequest.Create(url.Trim());
            req.Timeout = 10000;

            StreamReader reader = new StreamReader(req.GetResponse().GetResponseStream());
            while ((line = reader.ReadLine()) != null) {
                site += line;
            }

            return site;
        } catch (Exception ex) {
            MessageBox.Show("ERROR " + ex.Message);
        }

        return null;
    }
+2  A: 

You're not disposing of the response:

using (WebResponse response = req.GetResponse())
using (StreamReader reader = new StreamReader(response.GetResponseStream())
{
    while ((line = reader.ReadLine()) != null) {
        site += line;
    }
}

Basically there are pooled connections per server that you talk to. You're running out of them because you're not closing the response. The above should sort it out.

Additionally:

  • That's a potentially very slow way of building up a string. Use a StringBuilder to concatenate text content in a loop.
  • Do you really want to remove all the line breaks? If not, just use reader.ReadToEnd() instead.
Jon Skeet
I just tried that. Same stuff. On the 4th request it hangs.
blez
@blez, the code that @Jon showed is perfectly fine so try it with some other site. If it works then youtube is throttling your requests.
Darin Dimitrov
@Darin Dimitrov yes, it works for other sites. So it should be youtube. What should I do? Send IE/Firefox headers or?
blez
@blez, yes, try sending FireFox headers. If this doesn't work you are out of luck. Maybe look for some API that youtube exposes. I am not aware, never worked with it.
Darin Dimitrov
OK, blagodaria ;)
blez
@blez, molia :-)
Darin Dimitrov
I just used Firefox header, and the problem persists. BUT.. All works fine after a restart of the app. So the problem should be from my side?
blez
@blez, could you post the url you are testing with?
Darin Dimitrov
I use these links: http://pastie.org/1104156 and every time after the 3-4th link it fails until restarting.
blez
+2  A: 

I don't know if this solves your problem, but you should always dispose a HttpWebResponse (and other objects that implement IDisposable) when you're done:

public string RequestPage(string url)
{
    HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);
    req.Timeout = 10000;

    using (WebResponse resp = req.GetResponse())
    using (StreamReader reader = new StreamReader(resp.GetResponseStream()))
    {
        return reader.ReadToEnd();
    }
}

If you don't actually require all the features of HttpWebRequest, you can use WebClient instead:

public string RequestPage(string url)
{
    using (WebClient client = new WebClient())
    {
        return client.DownloadString(url);
    }
}
dtb
Using the your second method it sill hangs. Maybe it's some youtube bot protection?
blez
+1 for `WebClient`.
Darin Dimitrov