views:

137

answers:

1

Im experiencing problem with webclient to parse some very sensitive webpage. Somehow, webclient failed to do some task, and i dont even know why, no exception, no error, just simple not working (this is not the problem from the site)

I run webclient in a loop, however, only the first request was success, all the next requests are failed. When i restart the application, the same result appear, the first request always succeed, while all other request are failed.

Im sure all webclient are disposed properly, but i don't know what is the problem, is webclient saving information even after dispose?

Please guide me how to clear everything, reset everything and start brand new webclient.

for (int i = 1; i <= Count; i++)
{
    using (WebClient wc = new WebClient())
    {
        wc.Headers["Accept-Encoding"] = "gzip";
        wc.Headers["User-Agent"] = "Mozilla/4.0 (Compatible; Windows NT 5.1; MSIE 6.0) (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)";
        byte[] arr = wc.DownloadData(url);

        if (arr.Length > 0)
            Console.WriteLine(i.ToString() + ": SUCCESS");
        else
            Console.WriteLine(i.ToString() + ": FAILED");
    }
}
A: 

WebClient will not share any information between instances.
You probably have a different problem.

For example, the site might ignore requests that happen too frequently.

You should run Fiddler and check exactly what is happening.

Also, try adding Thread.Sleep(5000) in the loop and see whether anything changes. (In Fiddler)

SLaks
yes, in my program, i tried both fiddler and thread.sleep(5000) and it don't help. the site, in my actual request by hand, i can request to it without any problem. also, why it's only the first request is posible? i tried to run first request, close, run again, and it work totally fine.
DucDigital
Run Fiddler and see what the server is responding. (And whether there are any differences in the raw requests. Also, try making two requests using Fiddler's request builder.
SLaks
for the first answer, there's no different, but server respond Content-length 0. i tried to restart and run again, i can receive data, but all the next are content-length:0, if it's actually dispose everything, i suppose to loop successfully, right? but i need to restart the program and run again, very trouble. 2nd, request builder are not good for this case since i need to run a parse for some token before request the succeeding page...
DucDigital
Then you're probably getting the URL for the second page wrong.
SLaks
it's the same, the loop parse on the same page. i did this before and it work without a clue (i dont even know why it work) http://stackoverflow.com/questions/2123096/c-webclient-problem-with-looping-download/2123215#2123215 | but since there are some change on the target website, i need to rewrite code, since i dont have a clue what i did last time, now, i need to understand just why this happen.. :( i been write, rewrite code since 5h ago. but no luck
DucDigital