Read out a website every 5 seconds - how to do it fast?

Question

Hi,

I wanna read out a server status webpage every x seconds. Site is: http://www.ffxiv-status.com/

how to do it easily and fast?

I found this lines in Google, to read out the page source:

WebClient wClient = new WebClient();
string strSource = wClient.DownloadString("http://www.google.de");

I could split the page then wth string and so on and can have a look of the status and ms of the server.

Is that the fastes and resource best way or is there a better way?

(c#.net 3.5 VS 2010)

Answer 1

HTTP has support for a request header, If-Modified-Since, which will only download a page if it has changed since some specified date in the past. Why don't you use that to only retrieve the full status page in the event it bears new information?

Also, if you are polling every five seconds, something is wrong. You should probably set up a push notification system to avoid a constant waste of system resources.

Answer 2

It's every 5 seconds. Who cares. It's not going to take 5s, that's for sure. I'm not even sure if the CPU will wake from sleep mode to handle this process.

It's processing will be dominated by network traffic, and I use the word "dominated" very, very lightly.

If you were running this on an HP calculator with Kermit, you might want to think about some other potential optimizations. But, for real systems...just not an issue.

If it were me, I'd be looking at "wget" and grep/sed to find and clean up the data. But I'm lazy.

Answer 3

What you're trying to make is a small scraper. You can definitely get the page source and then analyze it. I suggest using regular expressions to look for the desired content. Of course you'll have to keep updating your parsing routines if the page design changes.