C# How to extract data from asp + javascript website

Question

Hello to all. I want to create C# program which would extract some data from site. This site is bookmaker: bet365.com/home/ and i need to get:

1. name of specific market + odds (example:Soccer - Barclays Premier League - Stoke v Fulham - 2.75|3.20|2.62)
2. another market + odds (example: Horse Racing - Lingfield - 12:30 - 3.2|4.3|5.2|6.7|8.62|..)
3. there are also different format of odds(decimal,fractional,american) i need to chose decimal

But when i open this site as text file - there is no all that data - so i can not just parse it with HTML parser. I need an advice - what is the best way to extract data from this kind of web pages?

Answer 1

What you're trying to do is called Scraping (or web scraping).

You'll have to do a bit of reading to learn about various methods. There's certainly not a simple way to do it.

I'd also venture to guess that a betting site has measures in place to stop you from doing it.

Answer 2

The data is most likely being loaded via AJAX. This means that the HTML page you're loading contains mostly static content. Once the HTML is displayed the data is loaded from another URL. You can use Firebug to analyze the data being loaded in.

Once you have the URL in hand, you can now scrape that URL for the data you're after. Scraping generally pulls in the raw text and uses Regular Expressions.