tags:

views:

40

answers:

1
+1  Q: 

scraping and parsing google data like page rank and more for a domain

Hi,

I need to scrape/parse some search engines related data for a given domain name(site).

I need

Google Page Rank (only for the domain name, not each pages).

Number of indexed results/pages (google, bing).

Number of Backlinks (google, bing, yahoo).

Traffic Rank (alexa).

Site thumbnail.

Could you provide me some pointers on where can I start? I tried to look around, but I was able to find only Google Ajax API which provides me number of indexed results only. That too expects valid headers which would mean, I have to be on that site, in order to make that work. So, I can't get data for any given domain.

Thanks

A: 

Their search API's are intentionally very limited so you will probably need to scrape this data directly from each website.

Plumo  2010-09-02 21:19:18
You're right that their api is intentionally limited. But, I find data hard to find where I can scrape.
cdpnet  2010-09-03 16:36:59

related questions

Displaying Flash content in a C# WinForms application
How to get the value of built, encoded ViewState?
Unhandled Exception Handler in .NET 1.1
How do I connect to a database and loop over a recordset in C#?
How do I most elegantly express left join with aggregate SQL as LINQ query
Get a new object instance from a Type in C#
.NET Testing Framework Advice
Automatically update version number
What is the difference between an int and an Integer in Java/C#?
How to write to Web.Config in Medium Trust ?
WinForms ComboBox data binding gotcha
How do you sort a C# dictionary by value?
Adding Scripting functionality to .NET applications
Floating Point Number parsing: Is there a Catch All algorithm?
How do I print an HTML document from a web service?
Decoding T-SQL CAST in C#/VB.net
Anatomy of a "Memory Leak"
How do I get a distinct, ordered list of names from a DataTable using Linq
Reliable Timer in a Console Application
How do I fill a DataSet or a DataTable from a LINQ query resultset ?
What's the difference between Math.Floor() and Math.Truncate() in .NET?
How do I calculate relative time?
How do I calculate someone's age in C#?
Are there any conversion tools for porting Visual J# code to C#?
When setting a form's opacity should I use a decimal or double?