views:

76

answers:

2

I'm working on an SEO app that (among other things) shows the number of incoming links to your site over time.

There are a few ways to get this data. Scraping Google "link:yoursite.com" results gives you some (not all) of the links they know about, but they aren't too happy if you are doing lots of scraping. Similarly Yahoo has their site explorer API calls which give you 20,000 queries per day I think, but it has a non commercial use limitation.

Are there any services out there which provide this information in some kind of API call? Probably a pay service would be best, need something reliable for commercial use. Basically it would have to be an index of the web that was updated fairly regularly selling API access to it. Or any other ways people get this info short of writing a SETI at home type app to make a bot army of scrapers?

A: 

Could you do the following:

Use google analytics, or logs to determine the referer websites to the page. (Which use the referer header to determine where there visitor has come from.)

Then you write a program to scan those websites, to determine the links on each page, referring to your website?

You might be able to use Yahoo's BOSS API, but that's going to be different to google's index.

Bravax
Thats a good point, the referrer log entries are a decent source. Although it does require a human or bot click the link first, and the user to paste some code into their site. That could work though.
Brian Armstrong
+1  A: 

Have a look at http://www.majesticseo.com.

They have reports for the incoming links to a website and you can even download them in CSV format. If you own the website (or doing it for a client who owns the site) then you can get the reports for free. Otherwise you can pay to obtain access statistics for other sites.

jtes0111
Yep, had just started speaking with them! This is the closest answer I've found so far. Thanks!
Brian Armstrong