Programmatically get Google results onto web page

Question

To save time Googling my website in order to see where my pages have changed I've decided to create a web page that monitors the position of my page under certain key-phrases (namely, my name). For example, if I were to type in my name and my web page were at the top I'd want the page to show the Google results given and my position under the term. I'll then be using the Google Analytics API to create a mash-up page to manipulate this data.

Before anyone starts freaking out this part is NOT against Google's rules. Google do not want you to scrape their web pages in order to get these results, even though for me this would be by far the easiest way to check this. There are already automated tools that output Google results into XML format, although this is again against Google's Terms of Service.

I know that there are other alternatives to this, such as Google Alerts, but I'd like to be able to work around the problems that Google has given and implement such a tool without having to break their TOS.

There was an easy way to do this a few years ago with the SOAP API. However, this has been stopped and Google now give out a "AJAX API" for developers to implement a widget of sorts. This is no replacement for a real API and I'd definitely prefer a method to actually put the data on my web page and be able to modify the results as I wish.

Does anyone have any idea how I could implement this in JavaScript or a server-side language? As far as I can tell it's already been done with a number of the pages Google has provided as examples so I'd like to see if anyone on here has any ideas on how this is done without scraping Google.

Answer 1

Google calls it an "AJAX API" because of all of the client-side code they provide for accessing the API directly from the browser.

Under the hood, it's just a RESTful HTTP-based API that returns JSON-formatted data rather than XML, so if your server-side code can open an HTTP connection and parse the JSON response, no browser-side code is required.

Here's an OSS library for using the Google API if you're using ASP.NET on the server:

http://code.google.com/p/google-api-for-dotnet/