How can I get a history of the number of pages in Google's index for a particular site?

Question

A Google search for "site:example.com" will tell you the number of pages of example.com that are currently in Google's index. Is it possible to find out how this number has changed over time?

Answer 1

HubSpot does this for you. It costs money but they do a lot of useful things like this.

Answer 2

If you don't mind waiting, you could have a cron parse your site:example.com results every day and wait for the data to build up.

Answer 3

I was going to suggest Google Webmaster Tools, but it doesn't appear to have this information. How irritating.

Anyway, to follow on from UltimateBrent's answer, this regular expression will extract the value from a google search:

\d+(?=</b> from <b>domain\.net</b>)

Obviously, changing domain\.net to whatever your domain is.

Answer 4

I set up a Python script as a cron job to parse the result from the Google results page and save it. I set it to run once per day for each of a set of sites. I wrote another script to produce a CSV spreadsheet from the data. I can open that in a spreadsheet program and quickly make charts to visualise trends.

I have similar scripts for monitoring PageRank.

This will still only give me data from the day I begin checking. I do not know of a way to access historical values.

Answer 5

you can use domaintools.com

Rahul http://Valdot.com