Which SEO practises are likely to be responsible for SO questions appearing so quickly in Google searches?

Question

Does anyone have some idea as to how come questions posted here on SO are showing up so quickly on Google?.

Sometimes questions submitted are appearing as the first 10 entries or so - on the first page within 30 minutes of submitting a question. Pray tell, what sort of magic is being wielded here?

Anybody have some ideas, suggestions?. My first thought is that they have info in their sitemap that tells google robots to trawl every N minutes or so - is that whats going on?

BTW, I am aware that simply instructing Googlebots to scan your site every N minutes will not work if you dont have quality information (that is constantly being updated on your site).

I'd just like to know if there is something else that SO may be doing right (apart from the marvelous content of course)

Answer 1

To put it simply, more popular websites with more quality content and more frequent changes are ranked higher with Google's algorithm, and are indexed and cached more frequently than sites that are less popular or change less frequently.

Answer 2

Also helped by SO providing an RSS feed, I think google likes feeds from reliable sources.

Answer 3

Broadly speaking, it's only content that does it. The size and quality of content has reached Google's threshold for "spider as fast as the site will permit". SO has to actively throttle the Googlebot; Jeff has said on Coding Horror that they were getting more then 50,000 requests per day from Google, and that was over a year ago.

If you scan through non-news sites from the Alexa top 500 you will find virtually all of those have results in Google that are just minutes old. (i.e. type site:archive.org into Google and choose "Latest" in the menu on the left)

So there's nothing practical you can do to your own site to speed up spidering, except to increase the amount of traffic to your site...

Answer 4

It is really simple.

SO is a PageRank 6 site that gives the world new information.

Google has a strong bias on new information. It will crawl the site many times a day and it will immediately add the pages to its index. It will favor a page (top 10) to say a specific query for a small period of time (a few days) and then it will stop favoring that page and rank it as normal.

This is standard G procedure and it happens with many many sites.

As you might guess, grayhat/blackhat seo uses that fact in many ways.