views:

302

answers:

5

It seems like Google can index certain sites or forums (I can't name any offhand as its been months since I last saw one) and when accessing you are prompted with a request to register or login.

How would I make my site open for Google to index and have a regular login for others?

A: 

Is this what you're looking for?

Google Custom Search Engine

Fiarr
Even though you've posed the answer in the form of a question, it's completely wrong!
Shadi Almosri
.........Sorry?
Fiarr
+3  A: 

This is frowned upon.

  1. Your page rank suffers if Google sees this (google "expert exchange google").
  2. People can still access your site with Google Cache
  3. You have to take into account every other search engine too, as you'll have to use "browser sniffing"

What you'd do is sniff for the Googlebot/2.1 (+http://www.google.com/bot.html) and Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) user-agents (at least), and deliver to that client different content altogether.

Another option, is to do what expersexchange does, and have all the information buried deep down on the page. That way you have your cake, and eat it too: users don't completely hate you, your page rank doesn't suffer, and you still discriminate non-registered users give a better experience to registered users.

voyager
at one point I remember that site blocked the real answers on IE, but not firefox
Domenic
+2  A: 

Is this what you are asking?

Can Google Site Search index my password-protected pages?

If so then here is the answer from Google

Google Site Search is only able to provide search results for pages that are publicly available for crawl, not websites that require password information. If you have protected web content such as intranet pages that you need to index and search, you might want to consider one of Google's universal search options. You can learn more about the Google Mini and Google Search Appliance here.

Shoban
+1  A: 

We open a backdoor for Google and other search engines.

We have a white-list of IPs used by major search engines. If we see the requests are from one of the IPs, we bypass authentication and display an abbreviated version of the page so Google can index it but not all information are available from Google's cache.

We tried to use User-Agent but found people abuse it. IP is much harder to hack.

ZZ Coder
A: 

Create sitemap for Google Indexing and upload it into Google.Please visit this link : Inside Google Sitemaps: Using Sitemap Index Files

Braveyard