views:

53

answers:

3

I'm working on this big website and I want to put it online before its fully finished...

I'm working locally and the database is getting really big so I wanted to upload the website and continue to work on it in the server, but allowing people to enter, so I can test.

The question is if this is good for SEO, I mean, there are a lot of things SEO related that are incomplete.. For example: there are no friendly URLs, no sitemap, no .htacces file, lot of 'in-construction' sections...

Will Google penalize me forever? How does it work? Google indexes and gets the structure of the site just once or is it constantly updating and checking for changes? Will using User-agent: * Disallow: in robots.txt fully stop Google from indexing it? Can I change the robots.txt file later and have Google index it again? What dp you recommend?

+1  A: 

Sure, just put a robots.txt file in your root so you can be safe that google doesn't start indexing it.

Like this:

User-agent: *
Disallow: /
Agos
it will fully stop google from indexing it? If I later change it, will google reindex my site?
Jonathan
This will stop all compliant search engines (including Google) in their tracks from touching your site. They will come back when your robots.txt changes.
Dolph
A: 

As long as your security is not beta quality, it's a good idea to get your site online as early as possible.

Google indexes your site periodically, and will index more frequently as it detects more frequent changes and/or your pagerank increases.

Dolph
+1  A: 

This is how i understand this issue:

Google will reach your website if someone submitted your website URL http://www.google.com/addurl/ or there is a link to your website in another already indexed website.

When google reach your website it will look at the robots.txt and will see what rules there, if you disallow indexing using code like the following, google will not index your website at the moment.

User-agent: *
Disallow: /

But google will visit your website again after some days may be, and will do the same as the first time, if you didn't find the robots.txt or found that you put rules that allow them to index the website using code like the following, they will start indexing the website pages and content.

User-agent: *
Allow: /

About putting the website online from now or not? if you will disallow google index using robots.txt, there no difference, go for which is better for you.

Note: I am not sure 100% from rules i mentioned in this answer as google always change their indexing technics.

Also what i said about Google is the same for other search engines such as yahoo and bing, but its not a rule for any search engine, its just a common way, so may be other search engine index all your website links while you have robots.txt disallow indexing.

And i used to put a stage version from my websites to test on the live environment before going on the real life version, and used to use the robots.txt and i never found any of these stage links in Google, Bing or Yahoo.

Amr ElGarhy