want 301 redirects to all except robots.txt

Question

Hi all, we have moved our website to new domain & want all pages of old site to get removed from search engines. Its the same site, same content, just a new domain, so search-engines are taking time because of duplicate content (maybe). We have added .htaccess 301 from our old site to new site as: redirect 301 / http://new-domain.com/

Now, to get removed our old site from Search Engines, we changed our robots.txt of old site as: User-agent: * Disallow: /

Now the problem is, the search engines are fetching robots.txt of new-domain.com because of .htaccess 301 redirect.

How do I restrict 301 redirect for robots.txt

Thanks.

Answer 1

Remove the Redirect directive and try this mod_rewrite rule:

RewriteEngine on
RewriteRule !^robots\.txt$ http://other.example.com%{REQUEST_URI} [L,R=301]

This will redirect any request except /robots.txt.

Answer 2

Thanks Gumbo,

It has solved the problem about not redirecting robots.txt, but now the problem is:

It redirects old-domain.com/any-page/ to new-domain.com, instead of new-domain.com/any-page/

just in case if required: my old domain: www.acme.0lx.net my new domain: http://theAcme.tk

-Thanks