I have wildcard A records (*
and *.*
) for my domain pointing to my dev machine. Basically, any subdomain that is not www
is pointed to my dev machine (except the root).
I never want my dev machine to be index or "followed" by search engines.
What I would like to do is simple set up a global URL Rewrite rule like so:
<rule name="Global robots.txt rewrite" stopProcessing="true">
<match url="^robots\.txt" ignoreCase="true" />
<action type="Rewrite" url="http://localhost/robots.txt" />
</rule>
The rule above will not work; though the following redirect rule does:
<rule name="GLobal robots.txt redirect" stopProcessing="true">
<match url="^robots\.txt" ignoreCase="true" />
<conditions>
<add input="{HTTP_HOST}" pattern="^localhost$" negate="true" />
</conditions>
<action type="Redirect" url="http://localhost/robots.txt" />
</rule>
.. but I'm not sure if 301 redirecting to a robots.txt actually works for the search engines.
Any ideas on how to accomplish what I'm attempting?