I have the domain www.mydomain.com and I set apache mod-rewrite so as to have www.mydomain.com/myappl. Where should I place the file robots.txt? thanks!
+2
A:
The robots.txt must be accessible to clients as http://www.mydomain.com/robots.txt . How to accomplish that depends on your mod-rewrite configuration. Please post the config if you need more help.
Also see http://en.wikipedia.org/wiki/Robots%5Fexclusion%5Fstandard for details.
sleske
2009-12-02 10:34:49
Unfortunately I dont't have the apache config (policy reason). I asked to be excluded from the "mod-rewrite" the robots and access as you said.
Angelo
2009-12-02 11:19:34
I don't really see how you're supposed to fix the server config without access to the (full!) config. Just tell that to your client.
sleske
2009-12-02 12:33:36
A:
I don't know ;-) But you can use the robots checker to find if your file is accessible. So if you don't know guess where it will be accessible and test it.
paradigmatic
2009-12-02 10:36:23