Where to put robots.txt file?

Question

Where to put robots.txt?

domainname.com/robots.txt

ot

domainname/public_html/robots.txt

I placed here

domainname.com/robots.txt

but it's not opening when i type this in browser

Answer 1

Where the file goes in your filesystem depends on what host you're using, so it's hard for us to give a specific answer about that.

The best description is: put it wherever the index.html (or index.php or whatever) file is that represents your homepage. If that's domainname/public_html/index.html, for example, put it in domainname/public_html/robots.txt.

Answer 2

In this case you should put it in domainname/public_html/robots.txt, as the public.html folder is where your index file will be.

Answer 3

i think the better way to describe it is to have it in the root web folder of your domain... so http://example.com/robots.txt you can also put your sitemap.xml in the root or refer to it with a Sitemap: http://example.com/fldr/smap.xml line in your robots.txt.

dont forget: you can use Google Webmaster Tools to check to make sure you haven't restricted anything you didnt mean to(you also get to see queries and links woohoo!).

suggestion: id consider using the <META NAME="ROBOTS" CONTENT="INDEX, NOFOLLOW"> if possible because you will still earn linkjuice for links on the page but it wont show up in googles index while a robots.txt directive can leave a plain url with do description in SERPs but will loose all value of links pointed to it because its robots.txted out (its ranking b/c of anchor text so get credit for it)

Answer 4

In the root of your web directory (where you put the files that show up on your website)