Htaccess/robots.txt to allow search bots to explore main domain but not directory on other domain

Question

Ok, I understand the Title didn't make any sense so here I've tried to explain it in detail.

I'm using a hosting that gives me space for my domain and lets me "add on" other domains on it. So lets say I have a domain A, and I add on a domain B. Basically my hosting gives me a public_html where I can put stuff that shows when someone visits website A. But, when I add the domain B, it lets me put the content of B, INSIDE of that public_html so that website B.com can also be visited by going to A.com/siteB... Thats all good, except that Google has started indexing B.com as well as A.com/siteB, I'm ok with it indexing B.com, but I somehow want to prevent it from indexing A.com/siteB so that when people search for B, it doesn't end up showing A.com/siteB.

Any ideas? Let me know if the question is still unclear.

Answer 1

Just put a

User-agent: *
Disallow: /siteB

in the robots.txt in the root of your main domain. All bots that have manners will accept this, but depending on the frequency they visit your site you might have to wait a bit before the search engines update their datbases.

Additionally you could add a .htaccess with a redirect rule to /siteB/... that redirects requests that have been done using the siteA domain to http://siteB/... This will make everyone use the canonical domain even if they browse to the lesser-favored one.