Multiple robots.txt for subdomains in rails

Question

I have a site with multiple subdomains and I want the named subdomains robots.txt to be different from the www one.

I tried to use .htaccess, but the FastCGI doesn't look at it.

So, I was trying to set up routes, but it doesn't seem that you can't do a direct rewrite since every routes needs a controller:

map.connect '/robots.txt', :controller => ?, :path => '/robots.www.txt', :conditions => { :subdomain => 'www' }
map.connect '/robots.txt', :controller => ?,  :path => '/robots.club.txt'

What would be the best way to approach this problem?

(I am using the request_routing plugin for subdomains)

Answer 1

If you can't configure your http server to do this before the request is sent to rails, I would just setup a 'robots' controller that renders a template like:

def show_robot
  subdomain = # get subdomain, escape
  render :text => open('robots.#{subdomain}.txt').read, :layout => false
end

Depending on what you're trying to accomplish you could also use a single template instead of a bunch of different files.

Answer 2

Actually, you probably want to set a mime type in mime_types.rb and do it in a respond_to block so it doesn't return it as 'text/html':

Mime::Type.register "text/plain", :txt

Then, your routes would look like this:

map.robots '/robots.txt', :controller => 'robots', :action => 'robots'

and the controller something like this (put the file(s) where ever you like):

class RobotsController < ApplicationController
  def robots
    subdomain = # get subdomain, escape
    robots = File.read(RAILS_ROOT + "/config/robots.#{subdomain}.txt")
    respond_to do |format|
      format.txt { render :text => robots, :layout => false }
    end
  end
end

at the risk of overengineering it, I might even be tempted to cache the file read operation...

Oh, yeah, you'll almost certainly have to remove/move the existing 'public/robots.txt' file.

Astute readers will notice that you can easily substitute RAILS_ENV for subdomain...