Why is Apache mod_rewrite not behaving as expected

Question

I want to redirect URLs from an old site that used raw URL requests to my new site which I have implemented in CodeIgniter. I simply want to redirect them to my index page. I also would like to get rid of "index.php" in my URLs so that my URLs can be as simple as example.com/this/that. So, this is the .htaccess file I have created:

RewriteEngine on
Options FollowSymLinks

RewriteBase /

RewriteCond $1 ^assets
RewriteRule ^(.*)$ example/production/$1

RewriteCond %{QUERY_STRING} .+
RewriteRule ^(.*)$ index.php? [R=301]

RewriteCond $1  !^(index\.php|example|robots\.txt)
RewriteRule ^(.*)$ index.php/$1 

It should also be noted that my index.php is actually a symlink to example/production/index.php.

Now, the first rule works as expected - all my styles and images show up just fine, it's the second two rules I'm having trouble with. The second rule is basically to destroy the query string and redirect to my index page (externally). So, I found this in the Apache manual:

Note: Query String

The Pattern will not be matched against the query string. Instead, you must use a RewriteCond with the %{QUERY_STRING} variable. You can, however, create URLs in the substitution string, containing a query string part. Simply use a question mark inside the substitution string, to indicate that the following text should be re-injected into the query string. When you want to erase an existing query string, end the substitution string with just a question mark. To combine a new query string with an old one, use the [QSA] flag.

However, when I try to access one of the old pages, instead of redirecting to my index page, I get a 404 page not found error. I have figured out a workaround by making it an internal redirect, but I would really like it to be external.

The next problem, and the one that has been baffling me the most is with the third rule. I would expect this to do something like the following. If I type in:

http://example.com/this/thing

I would expect it to re-route to

http://example.com/index.php/this/thing

Unfortunately, this does not work. Instead, no matter what I type in, it always routes to my index page as if nothing else was in the URL (it just goes to http://example.com/).

Furthermore, and even more confusing to me, if I replace that rule with the following:

RewriteCond $1 !^(index\.php|example|robots\.txt)
RewriteRule ^(.*)$ index.php/this/thing

If I type in a URL such as http://example.com/other/thing, then it will go to http://example.com/index.php/this/thing as expected, BUT if I type in http://example.com/this/thing it goes to http://example.com/ (my index page). I can't make heads or tails out of it. Any help would be greatly appreciated.

Answer 1

This should solve your index.php problem and it will simply detect if a robots.txt is available:

RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]

Answer 2

hmmm - this doesn't seem to work either. The problem is my URLs aren't really asking for a filename or directory anyway. For example: example.com/index.php/this/thing should call the 'thing' method of the 'this' controller. – Steven Oxley

The condition is: If request is NOT a file and NOT a directory, so that was right, what you should have done is combine the appending of the request string:

RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ index.php/$1 [L]