How do I get the [L] flag of RewriteRule (.htaccess) really working?

Question

To newcomers: While trying to comprehensively describe my problem and phrase my questions I produced huge ammount of text. If you don't want to read the whole thing, my observations about (read "proof of") [L] flag not working the misconception, from which it all sprung, is located in Additional observations section. Why I misunderstood apparent behaviour is described in my Answer as well as solution to given problem.

Setup

---

I have following code in my .htaccess file:

# disallow directory indexing
Options -Indexes

# turn mod_rewrite on
Options +FollowSymlinks
RewriteEngine on

# allow access to robots file
RewriteRule ^robots.txt$ robots.txt [NC,L]

# mangle core request handler address
RewriteRule ^core/(\?.+)?$ core/handleCoreRequest.php$1 [NC,L]

# mangle web file adresses (move them to application root folder)
# application root folder serves as application GUI address
RewriteRule ^$ web/index.html [L]
# allow access to images
RewriteRule ^(images/.+\.(ico|png|bmp|jpg|gif))$ web/$1 [NC,L]
# allow access to stylesheets
RewriteRule ^(css/.+\.css)$ web/$1 [NC,L]
# allow access to javascript
RewriteRule ^(js/.+\.js)$ web/$1 [NC,L]
# allow access to library scripts, styles and images
RewriteRule ^(lib/js/.+\.js)$ web/$1 [NC,L]
RewriteRule ^(lib/css/.+\.css)$ web/$1 [NC,L]
RewriteRule ^(lib/(.+/)?images/.+\.(ico|png|bmp|jpg|gif))$ web/$1 [NC,L]

# redirect all other requests to application address
# RewriteRule ^(.*)$ /foo/ [R]

My web application (and its .htaccess file) is located in foo subfolder of DOCUMENT_ROOT (accessed from browser as http://localhost/foo/). It has PHP core part located in foo/core and JavaScript GUI part located in foo/web. As can be seen from the code above, I want to allow access only to single core script that handles all requests from GUI and to 'safe' web files and redirect all other requests to base application address (last commented directive).

Problem

---

Behaviour

It works until I try the last part by uncommenting the last redirecting directive. If I comment some more lines, the appropriate page parts stop working, etc.

However, when I uncomment last line, which should be performed only when matching of all previous rules fails (at least that's what I understand), page goes into redirection cycle (Firefox throws error page with something like "This page isn't redirecting properly"), because it's redirecting to http://localhost/foo/ again and again and again, forever.

Questions

What I don't understand is this processing of this rule:

RewriteRule ^$ web/index.html [L],

specifically the [L] flag. The flag apparently doesn't work for me. When the last line is commented, it correctly redirects, but when I uncomment it, it is always processed, even though rewriting should stop on [L] flag. Anyone got any ideas?

Also, on a sidenote, I'd be thrilled to know why my following attempt at fixing it doesn't work either:

RewriteEngine on
RewriteRule ^core/(\?.+)?$ core/handleCoreRequest.php$1 [NC,L]
RewriteRule ^(.*)$ web/$1 [L]
RewriteRule ^.*$ /foo/ [L]

This actually doesn't work at all. Even if I remove the last line, it still doesn't redirect anything correctly. How does the redirecting work in the first example, if it doesn't work in the second?

It would also be of great benefit to me, if anybody knew any way to actually debug these directives. I spend hours on this without even the slightest clue what could possibly be wrong.

Additional observations

---

After trying the advice given by bbadour (not that I haven't tried it before, but now that I had a second opinion, I gave it another shot) and it didn't work, I've come up with the following observation. By rewriting last line to this:

RewriteRule ^(.*)$ /foo/?uri=$1 [R,L]

or this

RewriteRule ^(.*)$ /foo/?uri=%{REQUEST_URI} [R,L]

and using Firebug's Net panel, I found out more evidence, that the [L] flag is clearly not working as expected in the previously mentioned RewriteRule ^$ web/index.html [L] rule (let's call it THE RULE from now on). In first case I get [...]uri=web/index.html, in second case [...]uri=/foo/web/index.html. That means that THE RULE gets executed (rewrites ^$ to web/index.html), but the rewriting doesn't stop there. Any more ideas, please?

Answer 1

To debug, try simplifying your regex, and the url you ask for (a part of the full url you wanna match), and see if it's working, now step by step, add more bits to the regex adn the testing url, till you find where things are stopping to work properly.

Answer 2

Try using:

RewriteRule ^(.*)$ /foo/ [R,L]

If it still loops, put a RewriteCond in front of it to skip the rule if it is already /foo/

Answer 3

After hours of searching and testing, I finally found the real problem and solution. Hopefully this will help somebody else too, when they come across the same problem.

Cause of observed behavior

---

.htaccess file is processed after every redirect (even without [R] flag),

which means that after the RewriteRule ^$ web/index.html [L] is processed, mod_rewrite correctly stops rewriting, goes to the end of the file, redirects correctly to /foo/web/index.html, and then the server starts processing .htaccess file for the new location, which is the same file. Now only the last rewrite rule matches and redirects back to /foo/ (this time with [R], so the redirect can be observed in browser) ... and the .htaccess file is processed again, and again, and again...

Once more for clarity: Because only the hard redirects can be observed, it seems like the [L] flag is ignored, but it is not so. Instead, the .htaccess is processed two times redirecting back and forth between /foo/ and /foo/web/index.html.

Solution

---

Disallow direct access to subfolder

To virtually move subdirectory to application root directory, additional complex conditional rewrites must be used. Variable THE_REQUEST is useful for distinguishing between hard and soft redirects:

RewriteCond %{THE_REQUEST} ^GET\ /foo/web/
RewriteRule ^web/(.*) /foo/$1 [L,R]

For this rewrite rule to be matched, two conditions must apply. First, on second line, the "local URI" must start with web/ (which corresponds with absolute web URI /foo/web/). Second, on first line, the real request URI must start with /foo/web/ too. Together this means, that the rule only matches when the file inside the web/ subfolder is requested directly from the browser, in which case we want to do a hard redirect.

Redirect to allowed content from root to subfolder (soft)

RewriteCond $1 !^web/
RewriteCond $1 ^(.+\.(html|css|js|ico|png|bmp|jpg|gif))?$
RewriteRule ^(.*)$ web/$1 [L,NC]

We want to redirect to allowed content only if we haven't done it already, hence the first condition. Second condition specifies mask for allowed content. Anything matching this mask will be softly redirected, possibly returning 404 error if the content doesn't exist.

Hide all content not in subfolder or not allowed

RewriteRule !^web/ /foo/ [L,R]

This will do a hard redirect to application root for all URIs not beginning with web/ (and remember, only requests that can begin with web/ at this point are internal redirects for allowed content.

Real example

---

My code shown in my "question" after using solution tips mentioned above gradually transformed into the following:

# disallow directory indexing
Options -Indexes

# turn mod_rewrite on
Options +FollowSymlinks
RewriteEngine on

# allow access to robots file
RewriteRule ^robots.txt$ - [NC,L]

# mangle core request handler address
# disallow direct access to core request handler
RewriteCond %{THE_REQUEST} !^(GET|POST)\ /asm/core/handleCoreRequest.php
RewriteRule ^core/handleCoreRequest.php$ - [L]
# allow access to request handler under alias
RewriteRule ^core/$ core/handleCoreRequest.php [NC,QSA,L]

# mangle GUI files adressing (move to application root folder)
# disallow direct access to GUI subfolder
RewriteCond %{THE_REQUEST} ^GET\ /foo/web/
RewriteRule ^web/(.*) /foo/$1 [L,R]
# allow access only to correct filetypes in appropriate locations
RewriteCond $1 ^$ [OR]
RewriteCond $1 ^(images/.+\.(ico|png|bmp|jpg|gif))$ [OR]
RewriteCond $1 ^(css/.+\.css)$ [OR]
RewriteCond $1 ^(js/.+\.js)$ [OR]
RewriteCond $1 ^(lib/js/.+\.js)$ [OR]
RewriteCond $1 ^(lib/css/.+\.css)$ [OR]
RewriteCond $1 ^(lib/(.+/)?images/.+\.(ico|png|bmp|jpg|gif))$
RewriteRule ^(.*)$ web/$1 [L,NC]

# hide all files not in GUI subfolder that are not whitelisted above
RewriteRule !^web/ /foo/ [L,R]

What I don't like about this approach is that the application root folder must be hardcoded in .htaccess file (as far as I know), so the file must be generated on application install, not simply copied.