Is HTTP_REFERER supposed to be filled in when user tells a browser to "refresh"?

Question

I'm trying to use a referrer header check as a defence in depth (i.e. one of many security techniques while note solely relying on any one alone).

It seems that sometimes MSIE doesn't include the referrer on refresh. Is that expected behavior? Is there anyway I can detect refresh, so I that I know that a missing referrer is ok?

And yes, I know that referrers can be spoofed with custom browsers. Just saying, if a hacker does want to hack my little site, he will need to invest the time to set up an appropriate referrer.

Answer 1

Referrer is not set on bookmark clicks or when a user types in a URL.

It seems to me that this would also hold for refreshes, because otherwise it would look like you were getting a ton of incoming traffic from a single link even though the user had simply clicked the link once and then hit F5 a number of times.

Just my opinion though. It's such an unreliable indicator, it depends mostly on the behaviour of the individual browser client, and is usually customizable to boot.

Answer 2

Refreshing a page will usually invoke the browser to resend the request that was used to obtain the current page. (That’s also the reason why you’re asked if you want to resend a POST request.) So a Referer will only be send if the previous request did already contain a Referer too.

Answer 3

Most browser will resend the original request. That means the referer will be included if the original request has it.

Answer 4

Given that rfc2616 does not precisely specify what happens on "reload" (it only mentions that the Referer: MUST NOT be sent if the URI is obtained from the source that does not have its own URI), there might be some variability.

However, given that you talk within "security" context, I wonder if maybe you are expecting the Referer: to come when going from HTTPS-accessed page to the HTTP-accessed page and hitting the Reload, section 15.1.3 has a specific mention of this:

Clients SHOULD NOT include a Referer header field in a (non-secure)
HTTP request if the referring page was transferred with a secure
protocol.

I would not trust the Referer: from the security point of view. First, because the standard itself mentions that this is supposed to be adjustable (rfc2616, page 151):

The information sent in the From field might conflict with the user's
privacy interests or their site's security policy, and hence it
SHOULD NOT be transmitted without the user being able to disable,
enable, and modify the contents of the field. The user MUST be able
to set the contents of this field within a user preference or
application defaults configuration.

Second, from a practical angle, the control to tinker with this field e.g. for Firefox is in about:config, network.http.sendRefererHeader, documented e.g. at here.

And if one wants to send arbitrary referer header, then "more work" is merely a matter of downloading and starting curl with the option to set the referer to whatever they please. (--referer <URL>). So, trying to use it in any kind of security-like processing is a weak idea.

Answer 5

Beware that MSIE will not send the referer if you navigated to a link via JavaScript, this is a known bug (that hasn't seen any updates since IE6 development)

//e.g. this will not send the referer in IE
location.href = 'somepage.html';

//and I believe it also affects
location.reload();