Why does HttpCacheability.Private suppress ETags?

Question

While writing a custom IHttpHandler I came across a behavior that I didn't expect concerning the HttpCachePolicy object.

My handler calculates and sets an entity-tag (using the SetETag method on the HttpCachePolicy associated with the current response object). If I set the cache-control to public using the SetCacheability method everything works like a charm and the server sends along the e-tag header. If I set it to private the e-tag header will be suppressed.

Maybe I just haven't looked hard enough but I haven't seen anything in the HTTP/1.1 spec that would justify this behavior. Why wouldn't you want to send E-Tag to browsers while still prohibiting proxies from storing the data?

using System;
using System.Web;

public class Handler : IHttpHandler {
    public void ProcessRequest (HttpContext ctx) {
        ctx.Response.Cache.SetCacheability(HttpCacheability.Private);
        ctx.Response.Cache.SetETag("\"static\"");
        ctx.Response.ContentType = "text/plain";
        ctx.Response.Write("Hello World");
    }

    public bool IsReusable { get { return true; } }
}

Will return

Cache-Control: private
Content-Type: text/plain; charset=utf-8
Content-Length: 11

But if we change it to public it'll return

Cache-Control: public
Content-Type: text/plain; charset=utf-8
Content-Length: 11
Etag: "static"

I've run this on the ASP.NET development server and IIS6 so far with the same results. Also I'm unable to explicitly set the ETag using

Response.AppendHeader("ETag", "static")

Update: It's possible to append the ETag header manually when running in IIS7, I suspect this is caused by the tight integration between ASP.NET and the IIS7 pipeline.

Clarification: It's a long question but the core question is this: why does ASP.NET do this, how can I get around it and should I?

Update: I'm going to accept Tony's answer since it's essentially correct (go Tony!). I found that if you want to emulate the HttpCacheability.Private fully you can set the cacheability to ServerAndPrivate but you also have call cache.SetOmitVaryStar(true) otherwise the cache will add the Vary: * header to the output and you don't want that. I'll edit that into the answer when I get edit permissions (or if you see this Tony perhaps you could edit your answer to include that call?)

Answer 1

Unfortunately if you look at System.Web.HttpCachePolicy.UpdateCachedHeaders() in .NET Reflector you see that there's an if statement specifically checking that the Cacheability is not Private before doing any ETag stuff. In any case, I've always found that Last-Modified/If-Modified-Since works well for our data and is a bit easier to monitor in Fiddler anyway.

Answer 2

@Trumpi: Actually, as far as I know/can tell that's not what private means.

From the HTTP/1.1 spec

Private
Indicates that all or part of the response message is intended for a single user and MUST NOT be cached by a shared cache. This allows an origin server to state that the specified parts of the response are intended for only one user and are not a valid response for requests by other users. A private (non-shared) cache MAY cache the response.

That should mean that the browser is allowed to cache the response. Public means that both the browser and any intermediary proxies are allowed to cache.

The .NET documentation states

private: the response is cacheable only on the client and not by shared (proxy server) caches.

@Duncan: I don't understand why that would affect my ability to manually add the ETag though, do you?

Answer 3

I think you need to use HttpCacheability.ServerAndPrivate

That should give you cache-control: private in the headers and let you set an ETag.

The documentation on that needs to be a bit better.

Edit: Markus found that you also have call cache.SetOmitVaryStar(true) otherwise the cache will add the Vary: * header to the output and you don't want that.