Section 14.13 of the HTTP/1.1 spec detailed the Content-Length header, and says this:
Applications SHOULD use this field to
indicate the transfer-length of the
message-body, unless this is
prohibited by the rules in section
4.4.
The word 'SHOULD' has a very specific meaning in RFCs:
- SHOULD This word, or the adjective "RECOMMENDED", mean that there may exist valid reasons in particular circumstances to ignore a particular item, but the full implications must be understood and carefully weighed before choosing a different course.
So, you may not always see a Content-Length. Typically you might not see it for any content which is dynamically generated, since that might be too expensive to service an exploratory HEAD request. For example, a HEAD request to Apache for a static file will have a Content-Length, but a request for a PHP script may not.
For example, try Google...
telnet www.google.com 80
HEAD / HTTP/1.0
Host:www.google.co.uk
HTTP/1.0 200 OK
Date: Mon, 04 Oct 2010 11:56:02 GMT
Expires: -1
Cache-Control: private, max-age=0
Content-Type: text/html; charset=ISO-8859-1
Server: gws
X-XSS-Protection: 1; mode=block
No content-length there.