Content-Length header versus chunked encoding

Question

I'm trying to weigh the pros and cons of setting the Content-Length HTTP header versus using chunked encoding to return [possibly] large files from my server. One or the other is needed to be compliant with HTTP 1.1 specs using persistent connections. I see the advantage of the Content-Length header being :

• Download dialogs can show accurate progress bar
• Client knows upfront if the file may/may not be too large for them to ingest

The downside is having to calculate the size before you return the object which isn't always practical and could add to server/database utilization. The downside of chunked encoding is the small overhead of adding the chunk size before each chunk and the download progress bar. Any thoughts? Any other HTTP considerations for both methods that I may not have thought of?

Answer 1

Use Content-Length, definitely. The server utilization from this will be almost nonexistent and the benefit to your users will be large.

For dynamic content, it's also quite simple to add compressed response support (gzip). That requires output buffering, which in turn gives you the content length. (not practical with file downloads or already compressed content (sound,images)).

Consider also adding support for partial content/byte-range serving - that is, capability to restart downloads. See here for a byte-range example (the example is in PHP, but is applicable in any language). You need Content-Length when serving partial content.

Of course, those are not silver bullets: for streaming media, it's pointless to use output buffering or response size; for large files, output buffering doesn't make sense, but Content-Length and byte serving makes a lot of sense (restarting a failed download is possible).

Personally, I serve Content-Length whenever I know it; for file download, checking the filesize is insignificant in terms of resources. Result: user has a determinate progress bar (and dynamic pages download faster thanks to gzip).

Answer 2

If the content length is known beforehand, then I would certainly prefer it above sending in chunks. If there's means of static files at the local disk file system or in a database, then any self-respected programming language and RDBMS provides ways to get the content length beforehand. You should make use of it.

On the other hand, if the content length is really unpredictable beforehand (e.g. when your intent is to zip several files together and send it as one), then sending it in chunks may be faster than buffering it in server's memory or writing to local disk file system first. But this indeed impacts the user experience negatively because the download progress is unknown. The impatient may then abort the download and move along.

Another benefit of knowing the content length beforehand is the ability to resume downloads. I see in your post history that your main programming language is Java; you can find here an article with more technical background information and a Java Servlet example which does that.