What's Wrong With This HTTP/1.1 Request? Sometimes the Client Accepts it, and Sometimes it Rejects it.

Question

I am in the process of writing a small HTTP/1.1 web server. I have threading turned off and am not currently using persistent connections. For normal requests where I specify the Content-Length and write the bytes out to the socket, everything works great.

However, I need the ability to support chunked transfer encoding. When some of the browsers hit the uri for this, they make the request, immediately close the connection, then make the request again at which point it succeeds. Sometimes I have seen them close the connection multiple times, re-requesting multiple times, until it succeeds. Other times I have seen them close the connection and not retry.

Here is a log of the output that I am sending over the socket. The number in parentheses indicates the number of bytes that it actually attempted to send:

send(21): 'HTTP/1.1 100 Continue'
send(2):  '\r\n'
send(2):  '\r\n'
send(15): 'HTTP/1.1 200 OK'
send(2):  '\r\n'
send(26): 'Transfer-Encoding: chunked'
send(2):  '\r\n'
send(38): 'Content-Type: application/octet-stream'
send(2):  '\r\n'
send(52): 'Content-Disposition: attachment; filename="test.mp3"'
send(2):  '\r\n'
send(17): 'Connection: close'
send(2):  '\r\n'
send(35): 'Date: Mon, 20 Sep 2010 11:38:34 GMT'
send(2):  '\r\n'
send(2):  '\r\n'
send(4):  '4000'
send(2):  '\r\n'
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 64329)
...
    self.request.send("\r\n")
error: [Errno 32] Broken pipe
----------------------------------------

Note that each printed line there represents the bytes its about to send, not bytes that have actually been sent - thus it fails on that final "\r\n". Also note that sometimes it will successfully send a few chunks before the socket is closed. I can see in the browser that a few kilobytes have been received.

EDIT: Here's a request from Firefox 4 beta 6. My server was able to send back 16KB (according to FireBug) before the socket crapped out. Note that my server ignores the Range header, if that makes a difference:

GET /audios/11/download.ogg HTTP/1.1
Host: localhost:5000
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:2.0b6) Gecko/20100101 Firefox/4.0b6
Accept: audio/webm,audio/ogg,audio/wav,audio/*;q=0.9,application/ogg;q=0.7,video/*;q=0.6,*/*;q=0.5
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive
Range: bytes=0-
Cookie: player-61646d696e=1; player-64617665=4; JSESSIONID=6glsv8468cbp

Answer 1

Wild guess: the Connection:close-Header doesn't "feel" right. Actually, I think this is a request-header, not a response-header.

Edit: If I read this part of RFC2616 correctly, HTTP-Continue (HTTP 100) tells the client to continue with its request, so maybe this header should also be omitted.