Using python sockets to receive large http requests

Question

I am using python sockets to receive web style and soap requests. The code I have is

import socket
svrsocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
host = socket.gethostname();
svrsocket.bind((host,8091))
svrsocket.listen(1)
clientSocket, clientAddress = svrsocket.accept()
message = clientSocket.recv(4096)

Some of the soap requests I receive, however, are huge. 650k huge, and this could become several Mb. Instead of the single recv I tried

message = ''
while True:
  data = clientSocket.recv(4096)
  if len(data) == 0:
   break;
  message = message + data

but I never receive a 0 byte data chunk with firefox or safari, although the python socket how to says I should.

What can I do to get round this?

Answer 1

Unfortunately you can't solve this on the TCP level - HTTP defines its own connection management, see RFC 2616. This basically means you need to parse the stream (at least the headers) to figure out when a connection could be closed.

See related questions here - http://stackoverflow.com/search?q=http+connection