Python: How to shutdown a threaded HTTP server with persistent connections (how to kill readline() from another thread)?

Question

I'm using python2.6 with HTTPServer and the ThreadingMixIn, which will handle each request in a separate thread. I'm also using HTTP1.1 persistent connections ('Connection: keep-alive'), so neither the server or client will close a connection after a request.

Here's roughly what the request handler looks like

request, client_address = sock.accept()
rfile = request.makefile('rb', rbufsize)
wfile = request.makefile('wb', wbufsize)

global server_stopping
while not server_stopping:
    request_line = rfile.readline() # 'GET / HTTP/1.1'
    # etc - parse the full request, write to wfile with server response, etc
wfile.close()
rfile.close()
request.close()

The problem is that if I stop the server, there will still be a few threads waiting on rfile.readline().

I would put a select([rfile, closefile], [], []) above the readline() and write to closefile when I want to shutdown the server, but I don't think it would work on windows because select only works with sockets.

My other idea is to keep track of all the running requests and rfile.close() but I get Broken pipe errors.

Ideas?

Answer 1

If you set daemon_threads to true in your HTTPServer subclass, the activity of the threads will not prevent the server from exiting.

class ThreadedHTTPServer(ThreadingMixIn, HTTPServer):
    daemon_threads = True

Answer 2

You could work around the Windows problem by making closefile a socket, too -- after all, since it's presumably something that's opened by your main thread, it's up to you to decide whether to open it as a socket or a file;-).