HTTP Proxy/FastCGI/SCGI not closing connection when client disconnected - bug or feature?

Question

I'm working on Comet support for CppCMS framework via long XMLHttpRequest polls. In many cases, such request is closed by client before any response from server was given -- for example the page is closed, user moves to other page or it is just refeshed.

At the server side I expect that I would recieve the notification that connection is dropped. I tested the application via 3 connectors: FastCGI, SCGI and simple HTTP Proxy.

From 3 major UNIX web servers, Apache2, lighttpd and Nginx, only the last one had closed connection as expected allowing my application to remove the request from wait queue -- this worked for both FastCGI and HTTP Proxy connectors. (Nginx does not have scgi module by default).

Others, Apache and Lighttpd do not close connection or inform the backend about disconnected clients, the proceed as if the client is still on line. This happens for all 3 supported APIs: FastCGI, SCGI and HTTP Proxy.

I had opened an issue for Lighttpd, but what more conserns me is the fact that Apache -- mature and well supported web server as lighttpd and does not discloses the server backend that client had gone.

Questions:

1. Is this a bug or this is a feature? Is there any reason not to close the connection between web server and application backend?
2. Are there real life Comet application working behind these servers via FastCGI/SCGI/HTTP-Proxy backends?
3. If the above true, how do they deal with this issue? I understand that I can timeout all connections every 10 seconds, but I would like to keep them idle as far as client listens -- because this allows easier scale up -- each connection is very cheep -- the cost is only the opended socket.

Thanks!

Answer 1

http://ncannasse.fr/blog/tora_comet

I don't have any concrete information for you, but this article does mention that they can detect when the client has disconnected from Apache. See tora.Queue. And it sounds like the source is available in the neko CVS, so you might be able to find some clues there. Good luck.

Answer 2

(1) Feature. Or, more specifically, fallout from an implementation detail.

A TCP/IP connection does not involve a constant flow of traffic back and forth. Thus, there is no way to know that a client is gone without (a) the client telling you it is closing the connection or (b) a timeout.

(2) I'm not specifically familiar with Comet or CppCMS. But, yes, there are all kinds of CMS servers running behind the mentioned web servers and they all have to deal with this issue (and, yes, it is a pain).

(3) Timeouts are the only way, but you can mitigate the pain, so to speak. Have the client ping the server across the connection every N seconds when there is otherwise no activity. Doesn't have to do anything and you can tack stuff on the reply; notifications of concurrent edits or whatever you need.

You are correct in that it is surprising that mod_fastcgi doesn't support telling the backend that Apache has detected the disconnect or the connection timed out. And you aren't the first to be dismayed.

The second patch on this page should fix that particular issue:

http://osdir.com/ml/web.fastcgi.devel/2006-02/msg00015.html