views:

79

answers:

2

I've got an event-driven network server program. This program accepts connections from other processes on other hosts. There may be many short-lived connections from different ports on the same remote IP.

Currently, I've got a while(1) loop which calls accept() and then spawns a thread to process the new connection. Each connection is closed after the message is read. On the remote end, the connection is closed after a message is sent.

I want to eliminate the overhead of setting up and tearing down connections by caching the open socket FDs. On the sender side, this is easy - I just don't close the connections, and keep them around.

On the receiver side, it's a bit harder. I know I can store the FD returned by accept() in a structure and listen for messages across all such sockets using poll() or select(), but I want to simultaneously both listen for new connections via accept() and listen on all the cached connections.

If I use two threads, one on poll() and one on accept(), then when the accept() call returns (a new connection is opened), I have to wake up the other thread waiting on the old set of connections. I know I can do this with a signal and pselect(), but this whole mess seems like way too much work for something so simple.

Is there a call or superior methodology that will let me simultaneously handle new connections being opened and data being sent on old connections?

+1  A: 

Last time I checked, you could just listen on a socket and then select or poll to see if a connection came in. If so, accept it; it will not block (but you may want to really should set O_NONBLOCK just to be sure)

mvds
Hmm, that's a well-known race condition - the `accept(2)` will block if client drops connection attempt between two syscalls. You *need* the listening socket to be non-blocking.
Nikolai N Fetissov
This is correct - you can add your listening file descriptor to the `readfds` in your `select()` call, and `select()` will tell you the file descriptor is "readable" if it has a connection ready to `accept()`. @Nikolai is correct too - the listening socket should be nonblocking and the `accept()` call prepared to handle `EAGAIN`.
caf
Updated the wording of the answer to reflect this race condition more clearly.
mvds
A: 

I'd put a listener in separate process(thread) not to mess things up. And run a worker process on another to handle existing sockets. There's no need for non-blocking listener really. And no thread overhead running 2 threads.

It should work like that: you accept on your listener thread till it returns you a descriptor of client socket and pass it to worker which is doing all dirty read/write job on it.

If you want to listen several ports and don't want to hold one process per listener I suggest you set your socket in O_NONBLOCK and do someth like:

// loop through listeners here and poll'em for read, when read is successful call accept, get descriptor, pass it to worker and continue listen
    while(1){
        foreach( serverSocket in ServerSockets ){
             if( serverSocket.Poll( 10, SelectRead ) ){
                  clientSocket = serverSocket.Accept();
                  // pass to worker here and release
             }

        }
    }
hoodoos
That's spinning you've got there; no good. And one thread per socket is way too many threads.
Borealid
what's so bad about this spinning? :) it won't eat any CPU, besides it allows you to listen many ports in one thread.
hoodoos