Using accept() and select() at the same time?

Question

I've got an event-driven network server program. This program accepts connections from other processes on other hosts. There may be many short-lived connections from different ports on the same remote IP.

Currently, I've got a while(1) loop which calls accept() and then spawns a thread to process the new connection. Each connection is closed after the message is read. On the remote end, the connection is closed after a message is sent.

I want to eliminate the overhead of setting up and tearing down connections by caching the open socket FDs. On the sender side, this is easy - I just don't close the connections, and keep them around.

On the receiver side, it's a bit harder. I know I can store the FD returned by accept() in a structure and listen for messages across all such sockets using poll() or select(), but I want to simultaneously both listen for new connections via accept() and listen on all the cached connections.

If I use two threads, one on poll() and one on accept(), then when the accept() call returns (a new connection is opened), I have to wake up the other thread waiting on the old set of connections. I know I can do this with a signal and pselect(), but this whole mess seems like way too much work for something so simple.

Is there a call or superior methodology that will let me simultaneously handle new connections being opened and data being sent on old connections?

Answer 1

Last time I checked, you could just listen on a socket and then select or poll to see if a connection came in. If so, accept it; it will not block (but you may want to really should set O_NONBLOCK just to be sure)

Answer 2

I'd put a listener in separate process(thread) not to mess things up. And run a worker process on another to handle existing sockets. There's no need for non-blocking listener really. And no thread overhead running 2 threads.

It should work like that: you accept on your listener thread till it returns you a descriptor of client socket and pass it to worker which is doing all dirty read/write job on it.

If you want to listen several ports and don't want to hold one process per listener I suggest you set your socket in O_NONBLOCK and do someth like:

// loop through listeners here and poll'em for read, when read is successful call accept, get descriptor, pass it to worker and continue listen
    while(1){
        foreach( serverSocket in ServerSockets ){
             if( serverSocket.Poll( 10, SelectRead ) ){
                  clientSocket = serverSocket.Accept();
                  // pass to worker here and release
             }

        }
    }