How to cancel waiting in select() on Windows

Question

In my program there is one thread (receiving thread) that is responsible for receiving requests from a TCP socket and there are many threads (worker threads) that are responsible for processing the received requests. Once a request is processed I need to send an answer over TCP.

And here is a question. I would like to send TCP data in the same thread that I use for receiving data. This thread after receiving data usually waits for new data in select(). So once a worker thread finished processing a request and put an answer in the output queue it has to signal the receiving thread that there are data to send. The problem is that I don't know how to cancel waiting in select() in order to get out of waiting and to call send() .

Or shall I use another thread solely for sending data over TCP?

Updated

MSalters, Artyom thank you for you answers!

MSalters, having read your answer I found this site: Winsock 2 I/O Methods and read about WSAWaitForMultipleEvents(). My program in fact must work both on HP-UX and Windows I finally decided to use the approach that had been suggested by Artyom.

Answer 1

select is not the native API for Windows. The native way is WSAWaitForMultipleEvents. If you use this to create an alertable wait, you can use QueueUserAPC to instruct the waiting thread to send data. (This might also mean you don't have to implement your own output queue)

Answer 2

The typical model is for the worker to handle its own writing. Is there a reason why you want to send all the output-IO through selecting thread?

If you're sure of this model, you could have your workers send data back to the master thread using file descriptors as well (pipe(2)) and simply add those descriptors to your select() call.

And, if you're especially sure that you're not going to use pipes to send data back to your master process, the select call allows you to specify a timeout. You can busy-wait while checking your worker threads, and periodically call select to figure out which TCP sockets to read from.

Answer 3

You need to use something similar to safe-pipe trick, but in your case you need to use a pair of connected TCP sockets.

1. Create a pair of sockets.
2. Add one to the select and wait on it as well
3. Notify by writing to other socket from other threads.
4. Select is immediately waken-up as one of the sockets is readable, reads all the data in this special socket and check all data in queues to send/recv

How to create pair of sockets under Windows?

inline void pair(SOCKET fds[2])
{
    struct sockaddr_in inaddr;
    struct sockaddr addr;
    SOCKET lst=::socket(AF_INET, SOCK_STREAM,IPPROTO_TCP);
    memset(&inaddr, 0, sizeof(inaddr));
    memset(&addr, 0, sizeof(addr));
    inaddr.sin_family = AF_INET;
    inaddr.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
    inaddr.sin_port = 0;
    int yes=1;
    setsockopt(lst,SOL_SOCKET,SO_REUSEADDR,(char*)&yes,sizeof(yes));
    bind(lst,(struct sockaddr *)&inaddr,sizeof(inaddr));
    listen(lst,1);
    int len=sizeof(inaddr);
    getsockname(lst, &addr,&len);
    fds[0]=::socket(AF_INET, SOCK_STREAM,0);
    connect(fds[0],&addr,len);
    fds[1]=accept(lst,0,0);
    closesocket(lst);
}

Of course some checks should be added for return values.

Answer 4

Another quick&dirty solution is to add localhost sockets to the set. Now use those sockets as the inter-thread communication queues. Each worker thread simply sends something to its socket, which ends up on the corresponding socket in your receiving thread. This wakes up the select(), and your receiving thread can then echo the message on the appropriate outgoing socket.