Linux TCP/IP Non-blocking send for socket stream..what happens to the TCP recv buffer?

Question

This pertains to Linux kernel 2.6 TCP sockets.

I am sending a large amount of data, say 300 MB, with a non-blocking send to another client who receives 8 MB at a time.

After one 8 MB receive, the "receiver" stops receiving because it wants to perform other tasks, such as error handling. The sender would get a EWOULDBLOCK, but since it's asynchronous communication, the send would try and fill up the TCP recv buffer on the other end.

My question is: would there still be data in the TCP recv buffer even though the "sender" got a EWOULDBLOCK and the "receiver" stops receiving? The same socket is used for error handling, so would the "receiver" have to then clear the TCP recv buffer before trying to reuse the existing socket?

Answer 1

Yes. It is quite possible (and in fact likely) that when you get EWOULDBLOCK, some data you have already sent has not yet been read by the recieving application. This buffered data will be available to the next read on the socket.

This means that if your reciever then sends a "Ooops, don't send any more" message back to the sender, the sender can't act on that message and "un-send" the data. Once it's been passed to write()/send(), it's on its way and can't be recalled.

Your receiver will have to handle this eventuality by reading out the data it's no longer interested in and discarding it, which will mean you'll need some kind of transaction delimiters in your data stream.