What is stopping data flow with .NET 3.5 asynchronous System.Net.Sockets.Socket?

Question

I have a .NET 3.5 client/server socket interface using the asynchronous methods. The client connects to the server and the connection should remain open until the app terminates. The protocol consists of the following pattern:

1. send stx
2. receive ack
3. send data1
4. receive ack
5. send data2 (repeat 5-6 while more data)
6. receive ack
7. send etx

So a single transaction with two datablocks as above would consist of 4 sends from the client. After sending etx the client simply waits for more data to send out, then begins the next transmission with stx. I do not want to break the connection between individual exchanges or after each stx/data/etx payload.

Right now, after connection, the client can send the first stx, and get a single ack, but I can't put more data onto the wire after that. Neither side disconnects, the socket is still intact. The client code is seriously abbreviated as follows - I'm following the pattern commonly available in online code samples.

private void SendReceive(string data) {
    // ...
    SocketAsyncEventArgs completeArgs;
    completeArgs.Completed += new EventHandler<SocketAsyncEventArgs>(OnSend);
    clientSocket.SendAsync(completeArgs);
    // two AutoResetEvents, one for send, one for receive
    if ( !AutoResetEvent.WaitAll(autoSendReceiveEvents , -1) )
       Log("failed");
    else
       Log("success");
    // ...
}
private void OnSend( object sender , SocketAsyncEventArgs e ) {
// ...
    Socket s = e.UserToken as Socket;
    byte[] receiveBuffer = new byte[ 4096 ];
    e.SetBuffer(receiveBuffer , 0 , receiveBuffer.Length);
    e.Completed += new EventHandler<SocketAsyncEventArgs>(OnReceive);
    s.ReceiveAsync(e);
// ...
}
private void OnReceive( object sender , SocketAsyncEventArgs e ) {}
    // ...
    if ( e.BytesTransferred > 0 ) {
        Int32 bytesTransferred = e.BytesTransferred;
        String received = Encoding.ASCII.GetString(e.Buffer , e.Offset , bytesTransferred);
        dataReceived += received;
    }
    autoSendReceiveEvents[ SendOperation ].Set(); // could be moved elsewhere
    autoSendReceiveEvents[ ReceiveOperation ].Set(); // releases mutexes
}

The code on the server is very similar except that it receives first and then sends a response - the server is not doing anything (that I can tell) to modify the connection after it sends a response. The problem is that the second time I hit SendReceive in the client, the connection is already in a weird state.

Do I need to do something in the client to preserve the SocketAsyncEventArgs, and re-use the same object for the lifetime of the socket/connection? I'm not sure which eventargs object should hang around during the life of the connection or a given exchange.

Do I need to do something, or Not do something in the server to ensure it continues to Receive data?

The server setup and response processing looks like this:

void Start() {
    // ...
    listenSocket.Bind(...);
    listenSocket.Listen(0);
    StartAccept(null); // note accept as soon as we start. OK?
    mutex.WaitOne();
}
void StartAccept(SocketAsyncEventArgs acceptEventArg) {
    if ( acceptEventArg == null )
    {
        acceptEventArg = new SocketAsyncEventArgs();
        acceptEventArg.Completed += new EventHandler<SocketAsyncEventArgs>(OnAcceptCompleted);
    }
    Boolean willRaiseEvent = this.listenSocket.AcceptAsync(acceptEventArg);
    if ( !willRaiseEvent )
        ProcessAccept(acceptEventArg);
    // ...
}
private void OnAcceptCompleted( object sender , SocketAsyncEventArgs e ) {
    ProcessAccept(e);
}
private void ProcessAccept( SocketAsyncEventArgs e ) {
    // ...
    SocketAsyncEventArgs readEventArgs = new SocketAsyncEventArgs();
    readEventArgs.SetBuffer(dataBuffer , 0 , Int16.MaxValue);
    readEventArgs.Completed += new EventHandler<SocketAsyncEventArgs>(OnIOCompleted);
    readEventArgs.UserToken = e.AcceptSocket;
    dataReceived = ""; // note server is degraded for single client/thread use
    // As soon as the client is connected, post a receive to the connection.
    Boolean willRaiseEvent = e.AcceptSocket.ReceiveAsync(readEventArgs);
    if ( !willRaiseEvent )
        this.ProcessReceive(readEventArgs);
    // Accept the next connection request.
    this.StartAccept(e);
}
private void OnIOCompleted( object sender , SocketAsyncEventArgs e ) {
    // switch ( e.LastOperation )
    case SocketAsyncOperation.Receive:
        ProcessReceive(e); // similar to client code
        // operate on dataReceived here
    case SocketAsyncOperation.Send:
        ProcessSend(e); // similar to client code
}
// execute this when a data has been processed into a response (ack, etc)
private SendResponseToClient(string response) {
    // create buffer with response
    // currentEventArgs has class scope and is re-used
    currentEventArgs.SetBuffer(sendBuffer , 0 , sendBuffer.Length);
    Boolean willRaiseEvent = currentClient.SendAsync(currentEventArgs);
    if ( !willRaiseEvent )
        ProcessSend(currentEventArgs);
}

A .NET trace shows the following when sending ABC\r\n:

Socket#7588182::SendAsync()
Socket#7588182::SendAsync(True#1)
Data from Socket#7588182::FinishOperation(SendAsync)
00000000 : 41 42 43 0D 0A
Socket#7588182::ReceiveAsync()
Exiting Socket#7588182::ReceiveAsync()         -> True#1

And it stops there. It looks just like the first send from the client but the server shows no activity.

I think that could be info overload for now but I'll be happy to provide more details as required.

Thanks!

Answer 1

Firstly, I'm sure you know, but it's worth repeating; a TCP connection is a stream of bytes. Each read can return between 1 byte and the size of the buffer that you used depending on the amount of data that has arrived. Just because you send the data with 3 sends doesn't mean that you'll need 3 recvs to read it, you could get it all in one, or each recv could return a single byte. It's up to YOU to deal with message framing as you're the only one that knows about it. TCP is a stream of bytes.

Also WHY are you using those events? If you don't want to use asynchronous data flow then don't use the async functions, write something very simple with the sync socket functions and remove all the pointless complexity of using an async API and then using synchronisation primitives to hobble it.

Answer 2

Your protocol is fine, but you need to do Message Framing. For an example, see code showing how to send objects across the wire

Basically, your server (and client) are assuming that One Send == One receive (this is the same as what Len is saying above). This is not valid for TCP, because TCP communication is byte oriented. Therefore you need to send some sort of delimiter to delineate each sent packet. In my example, I use the length of the packet to delineate it, but you can also use a special token (for eg, FormFeed, LineFeed, CarriageReturn etc).