C# Socket.BeginSend Sometimes Seems to Behave Synchronously?

Question

While attempting to send a message for a queue through the BeginSend call seem te behave as a blocking call.

Specificly I have:

public void Send(MyMessage message)
{
  lock(SEND_LOCK){
    var state = ...
    try {
      log.Info("Begin Sending...");
      socket.BeginSend(message.AsBytes(),0, message.ByteLength, SocketFlags.None, 
         (r) => EndSend(r), state);
      log.Info("Begin Send Complete.");
    }
    catch (SocketException e) {
      ...
    }
  }
}

The callback would be something like this:

  private void EndSend(IAsyncResult result) {
     log.Info("EndSend: Ending send.");
     var state = (MySendState) result.AsyncState;
     ...

     state.Socket.EndSend(result, out code);

     log.Info("EndSend: Send ended.");

     WaitUntilNewMessageInQueue();
     SendNextMessage();
  }

Most of the time this works fine, but sometimes it hangs. Logging indicates this happens when BeginSend en EndSend are excecuted on the same Thread. The WaitUntilNewMessageInQueue blocks until there is a new message in the queue, so when there is no new message it can wait quit a while.

As far as I can tell this should not really be a problem, but in the some cases BeginSend blocks causing a deadlock situation where EndSend is blocking on WaitUntilNewMessageInQueue (expected), but Send is blocking on BeginSend in return as it seems te be waiting for the EndSend callback te return (not expected).

This behaviour was not what I was expecting. Why does BeginSend sometimes block if the callback does not return in timely fashion?

Answer 1

First of all, why are you locking in your Send method? The lock will be released before the send is complete since you are using BeginSend. The result is that multiple sends can be executing at the same time.

Secondly, do not write (r) => EndSend(r), just write EndSend (without any parameters).

Thrid: You do not need to include the socket in your state. Your EndSend method is working like any other instance method. You can therefore access the socket field directly.

As for your deadlocks, it's hard to tell. You delegate may have something to do with it (optimizations by the compiler / runner). But I have no knowledge in that area.

Need more help? Post more code. but I suggest that you fix the issues above (all four of them) and try again first.

Answer 2

Which operating system are you running on?

Are you sure you're seeing what you think you're seeing?

The notes on the MSDN page say that Send() CAN block if there's no OS buffer space to initiate your async send unless you have put the socket in non blocking mode. Could that be the case? Are you potentially sending data very quickly and filling the TCP window to the peer? If you break into the debugger what does the call stack show?

The rest is speculation based on my understanding of the underlying native technologies involved...

The notes for Send() are likely wrong about I/O being cancelled if the thread exits, this almost certainly depends on the underlying OS as it's a low level IO Completion Port/overlapped I/O issue that changed with Windows Vista (see here: http://www.lenholgate.com/archives/000763.html) and given that they're wrong about that then they could be wrong about how the completions (calls to EndSend() are dispatched on later operating systems). From Vista onwards it's possible that the completions could be dispatched on the issuing thread if the .Net sockets wrapper is enabling the correct options on the socket (see here where I talk about FILE_SKIP_COMPLETION_PORT_ON_SUCCESS)... However, if this were the case then it's likely that you'd see this behaviour a lot as initially most sends are likely to complete 'in line' and so you'd see most completions happening on the same thread - I'm pretty sure that this is NOT the case and that .Net does NOT enable this option without asking...