tags:

views:

1011

answers:

4

I recently wrote a quick-and-dirty proof-of-concept proxy server in C# as part of an effort to get a Java web application to communicate with a legacy VB6 application residing on another server. It's ridiculously simple:

The proxy server and clients both use the same message format; in the code I use a ProxyMessage class to represent both requests from clients and responses generated by the server:

public class ProxyMessage
{
   int Length; // message length (not including the length bytes themselves)
   string Body; // an XML string containing a request/response

   // writes this message instance in the proper network format to stream 
   // (helper for response messages)
   WriteToStream(Stream stream) { ... }
}

The messages are as simple as could be: the length of the body + the message body.

I have a separate ProxyClient class that represents a connection to a client. It handles all the interaction between the proxy and a single client.

What I'm wondering is are they are design patterns or best practices for simplifying the boilerplate code associated with asynchronous socket programming? For example, you need to take some care to manage the read buffer so that you don't accidentally lose bytes, and you need to keep track of how far along you are in the processing of the current message. In my current code, I do all of this work in my callback function for TcpClient.BeginRead, and manage the state of the buffer and the current message processing state with the help of a few instance variables.

The code for my callback function that I'm passing to BeginRead is below, along with the relevant instance variables for context. The code seems to work fine "as-is", but I'm wondering if it can be refactored a bit to make it clearer (or maybe it already is?).

private enum BufferStates 
{ 
    GetMessageLength, 
    GetMessageBody 
}
// The read buffer. Initially 4 bytes because we are initially
// waiting to receive the message length (a 32-bit int) from the client 
// on first connecting. By constraining the buffer length to exactly 4 bytes,
// we make the buffer management a bit simpler, because
// we don't have to worry about cases where the buffer might contain
// the message length plus a few bytes of the message body.
// Additional bytes will simply be buffered by the OS until we request them.
byte[] _buffer = new byte[4];

// A count of how many bytes read so far in a particular BufferState.
int _totalBytesRead = 0;

// The state of the our buffer processing. Initially, we want
// to read in the message length, as it's the first thing
// a client will send
BufferStates _bufferState = BufferStates.GetMessageLength;

// ...ADDITIONAL CODE OMITTED FOR BREVITY...

// This is called every time we receive data from
// the client.

private void ReadCallback(IAsyncResult ar)
{
    try
    {
        int bytesRead = _tcpClient.GetStream().EndRead(ar);

        if (bytesRead == 0)
        {
            // No more data/socket was closed.
            this.Dispose();
            return;
        }

        // The state passed to BeginRead is used to hold a ProxyMessage
        // instance that we use to build to up the message 
        // as it arrives.
        ProxyMessage message = (ProxyMessage)ar.AsyncState;

        if(message == null)
            message = new ProxyMessage();

        switch (_bufferState)
        {
            case BufferStates.GetMessageLength:

                _totalBytesRead += bytesRead;

                // if we have the message length (a 32-bit int)
                // read it in from the buffer, grow the buffer
                // to fit the incoming message, and change
                // state so that the next read will start appending
                // bytes to the message body

                if (_totalBytesRead == 4)
                {
                    int length = BitConverter.ToInt32(_buffer, 0);
                    message.Length = length;
                    _totalBytesRead = 0;
                    _buffer = new byte[message.Length];
                    _bufferState = BufferStates.GetMessageBody;
                }

                break;

            case BufferStates.GetMessageBody:

                string bodySegment = Encoding.ASCII.GetString(_buffer, _totalBytesRead, bytesRead);
                _totalBytesRead += bytesRead;

                message.Body += bodySegment;

                if (_totalBytesRead >= message.Length)
                {
                    // Got a complete message.
                    // Notify anyone interested.

                    // Pass a response ProxyMessage object to 
                    // with the event so that receivers of OnReceiveMessage
                    // can send a response back to the client after processing
                    // the request.
                    ProxyMessage response = new ProxyMessage();
                    OnReceiveMessage(this, new ProxyMessageEventArgs(message, response));
                    // Send the response to the client
                    response.WriteToStream(_tcpClient.GetStream());

                    // Re-initialize our state so that we're
                    // ready to receive additional requests...
                    message = new ProxyMessage();
                    _totalBytesRead = 0;
                    _buffer = new byte[4]; //message length is 32-bit int (4 bytes)
                    _bufferState = BufferStates.GetMessageLength;
                }

                break;
        }

        // Wait for more data...
        _tcpClient.GetStream().BeginRead(_buffer, 0, _buffer.Length, this.ReadCallback, message);
    }
    catch
    {
        // do nothing
    }

}

So far, my only real thought is to extract the buffer-related stuff into a separate MessageBuffer class and simply have my read callback append new bytes to it as they arrive. The MessageBuffer would then worry about things like the current BufferState and fire an event when it received a complete message, which the ProxyClient could then propagate further up to the main proxy server code, where the request can be processed.

+1  A: 

I think the design you've used is fine that's roughly how I would and have done the same sort of thing. I don't think you'd gain much by refactoring into additional classes/structs and from what I've seen you'd actually make the solution more complex by doing so.

The only comment I'd have is as to whether the two reads where the first is always the messgae length and the second always being the body is robust enough. I'm always wary of approaches like that as if they somehow get out of sync due to an unforseen circumstance (such as the other end sending the wrong length) it's very difficult to recover. Instead I'd do a single read with a big buffer so that I always get all the available data from the network and then inspect the buffer to extract out complete messages. That way if things do go wrong the current buffer can just be thrown away to get things back to a clean state and only the current messages are lost rather than stopping the whole service.

Actually at the moment you would have a problem if you message body was big and arrived in two seperate receives and the next message in line sent it's length at the same time as the second half of the previous body. If that happened your message length would end up appended to the body of the previous message and you'd been in the situation as desecribed in the previous paragraph.

sipwiz
Hmm. I agree that a client sending the wrong length will definitely mess things up given the way the code is written, and I debated how best to handle it. I'm leaning more towards if the client isn't sending a properly-formatted message, then too bad. Garbage in, garbage out ;-)
Mike Spross
Good catch in your last paragraph. Looking at my code again, I think you're right about two messages potentially overlapping. The irony is that I was resizing the buffer to avoid that problem in the first place, but didn't think about what would happen if a big message got split by the network. Oops
Mike Spross
Actually, after thinking about it some more, I think my approach still works. It's perfectly possible that the socket buffer at the OS level could contain the end of one message and the beginning of the next; however, BeginRead/EndRead guarantee to only read as many bytes as will fit...
Mike Spross
...in the buffer passed to BeginRead. Specfically, BeginRead won't read more than (buffer.Length - current_offset_in_buffer) bytes. If there are more bytes from another message, they will remain in the socket buffer at the OS level until the next BeginRead call. On top of that...
Mike Spross
...the TCP/IP protocol guarantees that the client will never send more bytes than the receiving socket can handle (the "sliding window"), so even the low-level socket buffer won't ever overflow, unless there's a bug in the TCP/IP stack. So, I think I'm OK. Your first point is still valid though.
Mike Spross
True you can never get more bytes than the size of your buffer but you will often get less and thats where you'll have a problem. If you get say half the messge on one read the next read will still have a buffer the same size available and you are giving it the full capacity each read.
sipwiz
+2  A: 

I've had to overcome similar problems. Here's my solution (modified to fit your own example).

We create a wrapper around Stream (a superclass of NetworkStream, which is a superclass of TcpClient or whatever). It monitors reads. When some data is read, it is buffered. When we receive a length indicator (4 bytes) we check if we have a full message (4 bytes + message body length). When we do, we raise a MessageReceived event with the message body, and remove the message from the buffer. This technique automatically handles fragmented messages and multiple-messages-per-packet situations.

public class MessageStream : IMessageStream, IDisposable
{
    public MessageStream(Stream stream)
    {
     if(stream == null)
      throw new ArgumentNullException("stream", "Stream must not be null");

     if(!stream.CanWrite || !stream.CanRead)
      throw new ArgumentException("Stream must be readable and writable", "stream");

     this.Stream = stream;
     this.readBuffer = new byte[512];
     messageBuffer = new List<byte>();
     stream.BeginRead(readBuffer, 0, readBuffer.Length, new AsyncCallback(ReadCallback), null);
    }

    // These belong to the ReadCallback thread only.
    private byte[] readBuffer;
    private List<byte> messageBuffer;

    private void ReadCallback(IAsyncResult result)
    {
     int bytesRead = Stream.EndRead(result);
     messageBuffer.AddRange(readBuffer.Take(bytesRead));

     if(messageBuffer.Count >= 4)
     {
      int length = BitConverter.ToInt32(messageBuffer.Take(4).ToArray(), 0); // 4 bytes per int32

      // Keep buffering until we get a full message.

      if(messageBuffer.Count >= length + 4)
      {
       messageBuffer.Skip(4);
       OnMessageReceived(new MessageEventArgs(messageBuffer.Take(length)));
       messageBuffer.Skip(length);
      }
     }

     // FIXME below is kinda hacky (I don't know the proper way of doing things...)

     // Don't bother reading again.  We don't have stream access.
     if(disposed)
      return;

     try
     {
      Stream.BeginRead(readBuffer, 0, readBuffer.Length, new AsyncCallback(ReadCallback), null);
     }
     catch(ObjectDisposedException)
     {
      // DO NOTHING
      // Ends read loop.
     }
    }

    public Stream Stream
    {
     get;
     private set;
    }

    public event EventHandler<MessageEventArgs> MessageReceived;

    protected virtual void OnMessageReceived(MessageEventArgs e)
    {
     var messageReceived = MessageReceived;

     if(messageReceived != null)
      messageReceived(this, e);
    }

    public virtual void SendMessage(Message message)
    {
     // Have fun ...
    }

    // Dispose stuff here
}
strager
I liked all the answers here, so +1 for everyone else that answered, but in the end I went with something very similar to this answer. It's simple and easy to understand, so I'll remember what I was doing a few months from now ;)
Mike Spross
+1  A: 

You can use yield return to automate the generation of a state machine for asynchronous callbacks. Jeffrey Richter promotes this technique through his AsyncEnumerator class, and I've played around with the idea here.

Daniel Earwicker
+1  A: 

There's nothing wrong with the way you've done it. For me, though, I like to separate the receiving of the data from the processing of it, which is what you seem to be thinking with your proposed MessageBuffer class. I have discussed that in detail here.

Matt Davis
Yeah, that's exactly what I was thinking: wanting to separate receiving from the actual message processing logic.
Mike Spross
What I didn't mention in my question was that the proxy server will eventually have to process different protocols (it will basically tunnel messages in a common proxy message format), so I guess I was looking at a good way to dispatch different logic based on the protocol being tunneled.
Mike Spross