.NET TCP server stability issues

Question

Hi,

I have build a basic .NET server-client infrastructure using TcpListener and SocketClient. It is multithreaded and asynchronious. The problem is, that the server crashes at some point when over 30 clients are connected at once.

I could not yet locate the cause for the crashes though I do use quite a few Try-Catch blocks to make sure to log all excepions.

So I'm thinking, I may be doing something wrong conceptually in the server code. I hope you guys can help me find the cause for those crashes. The code is the following:

Starting server and listening for a connection:

 public void StartServer()
    {
        isConnected = true;
        listener.Start();
        connectionThread = new Thread(new ThreadStart(ListenForConnection));
        connectionThread.Start();
    }

    private void ListenForConnection()
    {
        while (isConnected)
        {
            try
            {
                TcpClient client = listener.AcceptTcpClient();
                ClientConnection connection = new ClientConnection(this, client);
                connections.Add(connection);
            }
            catch (Exception ex)
            {
                log.Log("Exception in ListenForConnection: " + ex.Message, LogType.Exception);
            }
        }
    }

The ClientConnection class:

 public class ClientConnection : IClientConnection
{
    private TcpClient client;
    private ISocketServer server;
    private byte[] data;
    private object metaData;

    public TcpClient TcpClient
    {
        get { return client; }
    }

    internal ClientConnection(ISocketServer server, TcpClient client)
    {
        this.client = client;
        this.server = server;

        data = new byte[client.ReceiveBufferSize];

        lock (client.GetStream())
        {
            client.GetStream().BeginRead(data, 0, client.ReceiveBufferSize, ReceiveMessage, null);
        }
    }

    internal void ReceiveMessage(IAsyncResult ar)
    {
        int bytesRead;

        try
        {
            lock (client.GetStream())
            {
                bytesRead = client.GetStream().EndRead(ar);
            }

            if (bytesRead < 1)
                return;

            byte[] toSend = new byte[bytesRead];
            for (int i = 0; i < bytesRead; i++)
                toSend[i] = data[i];

            // Throws an Event with the data in the GUI Dispatcher Thread
            server.ReceiveDataFromClient(this, toSend);

            lock (client.GetStream())
            {
                client.GetStream().BeginRead(data, 0, client.ReceiveBufferSize, ReceiveMessage, null);
            }
        }
        catch (Exception ex)
        {
            Disconnect();
        }
    }

    public void Disconnect()
    {
        // Disconnect Client
    }


}

And sending data from the server to one or all clients:

public void SendDataToAll(byte[] data)
    {
        BinaryWriter writer;

        try
        {
            foreach (IClientConnection connection in connections)
            {
                writer = new BinaryWriter(connection.TcpClient.GetStream());
                writer.Write(data);
                writer.Flush();
            }
        }
        catch (Exception ex)
        {
            // Log
        }
    }

    public void SendDataToOne(IClientConnection client, byte[] data)
    {
        BinaryWriter writer;

        try
        {
            writer = new BinaryWriter(client.TcpClient.GetStream());
            writer.Write(data);
            writer.Flush();
        }
        catch (Exception ex)
        {
            // Log
        }
    }

At some point the server crashes and I really got no starting point where to even look for the problem. If needed I can provide more code.

Any help is very appreciated :-) Andrej

Answer 1

You should make access to the connections field thread safe.

In SendData, you are iterating over connections and sending data to each client. If a new client connects when the foreach loop is executing, you will get an exception with the message "Collection was modified; enumeration operation may not execute" since the collection is modified while you are iterating over it which is not allowed.

Modify the line in SendDataToAll to

foreach (IClientConnection connection in connections.ToList())

to make the problem go away (solution is courtesy of http://stackoverflow.com/questions/604831/collection-was-modified-enumeration-operation-may-not-execute).

Answer 2

It's possible that the Disconnect call in the catch block of the ClientConnection.ReceiveMessage method is throwing an exception, which then propogates out of the catch and is unhandled.

To be sure of catching all exceptions and logging them, try registering an event handler to the AppDomain.CurrentDomain.UnhandledException event. Also, if you're running the server as a Windows Service, there may be an .NET exception entry in the Application Event Log.