Let me summarize what I believe your problem is: Whenever you are hitting ctrl-c to close the client, the server exits. And you want this behavior to be handled, in a way that the server does not crash as above. If this is true, kindly read on, I think I may be able to shed some light on the actual issue, it does not really have much to do with using (or not using) a proper signal handle.
When you are closing the client using ctrl-c, OS will stop the client process and close all the sockets opened by it. As part of that, a FIN will be sent to the server.
Please review your code. I believe you would have written the code in a way that it is blocked in a read() call from the socket with the client concerned. Now, when the FIN sent above reaches the server side socket, this read will return with a value of 0.
If your code does not check for this condition and tries to write something back to the client, ultimately (I skip few steps that happen intermediately) it will receive a SIGPIPE. This will cause your server to crash.
If the above scenario is true, you have 2 way outs:
1. check for read() returning 0 and gracefully continue.
2. install a signal handler for SIGPIPE.
I prefer the 1st method, btw.