Uncaught exceptions in CLR code would not bring down the server. CLR code runs in a separate app domain, and is isolated. The most that could possibly happen (and this is unlikely) is that the exception would bring down the app domain, and cause the app domain to be unloaded. This would make it either a) reload on the next request, or b) be disabled till the problem is fixed or a new version of the assembly is deployed.
So at most that particular CLR assembly would get disabled, and calling sessions would get error messages. It's extremely unlikely that faulty CLR code would bring down an entire sql server instance; if it does, I would say that was a microsoft bug, not a bug in the CLR function or stored proc.
The highly likely case is that the uncaught .net exception would be translated into a tsql error message. At most, the tsql connection would be terminated; mostly likely it would result in an error message.