The U in UDP is often believed to stand for Unreliable (*). If you use UDP, then any time (and there will be some!) you need reliability in your server's networking you will need to program your own higher level protocols to detect packet loss, perform retry and so on. This is not simple, and it is particularly hard to do in a "user space" application. TCP deals with packet loss detection, out of order packet detection, retries, and so on ... without you having to worry about it at the application level.
Another issue is that a lot of firewalls, routers, gateways etc will block all UDP traffic by default.
IMO, you should only consider using UDP in situations where:
- loss of data is acceptable, and
- maximum network performance is an absolute, overriding requirement.
[* - Actually, UDP stands for User Datagram Protocol.]