I am using TCP for communicating with an arduino (just open a socket and wait for a connection)using an ethernet shield, While watching/reading about various other project that use some sort of network interface for communication they all seem to use UDP instead of TCP for communication. What I was wondering is what would be my gain if I use UDP instead?
A UDP stack is considerably simpler than a TCP stack. You can easily write a UDP stack from scratch on your own, TCP is a bit harder, doable but harder. TCP has built in retries and other things so you dont lose reliability with UDP directly, it is what you do with it that could compare. UDP is significantly faster than TCP and is why it is or was used for video and various things back in the day. Also things like video could stand to lose a packet here and there and didnt care. For embedded UDP is quite nice for being small, fast, etc. If you are using someone elses library then UDP is likely not going to save you much on memory/flash resources, it will still be a bit faster. It is when you implement your own UDP that you save quite a bit on memory, because you can cut corners. You can do things like only implement arp and udp and nothing else (although ping is useful but somehow painful), and you can cut corners on arp/rarp depending on what you need to do with this thing. You can implement support only for the packet size you are interested in. Numbering your packets and having the requesting side send two or three of everything and responding to every request can greatly reduce the lost packet problem. Keeping the packet size very small helps both the embedded resource problem and avoids any mtu or other problems along the way. For simplicity you can even force a specific packet length.
I always ask the question the other way, what would I stand to gain by using TCP. There are times when it is useful, embedded, desktop or server though I still ask that question every time and have to justify the use of TCP over UDP otherwise I wont use it.
You gain code space, data memory, and determinism.
A fair amount of memory is required to reassemble a TCP stream, unless you want to NAK every packet that's not in-order. They are never guaranteed to come in order....
An asynchronous command-response protocol with timeouts, where all commands and responses fit into a single UDP packet, and commands are idempotent (can be applied many times and maintain the correct result) is a pretty robust protocol.