I can see that stateful protocols lead to less botched together 'emulated state' like cookies.
but testing becomes a lot harder to ensure that your implementation is correct and reconnects, and session continuations can be very hard to handle.
Is it considered better practice to always use stateless protocols, or is it really domain specific?
I think that authentication becomes easier when dealing with stateful protocols, but are there any other reasons you should use a stateful protocol?
Another nice thing with stateless protocols is that it's easier to handle server failover situations and/or clustering/load-balancing situations.
A stateless protocol is easier to cluster since state never needs to be transfered from 1 server to another upon subsequent requests.
Advantages of stateless:
How important is state to your application? Do you need a constant flow of data between different machines or is it more useful to have bursts? If you're writing an IP Telephony type application then you're probably going to want something fairly stateful, if you can get away with stateless it's likely to be cheaper and easier to do it that way. Doing things statefully is necessarily more fragile because if either end of the connection gets dropped or the connection itself goes down you run a higher risk of data loss, whereas with a stateless connection you are more likely just to have to wait for a while and try again.
They really are different tools for different jobs, but given the ease and ubiquity of stateless technologies online it's logical to look in that direction when you have the option.
I would consider it domain specific. If you're writing the moral equivalent of ping, a stateless protocol is the right choice. On the other hand, if you are writing a VNC, stateful is surely the way to go.
As for when to choose which, there are two points to bear in mind. First, while the implementation choices are either/or, the problem space is a continuum. All real world tasks have at least a little state, the question is how much and is the overhead of passing it around worth the hassle of tracking it at both ends. And second, you are generally dealing with a protocol stack, not a single protocol; making sure that everything lives at the right level can simplify things enormously.
Stateful is better. Then you don't have to send the state all the time. The protocol then becomes simpler.
I'm not personally familiar with all of the design issues of stateful versus stateless, but I do know that NFSv4 is stateful after 15 years' worth of previous versions of NFS being stateless, so apparently statelessness became a significant limitation for the NFS designers.
A few minutes' Googling reveals several articles and blogs talking about NFSv4's statefulness; this should be interesting reading for some of the design issues involved.