This is my understanding of SSL, I am not an expert in the subject, but in the absence of other answers hopefully I can at least give you some things to think about.
When you create a self signed certificate for the server you need to add this to the client somehow, it doesn't get installed as soon as the client connects, otherwise any server could just become trusted by sending a self signed certificate to anything that tries to connect to it. In my application the server certificate is loaded into the truststore of the client when it is started up by specifying javax.net.ssl.truststore("path/to/server/cert");
Now, when the client connects to the server the handshake takes place. At this point the server will send it's certificate to the client and the client will confirm that it has in fact come from the server by checking it against its truststore (at this point it doesn't matter if it is self-signed or not, because the client should check the root certificates as well as any you have added). If the certificate sent by the server checks out the communication continues and data is shared.
There is some form of session behvaiour that takes place that allows the communication to proceed without having to exchange certificates every time. But I believe this is limited to the single connection, so as soon as you close the connection and create a new one the process has to be repeated, i.e. the server has to send its certificate for validation again.
So in summary: the self-signed server certificate has to be installed on the client outwith the SSL communication (like how root CA certificates are installed in a product from the start).
Every SSL connection made between client and server will require the server to send its certificate to the client so that it can check it against its truststore.
It is possible that the server will allow for sessions to be resumed, in which case the certificate will not be resent (but I am not sure under what conditions a session can be resumed, perhaps that is configurable on different servers).
Hope this at least gives you something to think about.